2024-09-12T01:06:27.320 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-12T01:06:27.345 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-12T01:06:27.443 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901536 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/suites/fsx}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7901536' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.dir_split install: ceph: flavor: default sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 481 sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc sleep_before_teardown: 0 subset: 57/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi001.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMxgmI+uZRzRU/LCwEn+4h1qvPB3Taq0SJpx3o7iFsiYNYOL2vS/MRJsPxDAz96eeRgq13FXAwA87W4MWmCD1TY= smithi053.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPpRAGQxZ00R7LWniWGTlKmZVT7klYy1UN33/S2Aqa6NMY+9b40e5DEMaSJy8DWWfRJRylAzLw0mDDRQoYXTmxY= smithi104.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB23/D5Pl9i4XO3CMB0ey/FQmpU0rEXR2wyDtBw4MyQlU4FDsPLKWRVPDPJXqSHVypZZsP/F2qWgp+UW41B9J9Q= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/fsx.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-11_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-12T01:06:27.443 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa; will attempt to use it 2024-09-12T01:06:27.444 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks 2024-09-12T01:06:27.444 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-12T01:06:27.446 INFO:teuthology.task.internal:Checking packages... 2024-09-12T01:06:27.466 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc' 2024-09-12T01:06:27.467 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-12T01:06:27.467 INFO:teuthology.packaging:ref: None 2024-09-12T01:06:27.467 INFO:teuthology.packaging:tag: None 2024-09-12T01:06:27.467 INFO:teuthology.packaging:branch: squid 2024-09-12T01:06:27.467 INFO:teuthology.packaging:sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T01:06:27.467 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-12T01:06:27.686 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-314.g086c737f 2024-09-12T01:06:27.687 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-12T01:06:27.698 INFO:teuthology.task.internal:no buildpackages task found 2024-09-12T01:06:27.698 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-12T01:06:27.716 INFO:teuthology.task.internal:Saving configuration 2024-09-12T01:06:27.735 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-12T01:06:27.752 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-12T01:06:27.785 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi001.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901536', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 01:01:34.211555', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMxgmI+uZRzRU/LCwEn+4h1qvPB3Taq0SJpx3o7iFsiYNYOL2vS/MRJsPxDAz96eeRgq13FXAwA87W4MWmCD1TY='} 2024-09-12T01:06:27.812 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi053.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901536', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 01:01:34.213315', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPpRAGQxZ00R7LWniWGTlKmZVT7klYy1UN33/S2Aqa6NMY+9b40e5DEMaSJy8DWWfRJRylAzLw0mDDRQoYXTmxY='} 2024-09-12T01:06:27.837 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi104.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901536', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 01:01:34.209597', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB23/D5Pl9i4XO3CMB0ey/FQmpU0rEXR2wyDtBw4MyQlU4FDsPLKWRVPDPJXqSHVypZZsP/F2qWgp+UW41B9J9Q='} 2024-09-12T01:06:27.837 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-12T01:06:27.846 INFO:teuthology.task.internal:roles: ubuntu@smithi001.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-12T01:06:27.846 INFO:teuthology.task.internal:roles: ubuntu@smithi053.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-12T01:06:27.846 INFO:teuthology.task.internal:roles: ubuntu@smithi104.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-12T01:06:27.846 INFO:teuthology.run_tasks:Running task console_log... 2024-09-12T01:06:27.984 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa351503be0>, signals=[15]) 2024-09-12T01:06:27.985 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-12T01:06:27.995 INFO:teuthology.task.internal:Opening connections... 2024-09-12T01:06:27.995 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi001.front.sepia.ceph.com 2024-09-12T01:06:27.997 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T01:06:28.097 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi053.front.sepia.ceph.com 2024-09-12T01:06:28.098 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T01:06:28.178 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi104.front.sepia.ceph.com 2024-09-12T01:06:28.179 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T01:06:28.263 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-12T01:06:28.271 DEBUG:teuthology.orchestra.run.smithi001:> uname -m 2024-09-12T01:06:28.289 INFO:teuthology.orchestra.run.smithi001.stdout:x86_64 2024-09-12T01:06:28.289 DEBUG:teuthology.orchestra.run.smithi001:> cat /etc/os-release 2024-09-12T01:06:28.348 INFO:teuthology.orchestra.run.smithi001.stdout:NAME="CentOS Stream" 2024-09-12T01:06:28.348 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION="9" 2024-09-12T01:06:28.348 INFO:teuthology.orchestra.run.smithi001.stdout:ID="centos" 2024-09-12T01:06:28.349 INFO:teuthology.orchestra.run.smithi001.stdout:ID_LIKE="rhel fedora" 2024-09-12T01:06:28.349 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION_ID="9" 2024-09-12T01:06:28.349 INFO:teuthology.orchestra.run.smithi001.stdout:PLATFORM_ID="platform:el9" 2024-09-12T01:06:28.349 INFO:teuthology.orchestra.run.smithi001.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T01:06:28.349 INFO:teuthology.orchestra.run.smithi001.stdout:ANSI_COLOR="0;31" 2024-09-12T01:06:28.349 INFO:teuthology.orchestra.run.smithi001.stdout:LOGO="fedora-logo-icon" 2024-09-12T01:06:28.349 INFO:teuthology.orchestra.run.smithi001.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T01:06:28.349 INFO:teuthology.orchestra.run.smithi001.stdout:HOME_URL="https://centos.org/" 2024-09-12T01:06:28.349 INFO:teuthology.orchestra.run.smithi001.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T01:06:28.349 INFO:teuthology.orchestra.run.smithi001.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T01:06:28.349 INFO:teuthology.orchestra.run.smithi001.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T01:06:28.350 INFO:teuthology.lock.ops:Updating smithi001.front.sepia.ceph.com on lock server 2024-09-12T01:06:28.380 DEBUG:teuthology.orchestra.run.smithi053:> uname -m 2024-09-12T01:06:28.398 INFO:teuthology.orchestra.run.smithi053.stdout:x86_64 2024-09-12T01:06:28.399 DEBUG:teuthology.orchestra.run.smithi053:> cat /etc/os-release 2024-09-12T01:06:28.456 INFO:teuthology.orchestra.run.smithi053.stdout:NAME="CentOS Stream" 2024-09-12T01:06:28.456 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION="9" 2024-09-12T01:06:28.456 INFO:teuthology.orchestra.run.smithi053.stdout:ID="centos" 2024-09-12T01:06:28.456 INFO:teuthology.orchestra.run.smithi053.stdout:ID_LIKE="rhel fedora" 2024-09-12T01:06:28.456 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION_ID="9" 2024-09-12T01:06:28.457 INFO:teuthology.orchestra.run.smithi053.stdout:PLATFORM_ID="platform:el9" 2024-09-12T01:06:28.457 INFO:teuthology.orchestra.run.smithi053.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T01:06:28.457 INFO:teuthology.orchestra.run.smithi053.stdout:ANSI_COLOR="0;31" 2024-09-12T01:06:28.457 INFO:teuthology.orchestra.run.smithi053.stdout:LOGO="fedora-logo-icon" 2024-09-12T01:06:28.457 INFO:teuthology.orchestra.run.smithi053.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T01:06:28.457 INFO:teuthology.orchestra.run.smithi053.stdout:HOME_URL="https://centos.org/" 2024-09-12T01:06:28.457 INFO:teuthology.orchestra.run.smithi053.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T01:06:28.457 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T01:06:28.457 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T01:06:28.457 INFO:teuthology.lock.ops:Updating smithi053.front.sepia.ceph.com on lock server 2024-09-12T01:06:28.488 DEBUG:teuthology.orchestra.run.smithi104:> uname -m 2024-09-12T01:06:28.507 INFO:teuthology.orchestra.run.smithi104.stdout:x86_64 2024-09-12T01:06:28.507 DEBUG:teuthology.orchestra.run.smithi104:> cat /etc/os-release 2024-09-12T01:06:28.565 INFO:teuthology.orchestra.run.smithi104.stdout:NAME="CentOS Stream" 2024-09-12T01:06:28.565 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION="9" 2024-09-12T01:06:28.565 INFO:teuthology.orchestra.run.smithi104.stdout:ID="centos" 2024-09-12T01:06:28.565 INFO:teuthology.orchestra.run.smithi104.stdout:ID_LIKE="rhel fedora" 2024-09-12T01:06:28.565 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION_ID="9" 2024-09-12T01:06:28.565 INFO:teuthology.orchestra.run.smithi104.stdout:PLATFORM_ID="platform:el9" 2024-09-12T01:06:28.565 INFO:teuthology.orchestra.run.smithi104.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T01:06:28.566 INFO:teuthology.orchestra.run.smithi104.stdout:ANSI_COLOR="0;31" 2024-09-12T01:06:28.566 INFO:teuthology.orchestra.run.smithi104.stdout:LOGO="fedora-logo-icon" 2024-09-12T01:06:28.566 INFO:teuthology.orchestra.run.smithi104.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T01:06:28.566 INFO:teuthology.orchestra.run.smithi104.stdout:HOME_URL="https://centos.org/" 2024-09-12T01:06:28.566 INFO:teuthology.orchestra.run.smithi104.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T01:06:28.566 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T01:06:28.566 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T01:06:28.566 INFO:teuthology.lock.ops:Updating smithi104.front.sepia.ceph.com on lock server 2024-09-12T01:06:28.596 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-12T01:06:28.606 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-12T01:06:28.615 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-12T01:06:28.616 DEBUG:teuthology.orchestra.run.smithi001:> test '!' -e /home/ubuntu/cephtest 2024-09-12T01:06:28.618 DEBUG:teuthology.orchestra.run.smithi053:> test '!' -e /home/ubuntu/cephtest 2024-09-12T01:06:28.621 DEBUG:teuthology.orchestra.run.smithi104:> test '!' -e /home/ubuntu/cephtest 2024-09-12T01:06:28.637 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-12T01:06:28.643 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-12T01:06:28.644 DEBUG:teuthology.orchestra.run.smithi001:> test -z $(ls -A /var/lib/ceph) 2024-09-12T01:06:28.676 DEBUG:teuthology.orchestra.run.smithi053:> test -z $(ls -A /var/lib/ceph) 2024-09-12T01:06:28.678 DEBUG:teuthology.orchestra.run.smithi104:> test -z $(ls -A /var/lib/ceph) 2024-09-12T01:06:28.712 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-12T01:06:28.812 INFO:teuthology.run_tasks:Running task kernel... 2024-09-12T01:06:28.829 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-12T01:06:28.829 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-09-12T01:06:28.830 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-09-12T01:06:28.830 DEBUG:teuthology.orchestra.run.smithi001:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T01:06:28.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T01:06:28.847 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-09-12T01:06:29.019 DEBUG:teuthology.task.kernel:sha1 for client.0 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-12T01:06:29.019 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-12T01:06:29.019 DEBUG:teuthology.orchestra.run.smithi001:> uname -r 2024-09-12T01:06:29.037 INFO:teuthology.orchestra.run.smithi001.stdout:5.14.0-503.el9.x86_64 2024-09-12T01:06:29.037 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-12T01:06:29.038 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-12T01:06:29.038 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum upgrade -y linux-firmware 2024-09-12T01:06:32.615 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - BaseOS 4.5 MB/s | 8.2 MB 00:01 2024-09-12T01:06:35.624 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - AppStream 11 MB/s | 20 MB 00:01 2024-09-12T01:06:40.275 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - CRB 8.9 MB/s | 6.5 MB 00:00 2024-09-12T01:06:41.851 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - Extras packages 66 kB/s | 19 kB 00:00 2024-09-12T01:06:42.763 INFO:teuthology.orchestra.run.smithi001.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-12T01:06:48.453 INFO:teuthology.orchestra.run.smithi001.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-12T01:06:50.639 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2024-09-12T01:06:50.640 INFO:teuthology.orchestra.run.smithi001.stdout:Nothing to do. 2024-09-12T01:06:50.640 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2024-09-12T01:06:50.753 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-12T01:06:50.753 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-12T01:06:50.914 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-12T01:06:50.914 DEBUG:teuthology.orchestra.run.smithi001:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-12T01:06:50.917 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi001.front.sepia.ceph.com to complete... 2024-09-12T01:07:02.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12 01:07:02 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-12T01:07:02.685 INFO:teuthology.orchestra.run.smithi001.stderr:FINISHED --2024-09-12 01:07:02-- 2024-09-12T01:07:02.685 INFO:teuthology.orchestra.run.smithi001.stderr:Total wall clock time: 12s 2024-09-12T01:07:02.685 INFO:teuthology.orchestra.run.smithi001.stderr:Downloaded: 1 files, 343M in 11s (30.0 MB/s) 2024-09-12T01:07:02.688 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-12T01:07:02.688 DEBUG:teuthology.orchestra.run.smithi001:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-12T01:07:04.900 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying... ######################################## 2024-09-12T01:07:05.051 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing... ######################################## 2024-09-12T01:07:05.180 INFO:teuthology.orchestra.run.smithi001.stdout:Updating / installing... 2024-09-12T01:07:13.591 INFO:teuthology.orchestra.run.smithi001.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-12T01:07:53.776 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi001.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-12T01:07:53.776 DEBUG:teuthology.orchestra.run.smithi001:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-12T01:07:53.862 INFO:teuthology.orchestra.run.smithi001.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-12T01:07:53.862 INFO:teuthology.orchestra.run.smithi001.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-12T01:07:53.862 INFO:teuthology.orchestra.run.smithi001.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-12T01:07:53.862 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-12T01:07:53.862 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-12T01:07:53.862 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-12T01:07:53.862 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-12T01:07:53.862 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-12T01:07:53.863 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-12T01:07:53.863 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-12T01:07:53.863 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-12T01:07:53.863 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-12T01:07:53.863 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-12T01:07:53.863 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-12T01:07:53.863 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-12T01:07:53.863 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-12T01:07:53.863 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-12T01:07:53.863 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-12T01:07:53.863 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-12T01:07:53.864 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-12T01:07:53.864 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-12T01:07:53.864 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-12T01:07:53.864 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-12T01:07:53.864 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-12T01:07:53.864 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-12T01:07:53.864 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-12T01:07:53.864 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-12T01:07:53.864 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-12T01:07:53.864 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-12T01:07:53.864 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-12T01:07:53.864 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-12T01:07:53.864 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-12T01:07:53.865 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-12T01:07:53.865 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-12T01:07:53.865 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-12T01:07:53.865 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-12T01:07:53.865 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-12T01:07:53.865 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-12T01:07:53.865 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-12T01:07:53.865 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-12T01:07:53.865 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-12T01:07:53.865 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-12T01:07:53.865 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-12T01:07:53.865 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-12T01:07:53.865 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-12T01:07:53.868 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-12T01:07:53.868 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-12T01:07:53.868 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-12T01:07:53.868 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-12T01:07:53.868 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-12T01:07:53.868 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-12T01:07:53.868 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-12T01:07:53.868 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-12T01:07:53.868 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-12T01:07:53.868 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-12T01:07:53.868 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-12T01:07:53.868 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-12T01:07:53.869 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-12T01:07:53.869 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-12T01:07:53.869 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-12T01:07:53.869 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-12T01:07:53.869 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-12T01:07:53.869 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-12T01:07:53.869 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-12T01:07:53.869 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-12T01:07:53.869 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-12T01:07:53.869 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-12T01:07:53.869 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-12T01:07:53.869 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-12T01:07:53.869 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-12T01:07:53.870 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-12T01:07:53.870 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-12T01:07:53.870 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-12T01:07:53.870 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-12T01:07:53.870 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-12T01:07:53.870 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-12T01:07:53.870 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-12T01:07:53.870 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-12T01:07:53.870 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-12T01:07:53.870 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-12T01:07:53.870 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-12T01:07:53.870 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-12T01:07:53.870 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-12T01:07:53.870 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-12T01:07:53.871 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-12T01:07:53.871 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-12T01:07:53.871 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-12T01:07:53.871 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-12T01:07:53.871 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-12T01:07:53.871 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-12T01:07:53.871 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-12T01:07:53.871 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-12T01:07:53.871 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-12T01:07:53.871 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-12T01:07:53.871 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-12T01:07:53.871 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-12T01:07:53.871 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-12T01:07:53.872 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-12T01:07:53.872 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-12T01:07:53.872 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-12T01:07:53.872 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-12T01:07:53.872 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-12T01:07:53.872 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-12T01:07:53.872 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-12T01:07:53.872 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-12T01:07:53.872 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-12T01:07:53.872 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-12T01:07:53.872 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-12T01:07:53.872 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-12T01:07:53.872 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-12T01:07:53.873 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-12T01:07:53.873 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-12T01:07:53.873 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-12T01:07:53.873 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-12T01:07:53.873 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-12T01:07:53.873 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-12T01:07:53.873 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-12T01:07:53.873 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-12T01:07:53.873 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-12T01:07:53.873 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-12T01:07:53.873 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-12T01:07:53.873 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-12T01:07:53.873 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-12T01:07:53.874 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-12T01:07:53.874 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-12T01:07:53.874 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-12T01:07:53.874 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-12T01:07:53.874 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-12T01:07:53.874 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-12T01:07:53.874 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-12T01:07:53.874 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-12T01:07:53.874 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-12T01:07:53.874 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-12T01:07:53.874 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-12T01:07:53.874 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-12T01:07:53.874 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-12T01:07:53.874 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-12T01:07:53.875 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-12T01:07:53.875 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-12T01:07:53.875 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-12T01:07:53.875 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-12T01:07:53.875 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-12T01:07:53.875 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-12T01:07:53.875 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-12T01:07:53.875 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-12T01:07:53.875 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-12T01:07:53.875 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-12T01:07:53.875 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-12T01:07:53.875 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-12T01:07:53.875 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-12T01:07:53.875 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-12T01:07:53.875 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-12T01:07:53.876 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-12T01:07:53.876 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-12T01:07:53.876 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-12T01:07:53.876 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-12T01:07:53.876 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-12T01:07:53.876 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-12T01:07:53.876 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-12T01:07:53.876 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-12T01:07:53.876 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-12T01:07:53.876 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-12T01:07:53.876 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-12T01:07:53.876 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-12T01:07:53.876 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-12T01:07:53.877 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-12T01:07:53.877 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-12T01:07:53.877 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-12T01:07:53.877 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-12T01:07:53.877 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-12T01:07:53.877 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-12T01:07:53.877 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-12T01:07:53.877 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-12T01:07:53.877 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-12T01:07:53.877 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-12T01:07:53.877 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-12T01:07:53.877 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-12T01:07:53.877 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-12T01:07:53.877 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-12T01:07:53.878 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-12T01:07:53.878 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-12T01:07:53.878 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-12T01:07:53.878 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-12T01:07:53.878 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-12T01:07:53.878 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-12T01:07:53.878 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-12T01:07:53.878 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-12T01:07:53.878 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-12T01:07:53.878 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-12T01:07:53.878 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-12T01:07:53.878 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-12T01:07:53.878 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-12T01:07:53.878 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-12T01:07:53.879 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-12T01:07:53.879 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-12T01:07:53.879 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-12T01:07:53.879 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-12T01:07:53.879 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-12T01:07:53.879 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-12T01:07:53.879 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-12T01:07:53.879 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-12T01:07:53.879 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-12T01:07:53.879 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-12T01:07:53.879 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-12T01:07:53.879 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-12T01:07:53.879 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-12T01:07:53.879 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-12T01:07:53.879 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-12T01:07:53.880 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-12T01:07:53.880 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-12T01:07:53.880 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-12T01:07:53.880 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-12T01:07:53.880 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-12T01:07:53.880 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-12T01:07:53.880 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-12T01:07:53.880 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-12T01:07:53.880 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-12T01:07:53.880 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-12T01:07:53.880 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-12T01:07:53.880 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-12T01:07:53.880 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-12T01:07:53.880 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-12T01:07:53.881 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-12T01:07:53.881 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-12T01:07:53.881 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-12T01:07:53.881 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-12T01:07:53.881 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-12T01:07:53.881 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-12T01:07:53.881 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-12T01:07:53.881 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-12T01:07:53.881 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-12T01:07:53.881 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-12T01:07:53.881 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-12T01:07:53.881 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-12T01:07:53.881 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-12T01:07:53.881 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-12T01:07:53.881 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-12T01:07:53.882 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-12T01:07:53.882 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-12T01:07:53.882 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-12T01:07:53.882 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-12T01:07:53.882 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-12T01:07:53.882 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-12T01:07:53.882 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-12T01:07:53.882 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-12T01:07:53.882 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-12T01:07:53.882 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-12T01:07:53.882 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-12T01:07:53.882 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-12T01:07:53.882 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-12T01:07:53.882 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-12T01:07:53.883 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-12T01:07:53.883 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-12T01:07:53.883 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-12T01:07:53.883 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-12T01:07:53.883 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-12T01:07:53.883 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-12T01:07:53.883 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-12T01:07:53.883 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-12T01:07:53.883 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-12T01:07:53.883 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-12T01:07:53.883 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-12T01:07:53.883 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-12T01:07:53.883 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-12T01:07:53.883 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-12T01:07:53.884 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-12T01:07:53.884 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-12T01:07:53.884 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-12T01:07:53.884 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-12T01:07:53.884 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-12T01:07:53.884 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-12T01:07:53.884 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-12T01:07:53.884 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-12T01:07:53.884 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-12T01:07:53.884 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-12T01:07:53.884 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-12T01:07:53.884 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-12T01:07:53.884 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-12T01:07:53.884 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-12T01:07:53.885 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-12T01:07:53.885 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-12T01:07:53.885 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-12T01:07:53.885 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-12T01:07:53.885 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-12T01:07:53.885 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-12T01:07:53.885 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-12T01:07:53.885 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-12T01:07:53.885 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-12T01:07:53.885 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-12T01:07:53.885 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-12T01:07:53.885 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-12T01:07:53.885 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-12T01:07:53.886 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-12T01:07:53.886 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-12T01:07:53.886 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-12T01:07:53.886 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-12T01:07:53.886 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-12T01:07:53.886 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-12T01:07:53.886 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-12T01:07:53.886 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-12T01:07:53.886 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-12T01:07:53.886 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-12T01:07:53.886 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-12T01:07:53.886 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-12T01:07:53.886 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-12T01:07:53.886 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-12T01:07:53.886 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-12T01:07:53.887 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-12T01:07:53.887 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-12T01:07:53.887 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-12T01:07:53.887 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-12T01:07:53.887 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-12T01:07:53.887 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-12T01:07:53.887 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-12T01:07:53.887 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-12T01:07:53.887 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-12T01:07:53.887 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-12T01:07:53.887 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-12T01:07:53.887 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-12T01:07:53.887 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-12T01:07:53.887 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-12T01:07:53.887 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-12T01:07:53.888 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-12T01:07:53.888 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-12T01:07:53.888 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-12T01:07:53.888 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-12T01:07:53.888 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-12T01:07:53.888 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-12T01:07:53.888 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-12T01:07:53.888 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-12T01:07:53.888 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-12T01:07:53.888 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-12T01:07:53.888 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-12T01:07:53.888 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-12T01:07:53.888 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-12T01:07:53.889 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-12T01:07:53.889 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-12T01:07:53.889 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-12T01:07:53.889 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-12T01:07:53.889 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-12T01:07:53.889 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-12T01:07:53.889 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-12T01:07:53.889 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-12T01:07:53.889 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-12T01:07:53.889 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-12T01:07:53.889 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-12T01:07:53.889 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-12T01:07:53.889 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-12T01:07:53.890 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-12T01:07:53.890 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-12T01:07:53.890 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-12T01:07:53.890 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-12T01:07:53.890 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-12T01:07:53.890 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-12T01:07:53.890 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-12T01:07:53.890 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-12T01:07:53.890 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-12T01:07:53.890 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-12T01:07:53.890 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-12T01:07:53.890 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-12T01:07:53.890 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-12T01:07:53.891 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-12T01:07:53.891 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-12T01:07:53.891 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-12T01:07:53.891 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-12T01:07:53.891 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-12T01:07:53.891 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-12T01:07:53.891 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-12T01:07:53.891 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-12T01:07:53.891 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-12T01:07:53.891 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-12T01:07:53.891 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-12T01:07:53.891 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-12T01:07:53.891 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-12T01:07:53.892 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-12T01:07:53.892 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-12T01:07:53.892 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-12T01:07:53.892 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-12T01:07:53.892 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-12T01:07:53.892 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-12T01:07:53.892 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-12T01:07:53.892 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-12T01:07:53.892 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-12T01:07:53.892 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-12T01:07:53.892 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-12T01:07:53.892 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-12T01:07:53.893 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-12T01:07:53.893 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-12T01:07:53.893 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-12T01:07:53.893 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-12T01:07:53.893 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-12T01:07:53.893 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-12T01:07:53.893 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-12T01:07:53.893 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-12T01:07:53.893 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-12T01:07:53.893 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-12T01:07:53.893 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-12T01:07:53.893 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-12T01:07:53.893 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-12T01:07:53.894 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-12T01:07:53.894 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-12T01:07:53.894 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-12T01:07:53.894 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-12T01:07:53.894 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-12T01:07:53.894 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-12T01:07:53.894 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-12T01:07:53.894 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-12T01:07:53.894 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-12T01:07:53.894 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-12T01:07:53.894 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-12T01:07:53.894 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-12T01:07:53.894 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-12T01:07:53.894 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-12T01:07:53.895 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-12T01:07:53.895 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-12T01:07:53.895 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-12T01:07:53.895 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-12T01:07:53.895 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-12T01:07:53.895 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-12T01:07:53.895 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-12T01:07:53.895 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-12T01:07:53.895 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-12T01:07:53.895 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-12T01:07:53.895 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-12T01:07:53.895 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-12T01:07:53.895 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-12T01:07:53.895 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-12T01:07:53.896 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-12T01:07:53.896 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-12T01:07:53.896 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-12T01:07:53.896 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-12T01:07:53.896 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-12T01:07:53.896 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-12T01:07:53.896 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-12T01:07:53.896 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-12T01:07:53.896 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-12T01:07:53.896 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-12T01:07:53.896 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-12T01:07:53.896 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-12T01:07:53.896 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-12T01:07:53.896 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-12T01:07:53.897 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-12T01:07:53.897 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-12T01:07:53.897 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-12T01:07:53.897 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-12T01:07:53.897 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-12T01:07:53.897 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-12T01:07:53.897 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-12T01:07:53.897 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-12T01:07:53.897 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-12T01:07:53.897 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-12T01:07:53.897 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-12T01:07:53.897 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-12T01:07:53.897 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-12T01:07:53.898 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-12T01:07:53.898 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-12T01:07:53.898 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-12T01:07:53.898 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-12T01:07:53.898 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-12T01:07:53.898 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-12T01:07:53.898 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-12T01:07:53.898 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-12T01:07:53.898 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-12T01:07:53.898 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-12T01:07:53.898 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-12T01:07:53.898 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-12T01:07:53.898 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-12T01:07:53.899 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-12T01:07:53.899 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-12T01:07:53.899 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-12T01:07:53.899 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-12T01:07:53.899 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-12T01:07:53.899 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-12T01:07:53.899 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-12T01:07:53.899 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-12T01:07:53.899 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-12T01:07:53.899 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-12T01:07:53.899 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-12T01:07:53.899 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-12T01:07:53.899 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-12T01:07:53.899 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-12T01:07:53.900 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-12T01:07:53.900 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-12T01:07:53.900 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-12T01:07:53.900 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-12T01:07:53.900 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-12T01:07:53.900 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-12T01:07:53.900 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-12T01:07:53.900 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-12T01:07:53.900 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-12T01:07:53.900 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-12T01:07:53.900 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-12T01:07:53.900 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-12T01:07:53.900 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-12T01:07:53.901 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-12T01:07:53.901 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-12T01:07:53.901 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-12T01:07:53.901 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-12T01:07:53.901 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-12T01:07:53.901 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-12T01:07:53.901 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-12T01:07:53.901 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-12T01:07:53.901 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-12T01:07:53.901 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-12T01:07:53.901 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-12T01:07:53.901 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-12T01:07:53.901 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-12T01:07:53.902 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-12T01:07:53.902 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-12T01:07:53.902 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-12T01:07:53.902 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-12T01:07:53.902 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-12T01:07:53.902 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-12T01:07:53.902 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-12T01:07:53.902 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-12T01:07:53.902 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-12T01:07:53.902 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-12T01:07:53.902 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-12T01:07:53.902 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-12T01:07:53.902 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-12T01:07:53.902 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-12T01:07:53.903 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-12T01:07:53.903 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-12T01:07:53.903 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-12T01:07:53.903 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-12T01:07:53.903 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-12T01:07:53.903 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-12T01:07:53.903 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-12T01:07:53.903 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-12T01:07:53.903 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-12T01:07:53.903 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-12T01:07:53.903 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-12T01:07:53.903 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-12T01:07:53.903 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-12T01:07:53.903 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-12T01:07:53.904 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-12T01:07:53.904 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-12T01:07:53.904 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-12T01:07:53.904 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-12T01:07:53.904 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-12T01:07:53.904 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-12T01:07:53.904 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-12T01:07:53.904 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-12T01:07:53.904 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-12T01:07:53.904 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-12T01:07:53.904 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-12T01:07:53.904 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-12T01:07:53.904 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-12T01:07:53.905 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-12T01:07:53.905 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-12T01:07:53.905 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-12T01:07:53.905 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-12T01:07:53.905 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-12T01:07:53.905 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-12T01:07:53.905 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-12T01:07:53.905 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-12T01:07:53.905 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-12T01:07:53.905 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-12T01:07:53.905 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-12T01:07:53.905 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-12T01:07:53.905 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-12T01:07:53.905 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-12T01:07:53.906 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-12T01:07:53.906 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-12T01:07:53.906 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-12T01:07:53.906 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-12T01:07:53.906 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-12T01:07:53.906 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-12T01:07:53.906 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-12T01:07:53.906 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-12T01:07:53.906 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-12T01:07:53.906 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-12T01:07:53.906 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-12T01:07:53.906 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-12T01:07:53.906 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-12T01:07:53.906 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-12T01:07:53.907 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-12T01:07:53.907 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-12T01:07:53.907 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-12T01:07:53.907 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-12T01:07:53.907 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-12T01:07:53.907 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-12T01:07:53.907 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-12T01:07:53.907 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-12T01:07:53.907 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-12T01:07:53.907 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-12T01:07:53.907 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-12T01:07:53.907 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-12T01:07:53.907 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-12T01:07:53.908 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-12T01:07:53.908 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-12T01:07:53.908 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-12T01:07:53.908 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-12T01:07:53.908 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-12T01:07:53.908 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-12T01:07:53.908 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-12T01:07:53.908 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-12T01:07:53.908 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-12T01:07:53.908 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-12T01:07:53.908 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-12T01:07:53.908 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-12T01:07:53.908 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-12T01:07:53.908 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-12T01:07:53.909 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-12T01:07:53.909 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-12T01:07:53.909 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-12T01:07:53.909 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-12T01:07:53.909 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-12T01:07:53.909 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-12T01:07:53.909 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-12T01:07:53.909 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-12T01:07:53.909 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-12T01:07:53.909 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-12T01:07:53.909 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-12T01:07:53.909 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-12T01:07:53.909 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-12T01:07:53.909 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-12T01:07:53.910 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-12T01:07:53.910 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-12T01:07:53.910 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-12T01:07:53.910 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-12T01:07:53.910 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-12T01:07:53.910 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-12T01:07:53.910 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-12T01:07:53.910 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-12T01:07:53.910 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-12T01:07:53.910 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-12T01:07:53.910 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-12T01:07:53.910 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-12T01:07:53.910 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-12T01:07:53.910 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-12T01:07:53.910 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-12T01:07:53.910 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-12T01:07:53.911 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-12T01:07:53.911 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-12T01:07:53.911 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-12T01:07:53.911 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-12T01:07:53.911 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-12T01:07:53.911 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-12T01:07:53.911 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-12T01:07:53.911 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-12T01:07:53.911 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-12T01:07:53.911 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-12T01:07:53.911 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-12T01:07:53.911 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-12T01:07:53.911 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-12T01:07:53.911 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-12T01:07:53.912 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-12T01:07:53.912 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-12T01:07:53.912 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-12T01:07:53.912 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-12T01:07:53.912 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-12T01:07:53.912 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-12T01:07:53.912 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-12T01:07:53.912 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-12T01:07:53.912 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-12T01:07:53.912 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-12T01:07:53.912 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-12T01:07:53.912 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-12T01:07:53.912 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-12T01:07:53.912 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-12T01:07:53.913 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-12T01:07:53.913 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-12T01:07:53.913 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-12T01:07:53.913 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-12T01:07:53.913 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-12T01:07:53.913 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-12T01:07:53.913 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-12T01:07:53.913 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-12T01:07:53.913 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-12T01:07:53.913 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-12T01:07:53.913 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-12T01:07:53.913 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-12T01:07:53.913 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-12T01:07:53.914 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-12T01:07:53.914 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-12T01:07:53.914 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-12T01:07:53.914 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-12T01:07:53.914 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-12T01:07:53.914 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-12T01:07:53.914 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-12T01:07:53.914 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-12T01:07:53.914 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-12T01:07:53.914 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-12T01:07:53.914 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-12T01:07:53.914 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-12T01:07:53.914 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-12T01:07:53.915 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-12T01:07:53.915 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-12T01:07:53.915 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-12T01:07:53.915 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-12T01:07:53.915 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-12T01:07:53.915 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-12T01:07:53.915 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-12T01:07:53.915 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-12T01:07:53.915 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-12T01:07:53.915 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-12T01:07:53.915 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-12T01:07:53.915 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-12T01:07:53.916 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-12T01:07:53.916 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-12T01:07:53.916 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-12T01:07:53.916 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-12T01:07:53.916 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-12T01:07:53.916 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-12T01:07:53.916 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-12T01:07:53.916 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-12T01:07:53.916 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-12T01:07:53.916 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-12T01:07:53.916 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-12T01:07:53.916 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-12T01:07:53.916 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-12T01:07:53.916 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-12T01:07:53.917 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-12T01:07:53.917 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-12T01:07:53.917 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-12T01:07:53.917 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-12T01:07:53.917 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-12T01:07:53.917 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-12T01:07:53.917 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-12T01:07:53.917 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-12T01:07:53.917 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-12T01:07:53.917 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-12T01:07:53.917 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-12T01:07:53.917 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-12T01:07:53.917 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-12T01:07:53.917 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-12T01:07:53.918 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-12T01:07:53.918 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-12T01:07:53.918 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-12T01:07:53.918 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-12T01:07:53.918 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-12T01:07:53.918 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-12T01:07:53.918 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-12T01:07:53.918 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-12T01:07:53.918 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-12T01:07:53.918 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-12T01:07:53.918 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-12T01:07:53.918 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-12T01:07:53.918 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-12T01:07:53.918 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-12T01:07:53.918 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-12T01:07:53.918 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-12T01:07:53.919 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-12T01:07:53.919 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-12T01:07:53.919 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-12T01:07:53.919 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-12T01:07:53.919 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-12T01:07:53.919 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-12T01:07:53.919 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-12T01:07:53.919 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-12T01:07:53.919 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-12T01:07:53.919 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-12T01:07:53.919 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-12T01:07:53.919 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-12T01:07:53.919 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-12T01:07:53.919 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-12T01:07:53.919 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-12T01:07:53.920 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-12T01:07:53.920 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-12T01:07:53.920 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-12T01:07:53.920 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-12T01:07:53.920 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-12T01:07:53.920 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-12T01:07:53.920 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-12T01:07:53.920 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-12T01:07:53.920 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-12T01:07:53.920 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-12T01:07:53.920 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-12T01:07:53.920 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-12T01:07:53.920 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-12T01:07:53.920 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-12T01:07:53.920 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-12T01:07:53.920 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-12T01:07:53.921 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-12T01:07:53.921 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-12T01:07:53.921 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-12T01:07:53.921 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-12T01:07:53.921 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-12T01:07:53.921 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-12T01:07:53.921 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-12T01:07:53.921 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-12T01:07:53.921 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-12T01:07:53.921 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-12T01:07:53.921 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-12T01:07:53.921 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-12T01:07:53.921 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-12T01:07:53.921 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-12T01:07:53.922 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-12T01:07:53.922 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-12T01:07:53.922 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-12T01:07:53.922 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-12T01:07:53.922 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-12T01:07:53.922 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-12T01:07:53.922 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-12T01:07:53.922 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-12T01:07:53.922 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-12T01:07:53.922 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-12T01:07:53.922 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-12T01:07:53.922 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-12T01:07:53.922 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-12T01:07:53.922 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-12T01:07:53.923 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-12T01:07:53.923 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-12T01:07:53.923 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-12T01:07:53.923 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-12T01:07:53.923 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-12T01:07:53.923 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-12T01:07:53.923 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-12T01:07:53.923 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-12T01:07:53.923 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-12T01:07:53.923 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-12T01:07:53.923 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-12T01:07:53.923 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-12T01:07:53.923 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-12T01:07:53.923 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-12T01:07:53.923 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-12T01:07:53.924 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-12T01:07:53.924 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-12T01:07:53.924 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-12T01:07:53.924 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-12T01:07:53.924 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-12T01:07:53.924 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-12T01:07:53.924 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-12T01:07:53.924 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-12T01:07:53.924 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-12T01:07:53.924 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-12T01:07:53.924 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-12T01:07:53.924 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-12T01:07:53.924 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-12T01:07:53.925 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-12T01:07:53.925 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-12T01:07:53.925 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-12T01:07:53.925 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-12T01:07:53.925 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-12T01:07:53.925 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-12T01:07:53.925 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-12T01:07:53.925 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-12T01:07:53.925 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-12T01:07:53.925 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-12T01:07:53.925 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-12T01:07:53.925 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-12T01:07:53.925 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-12T01:07:53.926 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-12T01:07:53.926 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-12T01:07:53.926 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-12T01:07:53.926 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-12T01:07:53.926 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-12T01:07:53.926 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-12T01:07:53.926 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-12T01:07:53.926 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-12T01:07:53.926 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-12T01:07:53.926 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-12T01:07:53.926 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-12T01:07:53.926 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-12T01:07:53.926 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-12T01:07:53.926 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-12T01:07:53.926 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-12T01:07:53.927 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-12T01:07:53.927 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-12T01:07:53.927 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-12T01:07:53.927 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-12T01:07:53.927 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-12T01:07:53.927 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-12T01:07:53.927 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-12T01:07:53.927 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-12T01:07:53.927 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-12T01:07:53.927 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-12T01:07:53.927 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-12T01:07:53.927 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-12T01:07:53.927 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-12T01:07:53.927 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-12T01:07:53.928 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-12T01:07:53.928 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-12T01:07:53.928 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-12T01:07:53.928 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-12T01:07:53.928 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-12T01:07:53.928 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-12T01:07:53.928 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-12T01:07:53.928 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-12T01:07:53.928 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-12T01:07:53.928 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-12T01:07:53.928 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-12T01:07:53.928 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-12T01:07:53.929 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-12T01:07:53.929 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-12T01:07:53.929 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-12T01:07:53.929 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-12T01:07:53.929 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-12T01:07:53.929 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-12T01:07:53.929 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-12T01:07:53.929 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-12T01:07:53.929 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-12T01:07:53.929 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-12T01:07:53.929 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-12T01:07:53.929 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-12T01:07:53.929 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-12T01:07:53.929 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-12T01:07:53.930 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-12T01:07:53.930 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-12T01:07:53.930 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-12T01:07:53.930 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-12T01:07:53.930 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-12T01:07:53.930 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-12T01:07:53.930 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-12T01:07:53.930 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-12T01:07:53.930 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-12T01:07:53.930 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-12T01:07:53.930 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-12T01:07:53.930 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-12T01:07:53.930 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-12T01:07:53.931 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-12T01:07:53.931 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-12T01:07:53.931 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-12T01:07:53.931 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-12T01:07:53.931 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-12T01:07:53.931 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-12T01:07:53.931 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-12T01:07:53.931 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-12T01:07:53.931 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-12T01:07:53.931 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-12T01:07:53.931 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-12T01:07:53.931 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-12T01:07:53.931 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-12T01:07:53.931 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-12T01:07:53.932 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-12T01:07:53.932 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-12T01:07:53.932 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-12T01:07:53.932 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-12T01:07:53.932 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-12T01:07:53.932 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-12T01:07:53.932 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-12T01:07:53.932 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-12T01:07:53.932 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-12T01:07:53.932 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-12T01:07:53.932 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-12T01:07:53.932 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-12T01:07:53.932 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-12T01:07:53.933 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-12T01:07:53.933 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-12T01:07:53.933 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-12T01:07:53.933 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-12T01:07:53.933 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-12T01:07:53.933 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-12T01:07:53.933 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-12T01:07:53.933 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-12T01:07:53.933 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-12T01:07:53.933 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-12T01:07:53.933 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-12T01:07:53.933 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-12T01:07:53.933 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-12T01:07:53.933 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-12T01:07:53.934 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-12T01:07:53.934 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-12T01:07:53.934 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-12T01:07:53.934 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-12T01:07:53.934 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-12T01:07:53.934 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-12T01:07:53.934 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-12T01:07:53.934 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-12T01:07:53.934 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-12T01:07:53.934 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-12T01:07:53.934 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-12T01:07:53.934 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-12T01:07:53.934 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-12T01:07:53.934 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-12T01:07:53.935 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-12T01:07:53.935 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-12T01:07:53.935 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-12T01:07:53.935 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-12T01:07:53.935 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-12T01:07:53.935 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-12T01:07:53.935 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-12T01:07:53.935 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-12T01:07:53.935 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-12T01:07:53.935 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-12T01:07:53.935 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-12T01:07:53.935 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-12T01:07:53.935 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-12T01:07:53.935 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-12T01:07:53.936 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-12T01:07:53.936 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-12T01:07:53.936 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-12T01:07:53.936 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-12T01:07:53.936 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-12T01:07:53.936 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-12T01:07:53.936 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-12T01:07:53.936 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-12T01:07:53.936 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-12T01:07:53.936 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-12T01:07:53.936 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-12T01:07:53.936 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-12T01:07:53.936 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-12T01:07:53.936 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-12T01:07:53.937 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-12T01:07:53.937 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-12T01:07:53.937 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-12T01:07:53.937 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-12T01:07:53.937 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-12T01:07:53.937 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-12T01:07:53.937 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-12T01:07:53.937 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-12T01:07:53.937 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-12T01:07:53.937 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-12T01:07:53.937 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-12T01:07:53.937 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-12T01:07:53.937 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-12T01:07:53.937 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-12T01:07:53.938 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-12T01:07:53.938 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-12T01:07:53.938 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-12T01:07:53.938 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-12T01:07:53.938 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-12T01:07:53.938 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-12T01:07:53.938 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-12T01:07:53.938 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-12T01:07:53.938 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-12T01:07:53.938 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-12T01:07:53.938 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-12T01:07:53.938 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-12T01:07:53.938 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-12T01:07:53.938 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-12T01:07:53.939 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-12T01:07:53.939 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-12T01:07:53.939 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-12T01:07:53.939 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-12T01:07:53.939 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-12T01:07:53.939 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-12T01:07:53.939 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-12T01:07:53.939 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-12T01:07:53.939 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-12T01:07:53.939 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-12T01:07:53.939 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-12T01:07:53.939 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-12T01:07:53.939 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-12T01:07:53.939 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-12T01:07:53.940 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-12T01:07:53.940 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-12T01:07:53.940 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-12T01:07:53.940 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-12T01:07:53.940 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-12T01:07:53.940 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-12T01:07:53.940 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-12T01:07:53.940 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-12T01:07:53.940 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-12T01:07:53.940 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-12T01:07:53.940 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-12T01:07:53.940 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-12T01:07:53.940 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-12T01:07:53.940 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-12T01:07:53.941 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-12T01:07:53.941 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-12T01:07:53.941 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-12T01:07:53.941 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-12T01:07:53.941 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-12T01:07:53.941 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-12T01:07:53.941 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-12T01:07:53.941 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-12T01:07:53.941 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-12T01:07:53.941 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-12T01:07:53.941 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-12T01:07:53.941 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-12T01:07:53.941 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-12T01:07:53.941 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-12T01:07:53.942 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-12T01:07:53.942 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-12T01:07:53.942 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-12T01:07:53.942 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-12T01:07:53.942 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-12T01:07:53.942 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-12T01:07:53.942 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-12T01:07:53.942 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-12T01:07:53.942 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-12T01:07:53.942 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-12T01:07:53.942 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-12T01:07:53.942 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-12T01:07:53.942 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-12T01:07:53.942 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-12T01:07:53.943 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-12T01:07:53.943 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-12T01:07:53.943 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-12T01:07:53.943 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-12T01:07:53.943 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-12T01:07:53.943 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-12T01:07:53.943 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-12T01:07:53.943 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-12T01:07:53.943 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-12T01:07:53.943 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-12T01:07:53.943 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-12T01:07:53.943 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-12T01:07:53.943 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-12T01:07:53.943 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-12T01:07:53.944 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-12T01:07:53.944 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-12T01:07:53.944 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-12T01:07:53.944 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-12T01:07:53.944 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-12T01:07:53.944 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-12T01:07:53.944 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-12T01:07:53.944 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-12T01:07:53.944 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-12T01:07:53.944 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-12T01:07:53.944 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-12T01:07:53.944 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-12T01:07:53.944 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-12T01:07:53.944 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-12T01:07:53.945 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-12T01:07:53.945 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-12T01:07:53.945 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-12T01:07:53.945 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-12T01:07:53.945 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-12T01:07:53.945 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-12T01:07:53.945 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-12T01:07:53.945 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-12T01:07:53.945 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-12T01:07:53.945 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-12T01:07:53.945 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-12T01:07:53.945 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-12T01:07:53.945 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-12T01:07:53.945 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-12T01:07:53.946 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-12T01:07:53.946 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-12T01:07:53.946 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-12T01:07:53.946 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-12T01:07:53.946 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-12T01:07:53.946 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-12T01:07:53.946 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-12T01:07:53.946 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-12T01:07:53.946 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-12T01:07:53.946 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-12T01:07:53.946 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-12T01:07:53.946 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-12T01:07:53.946 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-12T01:07:53.946 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-12T01:07:53.947 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-12T01:07:53.947 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-12T01:07:53.947 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-12T01:07:53.947 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-12T01:07:53.947 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-12T01:07:53.947 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-12T01:07:53.947 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-12T01:07:53.947 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-12T01:07:53.947 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-12T01:07:53.947 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-12T01:07:53.947 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-12T01:07:53.947 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-12T01:07:53.947 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-12T01:07:53.947 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-12T01:07:53.947 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-12T01:07:53.947 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-12T01:07:53.948 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-12T01:07:53.948 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-12T01:07:53.948 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-12T01:07:53.948 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-12T01:07:53.948 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-12T01:07:53.948 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-12T01:07:53.948 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-12T01:07:53.948 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-12T01:07:53.948 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-12T01:07:53.948 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-12T01:07:53.948 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-12T01:07:53.948 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-12T01:07:53.948 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-12T01:07:53.949 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-12T01:07:53.949 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-12T01:07:53.949 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-12T01:07:53.949 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-12T01:07:53.949 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-12T01:07:53.949 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-12T01:07:53.949 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-12T01:07:53.949 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-12T01:07:53.949 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-12T01:07:53.949 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-12T01:07:53.949 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-12T01:07:53.949 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-12T01:07:53.949 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-12T01:07:53.949 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-12T01:07:53.950 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-12T01:07:53.950 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-12T01:07:53.950 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-12T01:07:53.950 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-12T01:07:53.950 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-12T01:07:53.950 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-12T01:07:53.950 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-12T01:07:53.950 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-12T01:07:53.950 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-12T01:07:53.950 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-12T01:07:53.950 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-12T01:07:53.950 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-12T01:07:53.950 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-12T01:07:53.950 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-12T01:07:53.950 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-12T01:07:53.951 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-12T01:07:53.951 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-12T01:07:53.951 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-12T01:07:53.951 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-12T01:07:53.951 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-12T01:07:53.951 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-12T01:07:53.951 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-12T01:07:53.951 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-12T01:07:53.951 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-12T01:07:53.951 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-12T01:07:53.951 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-12T01:07:53.951 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-12T01:07:53.951 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-12T01:07:53.951 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-12T01:07:53.951 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-12T01:07:53.952 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-12T01:07:53.952 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-12T01:07:53.952 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-12T01:07:53.952 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-12T01:07:53.952 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-12T01:07:53.952 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-12T01:07:53.952 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-12T01:07:53.952 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-12T01:07:53.952 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-12T01:07:53.952 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-12T01:07:53.952 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-12T01:07:53.952 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-12T01:07:53.953 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-12T01:07:53.953 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-12T01:07:53.953 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-12T01:07:53.953 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-12T01:07:53.953 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-12T01:07:53.953 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-12T01:07:53.953 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-12T01:07:53.953 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-12T01:07:53.953 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-12T01:07:53.953 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-12T01:07:53.953 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-12T01:07:53.953 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-12T01:07:53.953 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-12T01:07:53.953 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-12T01:07:53.954 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-12T01:07:53.954 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-12T01:07:53.954 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-12T01:07:53.954 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-12T01:07:53.954 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-12T01:07:53.954 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-12T01:07:53.954 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-12T01:07:53.954 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-12T01:07:53.954 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-12T01:07:53.954 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-12T01:07:53.954 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-12T01:07:53.954 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-12T01:07:53.954 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-12T01:07:53.954 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-12T01:07:53.954 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-12T01:07:53.955 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-12T01:07:53.955 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-12T01:07:53.955 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-12T01:07:53.955 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-12T01:07:53.955 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-12T01:07:53.955 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-12T01:07:53.955 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-12T01:07:53.955 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-12T01:07:53.955 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-12T01:07:53.955 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-12T01:07:53.955 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-12T01:07:53.955 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-12T01:07:53.955 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-12T01:07:53.955 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-12T01:07:53.956 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-12T01:07:53.956 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-12T01:07:53.956 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-12T01:07:53.956 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-12T01:07:53.956 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-12T01:07:53.956 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-12T01:07:53.956 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-12T01:07:53.956 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-12T01:07:53.956 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-12T01:07:53.956 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-12T01:07:53.956 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-12T01:07:53.956 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-12T01:07:53.956 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-12T01:07:53.956 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-12T01:07:53.957 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-12T01:07:53.957 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-12T01:07:53.957 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-12T01:07:53.957 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-12T01:07:53.957 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-12T01:07:53.957 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-12T01:07:53.957 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-12T01:07:53.957 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-12T01:07:53.957 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-12T01:07:53.957 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-12T01:07:53.957 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-12T01:07:53.957 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-12T01:07:53.957 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-12T01:07:53.957 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-12T01:07:53.957 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-12T01:07:53.958 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-12T01:07:53.958 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-12T01:07:53.958 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-12T01:07:53.958 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-12T01:07:53.958 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-12T01:07:53.958 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-12T01:07:53.958 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-12T01:07:53.958 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-12T01:07:53.958 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-12T01:07:53.958 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-12T01:07:53.958 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-12T01:07:53.958 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-12T01:07:53.958 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-12T01:07:53.958 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-12T01:07:53.958 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-12T01:07:53.959 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-12T01:07:53.959 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-12T01:07:53.959 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-12T01:07:53.959 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-12T01:07:53.959 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-12T01:07:53.959 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-12T01:07:53.959 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-12T01:07:53.959 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-12T01:07:53.959 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-12T01:07:53.959 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-12T01:07:53.959 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-12T01:07:53.959 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-12T01:07:53.959 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-12T01:07:53.959 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-12T01:07:53.959 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-12T01:07:53.960 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-12T01:07:53.960 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-12T01:07:53.960 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-12T01:07:53.960 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-12T01:07:53.960 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-12T01:07:53.960 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-12T01:07:53.960 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-12T01:07:53.960 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-12T01:07:53.960 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-12T01:07:53.960 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-12T01:07:53.960 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-12T01:07:53.960 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-12T01:07:53.960 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-12T01:07:53.960 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-12T01:07:53.960 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-12T01:07:53.961 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-12T01:07:53.961 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-12T01:07:53.961 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-12T01:07:53.961 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-12T01:07:53.961 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-12T01:07:53.961 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-12T01:07:53.961 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-12T01:07:53.961 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-12T01:07:53.961 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-12T01:07:53.961 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-12T01:07:53.961 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-12T01:07:53.961 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-12T01:07:53.961 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-12T01:07:53.961 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-12T01:07:53.962 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-12T01:07:53.962 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-12T01:07:53.962 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-12T01:07:53.962 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-12T01:07:53.962 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-12T01:07:53.962 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-12T01:07:53.962 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-12T01:07:53.962 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-12T01:07:53.962 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-12T01:07:53.962 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-12T01:07:53.962 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-12T01:07:53.962 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-12T01:07:53.962 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-12T01:07:53.962 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-12T01:07:53.963 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-12T01:07:53.963 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-12T01:07:53.963 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-12T01:07:53.963 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-12T01:07:53.963 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-12T01:07:53.963 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-12T01:07:53.963 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-12T01:07:53.963 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-12T01:07:53.963 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-12T01:07:53.963 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-12T01:07:53.963 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-12T01:07:53.963 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-12T01:07:53.963 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-12T01:07:53.964 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-12T01:07:53.964 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-12T01:07:53.964 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-12T01:07:53.964 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-12T01:07:53.964 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-12T01:07:53.964 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-12T01:07:53.964 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-12T01:07:53.964 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-12T01:07:53.964 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-12T01:07:53.964 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-12T01:07:53.964 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-12T01:07:53.964 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-12T01:07:53.964 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-12T01:07:53.964 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-12T01:07:53.964 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-12T01:07:53.965 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-12T01:07:53.965 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-12T01:07:53.965 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-12T01:07:53.965 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-12T01:07:53.965 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-12T01:07:53.965 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-12T01:07:53.965 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-12T01:07:53.965 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-12T01:07:53.965 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-12T01:07:53.965 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-12T01:07:53.965 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-12T01:07:53.965 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-12T01:07:53.965 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-12T01:07:53.966 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-12T01:07:53.966 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-12T01:07:53.966 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-12T01:07:53.966 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-12T01:07:53.966 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-12T01:07:53.966 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-12T01:07:53.966 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-12T01:07:53.966 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-12T01:07:53.966 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-12T01:07:53.966 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-12T01:07:53.966 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-12T01:07:53.966 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-12T01:07:53.966 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-12T01:07:53.966 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-12T01:07:53.967 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-12T01:07:53.967 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-12T01:07:53.967 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-12T01:07:53.967 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-12T01:07:53.967 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-12T01:07:53.967 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-12T01:07:53.967 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-12T01:07:53.967 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-12T01:07:53.967 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-12T01:07:53.967 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-12T01:07:53.967 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-12T01:07:53.967 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-12T01:07:53.967 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-12T01:07:53.967 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-12T01:07:53.968 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-12T01:07:53.968 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-12T01:07:53.968 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-12T01:07:53.968 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-12T01:07:53.968 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-12T01:07:53.968 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-12T01:07:53.968 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-12T01:07:53.968 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-12T01:07:53.968 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-12T01:07:53.968 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-12T01:07:53.968 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-12T01:07:53.968 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-12T01:07:53.968 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-12T01:07:53.968 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-12T01:07:53.969 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-12T01:07:53.969 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-12T01:07:53.969 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-12T01:07:53.969 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-12T01:07:53.969 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-12T01:07:53.969 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-12T01:07:53.969 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-12T01:07:53.969 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-12T01:07:53.969 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-12T01:07:53.969 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-12T01:07:53.969 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-12T01:07:53.969 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-12T01:07:53.969 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-12T01:07:53.969 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-12T01:07:53.969 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-12T01:07:53.970 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-12T01:07:53.970 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-12T01:07:53.970 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-12T01:07:53.970 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-12T01:07:53.970 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-12T01:07:53.970 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-12T01:07:53.970 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-12T01:07:53.970 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-12T01:07:53.970 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-12T01:07:53.970 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-12T01:07:53.970 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-12T01:07:53.970 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-12T01:07:53.970 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-12T01:07:53.970 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-12T01:07:53.970 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-12T01:07:53.971 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-12T01:07:53.971 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-12T01:07:53.971 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-12T01:07:53.971 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-12T01:07:53.971 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-12T01:07:53.971 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-12T01:07:53.971 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-12T01:07:53.971 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-12T01:07:53.971 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-12T01:07:53.971 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-12T01:07:53.971 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-12T01:07:53.971 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-12T01:07:53.971 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-12T01:07:53.971 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-12T01:07:53.972 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-12T01:07:53.972 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-12T01:07:53.972 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-12T01:07:53.972 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-12T01:07:53.972 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-12T01:07:53.972 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-12T01:07:53.972 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-12T01:07:53.972 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-12T01:07:53.972 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-12T01:07:53.972 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-12T01:07:53.972 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-12T01:07:53.972 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-12T01:07:53.972 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-12T01:07:53.973 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-12T01:07:53.973 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-12T01:07:53.973 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-12T01:07:53.973 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-12T01:07:53.973 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-12T01:07:53.973 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-12T01:07:53.973 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-12T01:07:53.973 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-12T01:07:53.973 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-12T01:07:53.973 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-12T01:07:53.973 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-12T01:07:53.973 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-12T01:07:53.973 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-12T01:07:53.973 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-12T01:07:53.974 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-12T01:07:53.974 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-12T01:07:53.974 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-12T01:07:53.974 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-12T01:07:53.974 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-12T01:07:53.974 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-12T01:07:53.974 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-12T01:07:53.974 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-12T01:07:53.974 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-12T01:07:53.974 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-12T01:07:53.974 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-12T01:07:53.974 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-12T01:07:53.974 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-12T01:07:53.974 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-12T01:07:53.975 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-12T01:07:53.975 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-12T01:07:53.975 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-12T01:07:53.975 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-12T01:07:53.975 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-12T01:07:53.975 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-12T01:07:53.975 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-12T01:07:53.975 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-12T01:07:53.975 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-12T01:07:53.975 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-12T01:07:53.975 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-12T01:07:53.975 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-12T01:07:53.975 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-12T01:07:53.976 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-12T01:07:53.976 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-12T01:07:53.976 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-12T01:07:53.976 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-12T01:07:53.976 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-12T01:07:53.976 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-12T01:07:53.976 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-12T01:07:53.976 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-12T01:07:53.976 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-12T01:07:53.976 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-12T01:07:53.976 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-12T01:07:53.976 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-12T01:07:53.976 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-12T01:07:53.977 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-12T01:07:53.977 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-12T01:07:53.977 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-12T01:07:53.977 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-12T01:07:53.977 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-12T01:07:53.977 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-12T01:07:53.977 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-12T01:07:53.977 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-12T01:07:53.977 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-12T01:07:53.977 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-12T01:07:53.977 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-12T01:07:53.977 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-12T01:07:53.977 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-12T01:07:53.977 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-12T01:07:53.978 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-12T01:07:53.978 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-12T01:07:53.978 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-12T01:07:53.978 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-12T01:07:53.978 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-12T01:07:53.978 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-12T01:07:53.978 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-12T01:07:53.978 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-12T01:07:53.978 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-12T01:07:53.978 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-12T01:07:53.978 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-12T01:07:53.978 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-12T01:07:53.978 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-12T01:07:53.978 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-12T01:07:53.979 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-12T01:07:53.979 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-12T01:07:53.979 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-12T01:07:53.979 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-12T01:07:53.979 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-12T01:07:53.979 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-12T01:07:53.979 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-12T01:07:53.979 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-12T01:07:53.979 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-12T01:07:53.979 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-12T01:07:53.979 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-12T01:07:53.979 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-12T01:07:53.979 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-12T01:07:53.979 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-12T01:07:53.980 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-12T01:07:53.980 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-12T01:07:53.980 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-12T01:07:53.980 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-12T01:07:53.980 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-12T01:07:53.980 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-12T01:07:53.980 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-12T01:07:53.980 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-12T01:07:53.980 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-12T01:07:53.980 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-12T01:07:53.980 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-12T01:07:53.980 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-12T01:07:53.980 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-12T01:07:53.980 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-12T01:07:53.981 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-12T01:07:53.981 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-12T01:07:53.981 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-12T01:07:53.981 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-12T01:07:53.981 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-12T01:07:53.981 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-12T01:07:53.981 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-12T01:07:53.981 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-12T01:07:53.981 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-12T01:07:53.981 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-12T01:07:53.981 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-12T01:07:53.981 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-12T01:07:53.981 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-12T01:07:53.981 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-12T01:07:53.982 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-12T01:07:53.982 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-12T01:07:53.982 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-12T01:07:53.982 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-12T01:07:53.982 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-12T01:07:53.982 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-12T01:07:53.982 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-12T01:07:53.982 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-12T01:07:53.982 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-12T01:07:53.982 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-12T01:07:53.982 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-12T01:07:53.982 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-12T01:07:53.982 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-12T01:07:53.982 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-12T01:07:53.983 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-12T01:07:53.983 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-12T01:07:53.983 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-12T01:07:53.983 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-12T01:07:53.983 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-12T01:07:53.983 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-12T01:07:53.983 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-12T01:07:53.983 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-12T01:07:53.983 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-12T01:07:53.983 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-12T01:07:53.983 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-12T01:07:53.983 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-12T01:07:53.983 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-12T01:07:53.984 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-12T01:07:53.984 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-12T01:07:53.984 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-12T01:07:53.984 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-12T01:07:53.984 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-12T01:07:53.984 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-12T01:07:53.984 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-12T01:07:53.984 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-12T01:07:53.984 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-12T01:07:53.984 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-12T01:07:53.984 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-12T01:07:53.984 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-12T01:07:53.984 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-12T01:07:53.984 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-12T01:07:53.984 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-12T01:07:53.984 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-12T01:07:53.985 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-12T01:07:53.985 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-12T01:07:53.985 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-12T01:07:53.985 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-12T01:07:53.985 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-12T01:07:53.985 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-12T01:07:53.985 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-12T01:07:53.985 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-12T01:07:53.985 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-12T01:07:53.985 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-12T01:07:53.985 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-12T01:07:53.985 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-12T01:07:53.985 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-12T01:07:53.985 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-12T01:07:53.985 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-12T01:07:53.986 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-12T01:07:53.986 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-12T01:07:53.986 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-12T01:07:53.986 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-12T01:07:53.986 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-12T01:07:53.986 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-12T01:07:53.986 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-12T01:07:53.986 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-12T01:07:53.986 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-12T01:07:53.986 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-12T01:07:53.986 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-12T01:07:53.986 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-12T01:07:53.986 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-12T01:07:53.986 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-12T01:07:53.986 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-12T01:07:53.987 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-12T01:07:53.987 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-12T01:07:53.987 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-12T01:07:53.987 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-12T01:07:53.987 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-12T01:07:53.987 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-12T01:07:53.987 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-12T01:07:53.987 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-12T01:07:53.987 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-12T01:07:53.987 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-12T01:07:53.987 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-12T01:07:53.987 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-12T01:07:53.987 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-12T01:07:53.987 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-12T01:07:53.987 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-12T01:07:53.987 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-12T01:07:53.988 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-12T01:07:53.988 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-12T01:07:53.988 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-12T01:07:53.988 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-12T01:07:53.988 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-12T01:07:53.988 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-12T01:07:53.988 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-12T01:07:53.988 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-12T01:07:53.988 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-12T01:07:53.988 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-12T01:07:53.988 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-12T01:07:53.988 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-12T01:07:53.988 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-12T01:07:53.989 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-12T01:07:53.989 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-12T01:07:53.989 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-12T01:07:53.989 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-12T01:07:53.989 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-12T01:07:53.989 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-12T01:07:53.989 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-12T01:07:53.989 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-12T01:07:53.989 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-12T01:07:53.989 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-12T01:07:53.989 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-12T01:07:53.989 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-12T01:07:53.989 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-12T01:07:53.989 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-12T01:07:53.990 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-12T01:07:53.990 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-12T01:07:53.990 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-12T01:07:53.990 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-12T01:07:53.990 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-12T01:07:53.990 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-12T01:07:53.990 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-12T01:07:53.990 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-12T01:07:53.990 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-12T01:07:53.990 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-12T01:07:53.990 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-12T01:07:53.990 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-12T01:07:53.990 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-12T01:07:53.990 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-12T01:07:53.991 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-12T01:07:53.991 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-12T01:07:53.991 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-12T01:07:53.991 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-12T01:07:53.991 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-12T01:07:53.991 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-12T01:07:53.991 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-12T01:07:53.991 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-12T01:07:53.991 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-12T01:07:53.991 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-12T01:07:53.991 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-12T01:07:53.991 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-12T01:07:53.991 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-12T01:07:53.991 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-12T01:07:53.992 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-12T01:07:53.992 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-12T01:07:53.992 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-12T01:07:53.992 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-12T01:07:53.992 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-12T01:07:53.992 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-12T01:07:53.992 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-12T01:07:53.992 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-12T01:07:53.992 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-12T01:07:53.992 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-12T01:07:53.992 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-12T01:07:53.992 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-12T01:07:53.992 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-12T01:07:53.993 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-12T01:07:53.993 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-12T01:07:53.993 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-12T01:07:53.993 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-12T01:07:53.993 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-12T01:07:53.993 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-12T01:07:53.993 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-12T01:07:53.993 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-12T01:07:53.993 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-12T01:07:53.993 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-12T01:07:53.993 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-12T01:07:53.993 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-12T01:07:53.993 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-12T01:07:53.993 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-12T01:07:53.994 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-12T01:07:53.994 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-12T01:07:53.994 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-12T01:07:53.994 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-12T01:07:53.994 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-12T01:07:53.994 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-12T01:07:53.994 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-12T01:07:53.994 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-12T01:07:53.994 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-12T01:07:53.994 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-12T01:07:53.994 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-12T01:07:53.994 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-12T01:07:53.994 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-12T01:07:53.994 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-12T01:07:53.994 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-12T01:07:53.995 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-12T01:07:53.995 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-12T01:07:53.995 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-12T01:07:53.995 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-12T01:07:53.995 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-12T01:07:53.995 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-12T01:07:53.995 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-12T01:07:53.995 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-12T01:07:53.995 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-12T01:07:53.995 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-12T01:07:53.995 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-12T01:07:53.995 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-12T01:07:53.995 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-12T01:07:53.995 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-12T01:07:53.996 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-12T01:07:53.996 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-12T01:07:53.996 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-12T01:07:53.996 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-12T01:07:53.996 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-12T01:07:53.996 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-12T01:07:53.996 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-12T01:07:53.996 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-12T01:07:53.996 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-12T01:07:53.996 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-12T01:07:53.996 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-12T01:07:53.996 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-12T01:07:53.997 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-12T01:07:53.997 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-12T01:07:53.997 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-12T01:07:53.997 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-12T01:07:53.997 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-12T01:07:53.997 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-12T01:07:53.997 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-12T01:07:53.997 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-12T01:07:53.997 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-12T01:07:53.997 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-12T01:07:53.997 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-12T01:07:53.997 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-12T01:07:53.997 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-12T01:07:53.998 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-12T01:07:53.998 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-12T01:07:53.998 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-12T01:07:53.998 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-12T01:07:53.998 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-12T01:07:53.998 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-12T01:07:53.998 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-12T01:07:53.998 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-12T01:07:53.998 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-12T01:07:53.998 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-12T01:07:53.998 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-12T01:07:53.998 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-12T01:07:53.998 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-12T01:07:53.998 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-12T01:07:53.998 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-12T01:07:53.999 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-12T01:07:53.999 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-12T01:07:53.999 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-12T01:07:53.999 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-12T01:07:53.999 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-12T01:07:53.999 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-12T01:07:53.999 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-12T01:07:53.999 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-12T01:07:53.999 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-12T01:07:53.999 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-12T01:07:53.999 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-12T01:07:53.999 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-12T01:07:53.999 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-12T01:07:53.999 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-12T01:07:54.000 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-12T01:07:54.000 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-12T01:07:54.000 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-12T01:07:54.000 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-12T01:07:54.000 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-12T01:07:54.000 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-12T01:07:54.000 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-12T01:07:54.000 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-12T01:07:54.000 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-12T01:07:54.000 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-12T01:07:54.000 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-12T01:07:54.000 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-12T01:07:54.000 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-12T01:07:54.000 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-12T01:07:54.000 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-12T01:07:54.001 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-12T01:07:54.001 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-12T01:07:54.001 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-12T01:07:54.001 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-12T01:07:54.001 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-12T01:07:54.001 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-12T01:07:54.001 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-12T01:07:54.001 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-12T01:07:54.001 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-12T01:07:54.001 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-12T01:07:54.001 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-12T01:07:54.001 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-12T01:07:54.001 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-12T01:07:54.002 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-12T01:07:54.002 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-12T01:07:54.002 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-12T01:07:54.002 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-12T01:07:54.002 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-12T01:07:54.002 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-12T01:07:54.002 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-12T01:07:54.002 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-12T01:07:54.002 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-12T01:07:54.002 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-12T01:07:54.002 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-12T01:07:54.002 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-12T01:07:54.002 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-12T01:07:54.002 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-12T01:07:54.003 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-12T01:07:54.003 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-12T01:07:54.003 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-12T01:07:54.003 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-12T01:07:54.003 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-12T01:07:54.003 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-12T01:07:54.003 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-12T01:07:54.003 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-12T01:07:54.003 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-12T01:07:54.003 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-12T01:07:54.003 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-12T01:07:54.003 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-12T01:07:54.003 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-12T01:07:54.004 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-12T01:07:54.004 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-12T01:07:54.004 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-12T01:07:54.004 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-12T01:07:54.004 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-12T01:07:54.004 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-12T01:07:54.004 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-12T01:07:54.004 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-12T01:07:54.004 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-12T01:07:54.004 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-12T01:07:54.004 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-12T01:07:54.004 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-12T01:07:54.004 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-12T01:07:54.004 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-12T01:07:54.005 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-12T01:07:54.005 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-12T01:07:54.005 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-12T01:07:54.005 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-12T01:07:54.009 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-12T01:07:54.009 DEBUG:teuthology.orchestra.run.smithi001:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-12T01:07:54.033 INFO:teuthology.orchestra.run.smithi001.stdout:postinstall scriptlet (using /bin/sh): 2024-09-12T01:07:54.033 INFO:teuthology.orchestra.run.smithi001.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-12T01:07:54.033 INFO:teuthology.orchestra.run.smithi001.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-12T01:07:54.033 INFO:teuthology.orchestra.run.smithi001.stdout:fi 2024-09-12T01:07:54.033 INFO:teuthology.orchestra.run.smithi001.stdout:for file in vmlinuz System.map config; do 2024-09-12T01:07:54.033 INFO:teuthology.orchestra.run.smithi001.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-12T01:07:54.033 INFO:teuthology.orchestra.run.smithi001.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-12T01:07:54.033 INFO:teuthology.orchestra.run.smithi001.stdout: fi 2024-09-12T01:07:54.033 INFO:teuthology.orchestra.run.smithi001.stdout:done 2024-09-12T01:07:54.033 INFO:teuthology.orchestra.run.smithi001.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-12T01:07:54.034 INFO:teuthology.orchestra.run.smithi001.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-12T01:07:54.034 INFO:teuthology.orchestra.run.smithi001.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-12T01:07:54.034 INFO:teuthology.orchestra.run.smithi001.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-12T01:07:54.034 INFO:teuthology.orchestra.run.smithi001.stdout:fi 2024-09-12T01:07:54.034 INFO:teuthology.orchestra.run.smithi001.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-12T01:07:54.034 INFO:teuthology.orchestra.run.smithi001.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-12T01:07:54.034 INFO:teuthology.orchestra.run.smithi001.stdout:fi 2024-09-12T01:07:54.034 INFO:teuthology.orchestra.run.smithi001.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-12T01:07:54.034 INFO:teuthology.orchestra.run.smithi001.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-12T01:07:54.034 INFO:teuthology.orchestra.run.smithi001.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-12T01:07:54.034 INFO:teuthology.orchestra.run.smithi001.stdout:fi 2024-09-12T01:07:54.035 DEBUG:teuthology.orchestra.run.smithi001:> sudo rpm -qi grub2-tools 2024-09-12T01:07:54.115 INFO:teuthology.orchestra.run.smithi001.stdout:Name : grub2-tools 2024-09-12T01:07:54.116 INFO:teuthology.orchestra.run.smithi001.stdout:Epoch : 1 2024-09-12T01:07:54.116 INFO:teuthology.orchestra.run.smithi001.stdout:Version : 2.06 2024-09-12T01:07:54.116 INFO:teuthology.orchestra.run.smithi001.stdout:Release : 82.el9 2024-09-12T01:07:54.116 INFO:teuthology.orchestra.run.smithi001.stdout:Architecture: x86_64 2024-09-12T01:07:54.116 INFO:teuthology.orchestra.run.smithi001.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-12T01:07:54.116 INFO:teuthology.orchestra.run.smithi001.stdout:Group : Unspecified 2024-09-12T01:07:54.116 INFO:teuthology.orchestra.run.smithi001.stdout:Size : 8274670 2024-09-12T01:07:54.116 INFO:teuthology.orchestra.run.smithi001.stdout:License : GPLv3+ 2024-09-12T01:07:54.116 INFO:teuthology.orchestra.run.smithi001.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-12T01:07:54.116 INFO:teuthology.orchestra.run.smithi001.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-12T01:07:54.116 INFO:teuthology.orchestra.run.smithi001.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-12T01:07:54.117 INFO:teuthology.orchestra.run.smithi001.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-12T01:07:54.117 INFO:teuthology.orchestra.run.smithi001.stdout:Packager : builder@centos.org 2024-09-12T01:07:54.117 INFO:teuthology.orchestra.run.smithi001.stdout:Vendor : CentOS 2024-09-12T01:07:54.117 INFO:teuthology.orchestra.run.smithi001.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-12T01:07:54.117 INFO:teuthology.orchestra.run.smithi001.stdout:Summary : Support tools for GRUB. 2024-09-12T01:07:54.117 INFO:teuthology.orchestra.run.smithi001.stdout:Description : 2024-09-12T01:07:54.117 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:07:54.117 INFO:teuthology.orchestra.run.smithi001.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-12T01:07:54.117 INFO:teuthology.orchestra.run.smithi001.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-12T01:07:54.117 INFO:teuthology.orchestra.run.smithi001.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-12T01:07:54.117 INFO:teuthology.orchestra.run.smithi001.stdout:hardware devices. 2024-09-12T01:07:54.117 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:07:54.117 INFO:teuthology.orchestra.run.smithi001.stdout:This subpackage provides tools for support of all platforms. 2024-09-12T01:07:54.118 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-12T01:07:54.118 INFO:teuthology.task.kernel:Updating grub on smithi001 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-12T01:07:54.119 DEBUG:teuthology.orchestra.run.smithi001:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-12T01:07:54.711 INFO:teuthology.orchestra.run.smithi001.stderr:Generating grub configuration file ... 2024-09-12T01:07:56.106 INFO:teuthology.orchestra.run.smithi001.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-12T01:07:56.138 INFO:teuthology.orchestra.run.smithi001.stderr:done 2024-09-12T01:07:56.140 DEBUG:teuthology.orchestra.run.smithi001:> mktemp 2024-09-12T01:07:56.155 INFO:teuthology.orchestra.run.smithi001.stdout:/tmp/tmp.vdDKWSQcF4 2024-09-12T01:07:56.156 DEBUG:teuthology.orchestra.run.smithi001:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.vdDKWSQcF4 2024-09-12T01:07:56.221 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 0666 /tmp/tmp.vdDKWSQcF4 2024-09-12T01:07:56.366 DEBUG:teuthology.orchestra.remote:smithi001:/tmp/tmp.vdDKWSQcF4 is 6KB 2024-09-12T01:07:56.415 DEBUG:teuthology.orchestra.run.smithi001:> rm -fr /tmp/tmp.vdDKWSQcF4 2024-09-12T01:07:56.430 DEBUG:teuthology.orchestra.run.smithi001:> sudo /bin/ls /boot/loader/entries || true 2024-09-12T01:07:56.492 INFO:teuthology.orchestra.run.smithi001.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-12T01:07:56.492 INFO:teuthology.orchestra.run.smithi001.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-12T01:07:56.492 INFO:teuthology.orchestra.run.smithi001.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-12T01:07:56.492 INFO:teuthology.orchestra.run.smithi001.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-12T01:07:56.493 DEBUG:teuthology.orchestra.run.smithi001:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-12T01:07:56.667 DEBUG:teuthology.orchestra.run.smithi001:> sudo shutdown -r now 2024-09-12T01:08:26.670 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-12T01:08:26.671 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-09-12T01:08:26.672 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T01:08:45.249 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.1 2024-09-12T01:08:54.250 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-09-12T01:08:54.251 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T01:09:54.261 DEBUG:teuthology.orchestra.remote:timed out 2024-09-12T01:10:06.262 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-09-12T01:10:06.264 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T01:10:06.651 DEBUG:teuthology.orchestra.run.smithi001:> true 2024-09-12T01:10:07.163 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-09-12T01:10:07.164 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-12T01:10:07.164 DEBUG:teuthology.orchestra.run.smithi001:> uname -r 2024-09-12T01:10:07.198 INFO:teuthology.orchestra.run.smithi001.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-12T01:10:07.199 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-12T01:10:07.199 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-12T01:10:07.199 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-12T01:10:08.200 DEBUG:teuthology.parallel:result is None 2024-09-12T01:10:08.201 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-12T01:10:08.209 INFO:teuthology.task.internal:Creating test directory... 2024-09-12T01:10:08.209 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T01:10:08.214 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T01:10:08.216 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T01:10:08.248 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-12T01:10:08.254 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-12T01:10:08.281 INFO:teuthology.task.internal:Creating archive directory... 2024-09-12T01:10:08.281 DEBUG:teuthology.orchestra.run.smithi001:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T01:10:08.292 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T01:10:08.294 DEBUG:teuthology.orchestra.run.smithi104:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T01:10:08.340 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-12T01:10:08.347 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-12T01:10:08.347 DEBUG:teuthology.orchestra.run.smithi053:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T01:10:08.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T01:10:08.391 DEBUG:teuthology.orchestra.run.smithi104:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T01:10:08.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T01:10:08.407 DEBUG:teuthology.orchestra.run.smithi001:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T01:10:08.411 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T01:10:08.433 DEBUG:teuthology.orchestra.run.smithi104:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T01:10:08.465 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T01:10:08.485 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T01:10:08.506 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T01:10:08.513 INFO:teuthology.orchestra.run.smithi001.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T01:10:08.522 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T01:10:08.546 INFO:teuthology.orchestra.run.smithi001.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T01:10:08.549 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-12T01:10:08.558 INFO:teuthology.task.internal:Configuring sudo... 2024-09-12T01:10:08.558 DEBUG:teuthology.orchestra.run.smithi001:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T01:10:08.562 DEBUG:teuthology.orchestra.run.smithi053:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T01:10:08.564 DEBUG:teuthology.orchestra.run.smithi104:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T01:10:08.614 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-12T01:10:08.623 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-12T01:10:08.623 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T01:10:08.627 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T01:10:08.632 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T01:10:08.657 DEBUG:teuthology.orchestra.run.smithi001:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T01:10:08.765 DEBUG:teuthology.orchestra.run.smithi001:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T01:10:08.865 DEBUG:teuthology.orchestra.run.smithi001:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T01:10:08.951 DEBUG:teuthology.orchestra.run.smithi001:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T01:10:09.007 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-12T01:10:09.007 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T01:10:09.061 DEBUG:teuthology.orchestra.run.smithi053:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T01:10:09.115 DEBUG:teuthology.orchestra.run.smithi053:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T01:10:09.160 DEBUG:teuthology.orchestra.run.smithi053:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T01:10:09.253 DEBUG:teuthology.orchestra.run.smithi053:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T01:10:09.288 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-12T01:10:09.288 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T01:10:09.355 DEBUG:teuthology.orchestra.run.smithi104:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T01:10:09.412 DEBUG:teuthology.orchestra.run.smithi104:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T01:10:09.483 DEBUG:teuthology.orchestra.run.smithi104:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T01:10:09.538 DEBUG:teuthology.orchestra.run.smithi104:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T01:10:09.570 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-12T01:10:09.570 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T01:10:09.637 DEBUG:teuthology.orchestra.run.smithi001:> sudo service rsyslog restart 2024-09-12T01:10:09.641 DEBUG:teuthology.orchestra.run.smithi053:> sudo service rsyslog restart 2024-09-12T01:10:09.643 DEBUG:teuthology.orchestra.run.smithi104:> sudo service rsyslog restart 2024-09-12T01:10:09.685 INFO:teuthology.orchestra.run.smithi053.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T01:10:09.717 INFO:teuthology.orchestra.run.smithi104.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T01:10:09.733 INFO:teuthology.orchestra.run.smithi001.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T01:10:10.245 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-12T01:10:10.253 INFO:teuthology.task.internal:Starting timer... 2024-09-12T01:10:10.253 INFO:teuthology.run_tasks:Running task pcp... 2024-09-12T01:10:10.270 INFO:teuthology.run_tasks:Running task selinux... 2024-09-12T01:10:10.304 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-12T01:10:10.304 DEBUG:teuthology.orchestra.run.smithi001:> sudo service auditd rotate 2024-09-12T01:10:10.390 INFO:teuthology.orchestra.run.smithi001.stdout:Rotating logs: 2024-09-12T01:10:10.393 DEBUG:teuthology.orchestra.run.smithi053:> sudo service auditd rotate 2024-09-12T01:10:10.446 INFO:teuthology.orchestra.run.smithi053.stdout:Rotating logs: 2024-09-12T01:10:10.447 DEBUG:teuthology.orchestra.run.smithi104:> sudo service auditd rotate 2024-09-12T01:10:10.499 INFO:teuthology.orchestra.run.smithi104.stdout:Rotating logs: 2024-09-12T01:10:10.501 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-12T01:10:10.501 DEBUG:teuthology.orchestra.run.smithi001:> /usr/sbin/getenforce 2024-09-12T01:10:10.538 INFO:teuthology.orchestra.run.smithi001.stdout:Permissive 2024-09-12T01:10:10.538 DEBUG:teuthology.orchestra.run.smithi053:> /usr/sbin/getenforce 2024-09-12T01:10:10.561 INFO:teuthology.orchestra.run.smithi053.stdout:Permissive 2024-09-12T01:10:10.562 DEBUG:teuthology.orchestra.run.smithi104:> /usr/sbin/getenforce 2024-09-12T01:10:10.588 INFO:teuthology.orchestra.run.smithi104.stdout:Permissive 2024-09-12T01:10:10.589 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi001.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi053.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi104.front.sepia.ceph.com': 'permissive'} 2024-09-12T01:10:10.589 DEBUG:teuthology.orchestra.run.smithi001:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T01:10:10.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T01:10:10.647 DEBUG:teuthology.orchestra.run.smithi053:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T01:10:10.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T01:10:10.674 DEBUG:teuthology.orchestra.run.smithi104:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T01:10:10.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T01:10:10.701 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-12T01:10:10.702 DEBUG:teuthology.orchestra.run.smithi001:> sudo /usr/sbin/setenforce permissive 2024-09-12T01:10:10.787 DEBUG:teuthology.orchestra.run.smithi053:> sudo /usr/sbin/setenforce permissive 2024-09-12T01:10:10.814 DEBUG:teuthology.orchestra.run.smithi104:> sudo /usr/sbin/setenforce permissive 2024-09-12T01:10:10.852 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-12T01:10:10.861 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-12T01:10:10.864 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-12T01:10:10.982 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-12T01:10:10.993 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-12T01:10:10.994 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi001.front.sepia.ceph.com,smithi053.front.sepia.ceph.com,smithi104.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-12T01:14:54.261 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi001.front.sepia.ceph.com'), Remote(name='ubuntu@smithi053.front.sepia.ceph.com'), Remote(name='ubuntu@smithi104.front.sepia.ceph.com')] 2024-09-12T01:14:54.262 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-09-12T01:14:54.264 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T01:14:54.358 DEBUG:teuthology.orchestra.run.smithi001:> true 2024-09-12T01:14:54.466 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-09-12T01:14:54.467 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-09-12T01:14:54.467 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T01:14:54.550 DEBUG:teuthology.orchestra.run.smithi053:> true 2024-09-12T01:14:54.635 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-09-12T01:14:54.635 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-09-12T01:14:54.636 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T01:14:54.721 DEBUG:teuthology.orchestra.run.smithi104:> true 2024-09-12T01:14:54.800 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-09-12T01:14:54.801 INFO:teuthology.run_tasks:Running task clock... 2024-09-12T01:14:54.813 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-12T01:14:54.813 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T01:14:54.814 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T01:14:54.817 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T01:14:54.817 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T01:14:54.820 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T01:14:54.821 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T01:14:54.851 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T01:14:54.867 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T01:14:54.870 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T01:14:54.874 INFO:teuthology.orchestra.run.smithi001.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T01:14:54.884 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T01:14:54.895 INFO:teuthology.orchestra.run.smithi053.stderr:sudo: ntpd: command not found 2024-09-12T01:14:54.904 INFO:teuthology.orchestra.run.smithi001.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T01:14:54.907 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: ntpd: command not found 2024-09-12T01:14:54.908 INFO:teuthology.orchestra.run.smithi053.stdout:506 Cannot talk to daemon 2024-09-12T01:14:54.918 INFO:teuthology.orchestra.run.smithi104.stdout:506 Cannot talk to daemon 2024-09-12T01:14:54.924 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T01:14:54.932 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T01:14:54.939 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T01:14:54.946 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T01:14:54.956 INFO:teuthology.orchestra.run.smithi001.stderr:sudo: ntpd: command not found 2024-09-12T01:14:54.980 INFO:teuthology.orchestra.run.smithi001.stdout:506 Cannot talk to daemon 2024-09-12T01:14:54.994 INFO:teuthology.orchestra.run.smithi053.stderr:bash: line 1: ntpq: command not found 2024-09-12T01:14:54.998 INFO:teuthology.orchestra.run.smithi053.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T01:14:54.998 INFO:teuthology.orchestra.run.smithi053.stdout:=============================================================================== 2024-09-12T01:14:54.998 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:14:54.998 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:14:54.998 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:14:54.999 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:14:55.004 INFO:teuthology.orchestra.run.smithi104.stderr:bash: line 1: ntpq: command not found 2024-09-12T01:14:55.008 INFO:teuthology.orchestra.run.smithi104.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T01:14:55.008 INFO:teuthology.orchestra.run.smithi104.stdout:=============================================================================== 2024-09-12T01:14:55.008 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:14:55.008 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:14:55.008 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:14:55.008 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:14:55.009 INFO:teuthology.orchestra.run.smithi001.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T01:14:55.039 INFO:teuthology.orchestra.run.smithi001.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T01:14:55.138 INFO:teuthology.orchestra.run.smithi001.stderr:bash: line 1: ntpq: command not found 2024-09-12T01:14:55.145 INFO:teuthology.orchestra.run.smithi001.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T01:14:55.145 INFO:teuthology.orchestra.run.smithi001.stdout:=============================================================================== 2024-09-12T01:14:55.145 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:14:55.145 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:14:55.145 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:14:55.145 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:14:55.146 INFO:teuthology.run_tasks:Running task install... 2024-09-12T01:14:55.154 DEBUG:teuthology.task.install:project ceph 2024-09-12T01:14:55.154 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}} 2024-09-12T01:14:55.154 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'} 2024-09-12T01:14:55.155 INFO:teuthology.task.install:Using flavor: default 2024-09-12T01:14:55.160 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-12T01:14:55.160 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-12T01:14:55.161 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T01:14:55.161 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T01:14:55.163 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T01:14:55.163 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T01:14:55.164 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T01:14:55.164 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T01:14:55.319 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T01:14:55.319 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T01:14:55.330 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T01:14:55.330 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T01:14:55.342 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T01:14:55.342 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T01:14:55.508 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T01:14:55.508 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-12T01:14:55.508 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T01:14:55.508 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T01:14:55.509 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-12T01:14:55.509 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T01:14:55.536 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T01:14:55.536 DEBUG:teuthology.orchestra.run.smithi053:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T01:14:55.541 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T01:14:55.541 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-12T01:14:55.541 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T01:14:55.568 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T01:14:55.568 DEBUG:teuthology.orchestra.run.smithi001:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T01:14:55.569 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T01:14:55.569 DEBUG:teuthology.orchestra.run.smithi104:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T01:14:55.603 DEBUG:teuthology.orchestra.run.smithi053:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T01:14:55.636 DEBUG:teuthology.orchestra.run.smithi104:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T01:14:55.667 DEBUG:teuthology.orchestra.run.smithi001:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T01:14:55.679 DEBUG:teuthology.orchestra.run.smithi053:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T01:14:55.712 DEBUG:teuthology.orchestra.run.smithi104:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T01:14:55.745 INFO:teuthology.orchestra.run.smithi053.stdout:check_obsoletes = 1 2024-09-12T01:14:55.746 DEBUG:teuthology.orchestra.run.smithi001:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T01:14:55.747 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum clean all 2024-09-12T01:14:55.777 INFO:teuthology.orchestra.run.smithi104.stdout:check_obsoletes = 1 2024-09-12T01:14:55.778 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum clean all 2024-09-12T01:14:55.802 INFO:teuthology.orchestra.run.smithi001.stdout:check_obsoletes = 1 2024-09-12T01:14:55.805 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum clean all 2024-09-12T01:14:55.980 INFO:teuthology.orchestra.run.smithi053.stdout:45 files removed 2024-09-12T01:14:56.001 INFO:teuthology.orchestra.run.smithi104.stdout:45 files removed 2024-09-12T01:14:56.007 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T01:14:56.028 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T01:14:56.085 INFO:teuthology.orchestra.run.smithi001.stdout:45 files removed 2024-09-12T01:14:56.118 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T01:14:56.722 INFO:teuthology.orchestra.run.smithi053.stdout:ceph packages for x86_64 193 kB/s | 84 kB 00:00 2024-09-12T01:14:56.735 INFO:teuthology.orchestra.run.smithi104.stdout:ceph packages for x86_64 195 kB/s | 84 kB 00:00 2024-09-12T01:14:56.902 INFO:teuthology.orchestra.run.smithi001.stdout:ceph packages for x86_64 239 kB/s | 84 kB 00:00 2024-09-12T01:14:57.086 INFO:teuthology.orchestra.run.smithi104.stdout:ceph noarch packages 38 kB/s | 12 kB 00:00 2024-09-12T01:14:57.114 INFO:teuthology.orchestra.run.smithi053.stdout:ceph noarch packages 34 kB/s | 12 kB 00:00 2024-09-12T01:14:57.236 INFO:teuthology.orchestra.run.smithi001.stdout:ceph noarch packages 41 kB/s | 12 kB 00:00 2024-09-12T01:14:57.361 INFO:teuthology.orchestra.run.smithi104.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-12T01:14:57.390 INFO:teuthology.orchestra.run.smithi053.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-12T01:14:57.612 INFO:teuthology.orchestra.run.smithi001.stdout:ceph source packages 5.6 kB/s | 1.9 kB 00:00 2024-09-12T01:14:57.865 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - BaseOS 19 MB/s | 8.2 MB 00:00 2024-09-12T01:14:58.428 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - BaseOS 8.0 MB/s | 8.2 MB 00:01 2024-09-12T01:14:59.866 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - AppStream 24 MB/s | 20 MB 00:00 2024-09-12T01:15:00.270 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - BaseOS 3.2 MB/s | 8.2 MB 00:02 2024-09-12T01:15:04.475 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - CRB 9.6 MB/s | 6.5 MB 00:00 2024-09-12T01:15:04.753 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - AppStream 3.9 MB/s | 20 MB 00:05 2024-09-12T01:15:06.209 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 19 kB 00:00 2024-09-12T01:15:07.038 INFO:teuthology.orchestra.run.smithi053.stdout:Extra Packages for Enterprise Linux 32 MB/s | 23 MB 00:00 2024-09-12T01:15:07.879 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - AppStream 3.2 MB/s | 20 MB 00:06 2024-09-12T01:15:09.137 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-12T01:15:10.754 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 19 kB 00:00 2024-09-12T01:15:11.534 INFO:teuthology.orchestra.run.smithi104.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-12T01:15:12.803 INFO:teuthology.orchestra.run.smithi053.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-12T01:15:14.638 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - CRB 2.6 MB/s | 6.5 MB 00:02 2024-09-12T01:15:15.155 INFO:teuthology.orchestra.run.smithi053.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T01:15:15.156 INFO:teuthology.orchestra.run.smithi053.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T01:15:15.167 INFO:teuthology.orchestra.run.smithi053.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T01:15:15.168 INFO:teuthology.orchestra.run.smithi053.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T01:15:15.169 INFO:teuthology.orchestra.run.smithi053.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T01:15:15.170 INFO:teuthology.orchestra.run.smithi053.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T01:15:15.172 INFO:teuthology.orchestra.run.smithi053.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T01:15:15.174 INFO:teuthology.orchestra.run.smithi053.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T01:15:15.175 INFO:teuthology.orchestra.run.smithi053.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T01:15:15.241 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-12T01:15:15.266 INFO:teuthology.orchestra.run.smithi053.stdout:====================================================================================== 2024-09-12T01:15:15.266 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-09-12T01:15:15.267 INFO:teuthology.orchestra.run.smithi053.stdout:====================================================================================== 2024-09-12T01:15:15.267 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-12T01:15:15.267 INFO:teuthology.orchestra.run.smithi053.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T01:15:15.267 INFO:teuthology.orchestra.run.smithi053.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T01:15:15.267 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T01:15:15.267 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T01:15:15.267 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T01:15:15.267 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T01:15:15.267 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T01:15:15.267 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T01:15:15.267 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T01:15:15.268 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T01:15:15.268 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T01:15:15.268 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T01:15:15.268 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T01:15:15.268 INFO:teuthology.orchestra.run.smithi053.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T01:15:15.268 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T01:15:15.268 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T01:15:15.268 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T01:15:15.268 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T01:15:15.268 INFO:teuthology.orchestra.run.smithi053.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T01:15:15.268 INFO:teuthology.orchestra.run.smithi053.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T01:15:15.268 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T01:15:15.269 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T01:15:15.269 INFO:teuthology.orchestra.run.smithi053.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T01:15:15.269 INFO:teuthology.orchestra.run.smithi053.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T01:15:15.269 INFO:teuthology.orchestra.run.smithi053.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T01:15:15.269 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T01:15:15.269 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T01:15:15.269 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T01:15:15.269 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T01:15:15.269 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T01:15:15.269 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T01:15:15.269 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T01:15:15.269 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T01:15:15.269 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T01:15:15.270 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T01:15:15.270 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T01:15:15.270 INFO:teuthology.orchestra.run.smithi053.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T01:15:15.270 INFO:teuthology.orchestra.run.smithi053.stdout:Upgrading: 2024-09-12T01:15:15.270 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T01:15:15.270 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T01:15:15.270 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T01:15:15.270 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T01:15:15.270 INFO:teuthology.orchestra.run.smithi053.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T01:15:15.270 INFO:teuthology.orchestra.run.smithi053.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T01:15:15.270 INFO:teuthology.orchestra.run.smithi053.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T01:15:15.270 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T01:15:15.271 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-09-12T01:15:15.271 INFO:teuthology.orchestra.run.smithi053.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T01:15:15.271 INFO:teuthology.orchestra.run.smithi053.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T01:15:15.271 INFO:teuthology.orchestra.run.smithi053.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T01:15:15.271 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T01:15:15.271 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T01:15:15.271 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T01:15:15.271 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T01:15:15.271 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T01:15:15.271 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T01:15:15.271 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T01:15:15.271 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T01:15:15.271 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T01:15:15.272 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T01:15:15.272 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T01:15:15.272 INFO:teuthology.orchestra.run.smithi053.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T01:15:15.272 INFO:teuthology.orchestra.run.smithi053.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-12T01:15:15.272 INFO:teuthology.orchestra.run.smithi053.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T01:15:15.272 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T01:15:15.272 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T01:15:15.272 INFO:teuthology.orchestra.run.smithi053.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T01:15:15.272 INFO:teuthology.orchestra.run.smithi053.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T01:15:15.272 INFO:teuthology.orchestra.run.smithi053.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T01:15:15.272 INFO:teuthology.orchestra.run.smithi053.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T01:15:15.272 INFO:teuthology.orchestra.run.smithi053.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T01:15:15.272 INFO:teuthology.orchestra.run.smithi053.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T01:15:15.273 INFO:teuthology.orchestra.run.smithi053.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T01:15:15.273 INFO:teuthology.orchestra.run.smithi053.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T01:15:15.273 INFO:teuthology.orchestra.run.smithi053.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T01:15:15.273 INFO:teuthology.orchestra.run.smithi053.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T01:15:15.273 INFO:teuthology.orchestra.run.smithi053.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T01:15:15.273 INFO:teuthology.orchestra.run.smithi053.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T01:15:15.273 INFO:teuthology.orchestra.run.smithi053.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T01:15:15.273 INFO:teuthology.orchestra.run.smithi053.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T01:15:15.273 INFO:teuthology.orchestra.run.smithi053.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T01:15:15.273 INFO:teuthology.orchestra.run.smithi053.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T01:15:15.273 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T01:15:15.273 INFO:teuthology.orchestra.run.smithi053.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T01:15:15.273 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T01:15:15.274 INFO:teuthology.orchestra.run.smithi053.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T01:15:15.274 INFO:teuthology.orchestra.run.smithi053.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T01:15:15.274 INFO:teuthology.orchestra.run.smithi053.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T01:15:15.274 INFO:teuthology.orchestra.run.smithi053.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T01:15:15.274 INFO:teuthology.orchestra.run.smithi053.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T01:15:15.274 INFO:teuthology.orchestra.run.smithi053.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T01:15:15.274 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T01:15:15.274 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T01:15:15.274 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T01:15:15.274 INFO:teuthology.orchestra.run.smithi053.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T01:15:15.274 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T01:15:15.274 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T01:15:15.274 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T01:15:15.275 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T01:15:15.275 INFO:teuthology.orchestra.run.smithi053.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T01:15:15.275 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T01:15:15.275 INFO:teuthology.orchestra.run.smithi053.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T01:15:15.275 INFO:teuthology.orchestra.run.smithi053.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T01:15:15.275 INFO:teuthology.orchestra.run.smithi053.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T01:15:15.275 INFO:teuthology.orchestra.run.smithi053.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T01:15:15.275 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T01:15:15.275 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T01:15:15.275 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T01:15:15.275 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T01:15:15.275 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T01:15:15.275 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T01:15:15.276 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T01:15:15.276 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T01:15:15.276 INFO:teuthology.orchestra.run.smithi053.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T01:15:15.276 INFO:teuthology.orchestra.run.smithi053.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T01:15:15.276 INFO:teuthology.orchestra.run.smithi053.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T01:15:15.276 INFO:teuthology.orchestra.run.smithi053.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T01:15:15.276 INFO:teuthology.orchestra.run.smithi053.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T01:15:15.276 INFO:teuthology.orchestra.run.smithi053.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T01:15:15.276 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T01:15:15.276 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T01:15:15.276 INFO:teuthology.orchestra.run.smithi053.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T01:15:15.276 INFO:teuthology.orchestra.run.smithi053.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T01:15:15.276 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T01:15:15.277 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T01:15:15.277 INFO:teuthology.orchestra.run.smithi053.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T01:15:15.277 INFO:teuthology.orchestra.run.smithi053.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T01:15:15.277 INFO:teuthology.orchestra.run.smithi053.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T01:15:15.277 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T01:15:15.277 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T01:15:15.277 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T01:15:15.277 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T01:15:15.277 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T01:15:15.277 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T01:15:15.277 INFO:teuthology.orchestra.run.smithi053.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T01:15:15.277 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T01:15:15.277 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T01:15:15.278 INFO:teuthology.orchestra.run.smithi053.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T01:15:15.278 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T01:15:15.278 INFO:teuthology.orchestra.run.smithi053.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T01:15:15.278 INFO:teuthology.orchestra.run.smithi053.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T01:15:15.278 INFO:teuthology.orchestra.run.smithi053.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T01:15:15.278 INFO:teuthology.orchestra.run.smithi053.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T01:15:15.278 INFO:teuthology.orchestra.run.smithi053.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T01:15:15.278 INFO:teuthology.orchestra.run.smithi053.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T01:15:15.278 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T01:15:15.278 INFO:teuthology.orchestra.run.smithi053.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T01:15:15.278 INFO:teuthology.orchestra.run.smithi053.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T01:15:15.278 INFO:teuthology.orchestra.run.smithi053.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T01:15:15.278 INFO:teuthology.orchestra.run.smithi053.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T01:15:15.278 INFO:teuthology.orchestra.run.smithi053.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T01:15:15.279 INFO:teuthology.orchestra.run.smithi053.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T01:15:15.279 INFO:teuthology.orchestra.run.smithi053.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T01:15:15.279 INFO:teuthology.orchestra.run.smithi053.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T01:15:15.279 INFO:teuthology.orchestra.run.smithi053.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T01:15:15.279 INFO:teuthology.orchestra.run.smithi053.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T01:15:15.279 INFO:teuthology.orchestra.run.smithi053.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T01:15:15.279 INFO:teuthology.orchestra.run.smithi053.stdout:Installing weak dependencies: 2024-09-12T01:15:15.279 INFO:teuthology.orchestra.run.smithi053.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T01:15:15.279 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-12T01:15:15.279 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-12T01:15:15.279 INFO:teuthology.orchestra.run.smithi053.stdout:====================================================================================== 2024-09-12T01:15:15.279 INFO:teuthology.orchestra.run.smithi053.stdout:Install 148 Packages 2024-09-12T01:15:15.279 INFO:teuthology.orchestra.run.smithi053.stdout:Upgrade 8 Packages 2024-09-12T01:15:15.279 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-12T01:15:15.280 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 226 M 2024-09-12T01:15:15.280 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-12T01:15:15.946 INFO:teuthology.orchestra.run.smithi053.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 45 kB/s | 6.5 kB 00:00 2024-09-12T01:15:16.179 INFO:teuthology.orchestra.run.smithi053.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 3.8 MB/s | 897 kB 00:00 2024-09-12T01:15:16.246 INFO:teuthology.orchestra.run.smithi053.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 2.1 MB/s | 145 kB 00:00 2024-09-12T01:15:16.330 INFO:teuthology.orchestra.run.smithi053.stdout:(4/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 10 MB/s | 5.4 MB 00:00 2024-09-12T01:15:16.447 INFO:teuthology.orchestra.run.smithi053.stdout:(5/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 13 MB/s | 1.5 MB 00:00 2024-09-12T01:15:16.622 INFO:teuthology.orchestra.run.smithi053.stdout:(6/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 6.3 MB/s | 2.4 MB 00:00 2024-09-12T01:15:16.747 INFO:teuthology.orchestra.run.smithi053.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 16 MB/s | 4.7 MB 00:00 2024-09-12T01:15:16.897 INFO:teuthology.orchestra.run.smithi104.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-12T01:15:16.981 INFO:teuthology.orchestra.run.smithi053.stdout:(8/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 18 MB/s | 21 MB 00:01 2024-09-12T01:15:17.023 INFO:teuthology.orchestra.run.smithi053.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 610 kB/s | 25 kB 00:00 2024-09-12T01:15:17.039 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 19 kB 00:01 2024-09-12T01:15:17.340 INFO:teuthology.orchestra.run.smithi053.stdout:(10/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 18 MB/s | 11 MB 00:00 2024-09-12T01:15:17.398 INFO:teuthology.orchestra.run.smithi053.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 3.8 MB/s | 228 kB 00:00 2024-09-12T01:15:17.490 INFO:teuthology.orchestra.run.smithi053.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 295 kB/s | 27 kB 00:00 2024-09-12T01:15:17.574 INFO:teuthology.orchestra.run.smithi053.stdout:(13/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 380 kB/s | 32 kB 00:00 2024-09-12T01:15:17.682 INFO:teuthology.orchestra.run.smithi053.stdout:(14/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 6.8 MB/s | 752 kB 00:00 2024-09-12T01:15:17.758 INFO:teuthology.orchestra.run.smithi053.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 2.1 MB/s | 163 kB 00:00 2024-09-12T01:15:17.893 INFO:teuthology.orchestra.run.smithi001.stdout:Extra Packages for Enterprise Linux 32 MB/s | 23 MB 00:00 2024-09-12T01:15:17.966 INFO:teuthology.orchestra.run.smithi053.stdout:(16/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 13 MB/s | 17 MB 00:01 2024-09-12T01:15:17.991 INFO:teuthology.orchestra.run.smithi053.stdout:(17/156): librados-devel-19.1.1-298.gb53b6d3e.e 542 kB/s | 127 kB 00:00 2024-09-12T01:15:18.033 INFO:teuthology.orchestra.run.smithi053.stdout:(18/156): libradosstriper1-19.1.1-298.gb53b6d3e 7.4 MB/s | 506 kB 00:00 2024-09-12T01:15:18.100 INFO:teuthology.orchestra.run.smithi053.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 678 kB/s | 45 kB 00:00 2024-09-12T01:15:18.201 INFO:teuthology.orchestra.run.smithi053.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 1.3 MB/s | 136 kB 00:00 2024-09-12T01:15:18.276 INFO:teuthology.orchestra.run.smithi053.stdout:(21/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 19 MB/s | 5.3 MB 00:00 2024-09-12T01:15:18.301 INFO:teuthology.orchestra.run.smithi053.stdout:(22/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 1.6 MB/s | 162 kB 00:00 2024-09-12T01:15:18.326 INFO:teuthology.orchestra.run.smithi053.stdout:(23/156): python3-rados-19.1.1-298.gb53b6d3e.el 6.2 MB/s | 321 kB 00:00 2024-09-12T01:15:18.352 INFO:teuthology.orchestra.run.smithi053.stdout:(24/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 5.9 MB/s | 301 kB 00:00 2024-09-12T01:15:18.377 INFO:teuthology.orchestra.run.smithi053.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.9 MB/s | 100 kB 00:00 2024-09-12T01:15:18.403 INFO:teuthology.orchestra.run.smithi053.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.6 MB/s | 85 kB 00:00 2024-09-12T01:15:18.478 INFO:teuthology.orchestra.run.smithi053.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 2.2 MB/s | 171 kB 00:00 2024-09-12T01:15:18.537 INFO:teuthology.orchestra.run.smithi053.stdout:(28/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 19 MB/s | 3.1 MB 00:00 2024-09-12T01:15:18.562 INFO:teuthology.orchestra.run.smithi053.stdout:(29/156): ceph-grafana-dashboards-19.1.1-298.gb 381 kB/s | 32 kB 00:00 2024-09-12T01:15:18.595 INFO:teuthology.orchestra.run.smithi053.stdout:(30/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 2.4 MB/s | 146 kB 00:00 2024-09-12T01:15:18.812 INFO:teuthology.orchestra.run.smithi053.stdout:(31/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 14 MB/s | 3.6 MB 00:00 2024-09-12T01:15:18.938 INFO:teuthology.orchestra.run.smithi053.stdout:(32/156): ceph-mgr-diskprediction-local-19.1.1- 22 MB/s | 7.4 MB 00:00 2024-09-12T01:15:18.963 INFO:teuthology.orchestra.run.smithi053.stdout:(33/156): ceph-mgr-modules-core-19.1.1-298.gb53 1.6 MB/s | 249 kB 00:00 2024-09-12T01:15:18.996 INFO:teuthology.orchestra.run.smithi053.stdout:(34/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 844 kB/s | 49 kB 00:00 2024-09-12T01:15:19.013 INFO:teuthology.orchestra.run.smithi053.stdout:(35/156): ceph-prometheus-alerts-19.1.1-298.gb5 326 kB/s | 16 kB 00:00 2024-09-12T01:15:19.047 INFO:teuthology.orchestra.run.smithi053.stdout:(36/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 5.5 MB/s | 285 kB 00:00 2024-09-12T01:15:19.089 INFO:teuthology.orchestra.run.smithi053.stdout:(37/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 9.9 MB/s | 765 kB 00:00 2024-09-12T01:15:19.106 INFO:teuthology.orchestra.run.smithi053.stdout:(38/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 312 kB/s | 18 kB 00:00 2024-09-12T01:15:19.174 INFO:teuthology.orchestra.run.smithi104.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T01:15:19.174 INFO:teuthology.orchestra.run.smithi104.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T01:15:19.185 INFO:teuthology.orchestra.run.smithi104.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T01:15:19.186 INFO:teuthology.orchestra.run.smithi104.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T01:15:19.187 INFO:teuthology.orchestra.run.smithi104.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T01:15:19.188 INFO:teuthology.orchestra.run.smithi104.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T01:15:19.190 INFO:teuthology.orchestra.run.smithi104.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T01:15:19.192 INFO:teuthology.orchestra.run.smithi104.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T01:15:19.193 INFO:teuthology.orchestra.run.smithi104.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T01:15:19.255 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-09-12T01:15:19.270 INFO:teuthology.orchestra.run.smithi104.stdout:====================================================================================== 2024-09-12T01:15:19.270 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repository Size 2024-09-12T01:15:19.270 INFO:teuthology.orchestra.run.smithi104.stdout:====================================================================================== 2024-09-12T01:15:19.270 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-09-12T01:15:19.270 INFO:teuthology.orchestra.run.smithi104.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T01:15:19.270 INFO:teuthology.orchestra.run.smithi104.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T01:15:19.270 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T01:15:19.271 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T01:15:19.271 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T01:15:19.271 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T01:15:19.271 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T01:15:19.271 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T01:15:19.271 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T01:15:19.271 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T01:15:19.271 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T01:15:19.271 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T01:15:19.271 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T01:15:19.271 INFO:teuthology.orchestra.run.smithi104.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T01:15:19.271 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T01:15:19.271 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T01:15:19.272 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T01:15:19.272 INFO:teuthology.orchestra.run.smithi104.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T01:15:19.272 INFO:teuthology.orchestra.run.smithi104.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T01:15:19.272 INFO:teuthology.orchestra.run.smithi104.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T01:15:19.272 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T01:15:19.272 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T01:15:19.272 INFO:teuthology.orchestra.run.smithi104.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T01:15:19.272 INFO:teuthology.orchestra.run.smithi104.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T01:15:19.272 INFO:teuthology.orchestra.run.smithi104.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T01:15:19.272 INFO:teuthology.orchestra.run.smithi104.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T01:15:19.272 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T01:15:19.272 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T01:15:19.273 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T01:15:19.273 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T01:15:19.273 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T01:15:19.273 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T01:15:19.273 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T01:15:19.273 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T01:15:19.273 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T01:15:19.273 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T01:15:19.273 INFO:teuthology.orchestra.run.smithi104.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T01:15:19.273 INFO:teuthology.orchestra.run.smithi104.stdout:Upgrading: 2024-09-12T01:15:19.273 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T01:15:19.273 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T01:15:19.273 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T01:15:19.274 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T01:15:19.274 INFO:teuthology.orchestra.run.smithi104.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T01:15:19.274 INFO:teuthology.orchestra.run.smithi104.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T01:15:19.274 INFO:teuthology.orchestra.run.smithi104.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T01:15:19.274 INFO:teuthology.orchestra.run.smithi104.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T01:15:19.274 INFO:teuthology.orchestra.run.smithi104.stdout:Installing dependencies: 2024-09-12T01:15:19.274 INFO:teuthology.orchestra.run.smithi104.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T01:15:19.274 INFO:teuthology.orchestra.run.smithi104.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T01:15:19.274 INFO:teuthology.orchestra.run.smithi104.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T01:15:19.274 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T01:15:19.274 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T01:15:19.274 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T01:15:19.274 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T01:15:19.274 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T01:15:19.275 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T01:15:19.275 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T01:15:19.275 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T01:15:19.275 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T01:15:19.275 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T01:15:19.275 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T01:15:19.275 INFO:teuthology.orchestra.run.smithi104.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T01:15:19.275 INFO:teuthology.orchestra.run.smithi104.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-12T01:15:19.275 INFO:teuthology.orchestra.run.smithi104.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T01:15:19.275 INFO:teuthology.orchestra.run.smithi104.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T01:15:19.276 INFO:teuthology.orchestra.run.smithi104.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T01:15:19.276 INFO:teuthology.orchestra.run.smithi104.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T01:15:19.276 INFO:teuthology.orchestra.run.smithi104.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T01:15:19.276 INFO:teuthology.orchestra.run.smithi104.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T01:15:19.276 INFO:teuthology.orchestra.run.smithi104.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T01:15:19.276 INFO:teuthology.orchestra.run.smithi104.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T01:15:19.276 INFO:teuthology.orchestra.run.smithi104.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T01:15:19.276 INFO:teuthology.orchestra.run.smithi104.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T01:15:19.276 INFO:teuthology.orchestra.run.smithi104.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T01:15:19.276 INFO:teuthology.orchestra.run.smithi104.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T01:15:19.276 INFO:teuthology.orchestra.run.smithi104.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T01:15:19.276 INFO:teuthology.orchestra.run.smithi104.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T01:15:19.276 INFO:teuthology.orchestra.run.smithi104.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T01:15:19.277 INFO:teuthology.orchestra.run.smithi104.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T01:15:19.277 INFO:teuthology.orchestra.run.smithi104.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T01:15:19.277 INFO:teuthology.orchestra.run.smithi104.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T01:15:19.277 INFO:teuthology.orchestra.run.smithi104.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T01:15:19.277 INFO:teuthology.orchestra.run.smithi104.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T01:15:19.277 INFO:teuthology.orchestra.run.smithi104.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T01:15:19.277 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T01:15:19.277 INFO:teuthology.orchestra.run.smithi104.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T01:15:19.277 INFO:teuthology.orchestra.run.smithi104.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T01:15:19.277 INFO:teuthology.orchestra.run.smithi104.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T01:15:19.277 INFO:teuthology.orchestra.run.smithi104.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T01:15:19.277 INFO:teuthology.orchestra.run.smithi104.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T01:15:19.277 INFO:teuthology.orchestra.run.smithi104.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T01:15:19.278 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T01:15:19.278 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T01:15:19.278 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T01:15:19.278 INFO:teuthology.orchestra.run.smithi104.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T01:15:19.278 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T01:15:19.278 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T01:15:19.278 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T01:15:19.278 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T01:15:19.278 INFO:teuthology.orchestra.run.smithi104.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T01:15:19.278 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T01:15:19.278 INFO:teuthology.orchestra.run.smithi104.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T01:15:19.278 INFO:teuthology.orchestra.run.smithi104.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T01:15:19.278 INFO:teuthology.orchestra.run.smithi104.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T01:15:19.279 INFO:teuthology.orchestra.run.smithi104.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T01:15:19.279 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T01:15:19.279 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T01:15:19.279 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T01:15:19.279 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T01:15:19.279 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T01:15:19.279 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T01:15:19.279 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T01:15:19.279 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T01:15:19.279 INFO:teuthology.orchestra.run.smithi104.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T01:15:19.279 INFO:teuthology.orchestra.run.smithi104.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T01:15:19.279 INFO:teuthology.orchestra.run.smithi104.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T01:15:19.279 INFO:teuthology.orchestra.run.smithi104.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T01:15:19.279 INFO:teuthology.orchestra.run.smithi104.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T01:15:19.280 INFO:teuthology.orchestra.run.smithi104.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T01:15:19.280 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T01:15:19.280 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T01:15:19.280 INFO:teuthology.orchestra.run.smithi104.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T01:15:19.280 INFO:teuthology.orchestra.run.smithi104.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T01:15:19.280 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T01:15:19.280 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T01:15:19.280 INFO:teuthology.orchestra.run.smithi104.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T01:15:19.280 INFO:teuthology.orchestra.run.smithi104.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T01:15:19.280 INFO:teuthology.orchestra.run.smithi104.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T01:15:19.280 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T01:15:19.280 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T01:15:19.280 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T01:15:19.281 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T01:15:19.281 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T01:15:19.281 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T01:15:19.281 INFO:teuthology.orchestra.run.smithi104.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T01:15:19.281 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T01:15:19.281 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T01:15:19.281 INFO:teuthology.orchestra.run.smithi104.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T01:15:19.281 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T01:15:19.281 INFO:teuthology.orchestra.run.smithi104.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T01:15:19.281 INFO:teuthology.orchestra.run.smithi104.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T01:15:19.281 INFO:teuthology.orchestra.run.smithi104.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T01:15:19.281 INFO:teuthology.orchestra.run.smithi104.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T01:15:19.281 INFO:teuthology.orchestra.run.smithi104.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T01:15:19.281 INFO:teuthology.orchestra.run.smithi104.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T01:15:19.282 INFO:teuthology.orchestra.run.smithi104.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T01:15:19.282 INFO:teuthology.orchestra.run.smithi104.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T01:15:19.282 INFO:teuthology.orchestra.run.smithi104.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T01:15:19.282 INFO:teuthology.orchestra.run.smithi104.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T01:15:19.282 INFO:teuthology.orchestra.run.smithi104.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T01:15:19.282 INFO:teuthology.orchestra.run.smithi104.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T01:15:19.282 INFO:teuthology.orchestra.run.smithi104.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T01:15:19.282 INFO:teuthology.orchestra.run.smithi104.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T01:15:19.282 INFO:teuthology.orchestra.run.smithi104.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T01:15:19.282 INFO:teuthology.orchestra.run.smithi104.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T01:15:19.282 INFO:teuthology.orchestra.run.smithi104.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T01:15:19.282 INFO:teuthology.orchestra.run.smithi104.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T01:15:19.282 INFO:teuthology.orchestra.run.smithi104.stdout:Installing weak dependencies: 2024-09-12T01:15:19.282 INFO:teuthology.orchestra.run.smithi104.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T01:15:19.283 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-12T01:15:19.283 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-09-12T01:15:19.283 INFO:teuthology.orchestra.run.smithi104.stdout:====================================================================================== 2024-09-12T01:15:19.283 INFO:teuthology.orchestra.run.smithi104.stdout:Install 148 Packages 2024-09-12T01:15:19.283 INFO:teuthology.orchestra.run.smithi104.stdout:Upgrade 8 Packages 2024-09-12T01:15:19.283 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-12T01:15:19.283 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 226 M 2024-09-12T01:15:19.283 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-09-12T01:15:19.548 INFO:teuthology.orchestra.run.smithi053.stdout:(39/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 19 MB/s | 49 MB 00:02 2024-09-12T01:15:19.665 INFO:teuthology.orchestra.run.smithi053.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 139 kB/s | 80 kB 00:00 2024-09-12T01:15:19.690 INFO:teuthology.orchestra.run.smithi104.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 55 kB/s | 6.5 kB 00:00 2024-09-12T01:15:19.715 INFO:teuthology.orchestra.run.smithi053.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 191 kB 00:00 2024-09-12T01:15:19.774 INFO:teuthology.orchestra.run.smithi053.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 253 kB 00:00 2024-09-12T01:15:19.824 INFO:teuthology.orchestra.run.smithi053.stdout:(43/156): libgfortran-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 801 kB 00:00 2024-09-12T01:15:19.849 INFO:teuthology.orchestra.run.smithi053.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-09-12T01:15:19.883 INFO:teuthology.orchestra.run.smithi053.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-09-12T01:15:19.908 INFO:teuthology.orchestra.run.smithi053.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-12T01:15:19.924 INFO:teuthology.orchestra.run.smithi104.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 3.8 MB/s | 897 kB 00:00 2024-09-12T01:15:19.933 INFO:teuthology.orchestra.run.smithi053.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-12T01:15:19.967 INFO:teuthology.orchestra.run.smithi053.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 4.9 MB/s | 1.2 MB 00:00 2024-09-12T01:15:19.982 INFO:teuthology.orchestra.run.smithi104.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 2.4 MB/s | 145 kB 00:00 2024-09-12T01:15:20.074 INFO:teuthology.orchestra.run.smithi104.stdout:(4/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 11 MB/s | 5.4 MB 00:00 2024-09-12T01:15:20.084 INFO:teuthology.orchestra.run.smithi053.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 3.8 MB/s | 689 kB 00:00 2024-09-12T01:15:20.117 INFO:teuthology.orchestra.run.smithi053.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 5.0 MB/s | 943 kB 00:00 2024-09-12T01:15:20.143 INFO:teuthology.orchestra.run.smithi053.stdout:(51/156): boost-program-options-1.75.0-8.el9.x8 609 kB/s | 107 kB 00:00 2024-09-12T01:15:20.168 INFO:teuthology.orchestra.run.smithi053.stdout:(52/156): elfutils-libelf-devel-0.191-4.el9.x86 260 kB/s | 22 kB 00:00 2024-09-12T01:15:20.191 INFO:teuthology.orchestra.run.smithi104.stdout:(5/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 13 MB/s | 1.5 MB 00:00 2024-09-12T01:15:20.193 INFO:teuthology.orchestra.run.smithi053.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 4.1 MB/s | 313 kB 00:00 2024-09-12T01:15:20.218 INFO:teuthology.orchestra.run.smithi053.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 433 kB/s | 33 kB 00:00 2024-09-12T01:15:20.261 INFO:teuthology.orchestra.run.smithi053.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 272 kB/s | 18 kB 00:00 2024-09-12T01:15:20.266 INFO:teuthology.orchestra.run.smithi104.stdout:(6/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 8.4 MB/s | 2.4 MB 00:00 2024-09-12T01:15:20.303 INFO:teuthology.orchestra.run.smithi053.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 132 kB/s | 11 kB 00:00 2024-09-12T01:15:20.360 INFO:teuthology.orchestra.run.smithi053.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 16 MB/s | 3.0 MB 00:00 2024-09-12T01:15:20.385 INFO:teuthology.orchestra.run.smithi053.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 549 kB/s | 45 kB 00:00 2024-09-12T01:15:20.411 INFO:teuthology.orchestra.run.smithi053.stdout:(59/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-09-12T01:15:20.444 INFO:teuthology.orchestra.run.smithi053.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 662 kB 00:00 2024-09-12T01:15:20.478 INFO:teuthology.orchestra.run.smithi053.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 366 kB/s | 24 kB 00:00 2024-09-12T01:15:20.500 INFO:teuthology.orchestra.run.smithi104.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 15 MB/s | 4.7 MB 00:00 2024-09-12T01:15:20.520 INFO:teuthology.orchestra.run.smithi053.stdout:(62/156): libtool-2.4.6-46.el9.x86_64.rpm 4.2 MB/s | 581 kB 00:00 2024-09-12T01:15:20.545 INFO:teuthology.orchestra.run.smithi053.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 472 kB/s | 47 kB 00:00 2024-09-12T01:15:20.570 INFO:teuthology.orchestra.run.smithi053.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-09-12T01:15:20.596 INFO:teuthology.orchestra.run.smithi053.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-09-12T01:15:20.638 INFO:teuthology.orchestra.run.smithi053.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 417 kB/s | 38 kB 00:00 2024-09-12T01:15:20.746 INFO:teuthology.orchestra.run.smithi053.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 28 MB/s | 4.9 MB 00:00 2024-09-12T01:15:20.770 INFO:teuthology.orchestra.run.smithi104.stdout:(8/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 18 MB/s | 21 MB 00:01 2024-09-12T01:15:20.809 INFO:teuthology.orchestra.run.smithi104.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 607 kB/s | 25 kB 00:00 2024-09-12T01:15:20.843 INFO:teuthology.orchestra.run.smithi053.stdout:(68/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-09-12T01:15:20.889 INFO:teuthology.orchestra.run.smithi053.stdout:(69/156): postgresql-13.16-1.el9.x86_64.rpm 6.3 MB/s | 1.6 MB 00:00 2024-09-12T01:15:20.914 INFO:teuthology.orchestra.run.smithi053.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 1.9 MB/s | 137 kB 00:00 2024-09-12T01:15:20.939 INFO:teuthology.orchestra.run.smithi053.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 4.6 MB/s | 910 kB 00:00 2024-09-12T01:15:20.998 INFO:teuthology.orchestra.run.smithi053.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-09-12T01:15:21.023 INFO:teuthology.orchestra.run.smithi053.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-09-12T01:15:21.098 INFO:teuthology.orchestra.run.smithi053.stdout:(74/156): postgresql-server-13.16-1.el9.x86_64. 28 MB/s | 5.8 MB 00:00 2024-09-12T01:15:21.101 INFO:teuthology.orchestra.run.smithi104.stdout:(10/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 18 MB/s | 11 MB 00:00 2024-09-12T01:15:21.123 INFO:teuthology.orchestra.run.smithi053.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 611 kB/s | 61 kB 00:00 2024-09-12T01:15:21.176 INFO:teuthology.orchestra.run.smithi104.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 3.0 MB/s | 228 kB 00:00 2024-09-12T01:15:21.232 INFO:teuthology.orchestra.run.smithi053.stdout:(76/156): python3-babel-2.9.1-2.el9.noarch.rpm 26 MB/s | 6.0 MB 00:00 2024-09-12T01:15:21.251 INFO:teuthology.orchestra.run.smithi104.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 360 kB/s | 27 kB 00:00 2024-09-12T01:15:21.257 INFO:teuthology.orchestra.run.smithi053.stdout:(77/156): python3-devel-3.9.19-8.el9.x86_64.rpm 1.5 MB/s | 245 kB 00:00 2024-09-12T01:15:21.282 INFO:teuthology.orchestra.run.smithi053.stdout:(78/156): python3-jinja2-2.11.3-6.el9.noarch.rp 1.5 MB/s | 248 kB 00:00 2024-09-12T01:15:21.299 INFO:teuthology.orchestra.run.smithi053.stdout:(79/156): python3-jsonpatch-1.21-16.el9.noarch. 383 kB/s | 26 kB 00:00 2024-09-12T01:15:21.333 INFO:teuthology.orchestra.run.smithi053.stdout:(80/156): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-09-12T01:15:21.343 INFO:teuthology.orchestra.run.smithi104.stdout:(13/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 347 kB/s | 32 kB 00:00 2024-09-12T01:15:21.358 INFO:teuthology.orchestra.run.smithi053.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-09-12T01:15:21.392 INFO:teuthology.orchestra.run.smithi053.stdout:(82/156): python3-markupsafe-1.1.1-12.el9.x86_6 377 kB/s | 35 kB 00:00 2024-09-12T01:15:21.434 INFO:teuthology.orchestra.run.smithi053.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-09-12T01:15:21.501 INFO:teuthology.orchestra.run.smithi053.stdout:(84/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 30 MB/s | 5.1 MB 00:00 2024-09-12T01:15:21.510 INFO:teuthology.orchestra.run.smithi104.stdout:(14/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 14 MB/s | 17 MB 00:01 2024-09-12T01:15:21.517 INFO:teuthology.orchestra.run.smithi053.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 924 kB/s | 77 kB 00:00 2024-09-12T01:15:21.543 INFO:teuthology.orchestra.run.smithi053.stdout:(86/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 222 kB 00:00 2024-09-12T01:15:21.544 INFO:teuthology.orchestra.run.smithi104.stdout:(15/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 3.7 MB/s | 752 kB 00:00 2024-09-12T01:15:21.569 INFO:teuthology.orchestra.run.smithi104.stdout:(16/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 2.7 MB/s | 163 kB 00:00 2024-09-12T01:15:21.576 INFO:teuthology.orchestra.run.smithi053.stdout:(87/156): python3-prettytable-0.7.2-27.el9.noar 560 kB/s | 42 kB 00:00 2024-09-12T01:15:21.595 INFO:teuthology.orchestra.run.smithi104.stdout:(17/156): librados-devel-19.1.1-298.gb53b6d3e.e 2.5 MB/s | 127 kB 00:00 2024-09-12T01:15:21.601 INFO:teuthology.orchestra.run.smithi053.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 269 kB 00:00 2024-09-12T01:15:21.620 INFO:teuthology.orchestra.run.smithi104.stdout:(18/156): libradosstriper1-19.1.1-298.gb53b6d3e 9.8 MB/s | 506 kB 00:00 2024-09-12T01:15:21.627 INFO:teuthology.orchestra.run.smithi053.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2024-09-12T01:15:21.652 INFO:teuthology.orchestra.run.smithi053.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-09-12T01:15:21.677 INFO:teuthology.orchestra.run.smithi053.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 676 kB/s | 51 kB 00:00 2024-09-12T01:15:21.702 INFO:teuthology.orchestra.run.smithi053.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 710 kB/s | 54 kB 00:00 2024-09-12T01:15:21.735 INFO:teuthology.orchestra.run.smithi104.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 392 kB/s | 45 kB 00:00 2024-09-12T01:15:21.736 INFO:teuthology.orchestra.run.smithi053.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 712 kB/s | 42 kB 00:00 2024-09-12T01:15:21.769 INFO:teuthology.orchestra.run.smithi053.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 662 kB/s | 44 kB 00:00 2024-09-12T01:15:21.803 INFO:teuthology.orchestra.run.smithi053.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-09-12T01:15:21.828 INFO:teuthology.orchestra.run.smithi053.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 612 kB/s | 36 kB 00:00 2024-09-12T01:15:21.837 INFO:teuthology.orchestra.run.smithi104.stdout:(20/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 22 MB/s | 5.3 MB 00:00 2024-09-12T01:15:21.863 INFO:teuthology.orchestra.run.smithi104.stdout:(21/156): python3-ceph-common-19.1.1-298.gb53b6 1.0 MB/s | 136 kB 00:00 2024-09-12T01:15:21.870 INFO:teuthology.orchestra.run.smithi053.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 707 kB/s | 47 kB 00:00 2024-09-12T01:15:21.896 INFO:teuthology.orchestra.run.smithi104.stdout:(22/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 2.7 MB/s | 162 kB 00:00 2024-09-12T01:15:21.913 INFO:teuthology.orchestra.run.smithi053.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 3.5 MB/s | 304 kB 00:00 2024-09-12T01:15:21.922 INFO:teuthology.orchestra.run.smithi104.stdout:(23/156): python3-rados-19.1.1-298.gb53b6d3e.el 5.4 MB/s | 321 kB 00:00 2024-09-12T01:15:21.938 INFO:teuthology.orchestra.run.smithi053.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 864 kB/s | 58 kB 00:00 2024-09-12T01:15:21.947 INFO:teuthology.orchestra.run.smithi104.stdout:(24/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 5.9 MB/s | 301 kB 00:00 2024-09-12T01:15:21.972 INFO:teuthology.orchestra.run.smithi104.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.9 MB/s | 100 kB 00:00 2024-09-12T01:15:21.988 INFO:teuthology.orchestra.run.smithi053.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.2 MB/s | 96 kB 00:00 2024-09-12T01:15:21.998 INFO:teuthology.orchestra.run.smithi104.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-12T01:15:22.013 INFO:teuthology.orchestra.run.smithi053.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 846 kB/s | 64 kB 00:00 2024-09-12T01:15:22.047 INFO:teuthology.orchestra.run.smithi104.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 3.3 MB/s | 171 kB 00:00 2024-09-12T01:15:22.063 INFO:teuthology.orchestra.run.smithi053.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 595 kB/s | 45 kB 00:00 2024-09-12T01:15:22.114 INFO:teuthology.orchestra.run.smithi104.stdout:(28/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 22 MB/s | 3.1 MB 00:00 2024-09-12T01:15:22.139 INFO:teuthology.orchestra.run.smithi104.stdout:(29/156): ceph-grafana-dashboards-19.1.1-298.gb 347 kB/s | 32 kB 00:00 2024-09-12T01:15:22.147 INFO:teuthology.orchestra.run.smithi053.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 167 kB/s | 22 kB 00:00 2024-09-12T01:15:22.165 INFO:teuthology.orchestra.run.smithi104.stdout:(30/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 2.9 MB/s | 146 kB 00:00 2024-09-12T01:15:22.197 INFO:teuthology.orchestra.run.smithi053.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-12T01:15:22.231 INFO:teuthology.orchestra.run.smithi053.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 9.1 MB/s | 309 kB 00:00 2024-09-12T01:15:22.256 INFO:teuthology.orchestra.run.smithi053.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 779 kB/s | 19 kB 00:00 2024-09-12T01:15:22.423 INFO:teuthology.orchestra.run.smithi053.stdout:(107/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 21 MB/s | 16 MB 00:00 2024-09-12T01:15:22.473 INFO:teuthology.orchestra.run.smithi053.stdout:(108/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.1 MB/s | 863 kB 00:00 2024-09-12T01:15:22.507 INFO:teuthology.orchestra.run.smithi053.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 312 kB/s | 26 kB 00:00 2024-09-12T01:15:22.523 INFO:teuthology.orchestra.run.smithi104.stdout:(31/156): ceph-mgr-diskprediction-local-19.1.1- 21 MB/s | 7.4 MB 00:00 2024-09-12T01:15:22.532 INFO:teuthology.orchestra.run.smithi053.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 861 kB/s | 49 kB 00:00 2024-09-12T01:15:22.558 INFO:teuthology.orchestra.run.smithi053.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.4 MB/s | 67 kB 00:00 2024-09-12T01:15:22.583 INFO:teuthology.orchestra.run.smithi053.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-09-12T01:15:22.590 INFO:teuthology.orchestra.run.smithi104.stdout:(32/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 7.9 MB/s | 3.6 MB 00:00 2024-09-12T01:15:22.616 INFO:teuthology.orchestra.run.smithi104.stdout:(33/156): ceph-mgr-modules-core-19.1.1-298.gb53 2.7 MB/s | 249 kB 00:00 2024-09-12T01:15:22.625 INFO:teuthology.orchestra.run.smithi053.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-12T01:15:22.641 INFO:teuthology.orchestra.run.smithi104.stdout:(34/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 981 kB/s | 49 kB 00:00 2024-09-12T01:15:22.650 INFO:teuthology.orchestra.run.smithi053.stdout:(114/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 839 kB 00:00 2024-09-12T01:15:22.677 INFO:teuthology.orchestra.run.smithi104.stdout:(35/156): ceph-prometheus-alerts-19.1.1-298.gb5 285 kB/s | 16 kB 00:00 2024-09-12T01:15:22.678 INFO:teuthology.orchestra.run.smithi053.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 869 kB/s | 43 kB 00:00 2024-09-12T01:15:22.699 INFO:teuthology.orchestra.run.smithi104.stdout:(36/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 4.8 MB/s | 285 kB 00:00 2024-09-12T01:15:22.700 INFO:teuthology.orchestra.run.smithi053.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 642 kB/s | 32 kB 00:00 2024-09-12T01:15:22.726 INFO:teuthology.orchestra.run.smithi053.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-12T01:15:22.741 INFO:teuthology.orchestra.run.smithi104.stdout:(37/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 11 MB/s | 765 kB 00:00 2024-09-12T01:15:22.751 INFO:teuthology.orchestra.run.smithi053.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-09-12T01:15:22.766 INFO:teuthology.orchestra.run.smithi104.stdout:(38/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 272 kB/s | 18 kB 00:00 2024-09-12T01:15:22.785 INFO:teuthology.orchestra.run.smithi053.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-12T01:15:22.811 INFO:teuthology.orchestra.run.smithi053.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-12T01:15:22.844 INFO:teuthology.orchestra.run.smithi053.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-12T01:15:22.869 INFO:teuthology.orchestra.run.smithi053.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-12T01:15:22.894 INFO:teuthology.orchestra.run.smithi053.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 431 kB/s | 11 kB 00:00 2024-09-12T01:15:22.961 INFO:teuthology.orchestra.run.smithi053.stdout:(124/156): libarrow-9.0.0-11.el9.x86_64.rpm 6.3 MB/s | 4.4 MB 00:00 2024-09-12T01:15:22.986 INFO:teuthology.orchestra.run.smithi053.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-09-12T01:15:23.012 INFO:teuthology.orchestra.run.smithi053.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 463 kB/s | 23 kB 00:00 2024-09-12T01:15:23.037 INFO:teuthology.orchestra.run.smithi053.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-09-12T01:15:23.062 INFO:teuthology.orchestra.run.smithi053.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-12T01:15:23.096 INFO:teuthology.orchestra.run.smithi053.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.3 MB/s | 46 kB 00:00 2024-09-12T01:15:23.121 INFO:teuthology.orchestra.run.smithi053.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-12T01:15:23.146 INFO:teuthology.orchestra.run.smithi053.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-12T01:15:23.180 INFO:teuthology.orchestra.run.smithi053.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-12T01:15:23.205 INFO:teuthology.orchestra.run.smithi053.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 659 kB/s | 16 kB 00:00 2024-09-12T01:15:23.242 INFO:teuthology.orchestra.run.smithi104.stdout:(39/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 20 MB/s | 49 MB 00:02 2024-09-12T01:15:23.247 INFO:teuthology.orchestra.run.smithi053.stdout:(134/156): python3-kubernetes-26.1.0-2.el9.noar 4.9 MB/s | 1.0 MB 00:00 2024-09-12T01:15:23.267 INFO:teuthology.orchestra.run.smithi053.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.7 MB/s | 28 kB 00:00 2024-09-12T01:15:23.267 INFO:teuthology.orchestra.run.smithi104.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 152 kB/s | 80 kB 00:00 2024-09-12T01:15:23.289 INFO:teuthology.orchestra.run.smithi053.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-09-12T01:15:23.306 INFO:teuthology.orchestra.run.smithi053.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 738 kB/s | 31 kB 00:00 2024-09-12T01:15:23.323 INFO:teuthology.orchestra.run.smithi053.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 3.5 MB/s | 59 kB 00:00 2024-09-12T01:15:23.365 INFO:teuthology.orchestra.run.smithi053.stdout:(139/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.7 MB/s | 2.0 MB 00:00 2024-09-12T01:15:23.367 INFO:teuthology.orchestra.run.smithi104.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 1.3 MB/s | 801 kB 00:00 2024-09-12T01:15:23.382 INFO:teuthology.orchestra.run.smithi053.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 612 kB/s | 36 kB 00:00 2024-09-12T01:15:23.392 INFO:teuthology.orchestra.run.smithi104.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.0 MB/s | 253 kB 00:00 2024-09-12T01:15:23.407 INFO:teuthology.orchestra.run.smithi053.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-09-12T01:15:23.433 INFO:teuthology.orchestra.run.smithi053.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 1.1 MB/s | 74 kB 00:00 2024-09-12T01:15:23.434 INFO:teuthology.orchestra.run.smithi104.stdout:(43/156): libquadmath-11.5.0-2.el9.x86_64.rpm 994 kB/s | 191 kB 00:00 2024-09-12T01:15:23.450 INFO:teuthology.orchestra.run.smithi053.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 2.1 MB/s | 90 kB 00:00 2024-09-12T01:15:23.460 INFO:teuthology.orchestra.run.smithi104.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-09-12T01:15:23.475 INFO:teuthology.orchestra.run.smithi053.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-09-12T01:15:23.485 INFO:teuthology.orchestra.run.smithi104.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-12T01:15:23.500 INFO:teuthology.orchestra.run.smithi053.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-09-12T01:15:23.510 INFO:teuthology.orchestra.run.smithi104.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-12T01:15:23.525 INFO:teuthology.orchestra.run.smithi053.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-12T01:15:23.544 INFO:teuthology.orchestra.run.smithi104.stdout:(47/156): python3-cryptography-36.0.1-4.el9.x86 6.9 MB/s | 1.2 MB 00:00 2024-09-12T01:15:23.550 INFO:teuthology.orchestra.run.smithi053.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-09-12T01:15:23.569 INFO:teuthology.orchestra.run.smithi104.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-09-12T01:15:23.601 INFO:teuthology.orchestra.run.smithi053.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-12T01:15:23.703 INFO:teuthology.orchestra.run.smithi104.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 803 kB/s | 107 kB 00:00 2024-09-12T01:15:23.726 INFO:teuthology.orchestra.run.smithi053.stdout:(149/156): NetworkManager-1.49.90-1.el9.x86_64. 20 MB/s | 2.3 MB 00:00 2024-09-12T01:15:23.728 INFO:teuthology.orchestra.run.smithi104.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 3.1 MB/s | 689 kB 00:00 2024-09-12T01:15:23.753 INFO:teuthology.orchestra.run.smithi104.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 432 kB/s | 22 kB 00:00 2024-09-12T01:15:23.787 INFO:teuthology.orchestra.run.smithi104.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 3.8 MB/s | 943 kB 00:00 2024-09-12T01:15:23.801 INFO:teuthology.orchestra.run.smithi053.stdout:(150/156): librados2-19.1.1-298.gb53b6d3e.el9.x 12 MB/s | 3.4 MB 00:00 2024-09-12T01:15:23.812 INFO:teuthology.orchestra.run.smithi104.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 559 kB/s | 33 kB 00:00 2024-09-12T01:15:23.846 INFO:teuthology.orchestra.run.smithi104.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-12T01:15:23.852 INFO:teuthology.orchestra.run.smithi053.stdout:(151/156): NetworkManager-libnm-1.49.90-1.el9.x 14 MB/s | 1.8 MB 00:00 2024-09-12T01:15:23.871 INFO:teuthology.orchestra.run.smithi104.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 312 kB/s | 18 kB 00:00 2024-09-12T01:15:23.877 INFO:teuthology.orchestra.run.smithi053.stdout:(152/156): NetworkManager-team-1.49.90-1.el9.x8 543 kB/s | 41 kB 00:00 2024-09-12T01:15:23.897 INFO:teuthology.orchestra.run.smithi104.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-12T01:15:23.905 INFO:teuthology.orchestra.run.smithi053.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 4.9 MB/s | 249 kB 00:00 2024-09-12T01:15:23.922 INFO:teuthology.orchestra.run.smithi104.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-12T01:15:23.947 INFO:teuthology.orchestra.run.smithi104.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 907 kB/s | 45 kB 00:00 2024-09-12T01:15:23.969 INFO:teuthology.orchestra.run.smithi053.stdout:(154/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 7.6 MB/s | 3.2 MB 00:00 2024-09-12T01:15:24.005 INFO:teuthology.orchestra.run.smithi104.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 14 MB/s | 3.0 MB 00:00 2024-09-12T01:15:24.011 INFO:teuthology.orchestra.run.smithi053.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 10 MB/s | 1.4 MB 00:00 2024-09-12T01:15:24.031 INFO:teuthology.orchestra.run.smithi104.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 6.0 MB/s | 662 kB 00:00 2024-09-12T01:15:24.056 INFO:teuthology.orchestra.run.smithi104.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 489 kB/s | 24 kB 00:00 2024-09-12T01:15:24.061 INFO:teuthology.orchestra.run.smithi053.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 15 MB/s | 2.4 MB 00:00 2024-09-12T01:15:24.067 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-12T01:15:24.067 INFO:teuthology.orchestra.run.smithi053.stdout:Total 26 MB/s | 226 MB 00:08 2024-09-12T01:15:24.081 INFO:teuthology.orchestra.run.smithi104.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-09-12T01:15:24.115 INFO:teuthology.orchestra.run.smithi104.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-09-12T01:15:24.140 INFO:teuthology.orchestra.run.smithi104.stdout:(64/156): libtool-2.4.6-46.el9.x86_64.rpm 2.9 MB/s | 581 kB 00:00 2024-09-12T01:15:24.166 INFO:teuthology.orchestra.run.smithi104.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 766 kB/s | 38 kB 00:00 2024-09-12T01:15:24.191 INFO:teuthology.orchestra.run.smithi104.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-09-12T01:15:24.274 INFO:teuthology.orchestra.run.smithi104.stdout:(67/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-09-12T01:15:24.318 INFO:teuthology.orchestra.run.smithi104.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-12T01:15:24.350 INFO:teuthology.orchestra.run.smithi104.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 12 MB/s | 910 kB 00:00 2024-09-12T01:15:24.409 INFO:teuthology.orchestra.run.smithi104.stdout:(70/156): openblas-openmp-0.3.26-2.el9.x86_64.r 18 MB/s | 4.9 MB 00:00 2024-09-12T01:15:24.434 INFO:teuthology.orchestra.run.smithi104.stdout:(71/156): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-12T01:15:24.484 INFO:teuthology.orchestra.run.smithi104.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-09-12T01:15:24.507 INFO:teuthology.orchestra.run.smithi001.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-12T01:15:24.534 INFO:teuthology.orchestra.run.smithi104.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-09-12T01:15:24.610 INFO:teuthology.orchestra.run.smithi104.stdout:(74/156): postgresql-server-13.16-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-09-12T01:15:24.635 INFO:teuthology.orchestra.run.smithi104.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 611 kB/s | 61 kB 00:00 2024-09-12T01:15:24.660 INFO:teuthology.orchestra.run.smithi104.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 4.8 MB/s | 245 kB 00:00 2024-09-12T01:15:24.685 INFO:teuthology.orchestra.run.smithi104.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-12T01:15:24.711 INFO:teuthology.orchestra.run.smithi104.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-09-12T01:15:24.736 INFO:teuthology.orchestra.run.smithi104.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-09-12T01:15:24.761 INFO:teuthology.orchestra.run.smithi104.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-09-12T01:15:24.786 INFO:teuthology.orchestra.run.smithi104.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-09-12T01:15:24.887 INFO:teuthology.orchestra.run.smithi104.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-09-12T01:15:24.912 INFO:teuthology.orchestra.run.smithi104.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.0 MB/s | 250 kB 00:00 2024-09-12T01:15:24.946 INFO:teuthology.orchestra.run.smithi104.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-12T01:15:24.971 INFO:teuthology.orchestra.run.smithi104.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-12T01:15:24.996 INFO:teuthology.orchestra.run.smithi104.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 841 kB/s | 42 kB 00:00 2024-09-12T01:15:25.030 INFO:teuthology.orchestra.run.smithi104.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-12T01:15:25.097 INFO:teuthology.orchestra.run.smithi104.stdout:(88/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 5.1 MB 00:00 2024-09-12T01:15:25.122 INFO:teuthology.orchestra.run.smithi104.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-09-12T01:15:25.156 INFO:teuthology.orchestra.run.smithi104.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 870 kB/s | 51 kB 00:00 2024-09-12T01:15:25.181 INFO:teuthology.orchestra.run.smithi104.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 920 kB/s | 54 kB 00:00 2024-09-12T01:15:25.206 INFO:teuthology.orchestra.run.smithi104.stdout:(92/156): python3-pyasn1-modules-0.4.8-6.el9.no 1.5 MB/s | 279 kB 00:00 2024-09-12T01:15:25.231 INFO:teuthology.orchestra.run.smithi104.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 830 kB/s | 42 kB 00:00 2024-09-12T01:15:25.257 INFO:teuthology.orchestra.run.smithi104.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 886 kB/s | 44 kB 00:00 2024-09-12T01:15:25.290 INFO:teuthology.orchestra.run.smithi104.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-12T01:15:25.310 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-12T01:15:25.315 INFO:teuthology.orchestra.run.smithi104.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 615 kB/s | 36 kB 00:00 2024-09-12T01:15:25.341 INFO:teuthology.orchestra.run.smithi104.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-09-12T01:15:25.374 INFO:teuthology.orchestra.run.smithi104.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-09-12T01:15:25.400 INFO:teuthology.orchestra.run.smithi104.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 989 kB/s | 58 kB 00:00 2024-09-12T01:15:25.433 INFO:teuthology.orchestra.run.smithi104.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-09-12T01:15:25.459 INFO:teuthology.orchestra.run.smithi104.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-12T01:15:25.479 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-12T01:15:25.479 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-12T01:15:25.485 INFO:teuthology.orchestra.run.smithi104.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 896 kB/s | 45 kB 00:00 2024-09-12T01:15:25.576 INFO:teuthology.orchestra.run.smithi104.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 191 kB/s | 22 kB 00:00 2024-09-12T01:15:25.643 INFO:teuthology.orchestra.run.smithi104.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 8.0 MB/s | 548 kB 00:00 2024-09-12T01:15:25.677 INFO:teuthology.orchestra.run.smithi104.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 9.1 MB/s | 309 kB 00:00 2024-09-12T01:15:25.702 INFO:teuthology.orchestra.run.smithi104.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 779 kB/s | 19 kB 00:00 2024-09-12T01:15:25.727 INFO:teuthology.orchestra.run.smithi104.stdout:(107/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.5 MB/s | 863 kB 00:00 2024-09-12T01:15:25.886 INFO:teuthology.orchestra.run.smithi104.stdout:(108/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-09-12T01:15:25.919 INFO:teuthology.orchestra.run.smithi104.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 135 kB/s | 26 kB 00:00 2024-09-12T01:15:25.953 INFO:teuthology.orchestra.run.smithi104.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.1 MB/s | 67 kB 00:00 2024-09-12T01:15:25.978 INFO:teuthology.orchestra.run.smithi104.stdout:(111/156): liboath-2.6.7-2.el9.x86_64.rpm 540 kB/s | 49 kB 00:00 2024-09-12T01:15:26.012 INFO:teuthology.orchestra.run.smithi104.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-09-12T01:15:26.070 INFO:teuthology.orchestra.run.smithi104.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 9.2 MB/s | 548 kB 00:00 2024-09-12T01:15:26.095 INFO:teuthology.orchestra.run.smithi104.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-12T01:15:26.121 INFO:teuthology.orchestra.run.smithi104.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-12T01:15:26.146 INFO:teuthology.orchestra.run.smithi104.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 567 kB/s | 14 kB 00:00 2024-09-12T01:15:26.180 INFO:teuthology.orchestra.run.smithi104.stdout:(117/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.1 MB/s | 839 kB 00:00 2024-09-12T01:15:26.213 INFO:teuthology.orchestra.run.smithi104.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-09-12T01:15:26.247 INFO:teuthology.orchestra.run.smithi104.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-09-12T01:15:26.272 INFO:teuthology.orchestra.run.smithi104.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-12T01:15:26.331 INFO:teuthology.orchestra.run.smithi104.stdout:(121/156): libarrow-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 4.4 MB 00:00 2024-09-12T01:15:26.356 INFO:teuthology.orchestra.run.smithi104.stdout:(122/156): python3-google-auth-2.34.0-1.el9.noa 2.0 MB/s | 220 kB 00:00 2024-09-12T01:15:26.381 INFO:teuthology.orchestra.run.smithi104.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 431 kB/s | 11 kB 00:00 2024-09-12T01:15:26.407 INFO:teuthology.orchestra.run.smithi104.stdout:(124/156): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-09-12T01:15:26.423 INFO:teuthology.orchestra.run.smithi104.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 424 kB/s | 18 kB 00:00 2024-09-12T01:15:26.440 INFO:teuthology.orchestra.run.smithi104.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 692 kB/s | 23 kB 00:00 2024-09-12T01:15:26.457 INFO:teuthology.orchestra.run.smithi104.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 581 kB/s | 19 kB 00:00 2024-09-12T01:15:26.474 INFO:teuthology.orchestra.run.smithi104.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 590 kB/s | 20 kB 00:00 2024-09-12T01:15:26.491 INFO:teuthology.orchestra.run.smithi104.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 2.7 MB/s | 46 kB 00:00 2024-09-12T01:15:26.517 INFO:teuthology.orchestra.run.smithi104.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-12T01:15:26.533 INFO:teuthology.orchestra.run.smithi104.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 3.5 MB/s | 58 kB 00:00 2024-09-12T01:15:26.575 INFO:teuthology.orchestra.run.smithi104.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-09-12T01:15:26.601 INFO:teuthology.orchestra.run.smithi104.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-09-12T01:15:26.626 INFO:teuthology.orchestra.run.smithi104.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 325 kB/s | 16 kB 00:00 2024-09-12T01:15:26.651 INFO:teuthology.orchestra.run.smithi104.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-12T01:15:26.677 INFO:teuthology.orchestra.run.smithi104.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-12T01:15:26.702 INFO:teuthology.orchestra.run.smithi104.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-09-12T01:15:26.727 INFO:teuthology.orchestra.run.smithi104.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-12T01:15:26.753 INFO:teuthology.orchestra.run.smithi104.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-12T01:15:26.778 INFO:teuthology.orchestra.run.smithi104.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-09-12T01:15:26.887 INFO:teuthology.orchestra.run.smithi104.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 557 kB/s | 74 kB 00:00 2024-09-12T01:15:26.925 INFO:teuthology.orchestra.run.smithi104.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-12T01:15:26.937 INFO:teuthology.orchestra.run.smithi104.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.4 MB/s | 230 kB 00:00 2024-09-12T01:15:26.979 INFO:teuthology.orchestra.run.smithi104.stdout:(144/156): python3-grpcio-1.46.7-10.el9.x86_64. 2.9 MB/s | 2.0 MB 00:00 2024-09-12T01:15:27.004 INFO:teuthology.orchestra.run.smithi104.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 299 kB/s | 20 kB 00:00 2024-09-12T01:15:27.030 INFO:teuthology.orchestra.run.smithi104.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-12T01:15:27.063 INFO:teuthology.orchestra.run.smithi104.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.8 MB/s | 427 kB 00:00 2024-09-12T01:15:27.113 INFO:teuthology.orchestra.run.smithi104.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-12T01:15:27.145 INFO:teuthology.orchestra.run.smithi001.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T01:15:27.146 INFO:teuthology.orchestra.run.smithi001.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T01:15:27.160 INFO:teuthology.orchestra.run.smithi001.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T01:15:27.161 INFO:teuthology.orchestra.run.smithi001.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T01:15:27.162 INFO:teuthology.orchestra.run.smithi001.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T01:15:27.163 INFO:teuthology.orchestra.run.smithi001.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T01:15:27.165 INFO:teuthology.orchestra.run.smithi001.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T01:15:27.171 INFO:teuthology.orchestra.run.smithi001.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T01:15:27.171 INFO:teuthology.orchestra.run.smithi001.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T01:15:27.242 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2024-09-12T01:15:27.264 INFO:teuthology.orchestra.run.smithi104.stdout:(149/156): librados2-19.1.1-298.gb53b6d3e.el9.x 15 MB/s | 3.4 MB 00:00 2024-09-12T01:15:27.268 INFO:teuthology.orchestra.run.smithi001.stdout:====================================================================================== 2024-09-12T01:15:27.268 INFO:teuthology.orchestra.run.smithi001.stdout: Package Arch Version Repository Size 2024-09-12T01:15:27.268 INFO:teuthology.orchestra.run.smithi001.stdout:====================================================================================== 2024-09-12T01:15:27.268 INFO:teuthology.orchestra.run.smithi001.stdout:Installing: 2024-09-12T01:15:27.268 INFO:teuthology.orchestra.run.smithi001.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T01:15:27.268 INFO:teuthology.orchestra.run.smithi001.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T01:15:27.268 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T01:15:27.268 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T01:15:27.268 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T01:15:27.268 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T01:15:27.269 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T01:15:27.269 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T01:15:27.269 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T01:15:27.269 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T01:15:27.269 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T01:15:27.269 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T01:15:27.269 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T01:15:27.269 INFO:teuthology.orchestra.run.smithi001.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T01:15:27.269 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T01:15:27.269 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T01:15:27.269 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T01:15:27.269 INFO:teuthology.orchestra.run.smithi001.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T01:15:27.270 INFO:teuthology.orchestra.run.smithi001.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T01:15:27.270 INFO:teuthology.orchestra.run.smithi001.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T01:15:27.270 INFO:teuthology.orchestra.run.smithi001.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T01:15:27.270 INFO:teuthology.orchestra.run.smithi001.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T01:15:27.270 INFO:teuthology.orchestra.run.smithi001.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T01:15:27.270 INFO:teuthology.orchestra.run.smithi001.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T01:15:27.270 INFO:teuthology.orchestra.run.smithi001.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T01:15:27.270 INFO:teuthology.orchestra.run.smithi001.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T01:15:27.270 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T01:15:27.270 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T01:15:27.270 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T01:15:27.270 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T01:15:27.270 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T01:15:27.270 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T01:15:27.271 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T01:15:27.271 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T01:15:27.271 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T01:15:27.271 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T01:15:27.271 INFO:teuthology.orchestra.run.smithi001.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T01:15:27.271 INFO:teuthology.orchestra.run.smithi001.stdout:Upgrading: 2024-09-12T01:15:27.271 INFO:teuthology.orchestra.run.smithi001.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T01:15:27.271 INFO:teuthology.orchestra.run.smithi001.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T01:15:27.271 INFO:teuthology.orchestra.run.smithi001.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T01:15:27.271 INFO:teuthology.orchestra.run.smithi001.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T01:15:27.271 INFO:teuthology.orchestra.run.smithi001.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T01:15:27.271 INFO:teuthology.orchestra.run.smithi001.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T01:15:27.271 INFO:teuthology.orchestra.run.smithi001.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T01:15:27.271 INFO:teuthology.orchestra.run.smithi001.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T01:15:27.272 INFO:teuthology.orchestra.run.smithi001.stdout:Installing dependencies: 2024-09-12T01:15:27.272 INFO:teuthology.orchestra.run.smithi001.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T01:15:27.272 INFO:teuthology.orchestra.run.smithi001.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T01:15:27.272 INFO:teuthology.orchestra.run.smithi001.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T01:15:27.272 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T01:15:27.272 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T01:15:27.272 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T01:15:27.272 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T01:15:27.272 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T01:15:27.272 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T01:15:27.272 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T01:15:27.272 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T01:15:27.272 INFO:teuthology.orchestra.run.smithi001.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T01:15:27.273 INFO:teuthology.orchestra.run.smithi001.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T01:15:27.273 INFO:teuthology.orchestra.run.smithi001.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T01:15:27.273 INFO:teuthology.orchestra.run.smithi001.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T01:15:27.273 INFO:teuthology.orchestra.run.smithi001.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-12T01:15:27.273 INFO:teuthology.orchestra.run.smithi001.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T01:15:27.273 INFO:teuthology.orchestra.run.smithi001.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T01:15:27.273 INFO:teuthology.orchestra.run.smithi001.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T01:15:27.273 INFO:teuthology.orchestra.run.smithi001.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T01:15:27.273 INFO:teuthology.orchestra.run.smithi001.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T01:15:27.273 INFO:teuthology.orchestra.run.smithi001.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T01:15:27.273 INFO:teuthology.orchestra.run.smithi001.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T01:15:27.273 INFO:teuthology.orchestra.run.smithi001.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T01:15:27.273 INFO:teuthology.orchestra.run.smithi001.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T01:15:27.273 INFO:teuthology.orchestra.run.smithi001.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T01:15:27.274 INFO:teuthology.orchestra.run.smithi001.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T01:15:27.274 INFO:teuthology.orchestra.run.smithi001.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T01:15:27.274 INFO:teuthology.orchestra.run.smithi001.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T01:15:27.274 INFO:teuthology.orchestra.run.smithi001.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T01:15:27.274 INFO:teuthology.orchestra.run.smithi001.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T01:15:27.274 INFO:teuthology.orchestra.run.smithi001.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T01:15:27.274 INFO:teuthology.orchestra.run.smithi001.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T01:15:27.274 INFO:teuthology.orchestra.run.smithi001.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T01:15:27.274 INFO:teuthology.orchestra.run.smithi001.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T01:15:27.274 INFO:teuthology.orchestra.run.smithi001.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T01:15:27.274 INFO:teuthology.orchestra.run.smithi001.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T01:15:27.274 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T01:15:27.274 INFO:teuthology.orchestra.run.smithi001.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T01:15:27.274 INFO:teuthology.orchestra.run.smithi001.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T01:15:27.274 INFO:teuthology.orchestra.run.smithi001.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T01:15:27.275 INFO:teuthology.orchestra.run.smithi001.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T01:15:27.275 INFO:teuthology.orchestra.run.smithi001.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T01:15:27.275 INFO:teuthology.orchestra.run.smithi001.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T01:15:27.275 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T01:15:27.275 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T01:15:27.275 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T01:15:27.275 INFO:teuthology.orchestra.run.smithi001.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T01:15:27.275 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T01:15:27.275 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T01:15:27.275 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T01:15:27.275 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T01:15:27.275 INFO:teuthology.orchestra.run.smithi001.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T01:15:27.275 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T01:15:27.276 INFO:teuthology.orchestra.run.smithi001.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T01:15:27.276 INFO:teuthology.orchestra.run.smithi001.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T01:15:27.276 INFO:teuthology.orchestra.run.smithi001.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T01:15:27.276 INFO:teuthology.orchestra.run.smithi001.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T01:15:27.276 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T01:15:27.276 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T01:15:27.276 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T01:15:27.276 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T01:15:27.276 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T01:15:27.276 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T01:15:27.276 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T01:15:27.276 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T01:15:27.276 INFO:teuthology.orchestra.run.smithi001.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T01:15:27.276 INFO:teuthology.orchestra.run.smithi001.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T01:15:27.277 INFO:teuthology.orchestra.run.smithi001.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T01:15:27.277 INFO:teuthology.orchestra.run.smithi001.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T01:15:27.277 INFO:teuthology.orchestra.run.smithi001.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T01:15:27.277 INFO:teuthology.orchestra.run.smithi001.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T01:15:27.277 INFO:teuthology.orchestra.run.smithi001.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T01:15:27.277 INFO:teuthology.orchestra.run.smithi001.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T01:15:27.277 INFO:teuthology.orchestra.run.smithi001.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T01:15:27.277 INFO:teuthology.orchestra.run.smithi001.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T01:15:27.277 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T01:15:27.277 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T01:15:27.277 INFO:teuthology.orchestra.run.smithi001.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T01:15:27.277 INFO:teuthology.orchestra.run.smithi001.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T01:15:27.277 INFO:teuthology.orchestra.run.smithi001.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T01:15:27.277 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T01:15:27.278 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T01:15:27.278 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T01:15:27.278 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T01:15:27.278 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T01:15:27.278 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T01:15:27.278 INFO:teuthology.orchestra.run.smithi001.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T01:15:27.278 INFO:teuthology.orchestra.run.smithi001.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T01:15:27.278 INFO:teuthology.orchestra.run.smithi001.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T01:15:27.278 INFO:teuthology.orchestra.run.smithi001.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T01:15:27.278 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T01:15:27.278 INFO:teuthology.orchestra.run.smithi001.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T01:15:27.278 INFO:teuthology.orchestra.run.smithi001.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T01:15:27.278 INFO:teuthology.orchestra.run.smithi001.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T01:15:27.278 INFO:teuthology.orchestra.run.smithi001.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T01:15:27.279 INFO:teuthology.orchestra.run.smithi001.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T01:15:27.279 INFO:teuthology.orchestra.run.smithi001.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T01:15:27.279 INFO:teuthology.orchestra.run.smithi001.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T01:15:27.279 INFO:teuthology.orchestra.run.smithi001.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T01:15:27.279 INFO:teuthology.orchestra.run.smithi001.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T01:15:27.279 INFO:teuthology.orchestra.run.smithi001.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T01:15:27.279 INFO:teuthology.orchestra.run.smithi001.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T01:15:27.279 INFO:teuthology.orchestra.run.smithi001.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T01:15:27.279 INFO:teuthology.orchestra.run.smithi001.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T01:15:27.279 INFO:teuthology.orchestra.run.smithi001.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T01:15:27.279 INFO:teuthology.orchestra.run.smithi001.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T01:15:27.279 INFO:teuthology.orchestra.run.smithi001.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T01:15:27.279 INFO:teuthology.orchestra.run.smithi001.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T01:15:27.279 INFO:teuthology.orchestra.run.smithi001.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T01:15:27.280 INFO:teuthology.orchestra.run.smithi001.stdout:Installing weak dependencies: 2024-09-12T01:15:27.280 INFO:teuthology.orchestra.run.smithi001.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T01:15:27.280 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:15:27.280 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction Summary 2024-09-12T01:15:27.280 INFO:teuthology.orchestra.run.smithi001.stdout:====================================================================================== 2024-09-12T01:15:27.280 INFO:teuthology.orchestra.run.smithi001.stdout:Install 148 Packages 2024-09-12T01:15:27.280 INFO:teuthology.orchestra.run.smithi001.stdout:Upgrade 8 Packages 2024-09-12T01:15:27.280 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:15:27.281 INFO:teuthology.orchestra.run.smithi001.stdout:Total download size: 226 M 2024-09-12T01:15:27.281 INFO:teuthology.orchestra.run.smithi001.stdout:Downloading Packages: 2024-09-12T01:15:27.339 INFO:teuthology.orchestra.run.smithi104.stdout:(150/156): NetworkManager-1.49.90-1.el9.x86_64. 11 MB/s | 2.3 MB 00:00 2024-09-12T01:15:27.381 INFO:teuthology.orchestra.run.smithi104.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 983 kB/s | 41 kB 00:00 2024-09-12T01:15:27.386 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-12T01:15:27.387 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-12T01:15:27.423 INFO:teuthology.orchestra.run.smithi104.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 5.9 MB/s | 249 kB 00:00 2024-09-12T01:15:27.465 INFO:teuthology.orchestra.run.smithi104.stdout:(153/156): NetworkManager-libnm-1.49.90-1.el9.x 9.0 MB/s | 1.8 MB 00:00 2024-09-12T01:15:27.507 INFO:teuthology.orchestra.run.smithi104.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 16 MB/s | 1.4 MB 00:00 2024-09-12T01:15:27.565 INFO:teuthology.orchestra.run.smithi104.stdout:(155/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 6.3 MB/s | 3.2 MB 00:00 2024-09-12T01:15:27.616 INFO:teuthology.orchestra.run.smithi104.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 16 MB/s | 2.4 MB 00:00 2024-09-12T01:15:27.622 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-09-12T01:15:27.622 INFO:teuthology.orchestra.run.smithi104.stdout:Total 27 MB/s | 226 MB 00:08 2024-09-12T01:15:27.942 INFO:teuthology.orchestra.run.smithi001.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 52 kB/s | 6.5 kB 00:00 2024-09-12T01:15:28.134 INFO:teuthology.orchestra.run.smithi001.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 4.6 MB/s | 897 kB 00:00 2024-09-12T01:15:28.184 INFO:teuthology.orchestra.run.smithi001.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 2.8 MB/s | 145 kB 00:00 2024-09-12T01:15:28.293 INFO:teuthology.orchestra.run.smithi001.stdout:(4/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 11 MB/s | 5.4 MB 00:00 2024-09-12T01:15:28.385 INFO:teuthology.orchestra.run.smithi001.stdout:(5/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 17 MB/s | 1.5 MB 00:00 2024-09-12T01:15:28.458 INFO:teuthology.orchestra.run.smithi001.stdout:(6/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 8.7 MB/s | 2.4 MB 00:00 2024-09-12T01:15:28.610 INFO:teuthology.orchestra.run.smithi001.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 21 MB/s | 4.7 MB 00:00 2024-09-12T01:15:28.858 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-09-12T01:15:29.008 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-09-12T01:15:29.009 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-09-12T01:15:29.120 INFO:teuthology.orchestra.run.smithi001.stdout:(8/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 16 MB/s | 21 MB 00:01 2024-09-12T01:15:29.194 INFO:teuthology.orchestra.run.smithi001.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 339 kB/s | 25 kB 00:00 2024-09-12T01:15:29.361 INFO:teuthology.orchestra.run.smithi001.stdout:(10/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 14 MB/s | 11 MB 00:00 2024-09-12T01:15:29.420 INFO:teuthology.orchestra.run.smithi001.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 3.8 MB/s | 228 kB 00:00 2024-09-12T01:15:29.654 INFO:teuthology.orchestra.run.smithi001.stdout:(12/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 14 MB/s | 17 MB 00:01 2024-09-12T01:15:29.695 INFO:teuthology.orchestra.run.smithi001.stdout:(13/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 98 kB/s | 27 kB 00:00 2024-09-12T01:15:29.737 INFO:teuthology.orchestra.run.smithi001.stdout:(14/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 380 kB/s | 32 kB 00:00 2024-09-12T01:15:29.805 INFO:teuthology.orchestra.run.smithi001.stdout:(15/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 6.8 MB/s | 752 kB 00:00 2024-09-12T01:15:29.830 INFO:teuthology.orchestra.run.smithi001.stdout:(16/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 1.7 MB/s | 163 kB 00:00 2024-09-12T01:15:29.855 INFO:teuthology.orchestra.run.smithi001.stdout:(17/156): librados-devel-19.1.1-298.gb53b6d3e.e 2.5 MB/s | 127 kB 00:00 2024-09-12T01:15:29.905 INFO:teuthology.orchestra.run.smithi001.stdout:(18/156): libradosstriper1-19.1.1-298.gb53b6d3e 6.6 MB/s | 506 kB 00:00 2024-09-12T01:15:29.997 INFO:teuthology.orchestra.run.smithi001.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 492 kB/s | 45 kB 00:00 2024-09-12T01:15:30.072 INFO:teuthology.orchestra.run.smithi001.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 1.8 MB/s | 136 kB 00:00 2024-09-12T01:15:30.189 INFO:teuthology.orchestra.run.smithi001.stdout:(21/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 16 MB/s | 5.3 MB 00:00 2024-09-12T01:15:30.215 INFO:teuthology.orchestra.run.smithi001.stdout:(22/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 1.1 MB/s | 162 kB 00:00 2024-09-12T01:15:30.248 INFO:teuthology.orchestra.run.smithi001.stdout:(23/156): python3-rados-19.1.1-298.gb53b6d3e.el 5.4 MB/s | 321 kB 00:00 2024-09-12T01:15:30.274 INFO:teuthology.orchestra.run.smithi001.stdout:(24/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 5.0 MB/s | 301 kB 00:00 2024-09-12T01:15:30.299 INFO:teuthology.orchestra.run.smithi001.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.9 MB/s | 100 kB 00:00 2024-09-12T01:15:30.324 INFO:teuthology.orchestra.run.smithi001.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-12T01:15:30.375 INFO:teuthology.orchestra.run.smithi001.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 3.3 MB/s | 171 kB 00:00 2024-09-12T01:15:30.483 INFO:teuthology.orchestra.run.smithi001.stdout:(28/156): ceph-grafana-dashboards-19.1.1-298.gb 295 kB/s | 32 kB 00:00 2024-09-12T01:15:30.542 INFO:teuthology.orchestra.run.smithi001.stdout:(29/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 13 MB/s | 3.1 MB 00:00 2024-09-12T01:15:30.567 INFO:teuthology.orchestra.run.smithi001.stdout:(30/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 1.7 MB/s | 146 kB 00:00 2024-09-12T01:15:30.775 INFO:teuthology.orchestra.run.smithi001.stdout:(31/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 15 MB/s | 3.6 MB 00:00 2024-09-12T01:15:30.828 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-09-12T01:15:30.828 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-09-12T01:15:30.852 INFO:teuthology.orchestra.run.smithi001.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 3.2 MB/s | 249 kB 00:00 2024-09-12T01:15:30.935 INFO:teuthology.orchestra.run.smithi001.stdout:(33/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 592 kB/s | 49 kB 00:00 2024-09-12T01:15:31.010 INFO:teuthology.orchestra.run.smithi001.stdout:(34/156): ceph-prometheus-alerts-19.1.1-298.gb5 220 kB/s | 16 kB 00:00 2024-09-12T01:15:31.135 INFO:teuthology.orchestra.run.smithi001.stdout:(35/156): ceph-mgr-diskprediction-local-19.1.1- 13 MB/s | 7.4 MB 00:00 2024-09-12T01:15:31.160 INFO:teuthology.orchestra.run.smithi001.stdout:(36/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 1.9 MB/s | 285 kB 00:00 2024-09-12T01:15:31.202 INFO:teuthology.orchestra.run.smithi001.stdout:(37/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 11 MB/s | 765 kB 00:00 2024-09-12T01:15:31.245 INFO:teuthology.orchestra.run.smithi001.stdout:(38/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 218 kB/s | 18 kB 00:00 2024-09-12T01:15:31.829 INFO:teuthology.orchestra.run.smithi001.stdout:(39/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 18 MB/s | 49 MB 00:02 2024-09-12T01:15:32.470 INFO:teuthology.orchestra.run.smithi001.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 63 kB/s | 80 kB 00:01 2024-09-12T01:15:32.554 INFO:teuthology.orchestra.run.smithi001.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 264 kB/s | 191 kB 00:00 2024-09-12T01:15:32.680 INFO:teuthology.orchestra.run.smithi001.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 559 kB/s | 801 kB 00:01 2024-09-12T01:15:33.305 INFO:teuthology.orchestra.run.smithi001.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 304 kB/s | 253 kB 00:00 2024-09-12T01:15:33.355 INFO:teuthology.orchestra.run.smithi001.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 158 kB/s | 106 kB 00:00 2024-09-12T01:15:33.589 INFO:teuthology.orchestra.run.smithi001.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 1.2 MB/s | 1.2 MB 00:01 2024-09-12T01:15:33.717 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-12T01:15:33.849 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T01:15:33.988 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T01:15:34.014 INFO:teuthology.orchestra.run.smithi001.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 191 kB/s | 135 kB 00:00 2024-09-12T01:15:34.122 INFO:teuthology.orchestra.run.smithi001.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 163 kB/s | 125 kB 00:00 2024-09-12T01:15:34.134 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T01:15:34.175 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T01:15:34.199 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T01:15:34.253 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T01:15:34.439 INFO:teuthology.orchestra.run.smithi001.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 253 kB/s | 215 kB 00:00 2024-09-12T01:15:34.599 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T01:15:34.732 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T01:15:34.765 INFO:teuthology.orchestra.run.smithi001.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 918 kB/s | 689 kB 00:00 2024-09-12T01:15:34.898 INFO:teuthology.orchestra.run.smithi001.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 163 kB/s | 22 kB 00:00 2024-09-12T01:15:34.900 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T01:15:34.923 INFO:teuthology.orchestra.run.smithi001.stdout:(51/156): boost-program-options-1.75.0-8.el9.x8 221 kB/s | 107 kB 00:00 2024-09-12T01:15:34.934 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T01:15:35.015 INFO:teuthology.orchestra.run.smithi001.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-12T01:15:35.027 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T01:15:35.055 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T01:15:35.057 INFO:teuthology.orchestra.run.smithi001.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 245 kB/s | 33 kB 00:00 2024-09-12T01:15:35.108 INFO:teuthology.orchestra.run.smithi001.stdout:(54/156): bison-3.7.4-5.el9.x86_64.rpm 957 kB/s | 943 kB 00:00 2024-09-12T01:15:35.183 INFO:teuthology.orchestra.run.smithi001.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 145 kB/s | 18 kB 00:00 2024-09-12T01:15:35.212 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T01:15:35.233 INFO:teuthology.orchestra.run.smithi001.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 89 kB/s | 11 kB 00:00 2024-09-12T01:15:35.261 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T01:15:35.289 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T01:15:35.367 INFO:teuthology.orchestra.run.smithi001.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 340 kB/s | 45 kB 00:00 2024-09-12T01:15:35.392 INFO:teuthology.orchestra.run.smithi001.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 767 kB/s | 160 kB 00:00 2024-09-12T01:15:35.394 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T01:15:35.423 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T01:15:35.485 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T01:15:35.507 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T01:15:35.533 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T01:15:35.576 INFO:teuthology.orchestra.run.smithi001.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.1 MB/s | 662 kB 00:00 2024-09-12T01:15:35.684 INFO:teuthology.orchestra.run.smithi001.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 1.9 MB/s | 581 kB 00:00 2024-09-12T01:15:35.709 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T01:15:35.726 INFO:teuthology.orchestra.run.smithi001.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 163 kB/s | 24 kB 00:00 2024-09-12T01:15:35.802 INFO:teuthology.orchestra.run.smithi001.stdout:(62/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.8 MB/s | 3.0 MB 00:00 2024-09-12T01:15:35.844 INFO:teuthology.orchestra.run.smithi001.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 298 kB/s | 47 kB 00:00 2024-09-12T01:15:35.869 INFO:teuthology.orchestra.run.smithi001.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.0 MB/s | 292 kB 00:00 2024-09-12T01:15:35.890 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T01:15:35.969 INFO:teuthology.orchestra.run.smithi001.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 308 kB/s | 38 kB 00:00 2024-09-12T01:15:36.015 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T01:15:36.106 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T01:15:36.119 INFO:teuthology.orchestra.run.smithi001.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 594 kB/s | 188 kB 00:00 2024-09-12T01:15:36.173 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T01:15:36.398 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T01:15:36.509 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T01:15:36.603 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T01:15:36.637 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T01:15:36.672 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T01:15:36.747 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T01:15:36.798 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T01:15:36.838 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T01:15:36.882 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T01:15:36.914 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T01:15:36.950 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T01:15:37.022 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T01:15:37.073 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T01:15:37.102 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T01:15:37.138 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T01:15:37.211 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-09-12T01:15:37.217 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T01:15:37.249 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T01:15:37.293 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T01:15:37.294 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T01:15:37.321 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T01:15:37.332 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T01:15:37.370 INFO:teuthology.orchestra.run.smithi001.stdout:(67/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.2 MB/s | 4.4 MB 00:01 2024-09-12T01:15:37.394 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T01:15:37.402 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T01:15:37.436 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T01:15:37.439 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T01:15:37.463 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T01:15:37.504 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T01:15:37.525 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T01:15:37.672 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T01:15:37.731 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T01:15:37.827 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T01:15:37.946 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T01:15:38.076 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T01:15:38.105 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T01:15:38.195 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T01:15:38.225 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T01:15:38.281 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T01:15:38.365 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T01:15:38.375 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T01:15:38.399 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T01:15:38.431 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T01:15:38.441 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T01:15:38.445 INFO:teuthology.orchestra.run.smithi001.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 698 kB/s | 1.6 MB 00:02 2024-09-12T01:15:38.475 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T01:15:38.479 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T01:15:38.522 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T01:15:38.578 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T01:15:38.601 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T01:15:38.664 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T01:15:38.686 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T01:15:38.698 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T01:15:38.711 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T01:15:38.745 INFO:teuthology.orchestra.run.smithi001.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 662 kB/s | 910 kB 00:01 2024-09-12T01:15:38.749 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T01:15:38.770 INFO:teuthology.orchestra.run.smithi001.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 423 kB/s | 137 kB 00:00 2024-09-12T01:15:38.805 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T01:15:38.880 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T01:15:38.896 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T01:15:38.917 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T01:15:38.953 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T01:15:39.001 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T01:15:39.022 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T01:15:39.035 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T01:15:39.079 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T01:15:39.112 INFO:teuthology.orchestra.run.smithi001.stdout:(71/156): openblas-openmp-0.3.26-2.el9.x86_64.r 1.5 MB/s | 4.9 MB 00:03 2024-09-12T01:15:39.122 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T01:15:39.142 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T01:15:39.244 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T01:15:39.263 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T01:15:39.313 INFO:teuthology.orchestra.run.smithi001.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 438 kB/s | 87 kB 00:00 2024-09-12T01:15:39.326 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T01:15:39.405 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T01:15:39.447 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T01:15:39.546 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T01:15:39.554 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T01:15:39.588 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T01:15:39.628 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T01:15:39.653 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-12T01:15:39.656 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T01:15:39.677 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T01:15:39.743 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T01:15:39.756 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T01:15:39.782 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T01:15:39.816 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T01:15:39.884 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T01:15:39.892 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T01:15:39.935 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T01:15:39.937 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T01:15:39.963 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T01:15:39.969 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T01:15:39.997 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T01:15:40.012 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T01:15:40.036 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T01:15:40.044 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T01:15:40.073 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T01:15:40.077 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T01:15:40.106 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T01:15:40.141 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T01:15:40.142 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T01:15:40.174 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T01:15:40.195 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T01:15:40.216 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T01:15:40.226 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T01:15:40.250 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T01:15:40.278 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T01:15:40.313 INFO:teuthology.orchestra.run.smithi001.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 670 kB/s | 1.0 MB 00:01 2024-09-12T01:15:40.332 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T01:15:40.355 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T01:15:40.381 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T01:15:40.397 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T01:15:40.411 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T01:15:40.441 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T01:15:40.455 INFO:teuthology.orchestra.run.smithi001.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 432 kB/s | 61 kB 00:00 2024-09-12T01:15:40.469 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T01:15:40.519 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T01:15:40.548 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T01:15:40.572 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T01:15:40.593 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T01:15:40.667 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T01:15:40.803 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T01:15:40.870 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T01:15:41.172 INFO:teuthology.orchestra.run.smithi001.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 342 kB/s | 245 kB 00:00 2024-09-12T01:15:41.305 INFO:teuthology.orchestra.run.smithi001.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 1.8 MB/s | 248 kB 00:00 2024-09-12T01:15:41.334 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T01:15:41.384 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T01:15:41.418 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T01:15:41.419 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T01:15:41.447 INFO:teuthology.orchestra.run.smithi001.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 181 kB/s | 26 kB 00:00 2024-09-12T01:15:41.498 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T01:15:41.520 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T01:15:41.527 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T01:15:41.573 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T01:15:41.578 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T01:15:41.589 INFO:teuthology.orchestra.run.smithi001.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 135 kB/s | 19 kB 00:00 2024-09-12T01:15:41.618 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T01:15:41.624 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T01:15:41.674 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T01:15:41.708 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T01:15:41.753 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T01:15:41.823 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T01:15:41.873 INFO:teuthology.orchestra.run.smithi001.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 608 kB/s | 172 kB 00:00 2024-09-12T01:15:41.885 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T01:15:41.937 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T01:15:41.939 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T01:15:41.967 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T01:15:42.007 INFO:teuthology.orchestra.run.smithi001.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 261 kB/s | 35 kB 00:00 2024-09-12T01:15:42.012 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T01:15:42.044 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T01:15:42.048 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T01:15:42.080 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T01:15:42.135 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T01:15:42.185 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T01:15:42.230 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T01:15:42.273 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T01:15:42.406 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T01:15:42.449 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T01:15:42.491 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T01:15:42.594 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T01:15:42.633 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T01:15:42.731 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T01:15:42.764 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T01:15:42.845 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T01:15:42.941 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-12T01:15:42.965 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T01:15:43.042 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T01:15:43.165 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T01:15:43.197 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T01:15:43.232 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T01:15:43.275 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T01:15:43.314 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T01:15:43.346 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T01:15:43.376 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T01:15:43.410 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T01:15:43.443 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T01:15:43.485 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T01:15:43.520 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T01:15:43.605 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T01:15:43.658 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T01:15:43.698 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T01:15:43.800 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T01:15:43.848 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T01:15:44.151 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T01:15:44.230 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T01:15:44.259 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T01:15:44.291 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T01:15:44.564 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T01:15:44.612 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T01:15:44.621 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T01:15:44.646 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T01:15:44.654 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T01:15:44.729 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T01:15:44.753 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T01:15:44.759 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T01:15:44.765 INFO:teuthology.orchestra.run.smithi001.stdout:(81/156): postgresql-server-13.16-1.el9.x86_64. 982 kB/s | 5.8 MB 00:06 2024-09-12T01:15:44.795 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T01:15:44.809 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T01:15:44.854 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T01:15:44.943 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T01:15:44.982 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T01:15:45.140 INFO:teuthology.orchestra.run.smithi001.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 669 kB/s | 250 kB 00:00 2024-09-12T01:15:45.166 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T01:15:45.197 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T01:15:45.274 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T01:15:45.302 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T01:15:45.332 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T01:15:45.403 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T01:15:45.442 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T01:15:45.524 INFO:teuthology.orchestra.run.smithi001.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 579 kB/s | 222 kB 00:00 2024-09-12T01:15:45.608 INFO:teuthology.orchestra.run.smithi001.stdout:(84/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 1.4 MB/s | 5.1 MB 00:03 2024-09-12T01:15:45.741 INFO:teuthology.orchestra.run.smithi001.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 318 kB/s | 42 kB 00:00 2024-09-12T01:15:45.767 INFO:teuthology.orchestra.run.smithi001.stdout:(86/156): python3-packaging-20.9-5.el9.noarch.r 318 kB/s | 77 kB 00:00 2024-09-12T01:15:45.908 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T01:15:46.019 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T01:15:46.059 INFO:teuthology.orchestra.run.smithi001.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 545 kB/s | 159 kB 00:00 2024-09-12T01:15:46.201 INFO:teuthology.orchestra.run.smithi001.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 587 kB/s | 269 kB 00:00 2024-09-12T01:15:46.359 INFO:teuthology.orchestra.run.smithi001.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 322 kB/s | 51 kB 00:00 2024-09-12T01:15:46.493 INFO:teuthology.orchestra.run.smithi001.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 403 kB/s | 54 kB 00:00 2024-09-12T01:15:46.560 INFO:teuthology.orchestra.run.smithi001.stdout:(91/156): python3-pyasn1-modules-0.4.8-6.el9.no 557 kB/s | 279 kB 00:00 2024-09-12T01:15:46.719 INFO:teuthology.orchestra.run.smithi001.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 263 kB/s | 42 kB 00:00 2024-09-12T01:15:46.886 INFO:teuthology.orchestra.run.smithi001.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 265 kB/s | 44 kB 00:00 2024-09-12T01:15:47.227 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T01:15:47.269 INFO:teuthology.orchestra.run.smithi001.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 576 kB/s | 221 kB 00:00 2024-09-12T01:15:47.318 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T01:15:47.346 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T01:15:47.379 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T01:15:47.403 INFO:teuthology.orchestra.run.smithi001.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 269 kB/s | 36 kB 00:00 2024-09-12T01:15:47.528 INFO:teuthology.orchestra.run.smithi001.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 379 kB/s | 47 kB 00:00 2024-09-12T01:15:47.688 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T01:15:47.725 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T01:15:47.807 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T01:15:47.824 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T01:15:47.855 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T01:15:47.891 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T01:15:47.921 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T01:15:47.987 INFO:teuthology.orchestra.run.smithi001.stdout:(97/156): socat-1.7.4.1-6.el9.x86_64.rpm 663 kB/s | 304 kB 00:00 2024-09-12T01:15:48.138 INFO:teuthology.orchestra.run.smithi001.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 386 kB/s | 58 kB 00:00 2024-09-12T01:15:48.296 INFO:teuthology.orchestra.run.smithi001.stdout:(99/156): python3-babel-2.9.1-2.el9.noarch.rpm 680 kB/s | 6.0 MB 00:08 2024-09-12T01:15:48.369 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T01:15:48.379 INFO:teuthology.orchestra.run.smithi001.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 396 kB/s | 96 kB 00:00 2024-09-12T01:15:48.496 INFO:teuthology.orchestra.run.smithi001.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 384 kB/s | 45 kB 00:00 2024-09-12T01:15:48.499 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T01:15:48.536 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T01:15:48.538 INFO:teuthology.orchestra.run.smithi001.stdout:(102/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 263 kB/s | 64 kB 00:00 2024-09-12T01:15:48.541 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T01:15:48.608 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T01:15:48.622 INFO:teuthology.orchestra.run.smithi001.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 179 kB/s | 22 kB 00:00 2024-09-12T01:15:48.631 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T01:15:48.631 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T01:15:48.632 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T01:15:48.632 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-12T01:15:48.638 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T01:15:48.659 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T01:15:48.714 INFO:teuthology.orchestra.run.smithi001.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 5.9 MB/s | 548 kB 00:00 2024-09-12T01:15:48.747 INFO:teuthology.orchestra.run.smithi001.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 9.1 MB/s | 309 kB 00:00 2024-09-12T01:15:48.790 INFO:teuthology.orchestra.run.smithi001.stdout:(106/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.4 MB/s | 863 kB 00:00 2024-09-12T01:15:48.831 INFO:teuthology.orchestra.run.smithi001.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 232 kB/s | 19 kB 00:00 2024-09-12T01:15:48.899 INFO:teuthology.orchestra.run.smithi001.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 390 kB/s | 26 kB 00:00 2024-09-12T01:15:48.940 INFO:teuthology.orchestra.run.smithi001.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.1 MB/s | 49 kB 00:00 2024-09-12T01:15:48.966 INFO:teuthology.orchestra.run.smithi001.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-12T01:15:49.013 INFO:teuthology.orchestra.run.smithi001.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 3.2 MB/s | 151 kB 00:00 2024-09-12T01:15:49.066 INFO:teuthology.orchestra.run.smithi001.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 16 MB/s | 839 kB 00:00 2024-09-12T01:15:49.116 INFO:teuthology.orchestra.run.smithi001.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-09-12T01:15:49.158 INFO:teuthology.orchestra.run.smithi001.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-09-12T01:15:49.201 INFO:teuthology.orchestra.run.smithi001.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 771 kB/s | 32 kB 00:00 2024-09-12T01:15:49.242 INFO:teuthology.orchestra.run.smithi001.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 342 kB/s | 14 kB 00:00 2024-09-12T01:15:49.276 INFO:teuthology.orchestra.run.smithi001.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 5.0 MB/s | 172 kB 00:00 2024-09-12T01:15:49.310 INFO:teuthology.orchestra.run.smithi001.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 11 MB/s | 358 kB 00:00 2024-09-12T01:15:49.360 INFO:teuthology.orchestra.run.smithi001.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 6.0 MB/s | 303 kB 00:00 2024-09-12T01:15:49.394 INFO:teuthology.orchestra.run.smithi001.stdout:(120/156): python3-google-auth-2.34.0-1.el9.noa 6.5 MB/s | 220 kB 00:00 2024-09-12T01:15:49.469 INFO:teuthology.orchestra.run.smithi001.stdout:(121/156): libarrow-9.0.0-11.el9.x86_64.rpm 6.5 MB/s | 4.4 MB 00:00 2024-09-12T01:15:49.494 INFO:teuthology.orchestra.run.smithi001.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-12T01:15:49.545 INFO:teuthology.orchestra.run.smithi001.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 214 kB/s | 11 kB 00:00 2024-09-12T01:15:49.595 INFO:teuthology.orchestra.run.smithi001.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 356 kB/s | 18 kB 00:00 2024-09-12T01:15:49.645 INFO:teuthology.orchestra.run.smithi001.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 466 kB/s | 23 kB 00:00 2024-09-12T01:15:49.720 INFO:teuthology.orchestra.run.smithi001.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 259 kB/s | 19 kB 00:00 2024-09-12T01:15:49.770 INFO:teuthology.orchestra.run.smithi001.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 395 kB/s | 20 kB 00:00 2024-09-12T01:15:49.829 INFO:teuthology.orchestra.run.smithi001.stdout:(128/156): python3-kubernetes-26.1.0-2.el9.noar 18 MB/s | 1.0 MB 00:00 2024-09-12T01:15:49.879 INFO:teuthology.orchestra.run.smithi001.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 928 kB/s | 46 kB 00:00 2024-09-12T01:15:49.930 INFO:teuthology.orchestra.run.smithi001.stdout:(130/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.8 MB/s | 2.0 MB 00:00 2024-09-12T01:15:49.955 INFO:teuthology.orchestra.run.smithi001.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-09-12T01:15:50.019 INFO:teuthology.orchestra.run.smithi001.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 652 kB/s | 58 kB 00:00 2024-09-12T01:15:50.072 INFO:teuthology.orchestra.run.smithi001.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 308 kB/s | 16 kB 00:00 2024-09-12T01:15:50.097 INFO:teuthology.orchestra.run.smithi001.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.6 MB/s | 90 kB 00:00 2024-09-12T01:15:50.131 INFO:teuthology.orchestra.run.smithi001.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 1.5 MB/s | 272 kB 00:00 2024-09-12T01:15:50.173 INFO:teuthology.orchestra.run.smithi001.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 374 kB/s | 28 kB 00:00 2024-09-12T01:15:50.357 INFO:teuthology.orchestra.run.smithi001.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 137 kB/s | 31 kB 00:00 2024-09-12T01:15:50.399 INFO:teuthology.orchestra.run.smithi001.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-09-12T01:15:50.441 INFO:teuthology.orchestra.run.smithi001.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 863 kB/s | 36 kB 00:00 2024-09-12T01:15:50.466 INFO:teuthology.orchestra.run.smithi001.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 644 kB/s | 188 kB 00:00 2024-09-12T01:15:50.492 INFO:teuthology.orchestra.run.smithi001.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-12T01:15:50.517 INFO:teuthology.orchestra.run.smithi001.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.6 MB/s | 90 kB 00:00 2024-09-12T01:15:50.542 INFO:teuthology.orchestra.run.smithi001.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-12T01:15:50.584 INFO:teuthology.orchestra.run.smithi001.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 482 kB/s | 20 kB 00:00 2024-09-12T01:15:50.618 INFO:teuthology.orchestra.run.smithi001.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 4.1 MB/s | 427 kB 00:00 2024-09-12T01:15:50.643 INFO:teuthology.orchestra.run.smithi001.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-12T01:15:50.735 INFO:teuthology.orchestra.run.smithi001.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-12T01:15:50.872 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T01:15:50.985 INFO:teuthology.orchestra.run.smithi001.stdout:(148/156): librados2-19.1.1-298.gb53b6d3e.el9.x 10 MB/s | 3.4 MB 00:00 2024-09-12T01:15:51.067 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T01:15:51.085 INFO:teuthology.orchestra.run.smithi001.stdout:(149/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 9.0 MB/s | 3.2 MB 00:00 2024-09-12T01:15:51.617 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T01:15:51.662 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T01:15:51.745 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T01:15:51.746 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T01:15:51.746 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T01:15:51.746 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-12T01:15:51.789 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T01:15:52.136 INFO:teuthology.orchestra.run.smithi001.stdout:(150/156): NetworkManager-libnm-1.49.90-1.el9.x 1.7 MB/s | 1.8 MB 00:01 2024-09-12T01:15:52.186 INFO:teuthology.orchestra.run.smithi001.stdout:(151/156): NetworkManager-1.49.90-1.el9.x86_64. 1.9 MB/s | 2.3 MB 00:01 2024-09-12T01:15:52.778 INFO:teuthology.orchestra.run.smithi001.stdout:(152/156): NetworkManager-team-1.49.90-1.el9.x8 63 kB/s | 41 kB 00:00 2024-09-12T01:15:52.937 INFO:teuthology.orchestra.run.smithi001.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 332 kB/s | 249 kB 00:00 2024-09-12T01:15:53.812 INFO:teuthology.orchestra.run.smithi001.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 1.3 MB/s | 1.4 MB 00:01 2024-09-12T01:15:54.037 INFO:teuthology.orchestra.run.smithi001.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.2 MB/s | 2.4 MB 00:01 2024-09-12T01:16:00.977 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T01:16:00.977 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /sys 2024-09-12T01:16:00.977 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /proc 2024-09-12T01:16:00.977 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /mnt 2024-09-12T01:16:00.977 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /var/tmp 2024-09-12T01:16:00.977 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /home 2024-09-12T01:16:00.977 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /root 2024-09-12T01:16:00.978 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /tmp 2024-09-12T01:16:00.978 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-12T01:16:01.153 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T01:16:01.169 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T01:16:01.169 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:16:01.169 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T01:16:01.169 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-12T01:16:02.705 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T01:16:02.732 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T01:16:02.732 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:16:02.732 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T01:16:02.732 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T01:16:02.732 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T01:16:02.732 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-12T01:16:02.973 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T01:16:02.996 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T01:16:02.996 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:16:02.996 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T01:16:02.996 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T01:16:02.996 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T01:16:02.996 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-12T01:16:03.465 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T01:16:03.473 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T01:16:03.473 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /sys 2024-09-12T01:16:03.473 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /proc 2024-09-12T01:16:03.473 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /mnt 2024-09-12T01:16:03.473 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /var/tmp 2024-09-12T01:16:03.473 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /home 2024-09-12T01:16:03.473 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /root 2024-09-12T01:16:03.473 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /tmp 2024-09-12T01:16:03.473 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-12T01:16:03.488 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T01:16:03.488 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:16:03.488 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T01:16:03.488 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T01:16:03.489 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T01:16:03.489 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-12T01:16:03.598 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T01:16:03.641 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T01:16:03.655 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T01:16:03.655 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T01:16:03.656 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:16:03.656 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T01:16:03.656 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-12T01:16:03.717 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T01:16:03.920 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T01:16:03.966 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T01:16:04.047 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T01:16:04.136 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T01:16:04.295 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T01:16:04.341 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T01:16:04.372 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T01:16:04.400 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T01:16:04.792 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T01:16:04.815 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T01:16:04.838 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T01:16:04.903 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T01:16:04.929 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T01:16:05.141 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T01:16:05.169 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T01:16:05.169 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:16:05.169 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T01:16:05.169 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T01:16:05.169 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T01:16:05.169 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-12T01:16:05.184 INFO:teuthology.orchestra.run.smithi001.stdout:(156/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 890 kB/s | 16 MB 00:18 2024-09-12T01:16:05.189 INFO:teuthology.orchestra.run.smithi001.stdout:-------------------------------------------------------------------------------- 2024-09-12T01:16:05.189 INFO:teuthology.orchestra.run.smithi001.stdout:Total 6.0 MB/s | 226 MB 00:37 2024-09-12T01:16:05.401 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T01:16:05.425 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T01:16:05.425 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:16:05.425 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T01:16:05.425 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T01:16:05.426 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T01:16:05.426 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-12T01:16:05.869 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T01:16:05.894 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T01:16:05.894 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:16:05.894 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T01:16:05.894 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T01:16:05.894 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T01:16:05.894 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-12T01:16:05.999 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T01:16:06.035 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T01:16:06.064 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T01:16:06.081 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T01:16:06.120 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T01:16:06.305 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T01:16:06.364 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T01:16:06.439 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T01:16:06.537 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T01:16:06.677 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T01:16:06.737 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T01:16:06.776 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T01:16:06.804 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T01:16:07.099 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T01:16:07.132 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T01:16:07.239 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T01:16:07.255 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T01:16:07.286 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T01:16:07.317 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T01:16:07.320 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction check 2024-09-12T01:16:07.380 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T01:16:07.409 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T01:16:07.417 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T01:16:07.434 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T01:16:07.458 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T01:16:07.458 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:16:07.458 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T01:16:07.458 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T01:16:07.458 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T01:16:07.458 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-12T01:16:07.509 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T01:16:07.518 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction check succeeded. 2024-09-12T01:16:07.518 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction test 2024-09-12T01:16:07.521 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T01:16:07.623 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T01:16:07.667 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T01:16:08.454 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T01:16:08.502 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T01:16:08.643 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T01:16:08.678 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T01:16:08.702 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T01:16:08.702 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:16:08.702 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T01:16:08.702 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T01:16:08.702 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T01:16:08.703 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-12T01:16:08.745 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T01:16:08.768 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T01:16:08.769 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:16:08.769 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T01:16:08.769 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-12T01:16:08.821 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T01:16:08.844 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T01:16:08.844 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:16:08.844 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T01:16:08.844 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T01:16:08.844 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T01:16:08.844 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-12T01:16:09.163 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T01:16:09.187 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T01:16:09.187 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:16:09.187 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T01:16:09.187 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T01:16:09.187 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T01:16:09.187 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-12T01:16:09.463 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T01:16:09.494 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T01:16:09.596 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T01:16:09.755 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T01:16:09.788 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T01:16:09.812 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T01:16:09.812 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:16:09.812 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T01:16:09.812 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T01:16:09.812 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T01:16:09.812 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-12T01:16:09.864 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T01:16:09.876 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T01:16:09.971 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction test succeeded. 2024-09-12T01:16:09.972 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction 2024-09-12T01:16:09.972 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T01:16:10.028 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T01:16:10.958 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T01:16:10.999 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T01:16:11.023 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T01:16:11.023 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:16:11.023 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T01:16:11.023 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T01:16:11.024 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T01:16:11.024 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-12T01:16:11.066 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T01:16:11.090 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T01:16:11.090 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:16:11.090 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T01:16:11.090 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-12T01:16:11.142 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T01:16:11.169 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T01:16:11.169 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:16:11.169 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T01:16:11.169 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T01:16:11.169 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T01:16:11.170 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-12T01:16:11.476 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T01:16:11.500 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T01:16:11.500 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:16:11.501 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T01:16:11.501 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T01:16:11.501 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T01:16:11.501 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-12T01:16:14.009 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T01:16:14.051 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T01:16:14.092 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T01:16:14.123 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T01:16:14.150 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T01:16:14.344 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T01:16:14.396 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T01:16:14.427 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T01:16:14.457 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T01:16:14.502 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T01:16:15.130 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T01:16:15.166 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T01:16:15.188 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T01:16:16.029 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T01:16:16.070 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T01:16:16.093 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T01:16:16.120 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T01:16:16.169 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T01:16:16.195 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T01:16:16.204 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T01:16:16.246 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T01:16:16.333 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T01:16:16.365 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T01:16:16.377 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T01:16:16.390 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T01:16:16.416 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T01:16:16.416 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T01:16:16.425 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T01:16:16.456 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T01:16:16.485 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T01:16:16.500 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T01:16:16.500 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T01:16:16.531 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T01:16:16.568 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T01:16:16.593 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T01:16:16.598 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T01:16:16.598 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T01:16:16.778 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T01:16:16.778 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T01:16:16.862 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T01:16:16.928 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T01:16:16.985 INFO:teuthology.orchestra.run.smithi001.stdout: Preparing : 1/1 2024-09-12T01:16:17.088 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T01:16:17.129 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T01:16:17.151 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T01:16:17.155 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T01:16:17.217 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T01:16:17.285 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T01:16:17.329 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T01:16:17.359 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T01:16:17.427 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T01:16:17.858 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T01:16:17.968 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T01:16:17.987 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T01:16:18.075 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T01:16:18.116 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T01:16:18.116 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T01:16:18.160 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T01:16:18.197 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T01:16:18.228 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T01:16:18.253 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T01:16:18.280 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T01:16:18.280 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T01:16:18.308 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T01:16:18.342 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T01:16:18.372 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T01:16:18.372 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T01:16:18.464 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T01:16:18.489 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T01:16:18.495 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T01:16:18.495 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T01:16:18.578 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T01:16:18.647 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T01:16:18.673 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T01:16:18.674 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T01:16:18.674 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T01:16:18.750 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T01:16:18.802 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T01:16:18.814 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T01:16:18.833 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T01:16:18.920 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T01:16:18.950 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T01:16:18.993 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T01:16:19.244 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T01:16:19.437 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T01:16:19.624 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T01:16:19.744 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T01:16:19.875 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T01:16:20.153 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T01:16:20.316 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T01:16:20.431 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T01:16:20.484 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T01:16:20.527 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T01:16:20.630 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T01:16:20.697 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T01:16:20.743 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T01:16:20.798 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T01:16:20.836 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T01:16:20.887 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T01:16:20.979 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T01:16:21.048 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T01:16:21.081 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T01:16:21.132 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T01:16:21.229 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T01:16:21.268 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T01:16:21.313 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T01:16:21.349 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T01:16:21.443 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T01:16:21.494 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T01:16:21.589 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T01:16:21.760 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T01:16:21.825 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T01:16:22.498 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T01:16:22.543 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T01:16:22.543 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T01:16:22.543 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T01:16:22.543 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T01:16:22.543 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T01:16:22.543 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T01:16:22.543 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T01:16:22.543 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T01:16:22.543 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T01:16:22.543 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T01:16:22.544 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T01:16:22.544 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T01:16:22.544 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T01:16:22.544 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T01:16:22.544 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T01:16:22.544 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T01:16:22.544 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T01:16:22.544 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T01:16:22.544 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T01:16:22.544 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T01:16:22.544 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T01:16:22.544 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T01:16:22.545 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T01:16:22.545 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T01:16:22.545 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T01:16:22.545 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T01:16:22.545 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T01:16:22.545 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T01:16:22.557 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T01:16:22.557 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T01:16:22.557 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T01:16:22.557 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T01:16:22.557 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T01:16:22.557 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T01:16:22.557 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T01:16:22.557 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T01:16:22.558 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T01:16:22.558 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T01:16:22.558 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T01:16:22.558 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T01:16:22.558 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T01:16:22.558 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T01:16:22.558 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T01:16:22.558 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T01:16:22.558 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T01:16:22.558 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T01:16:22.558 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T01:16:22.558 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T01:16:22.559 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T01:16:22.559 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T01:16:22.559 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T01:16:22.559 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T01:16:22.559 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T01:16:22.559 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T01:16:22.559 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T01:16:22.559 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T01:16:22.559 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T01:16:22.559 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T01:16:22.559 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T01:16:22.559 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T01:16:22.560 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T01:16:22.560 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T01:16:22.560 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T01:16:22.560 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T01:16:22.560 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T01:16:22.560 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T01:16:22.560 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T01:16:22.560 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T01:16:22.560 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T01:16:22.560 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T01:16:22.560 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T01:16:22.560 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T01:16:22.560 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T01:16:22.561 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T01:16:22.561 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T01:16:22.561 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T01:16:22.561 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T01:16:22.561 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T01:16:22.561 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T01:16:22.561 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T01:16:22.561 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T01:16:22.561 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T01:16:22.561 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T01:16:22.561 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T01:16:22.561 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T01:16:22.561 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T01:16:22.562 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T01:16:22.562 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T01:16:22.562 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T01:16:22.562 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T01:16:22.562 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T01:16:22.562 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T01:16:22.562 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T01:16:22.562 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T01:16:22.562 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T01:16:22.562 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T01:16:22.562 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T01:16:22.562 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T01:16:22.562 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T01:16:22.562 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T01:16:22.563 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T01:16:22.563 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T01:16:22.563 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T01:16:22.563 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T01:16:22.563 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T01:16:22.563 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T01:16:22.563 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T01:16:22.563 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-12T01:16:22.563 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T01:16:22.563 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T01:16:22.563 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T01:16:22.563 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T01:16:22.563 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T01:16:22.563 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T01:16:22.563 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T01:16:22.564 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T01:16:22.564 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T01:16:22.564 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T01:16:22.564 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T01:16:22.564 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T01:16:22.564 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T01:16:22.564 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T01:16:22.564 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T01:16:22.564 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T01:16:22.564 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T01:16:22.564 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T01:16:22.564 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T01:16:22.564 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T01:16:22.565 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T01:16:22.565 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T01:16:22.565 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T01:16:22.565 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T01:16:22.565 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T01:16:22.565 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T01:16:22.565 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T01:16:22.565 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T01:16:22.565 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T01:16:22.565 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T01:16:22.565 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T01:16:22.565 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T01:16:22.565 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T01:16:22.566 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T01:16:22.566 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T01:16:22.566 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T01:16:22.566 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T01:16:22.566 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T01:16:22.566 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T01:16:22.566 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T01:16:22.566 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T01:16:22.566 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T01:16:22.566 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T01:16:22.566 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T01:16:22.566 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T01:16:22.566 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T01:16:22.567 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T01:16:22.567 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T01:16:22.567 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T01:16:22.567 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T01:16:22.567 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T01:16:22.567 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T01:16:22.567 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T01:16:22.567 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T01:16:22.567 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T01:16:22.567 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T01:16:22.619 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T01:16:22.661 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T01:16:22.704 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T01:16:22.743 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T01:16:22.796 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T01:16:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T01:16:23.062 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T01:16:23.132 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T01:16:23.219 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T01:16:23.263 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T01:16:23.300 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T01:16:23.359 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T01:16:23.398 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T01:16:23.445 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T01:16:23.495 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T01:16:23.703 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T01:16:23.760 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T01:16:23.805 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T01:16:23.937 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T01:16:23.977 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T01:16:24.020 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T01:16:24.066 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-12T01:16:24.091 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T01:16:24.197 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T01:16:24.326 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T01:16:24.326 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T01:16:24.326 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T01:16:24.326 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T01:16:24.326 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T01:16:24.327 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T01:16:24.327 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T01:16:24.327 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T01:16:24.327 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T01:16:24.327 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T01:16:24.327 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T01:16:24.327 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T01:16:24.327 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T01:16:24.327 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T01:16:24.327 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T01:16:24.327 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T01:16:24.328 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T01:16:24.328 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T01:16:24.328 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T01:16:24.328 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T01:16:24.328 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T01:16:24.328 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T01:16:24.328 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T01:16:24.328 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T01:16:24.328 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T01:16:24.328 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T01:16:24.328 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T01:16:24.328 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T01:16:24.328 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T01:16:24.329 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T01:16:24.329 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T01:16:24.329 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T01:16:24.329 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T01:16:24.329 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T01:16:24.329 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T01:16:24.329 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T01:16:24.329 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T01:16:24.329 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T01:16:24.329 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T01:16:24.329 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T01:16:24.329 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T01:16:24.330 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T01:16:24.330 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T01:16:24.330 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T01:16:24.330 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T01:16:24.330 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T01:16:24.330 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T01:16:24.330 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T01:16:24.330 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T01:16:24.330 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T01:16:24.330 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T01:16:24.330 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T01:16:24.330 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T01:16:24.331 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T01:16:24.331 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T01:16:24.331 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T01:16:24.331 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T01:16:24.331 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T01:16:24.331 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T01:16:24.331 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T01:16:24.332 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T01:16:24.332 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T01:16:24.332 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T01:16:24.332 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T01:16:24.332 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T01:16:24.332 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T01:16:24.332 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T01:16:24.332 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T01:16:24.332 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T01:16:24.332 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T01:16:24.332 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T01:16:24.332 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T01:16:24.333 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T01:16:24.333 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T01:16:24.333 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T01:16:24.333 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T01:16:24.333 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T01:16:24.333 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T01:16:24.333 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T01:16:24.333 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T01:16:24.333 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T01:16:24.333 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T01:16:24.333 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T01:16:24.333 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T01:16:24.333 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T01:16:24.333 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T01:16:24.334 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T01:16:24.334 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T01:16:24.334 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T01:16:24.334 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T01:16:24.334 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T01:16:24.334 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T01:16:24.334 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T01:16:24.334 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T01:16:24.334 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T01:16:24.334 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T01:16:24.334 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T01:16:24.334 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T01:16:24.334 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T01:16:24.335 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T01:16:24.335 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T01:16:24.335 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T01:16:24.335 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T01:16:24.335 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T01:16:24.335 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T01:16:24.335 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T01:16:24.335 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T01:16:24.335 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-12T01:16:24.335 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T01:16:24.335 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T01:16:24.335 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T01:16:24.336 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T01:16:24.336 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T01:16:24.336 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T01:16:24.336 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T01:16:24.336 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T01:16:24.336 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T01:16:24.336 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T01:16:24.336 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T01:16:24.336 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T01:16:24.336 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T01:16:24.336 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T01:16:24.336 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T01:16:24.336 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T01:16:24.337 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T01:16:24.337 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T01:16:24.337 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T01:16:24.337 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T01:16:24.337 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T01:16:24.337 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T01:16:24.337 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T01:16:24.337 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T01:16:24.337 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T01:16:24.337 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T01:16:24.337 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T01:16:24.337 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T01:16:24.337 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T01:16:24.338 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T01:16:24.338 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T01:16:24.338 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T01:16:24.338 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T01:16:24.338 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T01:16:24.338 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T01:16:24.338 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T01:16:24.338 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T01:16:24.338 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T01:16:24.338 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T01:16:24.338 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T01:16:24.338 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T01:16:24.338 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T01:16:24.338 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T01:16:24.339 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T01:16:24.339 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T01:16:24.339 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T01:16:24.339 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T01:16:24.339 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T01:16:24.339 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T01:16:24.339 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T01:16:24.339 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T01:16:24.339 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T01:16:24.339 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T01:16:24.339 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T01:16:24.339 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T01:16:24.339 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T01:16:24.376 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T01:16:24.424 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T01:16:24.461 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T01:16:24.503 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T01:16:24.576 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T01:16:24.625 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T01:16:24.662 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T01:16:24.718 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T01:16:24.771 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T01:16:24.824 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T01:16:24.856 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T01:16:24.984 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T01:16:25.040 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T01:16:25.076 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T01:16:25.205 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T01:16:25.273 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T01:16:25.321 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T01:16:25.321 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-12T01:16:25.321 INFO:teuthology.orchestra.run.smithi053.stdout:Upgraded: 2024-09-12T01:16:25.322 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T01:16:25.322 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T01:16:25.322 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T01:16:25.322 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T01:16:25.322 INFO:teuthology.orchestra.run.smithi053.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:25.322 INFO:teuthology.orchestra.run.smithi053.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:25.322 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T01:16:25.322 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T01:16:25.322 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-12T01:16:25.322 INFO:teuthology.orchestra.run.smithi053.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T01:16:25.322 INFO:teuthology.orchestra.run.smithi053.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T01:16:25.322 INFO:teuthology.orchestra.run.smithi053.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T01:16:25.322 INFO:teuthology.orchestra.run.smithi053.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T01:16:25.322 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:25.323 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:25.323 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:25.323 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:25.323 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:16:25.323 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:25.323 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:25.323 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:25.323 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:16:25.323 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:16:25.324 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:16:25.324 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:16:25.324 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:16:25.324 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:25.324 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:25.324 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:16:25.324 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:25.324 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:25.324 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:25.324 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:16:25.324 INFO:teuthology.orchestra.run.smithi053.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:16:25.324 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:25.324 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:25.324 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:16:25.325 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T01:16:25.325 INFO:teuthology.orchestra.run.smithi053.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T01:16:25.325 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T01:16:25.325 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T01:16:25.325 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T01:16:25.325 INFO:teuthology.orchestra.run.smithi053.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T01:16:25.325 INFO:teuthology.orchestra.run.smithi053.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-12T01:16:25.325 INFO:teuthology.orchestra.run.smithi053.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T01:16:25.325 INFO:teuthology.orchestra.run.smithi053.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T01:16:25.325 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T01:16:25.325 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T01:16:25.325 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:25.326 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:25.326 INFO:teuthology.orchestra.run.smithi053.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:25.326 INFO:teuthology.orchestra.run.smithi053.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T01:16:25.326 INFO:teuthology.orchestra.run.smithi053.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T01:16:25.326 INFO:teuthology.orchestra.run.smithi053.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T01:16:25.326 INFO:teuthology.orchestra.run.smithi053.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T01:16:25.326 INFO:teuthology.orchestra.run.smithi053.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T01:16:25.326 INFO:teuthology.orchestra.run.smithi053.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:25.326 INFO:teuthology.orchestra.run.smithi053.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:25.326 INFO:teuthology.orchestra.run.smithi053.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T01:16:25.326 INFO:teuthology.orchestra.run.smithi053.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:25.326 INFO:teuthology.orchestra.run.smithi053.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T01:16:25.326 INFO:teuthology.orchestra.run.smithi053.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T01:16:25.326 INFO:teuthology.orchestra.run.smithi053.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T01:16:25.327 INFO:teuthology.orchestra.run.smithi053.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T01:16:25.327 INFO:teuthology.orchestra.run.smithi053.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T01:16:25.327 INFO:teuthology.orchestra.run.smithi053.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T01:16:25.327 INFO:teuthology.orchestra.run.smithi053.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T01:16:25.327 INFO:teuthology.orchestra.run.smithi053.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T01:16:25.327 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T01:16:25.327 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T01:16:25.327 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T01:16:25.327 INFO:teuthology.orchestra.run.smithi053.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T01:16:25.327 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T01:16:25.327 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T01:16:25.327 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T01:16:25.327 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T01:16:25.328 INFO:teuthology.orchestra.run.smithi053.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T01:16:25.328 INFO:teuthology.orchestra.run.smithi053.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T01:16:25.328 INFO:teuthology.orchestra.run.smithi053.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T01:16:25.328 INFO:teuthology.orchestra.run.smithi053.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T01:16:25.328 INFO:teuthology.orchestra.run.smithi053.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T01:16:25.328 INFO:teuthology.orchestra.run.smithi053.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T01:16:25.328 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T01:16:25.328 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:25.328 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:25.328 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:25.328 INFO:teuthology.orchestra.run.smithi053.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T01:16:25.328 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T01:16:25.329 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T01:16:25.329 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T01:16:25.329 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T01:16:25.329 INFO:teuthology.orchestra.run.smithi053.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T01:16:25.329 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T01:16:25.329 INFO:teuthology.orchestra.run.smithi053.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T01:16:25.329 INFO:teuthology.orchestra.run.smithi053.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T01:16:25.329 INFO:teuthology.orchestra.run.smithi053.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T01:16:25.329 INFO:teuthology.orchestra.run.smithi053.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T01:16:25.329 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T01:16:25.329 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T01:16:25.329 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T01:16:25.329 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T01:16:25.330 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T01:16:25.330 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T01:16:25.330 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T01:16:25.330 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T01:16:25.330 INFO:teuthology.orchestra.run.smithi053.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T01:16:25.330 INFO:teuthology.orchestra.run.smithi053.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T01:16:25.330 INFO:teuthology.orchestra.run.smithi053.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T01:16:25.330 INFO:teuthology.orchestra.run.smithi053.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T01:16:25.330 INFO:teuthology.orchestra.run.smithi053.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T01:16:25.330 INFO:teuthology.orchestra.run.smithi053.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T01:16:25.331 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T01:16:25.331 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T01:16:25.331 INFO:teuthology.orchestra.run.smithi053.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T01:16:25.331 INFO:teuthology.orchestra.run.smithi053.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T01:16:25.331 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T01:16:25.331 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T01:16:25.331 INFO:teuthology.orchestra.run.smithi053.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T01:16:25.331 INFO:teuthology.orchestra.run.smithi053.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T01:16:25.331 INFO:teuthology.orchestra.run.smithi053.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T01:16:25.331 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T01:16:25.331 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T01:16:25.331 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T01:16:25.331 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T01:16:25.331 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T01:16:25.332 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T01:16:25.332 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:25.332 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:25.332 INFO:teuthology.orchestra.run.smithi053.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T01:16:25.332 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T01:16:25.332 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T01:16:25.332 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:25.332 INFO:teuthology.orchestra.run.smithi053.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T01:16:25.332 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T01:16:25.332 INFO:teuthology.orchestra.run.smithi053.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T01:16:25.332 INFO:teuthology.orchestra.run.smithi053.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T01:16:25.332 INFO:teuthology.orchestra.run.smithi053.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T01:16:25.332 INFO:teuthology.orchestra.run.smithi053.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T01:16:25.333 INFO:teuthology.orchestra.run.smithi053.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T01:16:25.333 INFO:teuthology.orchestra.run.smithi053.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T01:16:25.333 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T01:16:25.333 INFO:teuthology.orchestra.run.smithi053.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T01:16:25.333 INFO:teuthology.orchestra.run.smithi053.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T01:16:25.333 INFO:teuthology.orchestra.run.smithi053.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T01:16:25.333 INFO:teuthology.orchestra.run.smithi053.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T01:16:25.333 INFO:teuthology.orchestra.run.smithi053.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T01:16:25.333 INFO:teuthology.orchestra.run.smithi053.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T01:16:25.333 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:25.333 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:25.334 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:25.334 INFO:teuthology.orchestra.run.smithi053.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T01:16:25.334 INFO:teuthology.orchestra.run.smithi053.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T01:16:25.334 INFO:teuthology.orchestra.run.smithi053.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T01:16:25.334 INFO:teuthology.orchestra.run.smithi053.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T01:16:25.334 INFO:teuthology.orchestra.run.smithi053.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T01:16:25.334 INFO:teuthology.orchestra.run.smithi053.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T01:16:25.335 INFO:teuthology.orchestra.run.smithi053.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T01:16:25.335 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-12T01:16:25.335 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-12T01:16:25.774 DEBUG:teuthology.parallel:result is None 2024-09-12T01:16:26.225 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T01:16:26.282 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T01:16:26.316 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T01:16:26.476 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T01:16:26.533 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T01:16:26.595 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T01:16:26.713 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T01:16:26.759 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T01:16:27.004 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T01:16:27.004 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-12T01:16:27.004 INFO:teuthology.orchestra.run.smithi104.stdout:Upgraded: 2024-09-12T01:16:27.004 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T01:16:27.005 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T01:16:27.005 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T01:16:27.005 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T01:16:27.005 INFO:teuthology.orchestra.run.smithi104.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:27.005 INFO:teuthology.orchestra.run.smithi104.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:27.005 INFO:teuthology.orchestra.run.smithi104.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T01:16:27.005 INFO:teuthology.orchestra.run.smithi104.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T01:16:27.005 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-09-12T01:16:27.005 INFO:teuthology.orchestra.run.smithi104.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T01:16:27.005 INFO:teuthology.orchestra.run.smithi104.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T01:16:27.005 INFO:teuthology.orchestra.run.smithi104.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T01:16:27.005 INFO:teuthology.orchestra.run.smithi104.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T01:16:27.005 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:27.006 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:27.006 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:27.006 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:27.006 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:16:27.006 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:27.006 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:27.006 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:27.006 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:16:27.006 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:16:27.006 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:16:27.006 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:16:27.006 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:16:27.006 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:27.006 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:27.007 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:16:27.007 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:27.007 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:27.007 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:27.007 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:16:27.007 INFO:teuthology.orchestra.run.smithi104.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:16:27.007 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:27.007 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:27.007 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:16:27.007 INFO:teuthology.orchestra.run.smithi104.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T01:16:27.007 INFO:teuthology.orchestra.run.smithi104.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T01:16:27.007 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T01:16:27.007 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T01:16:27.008 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T01:16:27.008 INFO:teuthology.orchestra.run.smithi104.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T01:16:27.008 INFO:teuthology.orchestra.run.smithi104.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-12T01:16:27.008 INFO:teuthology.orchestra.run.smithi104.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T01:16:27.008 INFO:teuthology.orchestra.run.smithi104.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T01:16:27.008 INFO:teuthology.orchestra.run.smithi104.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T01:16:27.008 INFO:teuthology.orchestra.run.smithi104.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T01:16:27.008 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:27.008 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:27.008 INFO:teuthology.orchestra.run.smithi104.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:27.008 INFO:teuthology.orchestra.run.smithi104.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T01:16:27.009 INFO:teuthology.orchestra.run.smithi104.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T01:16:27.009 INFO:teuthology.orchestra.run.smithi104.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T01:16:27.009 INFO:teuthology.orchestra.run.smithi104.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T01:16:27.009 INFO:teuthology.orchestra.run.smithi104.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T01:16:27.009 INFO:teuthology.orchestra.run.smithi104.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:27.009 INFO:teuthology.orchestra.run.smithi104.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:27.009 INFO:teuthology.orchestra.run.smithi104.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T01:16:27.009 INFO:teuthology.orchestra.run.smithi104.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:27.009 INFO:teuthology.orchestra.run.smithi104.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T01:16:27.009 INFO:teuthology.orchestra.run.smithi104.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T01:16:27.010 INFO:teuthology.orchestra.run.smithi104.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T01:16:27.010 INFO:teuthology.orchestra.run.smithi104.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T01:16:27.010 INFO:teuthology.orchestra.run.smithi104.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T01:16:27.010 INFO:teuthology.orchestra.run.smithi104.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T01:16:27.010 INFO:teuthology.orchestra.run.smithi104.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T01:16:27.010 INFO:teuthology.orchestra.run.smithi104.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T01:16:27.010 INFO:teuthology.orchestra.run.smithi104.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T01:16:27.010 INFO:teuthology.orchestra.run.smithi104.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T01:16:27.010 INFO:teuthology.orchestra.run.smithi104.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T01:16:27.010 INFO:teuthology.orchestra.run.smithi104.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T01:16:27.010 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T01:16:27.010 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T01:16:27.010 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T01:16:27.010 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T01:16:27.010 INFO:teuthology.orchestra.run.smithi104.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T01:16:27.011 INFO:teuthology.orchestra.run.smithi104.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T01:16:27.011 INFO:teuthology.orchestra.run.smithi104.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T01:16:27.011 INFO:teuthology.orchestra.run.smithi104.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T01:16:27.011 INFO:teuthology.orchestra.run.smithi104.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T01:16:27.011 INFO:teuthology.orchestra.run.smithi104.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T01:16:27.011 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T01:16:27.011 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:27.011 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:27.011 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:27.011 INFO:teuthology.orchestra.run.smithi104.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T01:16:27.011 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T01:16:27.011 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T01:16:27.011 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T01:16:27.011 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T01:16:27.012 INFO:teuthology.orchestra.run.smithi104.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T01:16:27.012 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T01:16:27.012 INFO:teuthology.orchestra.run.smithi104.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T01:16:27.012 INFO:teuthology.orchestra.run.smithi104.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T01:16:27.012 INFO:teuthology.orchestra.run.smithi104.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T01:16:27.012 INFO:teuthology.orchestra.run.smithi104.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T01:16:27.012 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T01:16:27.012 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T01:16:27.012 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T01:16:27.012 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T01:16:27.012 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T01:16:27.013 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T01:16:27.013 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T01:16:27.013 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T01:16:27.013 INFO:teuthology.orchestra.run.smithi104.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T01:16:27.013 INFO:teuthology.orchestra.run.smithi104.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T01:16:27.013 INFO:teuthology.orchestra.run.smithi104.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T01:16:27.013 INFO:teuthology.orchestra.run.smithi104.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T01:16:27.013 INFO:teuthology.orchestra.run.smithi104.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T01:16:27.013 INFO:teuthology.orchestra.run.smithi104.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T01:16:27.013 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T01:16:27.013 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T01:16:27.013 INFO:teuthology.orchestra.run.smithi104.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T01:16:27.014 INFO:teuthology.orchestra.run.smithi104.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T01:16:27.014 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T01:16:27.014 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T01:16:27.014 INFO:teuthology.orchestra.run.smithi104.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T01:16:27.014 INFO:teuthology.orchestra.run.smithi104.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T01:16:27.014 INFO:teuthology.orchestra.run.smithi104.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T01:16:27.014 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T01:16:27.014 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T01:16:27.014 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T01:16:27.014 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T01:16:27.014 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T01:16:27.014 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T01:16:27.014 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:27.014 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:27.015 INFO:teuthology.orchestra.run.smithi104.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T01:16:27.015 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T01:16:27.015 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T01:16:27.015 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:27.015 INFO:teuthology.orchestra.run.smithi104.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T01:16:27.015 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T01:16:27.015 INFO:teuthology.orchestra.run.smithi104.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T01:16:27.015 INFO:teuthology.orchestra.run.smithi104.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T01:16:27.015 INFO:teuthology.orchestra.run.smithi104.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T01:16:27.015 INFO:teuthology.orchestra.run.smithi104.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T01:16:27.015 INFO:teuthology.orchestra.run.smithi104.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T01:16:27.015 INFO:teuthology.orchestra.run.smithi104.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T01:16:27.015 INFO:teuthology.orchestra.run.smithi104.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T01:16:27.015 INFO:teuthology.orchestra.run.smithi104.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T01:16:27.015 INFO:teuthology.orchestra.run.smithi104.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T01:16:27.016 INFO:teuthology.orchestra.run.smithi104.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T01:16:27.016 INFO:teuthology.orchestra.run.smithi104.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T01:16:27.016 INFO:teuthology.orchestra.run.smithi104.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T01:16:27.016 INFO:teuthology.orchestra.run.smithi104.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T01:16:27.016 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:27.016 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:27.016 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:16:27.016 INFO:teuthology.orchestra.run.smithi104.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T01:16:27.016 INFO:teuthology.orchestra.run.smithi104.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T01:16:27.016 INFO:teuthology.orchestra.run.smithi104.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T01:16:27.016 INFO:teuthology.orchestra.run.smithi104.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T01:16:27.016 INFO:teuthology.orchestra.run.smithi104.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T01:16:27.016 INFO:teuthology.orchestra.run.smithi104.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T01:16:27.017 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T01:16:27.017 INFO:teuthology.orchestra.run.smithi104.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T01:16:27.017 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-12T01:16:27.017 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-09-12T01:16:27.040 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T01:16:27.138 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T01:16:27.434 DEBUG:teuthology.parallel:result is None 2024-09-12T01:16:28.005 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T01:16:28.158 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T01:16:29.850 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T01:16:29.952 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T01:16:29.989 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T01:16:30.023 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T01:16:30.389 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T01:16:30.435 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T01:16:30.581 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T01:16:30.616 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T01:16:30.682 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T01:16:31.241 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T01:16:31.279 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T01:16:31.354 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T01:16:31.381 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T01:16:34.109 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T01:16:34.145 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T01:16:34.843 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T01:16:34.899 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T01:16:35.026 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T01:16:35.027 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T01:16:35.027 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T01:16:35.027 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:16:35.074 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T01:16:48.382 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T01:16:48.382 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /sys 2024-09-12T01:16:48.382 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /proc 2024-09-12T01:16:48.382 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /mnt 2024-09-12T01:16:48.382 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /var/tmp 2024-09-12T01:16:48.383 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /home 2024-09-12T01:16:48.383 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /root 2024-09-12T01:16:48.383 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /tmp 2024-09-12T01:16:48.383 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:16:48.605 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T01:16:48.639 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T01:16:48.639 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:16:48.639 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T01:16:48.639 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:16:50.306 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T01:16:50.365 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T01:16:50.365 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:16:50.365 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T01:16:50.365 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T01:16:50.365 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T01:16:50.365 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:16:50.631 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T01:16:50.683 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T01:16:50.683 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:16:50.683 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T01:16:50.683 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T01:16:50.683 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T01:16:50.683 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:16:51.191 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T01:16:51.242 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T01:16:51.242 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:16:51.242 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T01:16:51.242 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T01:16:51.243 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T01:16:51.243 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:16:51.375 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T01:16:51.436 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T01:16:51.509 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T01:16:51.747 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T01:16:51.820 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T01:16:51.909 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T01:16:52.022 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T01:16:52.208 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T01:16:52.261 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T01:16:52.309 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T01:16:52.351 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T01:16:52.955 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T01:16:52.984 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T01:16:53.026 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T01:16:53.104 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T01:16:53.139 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T01:16:54.434 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T01:16:54.483 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T01:16:55.640 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T01:16:55.675 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T01:16:55.820 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T01:16:56.007 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T01:16:56.035 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T01:16:56.088 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T01:16:56.089 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:16:56.089 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T01:16:56.089 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T01:16:56.089 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T01:16:56.089 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:16:56.153 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T01:16:56.180 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T01:16:56.290 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T01:16:56.357 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T01:16:57.446 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T01:16:57.489 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T01:16:57.541 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T01:16:57.541 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:16:57.541 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T01:16:57.541 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T01:16:57.541 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T01:16:57.541 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:16:57.589 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T01:16:57.640 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T01:16:57.640 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:16:57.640 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T01:16:57.640 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:16:57.699 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T01:16:57.750 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T01:16:57.750 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:16:57.750 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T01:16:57.750 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T01:16:57.750 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T01:16:57.750 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:16:58.108 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T01:16:58.160 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T01:16:58.160 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:16:58.160 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T01:16:58.160 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T01:16:58.160 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T01:16:58.160 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:17:03.422 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T01:17:03.464 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T01:17:03.508 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T01:17:03.544 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T01:17:03.580 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T01:17:03.829 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T01:17:03.885 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T01:17:03.914 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T01:17:03.954 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T01:17:04.031 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T01:17:04.836 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T01:17:04.896 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T01:17:04.942 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T01:17:06.342 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T01:17:06.452 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T01:17:06.509 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T01:17:06.626 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T01:17:06.681 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T01:17:06.722 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T01:17:06.758 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T01:17:06.758 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T01:17:06.873 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T01:17:06.873 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T01:17:07.016 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T01:17:07.040 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T01:17:07.058 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T01:17:07.058 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T01:17:07.342 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T01:17:07.342 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T01:17:07.486 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T01:17:07.570 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T01:17:07.823 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T01:17:07.823 INFO:teuthology.orchestra.run.smithi001.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-12T01:17:07.823 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:17:13.650 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T01:17:13.650 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T01:17:13.650 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T01:17:13.650 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T01:17:13.651 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T01:17:13.651 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T01:17:13.651 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T01:17:13.651 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T01:17:13.651 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T01:17:13.651 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T01:17:13.651 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T01:17:13.651 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T01:17:13.651 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T01:17:13.652 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T01:17:13.652 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T01:17:13.652 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T01:17:13.652 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T01:17:13.652 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T01:17:13.652 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T01:17:13.652 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T01:17:13.652 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T01:17:13.652 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T01:17:13.652 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T01:17:13.653 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T01:17:13.653 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T01:17:13.653 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T01:17:13.653 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T01:17:13.653 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T01:17:13.653 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T01:17:13.653 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T01:17:13.653 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T01:17:13.653 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T01:17:13.654 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T01:17:13.654 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T01:17:13.654 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T01:17:13.654 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T01:17:13.654 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T01:17:13.654 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T01:17:13.654 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T01:17:13.654 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T01:17:13.654 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T01:17:13.654 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T01:17:13.655 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T01:17:13.655 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T01:17:13.655 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T01:17:13.655 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T01:17:13.655 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T01:17:13.655 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T01:17:13.655 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T01:17:13.655 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T01:17:13.655 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T01:17:13.655 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T01:17:13.656 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T01:17:13.656 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T01:17:13.656 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T01:17:13.656 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T01:17:13.656 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T01:17:13.656 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T01:17:13.656 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T01:17:13.656 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T01:17:13.656 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T01:17:13.657 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T01:17:13.657 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T01:17:13.657 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T01:17:13.657 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T01:17:13.657 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T01:17:13.657 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T01:17:13.657 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T01:17:13.657 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T01:17:13.657 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T01:17:13.657 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T01:17:13.658 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T01:17:13.658 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T01:17:13.658 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T01:17:13.658 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T01:17:13.658 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T01:17:13.658 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T01:17:13.658 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T01:17:13.658 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T01:17:13.658 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T01:17:13.658 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T01:17:13.658 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T01:17:13.659 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T01:17:13.659 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T01:17:13.659 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T01:17:13.659 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T01:17:13.659 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T01:17:13.659 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T01:17:13.659 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T01:17:13.659 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T01:17:13.659 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T01:17:13.659 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T01:17:13.659 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T01:17:13.659 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T01:17:13.660 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T01:17:13.660 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T01:17:13.660 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T01:17:13.660 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T01:17:13.660 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T01:17:13.660 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T01:17:13.660 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T01:17:13.661 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T01:17:13.661 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T01:17:13.661 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T01:17:13.661 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T01:17:13.661 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T01:17:13.665 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-12T01:17:13.665 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T01:17:13.665 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T01:17:13.665 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T01:17:13.666 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T01:17:13.666 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T01:17:13.666 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T01:17:13.666 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T01:17:13.666 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T01:17:13.666 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T01:17:13.666 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T01:17:13.666 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T01:17:13.666 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T01:17:13.666 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T01:17:13.666 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T01:17:13.666 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T01:17:13.667 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T01:17:13.667 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T01:17:13.667 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T01:17:13.667 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T01:17:13.667 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T01:17:13.667 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T01:17:13.667 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T01:17:13.667 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T01:17:13.667 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T01:17:13.667 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T01:17:13.667 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T01:17:13.667 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T01:17:13.667 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T01:17:13.667 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T01:17:13.667 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T01:17:13.668 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T01:17:13.668 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T01:17:13.668 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T01:17:13.668 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T01:17:13.669 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T01:17:13.669 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T01:17:13.669 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T01:17:13.669 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T01:17:13.669 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T01:17:13.669 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T01:17:13.669 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T01:17:13.669 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T01:17:13.669 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T01:17:13.669 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T01:17:13.669 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T01:17:13.669 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T01:17:13.670 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T01:17:13.670 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T01:17:13.670 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T01:17:13.670 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T01:17:13.670 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T01:17:13.670 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T01:17:13.670 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T01:17:13.670 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T01:17:13.670 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T01:17:13.670 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T01:17:16.010 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T01:17:16.011 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:17:16.011 INFO:teuthology.orchestra.run.smithi001.stdout:Upgraded: 2024-09-12T01:17:16.011 INFO:teuthology.orchestra.run.smithi001.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T01:17:16.011 INFO:teuthology.orchestra.run.smithi001.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T01:17:16.011 INFO:teuthology.orchestra.run.smithi001.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T01:17:16.011 INFO:teuthology.orchestra.run.smithi001.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T01:17:16.011 INFO:teuthology.orchestra.run.smithi001.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:17:16.011 INFO:teuthology.orchestra.run.smithi001.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:17:16.011 INFO:teuthology.orchestra.run.smithi001.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T01:17:16.011 INFO:teuthology.orchestra.run.smithi001.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T01:17:16.011 INFO:teuthology.orchestra.run.smithi001.stdout:Installed: 2024-09-12T01:17:16.011 INFO:teuthology.orchestra.run.smithi001.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T01:17:16.011 INFO:teuthology.orchestra.run.smithi001.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T01:17:16.012 INFO:teuthology.orchestra.run.smithi001.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T01:17:16.012 INFO:teuthology.orchestra.run.smithi001.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T01:17:16.012 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:17:16.012 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:17:16.012 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:17:16.012 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:17:16.012 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:17:16.012 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:17:16.012 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:17:16.012 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:17:16.012 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:17:16.012 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:17:16.013 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:17:16.013 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:17:16.013 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:17:16.016 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:17:16.016 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:17:16.016 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:17:16.017 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:17:16.017 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:17:16.017 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:17:16.017 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:17:16.017 INFO:teuthology.orchestra.run.smithi001.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:17:16.017 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:17:16.017 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:17:16.017 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:17:16.017 INFO:teuthology.orchestra.run.smithi001.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T01:17:16.017 INFO:teuthology.orchestra.run.smithi001.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T01:17:16.017 INFO:teuthology.orchestra.run.smithi001.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T01:17:16.017 INFO:teuthology.orchestra.run.smithi001.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T01:17:16.018 INFO:teuthology.orchestra.run.smithi001.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T01:17:16.018 INFO:teuthology.orchestra.run.smithi001.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T01:17:16.018 INFO:teuthology.orchestra.run.smithi001.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-12T01:17:16.018 INFO:teuthology.orchestra.run.smithi001.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T01:17:16.018 INFO:teuthology.orchestra.run.smithi001.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T01:17:16.018 INFO:teuthology.orchestra.run.smithi001.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T01:17:16.018 INFO:teuthology.orchestra.run.smithi001.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T01:17:16.018 INFO:teuthology.orchestra.run.smithi001.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:17:16.018 INFO:teuthology.orchestra.run.smithi001.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:17:16.018 INFO:teuthology.orchestra.run.smithi001.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:17:16.018 INFO:teuthology.orchestra.run.smithi001.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T01:17:16.018 INFO:teuthology.orchestra.run.smithi001.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T01:17:16.018 INFO:teuthology.orchestra.run.smithi001.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T01:17:16.018 INFO:teuthology.orchestra.run.smithi001.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T01:17:16.019 INFO:teuthology.orchestra.run.smithi001.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T01:17:16.019 INFO:teuthology.orchestra.run.smithi001.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:17:16.019 INFO:teuthology.orchestra.run.smithi001.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:17:16.019 INFO:teuthology.orchestra.run.smithi001.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T01:17:16.019 INFO:teuthology.orchestra.run.smithi001.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:17:16.019 INFO:teuthology.orchestra.run.smithi001.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T01:17:16.019 INFO:teuthology.orchestra.run.smithi001.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T01:17:16.019 INFO:teuthology.orchestra.run.smithi001.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T01:17:16.019 INFO:teuthology.orchestra.run.smithi001.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T01:17:16.019 INFO:teuthology.orchestra.run.smithi001.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T01:17:16.019 INFO:teuthology.orchestra.run.smithi001.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T01:17:16.019 INFO:teuthology.orchestra.run.smithi001.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T01:17:16.019 INFO:teuthology.orchestra.run.smithi001.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T01:17:16.020 INFO:teuthology.orchestra.run.smithi001.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T01:17:16.020 INFO:teuthology.orchestra.run.smithi001.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T01:17:16.020 INFO:teuthology.orchestra.run.smithi001.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T01:17:16.020 INFO:teuthology.orchestra.run.smithi001.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T01:17:16.020 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T01:17:16.020 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T01:17:16.020 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T01:17:16.020 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T01:17:16.020 INFO:teuthology.orchestra.run.smithi001.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T01:17:16.020 INFO:teuthology.orchestra.run.smithi001.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T01:17:16.021 INFO:teuthology.orchestra.run.smithi001.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T01:17:16.021 INFO:teuthology.orchestra.run.smithi001.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T01:17:16.021 INFO:teuthology.orchestra.run.smithi001.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T01:17:16.021 INFO:teuthology.orchestra.run.smithi001.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T01:17:16.021 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T01:17:16.021 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:17:16.021 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:17:16.021 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:17:16.021 INFO:teuthology.orchestra.run.smithi001.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T01:17:16.021 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T01:17:16.021 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T01:17:16.021 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T01:17:16.021 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T01:17:16.022 INFO:teuthology.orchestra.run.smithi001.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T01:17:16.022 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T01:17:16.022 INFO:teuthology.orchestra.run.smithi001.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T01:17:16.022 INFO:teuthology.orchestra.run.smithi001.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T01:17:16.022 INFO:teuthology.orchestra.run.smithi001.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T01:17:16.022 INFO:teuthology.orchestra.run.smithi001.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T01:17:16.022 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T01:17:16.022 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T01:17:16.022 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T01:17:16.022 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T01:17:16.022 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T01:17:16.022 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T01:17:16.022 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T01:17:16.022 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T01:17:16.022 INFO:teuthology.orchestra.run.smithi001.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T01:17:16.023 INFO:teuthology.orchestra.run.smithi001.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T01:17:16.023 INFO:teuthology.orchestra.run.smithi001.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T01:17:16.023 INFO:teuthology.orchestra.run.smithi001.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T01:17:16.023 INFO:teuthology.orchestra.run.smithi001.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T01:17:16.023 INFO:teuthology.orchestra.run.smithi001.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T01:17:16.023 INFO:teuthology.orchestra.run.smithi001.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T01:17:16.023 INFO:teuthology.orchestra.run.smithi001.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T01:17:16.023 INFO:teuthology.orchestra.run.smithi001.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T01:17:16.023 INFO:teuthology.orchestra.run.smithi001.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T01:17:16.023 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T01:17:16.023 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T01:17:16.023 INFO:teuthology.orchestra.run.smithi001.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T01:17:16.023 INFO:teuthology.orchestra.run.smithi001.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T01:17:16.023 INFO:teuthology.orchestra.run.smithi001.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T01:17:16.024 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T01:17:16.024 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T01:17:16.024 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T01:17:16.024 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T01:17:16.024 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T01:17:16.024 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T01:17:16.024 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:17:16.024 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:17:16.024 INFO:teuthology.orchestra.run.smithi001.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T01:17:16.024 INFO:teuthology.orchestra.run.smithi001.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T01:17:16.024 INFO:teuthology.orchestra.run.smithi001.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T01:17:16.024 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:17:16.025 INFO:teuthology.orchestra.run.smithi001.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T01:17:16.025 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T01:17:16.025 INFO:teuthology.orchestra.run.smithi001.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T01:17:16.025 INFO:teuthology.orchestra.run.smithi001.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T01:17:16.025 INFO:teuthology.orchestra.run.smithi001.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T01:17:16.025 INFO:teuthology.orchestra.run.smithi001.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T01:17:16.025 INFO:teuthology.orchestra.run.smithi001.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T01:17:16.025 INFO:teuthology.orchestra.run.smithi001.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T01:17:16.025 INFO:teuthology.orchestra.run.smithi001.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T01:17:16.025 INFO:teuthology.orchestra.run.smithi001.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T01:17:16.025 INFO:teuthology.orchestra.run.smithi001.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T01:17:16.025 INFO:teuthology.orchestra.run.smithi001.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T01:17:16.025 INFO:teuthology.orchestra.run.smithi001.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T01:17:16.026 INFO:teuthology.orchestra.run.smithi001.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T01:17:16.026 INFO:teuthology.orchestra.run.smithi001.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T01:17:16.026 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:17:16.026 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:17:16.026 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:17:16.026 INFO:teuthology.orchestra.run.smithi001.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T01:17:16.026 INFO:teuthology.orchestra.run.smithi001.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T01:17:16.026 INFO:teuthology.orchestra.run.smithi001.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T01:17:16.026 INFO:teuthology.orchestra.run.smithi001.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T01:17:16.026 INFO:teuthology.orchestra.run.smithi001.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T01:17:16.026 INFO:teuthology.orchestra.run.smithi001.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T01:17:16.027 INFO:teuthology.orchestra.run.smithi001.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T01:17:16.027 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:17:16.027 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2024-09-12T01:17:16.675 DEBUG:teuthology.parallel:result is None 2024-09-12T01:17:16.676 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T01:17:16.833 DEBUG:teuthology.orchestra.run.smithi001:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T01:17:16.878 INFO:teuthology.orchestra.run.smithi001.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T01:17:16.879 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T01:17:16.879 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T01:17:16.881 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T01:17:17.037 DEBUG:teuthology.orchestra.run.smithi053:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T01:17:17.063 INFO:teuthology.orchestra.run.smithi053.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T01:17:17.063 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T01:17:17.064 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T01:17:17.065 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T01:17:17.211 DEBUG:teuthology.orchestra.run.smithi104:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T01:17:17.238 INFO:teuthology.orchestra.run.smithi104.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T01:17:17.238 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T01:17:17.238 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T01:17:17.240 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-12T01:17:17.240 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-12T01:17:17.240 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T01:17:17.300 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-12T01:17:17.300 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T01:17:17.332 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-12T01:17:17.333 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T01:17:17.365 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-12T01:17:17.366 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-12T01:17:17.366 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T01:17:17.423 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T01:17:17.519 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-12T01:17:17.520 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T01:17:17.549 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T01:17:17.621 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-12T01:17:17.621 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T01:17:17.647 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T01:17:17.713 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-12T01:17:17.713 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-12T01:17:17.714 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T01:17:17.772 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T01:17:17.870 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-12T01:17:17.870 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T01:17:17.896 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T01:17:17.961 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-12T01:17:17.961 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T01:17:17.986 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T01:17:18.052 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-12T01:17:18.053 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-12T01:17:18.053 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T01:17:18.109 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T01:17:18.208 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-12T01:17:18.209 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T01:17:18.234 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T01:17:18.299 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-12T01:17:18.299 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T01:17:18.324 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T01:17:18.390 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-12T01:17:18.494 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-12T01:17:18.494 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T01:17:18.495 INFO:tasks.cephadm:Cluster fsid is c0e4f180-70a4-11ef-bcea-c7b262605968 2024-09-12T01:17:18.495 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-12T01:17:18.495 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.1', 'mon.b': '172.21.15.53', 'mon.c': '172.21.15.104'} 2024-09-12T01:17:18.495 INFO:tasks.cephadm:First mon is mon.a on smithi001 2024-09-12T01:17:18.495 INFO:tasks.cephadm:First mgr is x 2024-09-12T01:17:18.495 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-12T01:17:18.496 DEBUG:teuthology.orchestra.run.smithi001:> sudo hostname $(hostname -s) 2024-09-12T01:17:18.560 DEBUG:teuthology.orchestra.run.smithi053:> sudo hostname $(hostname -s) 2024-09-12T01:17:18.585 DEBUG:teuthology.orchestra.run.smithi104:> sudo hostname $(hostname -s) 2024-09-12T01:17:18.611 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-12T01:17:18.611 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T01:17:18.756 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56038/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-298-gb53b6d3e', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-298.gb53b6d3e'}, 'url': 'https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-09 17:24:58.599585', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-12T01:17:18.901 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref squid, sha1 b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T01:17:18.903 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-12T01:17:18.903 INFO:tasks.cephadm:Downloading cephadm from url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-12T01:17:18.903 DEBUG:teuthology.orchestra.run.smithi001:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-12T01:17:19.332 INFO:teuthology.orchestra.run.smithi001.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 12 01:17 /home/ubuntu/cephtest/cephadm 2024-09-12T01:17:19.333 DEBUG:teuthology.orchestra.run.smithi053:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-12T01:17:19.716 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 12 01:17 /home/ubuntu/cephtest/cephadm 2024-09-12T01:17:19.716 DEBUG:teuthology.orchestra.run.smithi104:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-12T01:17:20.111 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 12 01:17 /home/ubuntu/cephtest/cephadm 2024-09-12T01:17:20.111 DEBUG:teuthology.orchestra.run.smithi001:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-12T01:17:20.152 DEBUG:teuthology.orchestra.run.smithi053:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-12T01:17:20.172 DEBUG:teuthology.orchestra.run.smithi104:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-12T01:17:20.219 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc on all hosts... 2024-09-12T01:17:20.220 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-12T01:17:20.224 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-12T01:17:20.227 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-12T01:17:21.751 INFO:teuthology.orchestra.run.smithi053.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T01:17:21.778 INFO:teuthology.orchestra.run.smithi104.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T01:17:22.297 INFO:teuthology.orchestra.run.smithi001.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T01:17:57.038 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-12T01:17:57.039 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T01:17:57.039 INFO:teuthology.orchestra.run.smithi001.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-12T01:17:57.039 INFO:teuthology.orchestra.run.smithi001.stdout: "repo_digests": [ 2024-09-12T01:17:57.039 INFO:teuthology.orchestra.run.smithi001.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-12T01:17:57.040 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-12T01:17:57.040 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-12T01:18:01.335 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-12T01:18:01.335 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T01:18:01.335 INFO:teuthology.orchestra.run.smithi104.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-12T01:18:01.335 INFO:teuthology.orchestra.run.smithi104.stdout: "repo_digests": [ 2024-09-12T01:18:01.335 INFO:teuthology.orchestra.run.smithi104.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-12T01:18:01.335 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-12T01:18:01.335 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-12T01:18:07.685 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-12T01:18:07.685 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T01:18:07.685 INFO:teuthology.orchestra.run.smithi053.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-12T01:18:07.686 INFO:teuthology.orchestra.run.smithi053.stdout: "repo_digests": [ 2024-09-12T01:18:07.686 INFO:teuthology.orchestra.run.smithi053.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-12T01:18:07.686 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-12T01:18:07.686 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-12T01:18:07.711 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /etc/ceph 2024-09-12T01:18:07.776 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /etc/ceph 2024-09-12T01:18:07.804 DEBUG:teuthology.orchestra.run.smithi104:> sudo mkdir -p /etc/ceph 2024-09-12T01:18:07.838 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 777 /etc/ceph 2024-09-12T01:18:07.896 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 777 /etc/ceph 2024-09-12T01:18:07.921 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 777 /etc/ceph 2024-09-12T01:18:07.950 INFO:tasks.cephadm:Writing seed config... 2024-09-12T01:18:07.951 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-12T01:18:07.951 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-12T01:18:07.951 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-12T01:18:07.951 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-12T01:18:07.951 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-12T01:18:07.952 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-12T01:18:07.952 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-12T01:18:07.952 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-12T01:18:07.952 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-12T01:18:07.952 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-12T01:18:07.952 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-12T01:18:07.952 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-12T01:18:07.952 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-12T01:18:07.952 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-12T01:18:07.952 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-12T01:18:07.952 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-12T01:18:07.952 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-12T01:18:07.953 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-09-12T01:18:07.953 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-12T01:18:07.953 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-12T01:18:07.953 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-12T01:18:07.953 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-12T01:18:07.953 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-12T01:18:07.954 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-12T01:18:07.954 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-12T01:18:07.954 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-12T01:18:07.954 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-12T01:18:07.954 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-12T01:18:07.955 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-12T01:18:07.955 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-12T01:18:07.955 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-12T01:18:07.955 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-12T01:18:07.955 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-12T01:18:07.955 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-12T01:18:07.955 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-12T01:18:07.955 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-12T01:18:07.955 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-12T01:18:07.955 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-12T01:18:07.955 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-12T01:18:07.956 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-12T01:18:07.956 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-12T01:18:07.956 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-12T01:18:07.956 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-12T01:18:07.956 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-12T01:18:07.956 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-12T01:18:07.957 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-12T01:18:07.957 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-12T01:18:07.957 DEBUG:teuthology.orchestra.run.smithi001:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-12T01:18:07.994 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = c0e4f180-70a4-11ef-bcea-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-12T01:18:07.995 DEBUG:teuthology.orchestra.run.smithi001:mon.a> sudo journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mon.a.service 2024-09-12T01:18:08.039 DEBUG:teuthology.orchestra.run.smithi001:mgr.x> sudo journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mgr.x.service 2024-09-12T01:18:08.083 INFO:tasks.cephadm:Bootstrapping... 2024-09-12T01:18:08.084 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc -v bootstrap --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.1 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-12T01:18:08.346 INFO:teuthology.orchestra.run.smithi001.stdout:-------------------------------------------------------------------------------- 2024-09-12T01:18:08.346 INFO:teuthology.orchestra.run.smithi001.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', '-v', 'bootstrap', '--fsid', 'c0e4f180-70a4-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.1', '--skip-admin-label'] 2024-09-12T01:18:08.347 INFO:teuthology.orchestra.run.smithi001.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-12T01:18:08.347 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying podman|docker is present... 2024-09-12T01:18:08.391 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stdout 5.2.2 2024-09-12T01:18:08.392 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying lvm2 is present... 2024-09-12T01:18:08.392 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying time synchronization is in place... 2024-09-12T01:18:08.406 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-12T01:18:08.406 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-12T01:18:08.418 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-12T01:18:08.419 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout inactive 2024-09-12T01:18:08.432 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout enabled 2024-09-12T01:18:08.445 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout active 2024-09-12T01:18:08.445 INFO:teuthology.orchestra.run.smithi001.stdout:Unit chronyd.service is enabled and running 2024-09-12T01:18:08.445 INFO:teuthology.orchestra.run.smithi001.stdout:Repeating the final host check... 2024-09-12T01:18:08.490 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stdout 5.2.2 2024-09-12T01:18:08.490 INFO:teuthology.orchestra.run.smithi001.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-12T01:18:08.490 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl is present 2024-09-12T01:18:08.490 INFO:teuthology.orchestra.run.smithi001.stdout:lvcreate is present 2024-09-12T01:18:08.503 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-12T01:18:08.503 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-12T01:18:08.516 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-12T01:18:08.516 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout inactive 2024-09-12T01:18:08.529 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout enabled 2024-09-12T01:18:08.542 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout active 2024-09-12T01:18:08.542 INFO:teuthology.orchestra.run.smithi001.stdout:Unit chronyd.service is enabled and running 2024-09-12T01:18:08.542 INFO:teuthology.orchestra.run.smithi001.stdout:Host looks OK 2024-09-12T01:18:08.542 INFO:teuthology.orchestra.run.smithi001.stdout:Cluster fsid: c0e4f180-70a4-11ef-bcea-c7b262605968 2024-09-12T01:18:08.542 INFO:teuthology.orchestra.run.smithi001.stdout:Acquiring lock 140541863424880 on /run/cephadm/c0e4f180-70a4-11ef-bcea-c7b262605968.lock 2024-09-12T01:18:08.543 INFO:teuthology.orchestra.run.smithi001.stdout:Lock 140541863424880 acquired on /run/cephadm/c0e4f180-70a4-11ef-bcea-c7b262605968.lock 2024-09-12T01:18:08.543 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying IP 172.21.15.1 port 3300 ... 2024-09-12T01:18:08.543 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying IP 172.21.15.1 port 6789 ... 2024-09-12T01:18:08.543 INFO:teuthology.orchestra.run.smithi001.stdout:Base mon IP(s) is [172.21.15.1:3300, 172.21.15.1:6789], mon addrv is [v2:172.21.15.1:3300,v1:172.21.15.1:6789] 2024-09-12T01:18:08.550 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.1 metric 100 2024-09-12T01:18:08.550 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.1 metric 100 2024-09-12T01:18:08.556 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-12T01:18:08.558 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-12T01:18:08.559 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1788sec hoplimit 64 pref medium 2024-09-12T01:18:08.562 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-12T01:18:08.562 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-12T01:18:08.562 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-12T01:18:08.563 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-12T01:18:08.563 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15e8/64 scope link 2024-09-12T01:18:08.563 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-12T01:18:08.563 INFO:teuthology.orchestra.run.smithi001.stdout:Mon IP `172.21.15.1` is in CIDR network `172.21.0.0/20` 2024-09-12T01:18:08.564 INFO:teuthology.orchestra.run.smithi001.stdout:Mon IP `172.21.15.1` is in CIDR network `172.21.0.0/20` 2024-09-12T01:18:08.564 INFO:teuthology.orchestra.run.smithi001.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-12T01:18:08.564 INFO:teuthology.orchestra.run.smithi001.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-12T01:18:08.565 INFO:teuthology.orchestra.run.smithi001.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T01:18:09.101 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stdout 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-12T01:18:09.102 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T01:18:09.102 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stderr Getting image source signatures 2024-09-12T01:18:09.102 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stderr Copying blob sha256:60180723200353b4e13b39fd162aecaf1bc097c859c7c6537324229e97c78567 2024-09-12T01:18:09.102 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-12T01:18:09.102 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stderr Copying config sha256:54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-12T01:18:09.103 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-12T01:18:10.229 INFO:teuthology.orchestra.run.smithi001.stdout:ceph: stdout ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-12T01:18:10.229 INFO:teuthology.orchestra.run.smithi001.stdout:Ceph version: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-12T01:18:10.230 INFO:teuthology.orchestra.run.smithi001.stdout:Extracting ceph user uid/gid from container image... 2024-09-12T01:18:11.495 INFO:teuthology.orchestra.run.smithi001.stdout:stat: stdout 167 167 2024-09-12T01:18:11.495 INFO:teuthology.orchestra.run.smithi001.stdout:Creating initial keys... 2024-09-12T01:18:12.572 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph-authtool: stdout AQBTQeJm69TCMxAAbinJbQCRlDVLVDDq8epINA== 2024-09-12T01:18:13.664 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph-authtool: stdout AQBVQeJmgLtwABAALArobGT49u5lYLZkjRkBsw== 2024-09-12T01:18:14.819 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph-authtool: stdout AQBWQeJm4/FZAxAAxnhUcpQqUqYDbq4FW3Sn0Q== 2024-09-12T01:18:14.821 INFO:teuthology.orchestra.run.smithi001.stdout:Creating initial monmap... 2024-09-12T01:18:15.850 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-12T01:18:15.850 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-12T01:18:15.850 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to c0e4f180-70a4-11ef-bcea-c7b262605968 2024-09-12T01:18:15.850 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-12T01:18:15.850 INFO:teuthology.orchestra.run.smithi001.stdout:monmaptool for a [v2:172.21.15.1:3300,v1:172.21.15.1:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-12T01:18:15.850 INFO:teuthology.orchestra.run.smithi001.stdout:setting min_mon_release = quincy 2024-09-12T01:18:15.850 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/monmaptool: set fsid to c0e4f180-70a4-11ef-bcea-c7b262605968 2024-09-12T01:18:15.851 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-12T01:18:15.851 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:18:15.851 INFO:teuthology.orchestra.run.smithi001.stdout:Creating mon... 2024-09-12T01:18:17.273 INFO:teuthology.orchestra.run.smithi001.stdout:create mon.a on 2024-09-12T01:18:17.560 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-12T01:18:17.781 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-12T01:18:18.046 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-c0e4f180-70a4-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-c0e4f180-70a4-11ef-bcea-c7b262605968.target. 2024-09-12T01:18:18.046 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-c0e4f180-70a4-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-c0e4f180-70a4-11ef-bcea-c7b262605968.target. 2024-09-12T01:18:18.395 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mon.a 2024-09-12T01:18:18.395 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-12T01:18:18.663 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c0e4f180-70a4-11ef-bcea-c7b262605968.target.wants/ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@.service. 2024-09-12T01:18:19.277 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:19 smithi001 podman[30012]: 2024-09-12 01:18:19.27177919 +0000 UTC m=+0.369734836 container init 564d21c06d1f7fa0036994ccc0191a4ddc146c97d91d6beff2d6e791c651eb4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-12T01:18:19.278 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:19 smithi001 podman[30012]: 2024-09-12 01:18:19.278217415 +0000 UTC m=+0.376173060 container start 564d21c06d1f7fa0036994ccc0191a4ddc146c97d91d6beff2d6e791c651eb4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=) 2024-09-12T01:18:19.381 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T01:18:19.382 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout disabled 2024-09-12T01:18:19.395 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T01:18:19.395 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout inactive 2024-09-12T01:18:19.395 INFO:teuthology.orchestra.run.smithi001.stdout:firewalld.service is not enabled 2024-09-12T01:18:19.395 INFO:teuthology.orchestra.run.smithi001.stdout:Not possible to enable service . firewalld.service is not available 2024-09-12T01:18:19.396 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for mon to start... 2024-09-12T01:18:19.396 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for mon... 2024-09-12T01:18:19.537 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:19 smithi001 ceph-mon[30030]: mkfs c0e4f180-70a4-11ef-bcea-c7b262605968 2024-09-12T01:18:19.537 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:19 smithi001 ceph-mon[30030]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-12T01:18:19.537 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:19 smithi001 bash[30012]: 564d21c06d1f7fa0036994ccc0191a4ddc146c97d91d6beff2d6e791c651eb4f 2024-09-12T01:18:19.537 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:19 smithi001 systemd[1]: Started Ceph mon.a for c0e4f180-70a4-11ef-bcea-c7b262605968. 2024-09-12T01:18:20.635 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:20 smithi001 ceph-mon[30030]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-12T01:18:20.635 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:20 smithi001 ceph-mon[30030]: monmap epoch 1 2024-09-12T01:18:20.635 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:20 smithi001 ceph-mon[30030]: fsid c0e4f180-70a4-11ef-bcea-c7b262605968 2024-09-12T01:18:20.636 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:20 smithi001 ceph-mon[30030]: last_changed 2024-09-12T01:18:15.215734+0000 2024-09-12T01:18:20.636 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:20 smithi001 ceph-mon[30030]: created 2024-09-12T01:18:15.215734+0000 2024-09-12T01:18:20.636 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:20 smithi001 ceph-mon[30030]: min_mon_release 19 (squid) 2024-09-12T01:18:20.636 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:20 smithi001 ceph-mon[30030]: election_strategy: 1 2024-09-12T01:18:20.636 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:20 smithi001 ceph-mon[30030]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-09-12T01:18:20.636 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:20 smithi001 ceph-mon[30030]: fsmap 2024-09-12T01:18:20.636 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:20 smithi001 ceph-mon[30030]: osdmap e1: 0 total, 0 up, 0 in 2024-09-12T01:18:20.636 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:20 smithi001 ceph-mon[30030]: mgrmap e1: no daemons active 2024-09-12T01:18:20.636 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:20 smithi001 ceph-mon[30030]: from='client.? 172.21.15.1:0/1640650265' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-12T01:18:20.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout cluster: 2024-09-12T01:18:20.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout id: c0e4f180-70a4-11ef-bcea-c7b262605968 2024-09-12T01:18:20.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-12T01:18:20.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-09-12T01:18:20.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout services: 2024-09-12T01:18:20.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.652003s) 2024-09-12T01:18:20.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-12T01:18:20.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-12T01:18:20.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-09-12T01:18:20.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout data: 2024-09-12T01:18:20.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-12T01:18:20.648 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-12T01:18:20.648 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-12T01:18:20.648 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout pgs: 2024-09-12T01:18:20.648 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-09-12T01:18:20.648 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.909+0000 7fc5b3200640 1 Processor -- start 2024-09-12T01:18:20.648 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.909+0000 7fc5b3200640 1 -- start start 2024-09-12T01:18:20.648 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.910+0000 7fc5b3200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5ac1098c0 0x7fc5ac109cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:18:20.648 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.910+0000 7fc5b3200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac10a290 con 0x7fc5ac1098c0 2024-09-12T01:18:20.648 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.910+0000 7fc5abe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5ac1098c0 0x7fc5ac109cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:18:20.648 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.910+0000 7fc5abe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5ac1098c0 0x7fc5ac109cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54230/0 (socket says 172.21.15.1:54230) 2024-09-12T01:18:20.648 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.910+0000 7fc5abe00640 1 -- 172.21.15.1:0/2543942187 learned_addr learned my addr 172.21.15.1:0/2543942187 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:18:20.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.911+0000 7fc5abe00640 1 -- 172.21.15.1:0/2543942187 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5ac10a9b0 con 0x7fc5ac1098c0 2024-09-12T01:18:20.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.914+0000 7fc5abe00640 1 --2- 172.21.15.1:0/2543942187 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5ac1098c0 0x7fc5ac109cc0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc59c009820 tx=0x7fc59c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b303315ee132a746 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:18:20.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.914+0000 7fc5ab400640 1 -- 172.21.15.1:0/2543942187 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc59c037070 con 0x7fc5ac1098c0 2024-09-12T01:18:20.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.914+0000 7fc5ab400640 1 -- 172.21.15.1:0/2543942187 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc59c032440 con 0x7fc5ac1098c0 2024-09-12T01:18:20.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.914+0000 7fc5ab400640 1 -- 172.21.15.1:0/2543942187 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc59c032740 con 0x7fc5ac1098c0 2024-09-12T01:18:20.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.915+0000 7fc5b3200640 1 -- 172.21.15.1:0/2543942187 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5ac1098c0 msgr2=0x7fc5ac109cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:18:20.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.915+0000 7fc5b3200640 1 --2- 172.21.15.1:0/2543942187 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5ac1098c0 0x7fc5ac109cc0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc59c009820 tx=0x7fc59c02a270 comp rx=0 tx=0).stop 2024-09-12T01:18:20.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.915+0000 7fc5b3200640 1 -- 172.21.15.1:0/2543942187 shutdown_connections 2024-09-12T01:18:20.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.915+0000 7fc5b3200640 1 --2- 172.21.15.1:0/2543942187 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5ac1098c0 0x7fc5ac109cc0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:20.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.915+0000 7fc5b3200640 1 -- 172.21.15.1:0/2543942187 >> 172.21.15.1:0/2543942187 conn(0x7fc5ac07ce20 msgr2=0x7fc5ac07d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:18:20.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.915+0000 7fc5b3200640 1 -- 172.21.15.1:0/2543942187 shutdown_connections 2024-09-12T01:18:20.650 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.915+0000 7fc5b3200640 1 -- 172.21.15.1:0/2543942187 wait complete. 2024-09-12T01:18:20.650 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.916+0000 7fc5b3200640 1 Processor -- start 2024-09-12T01:18:20.650 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.916+0000 7fc5b3200640 1 -- start start 2024-09-12T01:18:20.650 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.916+0000 7fc5b3200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5ac1098c0 0x7fc5ac19d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:18:20.650 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.916+0000 7fc5b3200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac10aee0 con 0x7fc5ac1098c0 2024-09-12T01:18:20.650 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.916+0000 7fc5abe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5ac1098c0 0x7fc5ac19d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:18:20.650 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.917+0000 7fc5abe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5ac1098c0 0x7fc5ac19d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54238/0 (socket says 172.21.15.1:54238) 2024-09-12T01:18:20.650 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.917+0000 7fc5abe00640 1 -- 172.21.15.1:0/1640650265 learned_addr learned my addr 172.21.15.1:0/1640650265 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:18:20.650 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.917+0000 7fc5abe00640 1 -- 172.21.15.1:0/1640650265 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5ac19d7c0 con 0x7fc5ac1098c0 2024-09-12T01:18:20.650 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.917+0000 7fc5abe00640 1 --2- 172.21.15.1:0/1640650265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5ac1098c0 0x7fc5ac19d280 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc59c0090b0 tx=0x7fc59c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:18:20.650 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.917+0000 7fc5a9600640 1 -- 172.21.15.1:0/1640650265 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc59c041070 con 0x7fc5ac1098c0 2024-09-12T01:18:20.651 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.917+0000 7fc5a9600640 1 -- 172.21.15.1:0/1640650265 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc59c03c7a0 con 0x7fc5ac1098c0 2024-09-12T01:18:20.651 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.918+0000 7fc5a9600640 1 -- 172.21.15.1:0/1640650265 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc59c037040 con 0x7fc5ac1098c0 2024-09-12T01:18:20.651 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.918+0000 7fc5b3200640 1 -- 172.21.15.1:0/1640650265 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5ac19da50 con 0x7fc5ac1098c0 2024-09-12T01:18:20.651 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.918+0000 7fc5b3200640 1 -- 172.21.15.1:0/1640650265 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5ac1a0740 con 0x7fc5ac1098c0 2024-09-12T01:18:20.651 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.918+0000 7fc5a9600640 1 -- 172.21.15.1:0/1640650265 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc59c044070 con 0x7fc5ac1098c0 2024-09-12T01:18:20.651 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.918+0000 7fc5a9600640 1 -- 172.21.15.1:0/1640650265 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc59c03d3f0 con 0x7fc5ac1098c0 2024-09-12T01:18:20.651 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.919+0000 7fc5b3200640 1 -- 172.21.15.1:0/1640650265 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5ac10b530 con 0x7fc5ac1098c0 2024-09-12T01:18:20.651 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.922+0000 7fc5a9600640 1 -- 172.21.15.1:0/1640650265 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fc59c046450 con 0x7fc5ac1098c0 2024-09-12T01:18:20.651 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.987+0000 7fc5b3200640 1 -- 172.21.15.1:0/1640650265 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fc5ac067ba0 con 0x7fc5ac1098c0 2024-09-12T01:18:20.651 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.987+0000 7fc5a9600640 1 -- 172.21.15.1:0/1640650265 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fc5ac067ba0 con 0x7fc5ac1098c0 2024-09-12T01:18:20.651 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.988+0000 7fc5b3200640 1 -- 172.21.15.1:0/1640650265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5ac1098c0 msgr2=0x7fc5ac19d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:18:20.651 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.988+0000 7fc5b3200640 1 --2- 172.21.15.1:0/1640650265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5ac1098c0 0x7fc5ac19d280 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc59c0090b0 tx=0x7fc59c0046c0 comp rx=0 tx=0).stop 2024-09-12T01:18:20.651 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.988+0000 7fc5b3200640 1 -- 172.21.15.1:0/1640650265 shutdown_connections 2024-09-12T01:18:20.652 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.988+0000 7fc5b3200640 1 --2- 172.21.15.1:0/1640650265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5ac1098c0 0x7fc5ac19d280 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:20.652 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.988+0000 7fc5b3200640 1 -- 172.21.15.1:0/1640650265 >> 172.21.15.1:0/1640650265 conn(0x7fc5ac07ce20 msgr2=0x7fc5ac193cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:18:20.652 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.988+0000 7fc5b3200640 1 -- 172.21.15.1:0/1640650265 shutdown_connections 2024-09-12T01:18:20.652 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:19.988+0000 7fc5b3200640 1 -- 172.21.15.1:0/1640650265 wait complete. 2024-09-12T01:18:20.652 INFO:teuthology.orchestra.run.smithi001.stdout:mon is available 2024-09-12T01:18:20.652 INFO:teuthology.orchestra.run.smithi001.stdout:Assimilating anything we can from ceph.conf... 2024-09-12T01:18:21.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:21 smithi001 ceph-mon[30030]: from='client.? 172.21.15.1:0/3196401615' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-12T01:18:21.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:21 smithi001 ceph-mon[30030]: from='client.? 172.21.15.1:0/3196401615' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-12T01:18:21.853 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-09-12T01:18:21.853 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout [global] 2024-09-12T01:18:21.853 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout fsid = c0e4f180-70a4-11ef-bcea-c7b262605968 2024-09-12T01:18:21.853 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.1:3300,v1:172.21.15.1:6789] 2024-09-12T01:18:21.853 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-12T01:18:21.853 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-12T01:18:21.853 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-12T01:18:21.853 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-12T01:18:21.853 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-09-12T01:18:21.853 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-12T01:18:21.853 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-12T01:18:21.854 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-09-12T01:18:21.854 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout [osd] 2024-09-12T01:18:21.854 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-12T01:18:21.854 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-12T01:18:21.854 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-12T01:18:21.854 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-12T01:18:21.854 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-12T01:18:21.854 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.097+0000 7f5bb1e00640 1 Processor -- start 2024-09-12T01:18:21.854 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.098+0000 7f5bb1e00640 1 -- start start 2024-09-12T01:18:21.854 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.098+0000 7f5bb1e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5bac109860 0x7f5bac109c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:18:21.854 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.098+0000 7f5bb1e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5bac10a230 con 0x7f5bac109860 2024-09-12T01:18:21.854 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.098+0000 7f5baaa00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5bac109860 0x7f5bac109c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:18:21.854 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.098+0000 7f5baaa00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5bac109860 0x7f5bac109c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54250/0 (socket says 172.21.15.1:54250) 2024-09-12T01:18:21.854 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.098+0000 7f5baaa00640 1 -- 172.21.15.1:0/1209655042 learned_addr learned my addr 172.21.15.1:0/1209655042 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:18:21.855 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.099+0000 7f5baaa00640 1 -- 172.21.15.1:0/1209655042 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bac10a9e0 con 0x7f5bac109860 2024-09-12T01:18:21.855 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.099+0000 7f5baaa00640 1 --2- 172.21.15.1:0/1209655042 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5bac109860 0x7f5bac109c60 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c009820 tx=0x7f5b9c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c64bd06e1eeaf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:18:21.855 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.100+0000 7f5ba9600640 1 -- 172.21.15.1:0/1209655042 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b9c037070 con 0x7f5bac109860 2024-09-12T01:18:21.855 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.100+0000 7f5ba9600640 1 -- 172.21.15.1:0/1209655042 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b9c032440 con 0x7f5bac109860 2024-09-12T01:18:21.855 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.100+0000 7f5ba9600640 1 -- 172.21.15.1:0/1209655042 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b9c032740 con 0x7f5bac109860 2024-09-12T01:18:21.855 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.100+0000 7f5bb1e00640 1 -- 172.21.15.1:0/1209655042 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5bac109860 msgr2=0x7f5bac109c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:18:21.855 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.100+0000 7f5bb1e00640 1 --2- 172.21.15.1:0/1209655042 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5bac109860 0x7f5bac109c60 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c009820 tx=0x7f5b9c02a270 comp rx=0 tx=0).stop 2024-09-12T01:18:21.855 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.100+0000 7f5bb1e00640 1 -- 172.21.15.1:0/1209655042 shutdown_connections 2024-09-12T01:18:21.855 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.100+0000 7f5bb1e00640 1 --2- 172.21.15.1:0/1209655042 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5bac109860 0x7f5bac109c60 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:21.855 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.100+0000 7f5bb1e00640 1 -- 172.21.15.1:0/1209655042 >> 172.21.15.1:0/1209655042 conn(0x7f5bac07cee0 msgr2=0x7f5bac07d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:18:21.855 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.101+0000 7f5bb1e00640 1 -- 172.21.15.1:0/1209655042 shutdown_connections 2024-09-12T01:18:21.855 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.101+0000 7f5bb1e00640 1 -- 172.21.15.1:0/1209655042 wait complete. 2024-09-12T01:18:21.855 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.101+0000 7f5bb1e00640 1 Processor -- start 2024-09-12T01:18:21.856 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.102+0000 7f5bb1e00640 1 -- start start 2024-09-12T01:18:21.856 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.102+0000 7f5bb1e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5bac109860 0x7f5bac19d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:18:21.856 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.102+0000 7f5bb1e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5bac10af80 con 0x7f5bac109860 2024-09-12T01:18:21.856 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.102+0000 7f5baaa00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5bac109860 0x7f5bac19d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:18:21.856 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.102+0000 7f5baaa00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5bac109860 0x7f5bac19d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54266/0 (socket says 172.21.15.1:54266) 2024-09-12T01:18:21.856 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.102+0000 7f5baaa00640 1 -- 172.21.15.1:0/3196401615 learned_addr learned my addr 172.21.15.1:0/3196401615 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:18:21.856 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.103+0000 7f5baaa00640 1 -- 172.21.15.1:0/3196401615 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bac19d770 con 0x7f5bac109860 2024-09-12T01:18:21.856 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.103+0000 7f5baaa00640 1 --2- 172.21.15.1:0/3196401615 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5bac109860 0x7f5bac19d230 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c0090b0 tx=0x7f5b9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:18:21.856 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.103+0000 7f5bb0a00640 1 -- 172.21.15.1:0/3196401615 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b9c041070 con 0x7f5bac109860 2024-09-12T01:18:21.856 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.103+0000 7f5bb0a00640 1 -- 172.21.15.1:0/3196401615 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b9c03c7a0 con 0x7f5bac109860 2024-09-12T01:18:21.856 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.103+0000 7f5bb1e00640 1 -- 172.21.15.1:0/3196401615 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bac19da00 con 0x7f5bac109860 2024-09-12T01:18:21.856 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.103+0000 7f5bb0a00640 1 -- 172.21.15.1:0/3196401615 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b9c037040 con 0x7f5bac109860 2024-09-12T01:18:21.856 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.103+0000 7f5bb1e00640 1 -- 172.21.15.1:0/3196401615 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bac1a06f0 con 0x7f5bac109860 2024-09-12T01:18:21.857 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.104+0000 7f5bb0a00640 1 -- 172.21.15.1:0/3196401615 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5b9c044070 con 0x7f5bac109860 2024-09-12T01:18:21.857 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.104+0000 7f5bb0a00640 1 -- 172.21.15.1:0/3196401615 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5b9c03d3f0 con 0x7f5bac109860 2024-09-12T01:18:21.857 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.104+0000 7f5bb1e00640 1 -- 172.21.15.1:0/3196401615 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bac10b5d0 con 0x7f5bac109860 2024-09-12T01:18:21.857 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.107+0000 7f5bb0a00640 1 -- 172.21.15.1:0/3196401615 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f5b9c046e20 con 0x7f5bac109860 2024-09-12T01:18:21.857 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.170+0000 7f5bb1e00640 1 -- 172.21.15.1:0/3196401615 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f5bac19dee0 con 0x7f5bac109860 2024-09-12T01:18:21.857 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.173+0000 7f5bb0a00640 1 -- 172.21.15.1:0/3196401615 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+427 (secure 0 0 0) 0x7f5bac19dee0 con 0x7f5bac109860 2024-09-12T01:18:21.857 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.174+0000 7f5bb0a00640 1 -- 172.21.15.1:0/3196401615 <== mon.0 v2:172.21.15.1:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5b9c056070 con 0x7f5bac109860 2024-09-12T01:18:21.857 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.175+0000 7f5bb1e00640 1 -- 172.21.15.1:0/3196401615 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5bac109860 msgr2=0x7f5bac19d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:18:21.857 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.175+0000 7f5bb1e00640 1 --2- 172.21.15.1:0/3196401615 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5bac109860 0x7f5bac19d230 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c0090b0 tx=0x7f5b9c0046c0 comp rx=0 tx=0).stop 2024-09-12T01:18:21.857 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.175+0000 7f5bb1e00640 1 -- 172.21.15.1:0/3196401615 shutdown_connections 2024-09-12T01:18:21.857 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.175+0000 7f5bb1e00640 1 --2- 172.21.15.1:0/3196401615 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5bac109860 0x7f5bac19d230 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:21.857 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.175+0000 7f5bb1e00640 1 -- 172.21.15.1:0/3196401615 >> 172.21.15.1:0/3196401615 conn(0x7f5bac07cee0 msgr2=0x7f5bac193c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:18:21.857 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.175+0000 7f5bb1e00640 1 -- 172.21.15.1:0/3196401615 shutdown_connections 2024-09-12T01:18:21.857 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:21.175+0000 7f5bb1e00640 1 -- 172.21.15.1:0/3196401615 wait complete. 2024-09-12T01:18:21.858 INFO:teuthology.orchestra.run.smithi001.stdout:Generating new minimal ceph.conf... 2024-09-12T01:18:22.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:22 smithi001 ceph-mon[30030]: from='client.? 172.21.15.1:0/1106044765' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:18:23.048 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.327+0000 7fdf48800640 1 Processor -- start 2024-09-12T01:18:23.049 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.327+0000 7fdf48800640 1 -- start start 2024-09-12T01:18:23.049 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.328+0000 7fdf48800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf4007dd30 0x7fdf4007c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:18:23.049 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.328+0000 7fdf48800640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4007c7e0 con 0x7fdf4007dd30 2024-09-12T01:18:23.049 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.328+0000 7fdf45e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf4007dd30 0x7fdf4007c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:18:23.049 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.328+0000 7fdf45e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf4007dd30 0x7fdf4007c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54278/0 (socket says 172.21.15.1:54278) 2024-09-12T01:18:23.049 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.328+0000 7fdf45e00640 1 -- 172.21.15.1:0/2917500597 learned_addr learned my addr 172.21.15.1:0/2917500597 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:18:23.049 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.329+0000 7fdf45e00640 1 -- 172.21.15.1:0/2917500597 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf4007c960 con 0x7fdf4007dd30 2024-09-12T01:18:23.049 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.329+0000 7fdf45e00640 1 --2- 172.21.15.1:0/2917500597 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf4007dd30 0x7fdf4007c2a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdf34009820 tx=0x7fdf3402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc791880728f70c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:18:23.049 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.330+0000 7fdf44a00640 1 -- 172.21.15.1:0/2917500597 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf34037070 con 0x7fdf4007dd30 2024-09-12T01:18:23.049 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.330+0000 7fdf44a00640 1 -- 172.21.15.1:0/2917500597 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdf34032900 con 0x7fdf4007dd30 2024-09-12T01:18:23.049 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.330+0000 7fdf44a00640 1 -- 172.21.15.1:0/2917500597 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf34030850 con 0x7fdf4007dd30 2024-09-12T01:18:23.049 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.330+0000 7fdf48800640 1 -- 172.21.15.1:0/2917500597 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf4007dd30 msgr2=0x7fdf4007c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:18:23.050 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.330+0000 7fdf48800640 1 --2- 172.21.15.1:0/2917500597 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf4007dd30 0x7fdf4007c2a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdf34009820 tx=0x7fdf3402a270 comp rx=0 tx=0).stop 2024-09-12T01:18:23.050 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.330+0000 7fdf48800640 1 -- 172.21.15.1:0/2917500597 shutdown_connections 2024-09-12T01:18:23.050 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.330+0000 7fdf48800640 1 --2- 172.21.15.1:0/2917500597 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf4007dd30 0x7fdf4007c2a0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:23.050 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.330+0000 7fdf48800640 1 -- 172.21.15.1:0/2917500597 >> 172.21.15.1:0/2917500597 conn(0x7fdf40102e50 msgr2=0x7fdf40105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:18:23.050 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.331+0000 7fdf48800640 1 -- 172.21.15.1:0/2917500597 shutdown_connections 2024-09-12T01:18:23.050 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.331+0000 7fdf48800640 1 -- 172.21.15.1:0/2917500597 wait complete. 2024-09-12T01:18:23.050 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.331+0000 7fdf48800640 1 Processor -- start 2024-09-12T01:18:23.050 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.332+0000 7fdf48800640 1 -- start start 2024-09-12T01:18:23.050 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.332+0000 7fdf48800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf4007dd30 0x7fdf4019d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:18:23.050 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.332+0000 7fdf48800640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4010db80 con 0x7fdf4007dd30 2024-09-12T01:18:23.050 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.332+0000 7fdf45e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf4007dd30 0x7fdf4019d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:18:23.051 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.332+0000 7fdf45e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf4007dd30 0x7fdf4019d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54280/0 (socket says 172.21.15.1:54280) 2024-09-12T01:18:23.051 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.332+0000 7fdf45e00640 1 -- 172.21.15.1:0/1106044765 learned_addr learned my addr 172.21.15.1:0/1106044765 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:18:23.051 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.332+0000 7fdf45e00640 1 -- 172.21.15.1:0/1106044765 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf4019dd00 con 0x7fdf4007dd30 2024-09-12T01:18:23.051 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.333+0000 7fdf45e00640 1 --2- 172.21.15.1:0/1106044765 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf4007dd30 0x7fdf4019d7c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdf340097f0 tx=0x7fdf34030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:18:23.051 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.333+0000 7fdf3aa00640 1 -- 172.21.15.1:0/1106044765 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf34037040 con 0x7fdf4007dd30 2024-09-12T01:18:23.051 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.333+0000 7fdf3aa00640 1 -- 172.21.15.1:0/1106044765 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdf34040070 con 0x7fdf4007dd30 2024-09-12T01:18:23.051 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.333+0000 7fdf3aa00640 1 -- 172.21.15.1:0/1106044765 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf3403a390 con 0x7fdf4007dd30 2024-09-12T01:18:23.051 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.333+0000 7fdf48800640 1 -- 172.21.15.1:0/1106044765 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf4019df90 con 0x7fdf4007dd30 2024-09-12T01:18:23.051 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.333+0000 7fdf48800640 1 -- 172.21.15.1:0/1106044765 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf401a0c80 con 0x7fdf4007dd30 2024-09-12T01:18:23.051 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.334+0000 7fdf3aa00640 1 -- 172.21.15.1:0/1106044765 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdf3403a5f0 con 0x7fdf4007dd30 2024-09-12T01:18:23.051 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.334+0000 7fdf3aa00640 1 -- 172.21.15.1:0/1106044765 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdf34045960 con 0x7fdf4007dd30 2024-09-12T01:18:23.051 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.335+0000 7fdf48800640 1 -- 172.21.15.1:0/1106044765 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf4007c320 con 0x7fdf4007dd30 2024-09-12T01:18:23.051 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.337+0000 7fdf3aa00640 1 -- 172.21.15.1:0/1106044765 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fdf34045460 con 0x7fdf4007dd30 2024-09-12T01:18:23.052 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.400+0000 7fdf48800640 1 -- 172.21.15.1:0/1106044765 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fdf401a0e50 con 0x7fdf4007dd30 2024-09-12T01:18:23.052 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.400+0000 7fdf3aa00640 1 -- 172.21.15.1:0/1106044765 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+173 (secure 0 0 0) 0x7fdf3403b430 con 0x7fdf4007dd30 2024-09-12T01:18:23.052 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.401+0000 7fdf48800640 1 -- 172.21.15.1:0/1106044765 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf4007dd30 msgr2=0x7fdf4019d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:18:23.052 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.401+0000 7fdf48800640 1 --2- 172.21.15.1:0/1106044765 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf4007dd30 0x7fdf4019d7c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdf340097f0 tx=0x7fdf34030d80 comp rx=0 tx=0).stop 2024-09-12T01:18:23.052 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.401+0000 7fdf48800640 1 -- 172.21.15.1:0/1106044765 shutdown_connections 2024-09-12T01:18:23.052 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.401+0000 7fdf48800640 1 --2- 172.21.15.1:0/1106044765 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf4007dd30 0x7fdf4019d7c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:23.052 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.401+0000 7fdf48800640 1 -- 172.21.15.1:0/1106044765 >> 172.21.15.1:0/1106044765 conn(0x7fdf40102e50 msgr2=0x7fdf40103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:18:23.052 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.401+0000 7fdf48800640 1 -- 172.21.15.1:0/1106044765 shutdown_connections 2024-09-12T01:18:23.052 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:22.401+0000 7fdf48800640 1 -- 172.21.15.1:0/1106044765 wait complete. 2024-09-12T01:18:23.052 INFO:teuthology.orchestra.run.smithi001.stdout:Restarting the monitor... 2024-09-12T01:18:23.416 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:23 smithi001 systemd[1]: Stopping Ceph mon.a for c0e4f180-70a4-11ef-bcea-c7b262605968... 2024-09-12T01:18:23.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:23 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mon-a[30026]: 2024-09-12T01:18:23.416+0000 7f08bdc00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:18:23.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:23 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mon-a[30026]: 2024-09-12T01:18:23.416+0000 7f08bdc00640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-12T01:18:23.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:23 smithi001 podman[30267]: 2024-09-12 01:18:23.632535097 +0000 UTC m=+0.345019961 container died 564d21c06d1f7fa0036994ccc0191a4ddc146c97d91d6beff2d6e791c651eb4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T01:18:24.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:24 smithi001 podman[30267]: 2024-09-12 01:18:24.074833847 +0000 UTC m=+0.787318716 container cleanup 564d21c06d1f7fa0036994ccc0191a4ddc146c97d91d6beff2d6e791c651eb4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-12T01:18:24.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:24 smithi001 bash[30267]: ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mon-a 2024-09-12T01:18:24.548 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:24 smithi001 podman[30280]: 2024-09-12 01:18:24.285943535 +0000 UTC m=+0.641729090 container remove 564d21c06d1f7fa0036994ccc0191a4ddc146c97d91d6beff2d6e791c651eb4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-12T01:18:24.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:24 smithi001 systemd[1]: ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-12T01:18:24.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:24 smithi001 systemd[1]: Stopped Ceph mon.a for c0e4f180-70a4-11ef-bcea-c7b262605968. 2024-09-12T01:18:24.881 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:24 smithi001 systemd[1]: ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mon.a.service: Consumed 1.226s CPU time. 2024-09-12T01:18:24.881 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:24 smithi001 systemd[1]: Starting Ceph mon.a for c0e4f180-70a4-11ef-bcea-c7b262605968... 2024-09-12T01:18:25.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:24 smithi001 podman[30369]: 2024-09-12 01:18:24.786257379 +0000 UTC m=+0.028181155 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T01:18:25.908 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 podman[30369]: 2024-09-12 01:18:25.653769386 +0000 UTC m=+0.895693150 container create 72a843457ce67b7e125e8b9fb9e94d917efcc71d6d6ce00727dbb3f1b9e267b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mon-a, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T01:18:25.908 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 podman[30369]: 2024-09-12 01:18:25.864476503 +0000 UTC m=+1.106400279 container init 72a843457ce67b7e125e8b9fb9e94d917efcc71d6d6ce00727dbb3f1b9e267b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux ) 2024-09-12T01:18:25.908 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 podman[30369]: 2024-09-12 01:18:25.871731772 +0000 UTC m=+1.113655538 container start 72a843457ce67b7e125e8b9fb9e94d917efcc71d6d6ce00727dbb3f1b9e267b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-12T01:18:26.005 INFO:teuthology.orchestra.run.smithi001.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-12T01:18:26.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: set uid:gid to 167:167 (ceph:ceph) 2024-09-12T01:18:26.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable), process ceph-mon, pid 2 2024-09-12T01:18:26.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: pidfile_write: ignore empty --pid-file 2024-09-12T01:18:26.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: load: jerasure load: lrc 2024-09-12T01:18:26.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: RocksDB version: 7.9.2 2024-09-12T01:18:26.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Git sha 0 2024-09-12T01:18:26.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Compile date 2024-09-09 16:23:28 2024-09-12T01:18:26.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: DB SUMMARY 2024-09-12T01:18:26.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: DB Session ID: CYNRQC5CFIO6C29CUBUS 2024-09-12T01:18:26.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: CURRENT file: CURRENT 2024-09-12T01:18:26.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: IDENTITY file: IDENTITY 2024-09-12T01:18:26.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-12T01:18:26.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-12T01:18:26.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104804 ; 2024-09-12T01:18:26.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.error_if_exists: 0 2024-09-12T01:18:26.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.create_if_missing: 0 2024-09-12T01:18:26.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.paranoid_checks: 1 2024-09-12T01:18:26.180 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-12T01:18:26.180 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-12T01:18:26.180 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-12T01:18:26.180 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.env: 0x55f138a61940 2024-09-12T01:18:26.180 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.fs: PosixFileSystem 2024-09-12T01:18:26.180 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.info_log: 0x55f140f916a0 2024-09-12T01:18:26.180 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.max_file_opening_threads: 16 2024-09-12T01:18:26.180 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.statistics: (nil) 2024-09-12T01:18:26.180 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.use_fsync: 0 2024-09-12T01:18:26.180 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.max_log_file_size: 0 2024-09-12T01:18:26.180 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-12T01:18:26.180 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-12T01:18:26.180 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.keep_log_file_num: 1000 2024-09-12T01:18:26.181 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.recycle_log_file_num: 0 2024-09-12T01:18:26.181 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.allow_fallocate: 1 2024-09-12T01:18:26.181 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.allow_mmap_reads: 0 2024-09-12T01:18:26.181 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.allow_mmap_writes: 0 2024-09-12T01:18:26.181 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.use_direct_reads: 0 2024-09-12T01:18:26.181 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-12T01:18:26.181 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.create_missing_column_families: 0 2024-09-12T01:18:26.181 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.db_log_dir: 2024-09-12T01:18:26.181 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.wal_dir: 2024-09-12T01:18:26.181 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-12T01:18:26.181 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-12T01:18:26.181 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-12T01:18:26.181 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-12T01:18:26.181 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-12T01:18:26.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-12T01:18:26.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.advise_random_on_open: 1 2024-09-12T01:18:26.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.db_write_buffer_size: 0 2024-09-12T01:18:26.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.write_buffer_manager: 0x55f140fa0dc0 2024-09-12T01:18:26.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-12T01:18:26.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-12T01:18:26.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-12T01:18:26.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.rate_limiter: (nil) 2024-09-12T01:18:26.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-12T01:18:26.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.wal_recovery_mode: 2 2024-09-12T01:18:26.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.enable_thread_tracking: 0 2024-09-12T01:18:26.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.enable_pipelined_write: 0 2024-09-12T01:18:26.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.unordered_write: 0 2024-09-12T01:18:26.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-12T01:18:26.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-12T01:18:26.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-12T01:18:26.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-12T01:18:26.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.row_cache: None 2024-09-12T01:18:26.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.wal_filter: None 2024-09-12T01:18:26.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-12T01:18:26.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.allow_ingest_behind: 0 2024-09-12T01:18:26.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.two_write_queues: 0 2024-09-12T01:18:26.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.manual_wal_flush: 0 2024-09-12T01:18:26.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.wal_compression: 0 2024-09-12T01:18:26.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.atomic_flush: 0 2024-09-12T01:18:26.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-12T01:18:26.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-12T01:18:26.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-12T01:18:26.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.log_readahead_size: 0 2024-09-12T01:18:26.184 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-12T01:18:26.184 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.best_efforts_recovery: 0 2024-09-12T01:18:26.184 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-12T01:18:26.184 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-12T01:18:26.184 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.allow_data_in_errors: 0 2024-09-12T01:18:26.184 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.db_host_id: __hostname__ 2024-09-12T01:18:26.184 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-12T01:18:26.185 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.max_background_jobs: 2 2024-09-12T01:18:26.185 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.max_background_compactions: -1 2024-09-12T01:18:26.185 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.max_subcompactions: 1 2024-09-12T01:18:26.185 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-12T01:18:26.186 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-12T01:18:26.186 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-12T01:18:26.186 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.max_total_wal_size: 0 2024-09-12T01:18:26.186 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-12T01:18:26.186 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-12T01:18:26.186 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-12T01:18:26.186 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-12T01:18:26.186 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.max_open_files: -1 2024-09-12T01:18:26.186 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.bytes_per_sync: 0 2024-09-12T01:18:26.186 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-12T01:18:26.186 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-12T01:18:26.186 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.compaction_readahead_size: 0 2024-09-12T01:18:26.186 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.max_background_flushes: -1 2024-09-12T01:18:26.186 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Compression algorithms supported: 2024-09-12T01:18:26.187 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: kZSTD supported: 0 2024-09-12T01:18:26.187 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: kXpressCompression supported: 0 2024-09-12T01:18:26.187 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: kBZip2Compression supported: 0 2024-09-12T01:18:26.187 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-12T01:18:26.187 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: kLZ4Compression supported: 1 2024-09-12T01:18:26.187 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: kZlibCompression supported: 1 2024-09-12T01:18:26.187 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: kLZ4HCCompression supported: 1 2024-09-12T01:18:26.187 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: kSnappyCompression supported: 1 2024-09-12T01:18:26.187 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-12T01:18:26.187 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-12T01:18:26.187 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-12T01:18:26.187 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-12T01:18:26.187 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-12T01:18:26.187 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.merge_operator: 2024-09-12T01:18:26.188 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.compaction_filter: None 2024-09-12T01:18:26.188 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.compaction_filter_factory: None 2024-09-12T01:18:26.188 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.sst_partitioner_factory: None 2024-09-12T01:18:26.188 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-12T01:18:26.188 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-12T01:18:26.188 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f140f91300) 2024-09-12T01:18:26.188 INFO:journalctl@ceph.mon.a.smithi001.stdout: cache_index_and_filter_blocks: 1 2024-09-12T01:18:26.188 INFO:journalctl@ceph.mon.a.smithi001.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-12T01:18:26.188 INFO:journalctl@ceph.mon.a.smithi001.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-12T01:18:26.188 INFO:journalctl@ceph.mon.a.smithi001.stdout: pin_top_level_index_and_filter: 1 2024-09-12T01:18:26.188 INFO:journalctl@ceph.mon.a.smithi001.stdout: index_type: 0 2024-09-12T01:18:26.188 INFO:journalctl@ceph.mon.a.smithi001.stdout: data_block_index_type: 0 2024-09-12T01:18:26.188 INFO:journalctl@ceph.mon.a.smithi001.stdout: index_shortening: 1 2024-09-12T01:18:26.188 INFO:journalctl@ceph.mon.a.smithi001.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-12T01:18:26.189 INFO:journalctl@ceph.mon.a.smithi001.stdout: checksum: 4 2024-09-12T01:18:26.189 INFO:journalctl@ceph.mon.a.smithi001.stdout: no_block_cache: 0 2024-09-12T01:18:26.189 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_cache: 0x55f140f851f0 2024-09-12T01:18:26.189 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_cache_name: BinnedLRUCache 2024-09-12T01:18:26.189 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_cache_options: 2024-09-12T01:18:26.189 INFO:journalctl@ceph.mon.a.smithi001.stdout: capacity : 536870912 2024-09-12T01:18:26.189 INFO:journalctl@ceph.mon.a.smithi001.stdout: num_shard_bits : 4 2024-09-12T01:18:26.189 INFO:journalctl@ceph.mon.a.smithi001.stdout: strict_capacity_limit : 0 2024-09-12T01:18:26.189 INFO:journalctl@ceph.mon.a.smithi001.stdout: high_pri_pool_ratio: 0.000 2024-09-12T01:18:26.189 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_cache_compressed: (nil) 2024-09-12T01:18:26.189 INFO:journalctl@ceph.mon.a.smithi001.stdout: persistent_cache: (nil) 2024-09-12T01:18:26.189 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_size: 4096 2024-09-12T01:18:26.189 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_size_deviation: 10 2024-09-12T01:18:26.189 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_restart_interval: 16 2024-09-12T01:18:26.190 INFO:journalctl@ceph.mon.a.smithi001.stdout: index_block_restart_interval: 1 2024-09-12T01:18:26.190 INFO:journalctl@ceph.mon.a.smithi001.stdout: metadata_block_size: 4096 2024-09-12T01:18:26.190 INFO:journalctl@ceph.mon.a.smithi001.stdout: partition_filters: 0 2024-09-12T01:18:26.190 INFO:journalctl@ceph.mon.a.smithi001.stdout: use_delta_encoding: 1 2024-09-12T01:18:26.190 INFO:journalctl@ceph.mon.a.smithi001.stdout: filter_policy: bloomfilter 2024-09-12T01:18:26.190 INFO:journalctl@ceph.mon.a.smithi001.stdout: whole_key_filtering: 1 2024-09-12T01:18:26.190 INFO:journalctl@ceph.mon.a.smithi001.stdout: verify_compression: 0 2024-09-12T01:18:26.190 INFO:journalctl@ceph.mon.a.smithi001.stdout: read_amp_bytes_per_bit: 0 2024-09-12T01:18:26.190 INFO:journalctl@ceph.mon.a.smithi001.stdout: format_version: 5 2024-09-12T01:18:26.190 INFO:journalctl@ceph.mon.a.smithi001.stdout: enable_index_compression: 1 2024-09-12T01:18:26.190 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_align: 0 2024-09-12T01:18:26.190 INFO:journalctl@ceph.mon.a.smithi001.stdout: max_auto_readahead_size: 262144 2024-09-12T01:18:26.190 INFO:journalctl@ceph.mon.a.smithi001.stdout: prepopulate_block_cache: 0 2024-09-12T01:18:26.190 INFO:journalctl@ceph.mon.a.smithi001.stdout: initial_auto_readahead_size: 8192 2024-09-12T01:18:26.191 INFO:journalctl@ceph.mon.a.smithi001.stdout: num_file_reads_for_auto_readahead: 2 2024-09-12T01:18:26.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.write_buffer_size: 33554432 2024-09-12T01:18:26.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.max_write_buffer_number: 2 2024-09-12T01:18:26.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.compression: NoCompression 2024-09-12T01:18:26.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.bottommost_compression: Disabled 2024-09-12T01:18:26.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.prefix_extractor: nullptr 2024-09-12T01:18:26.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-12T01:18:26.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.num_levels: 7 2024-09-12T01:18:26.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-12T01:18:26.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-12T01:18:26.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-12T01:18:26.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-12T01:18:26.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-12T01:18:26.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-12T01:18:26.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-12T01:18:26.192 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-12T01:18:26.192 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-12T01:18:26.192 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-12T01:18:26.192 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-12T01:18:26.192 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-12T01:18:26.192 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-12T01:18:26.192 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.compression_opts.level: 32767 2024-09-12T01:18:26.192 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.compression_opts.strategy: 0 2024-09-12T01:18:26.192 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-12T01:18:26.192 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-12T01:18:26.192 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-12T01:18:26.192 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-12T01:18:26.192 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.compression_opts.enabled: false 2024-09-12T01:18:26.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-12T01:18:26.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-12T01:18:26.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-12T01:18:26.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-12T01:18:26.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.target_file_size_base: 67108864 2024-09-12T01:18:26.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-12T01:18:26.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-12T01:18:26.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-12T01:18:26.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-12T01:18:26.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-12T01:18:26.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-12T01:18:26.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-12T01:18:26.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-12T01:18:26.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-12T01:18:26.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-12T01:18:26.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-12T01:18:26.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-12T01:18:26.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-12T01:18:26.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-12T01:18:26.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.arena_block_size: 1048576 2024-09-12T01:18:26.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-12T01:18:26.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-12T01:18:26.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.disable_auto_compactions: 0 2024-09-12T01:18:26.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-12T01:18:26.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-12T01:18:26.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-12T01:18:26.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-12T01:18:26.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-12T01:18:26.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-12T01:18:26.195 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-12T01:18:26.195 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-12T01:18:26.195 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-12T01:18:26.195 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-12T01:18:26.195 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-12T01:18:26.195 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.inplace_update_support: 0 2024-09-12T01:18:26.195 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-12T01:18:26.195 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-12T01:18:26.195 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-12T01:18:26.195 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-12T01:18:26.195 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.bloom_locality: 0 2024-09-12T01:18:26.195 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.max_successive_merges: 0 2024-09-12T01:18:26.195 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-12T01:18:26.195 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.paranoid_file_checks: 0 2024-09-12T01:18:26.196 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.force_consistency_checks: 1 2024-09-12T01:18:26.196 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.report_bg_io_stats: 0 2024-09-12T01:18:26.196 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.ttl: 2592000 2024-09-12T01:18:26.196 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-12T01:18:26.196 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-12T01:18:26.196 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-12T01:18:26.196 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.enable_blob_files: false 2024-09-12T01:18:26.196 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.min_blob_size: 0 2024-09-12T01:18:26.196 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.blob_file_size: 268435456 2024-09-12T01:18:26.196 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-12T01:18:26.196 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-12T01:18:26.196 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-12T01:18:26.196 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-12T01:18:26.196 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-12T01:18:26.196 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.blob_file_starting_level: 0 2024-09-12T01:18:26.197 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-12T01:18:26.197 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-12T01:18:26.197 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-12T01:18:26.197 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: dd7a765f-303e-4b33-be9e-52ee11e9f75a 2024-09-12T01:18:26.197 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726103905917705, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-12T01:18:26.197 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-12T01:18:26.197 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726103905919620, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99940, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 97824, "index_size": 308, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14189, "raw_average_key_size": 49, "raw_value_size": 90120, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726103905, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "dd7a765f-303e-4b33-be9e-52ee11e9f75a", "db_session_id": "CYNRQC5CFIO6C29CUBUS", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-12T01:18:26.197 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726103905919808, "job": 1, "event": "recovery_finished"} 2024-09-12T01:18:26.197 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-12T01:18:26.197 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-12T01:18:26.197 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f140fb4e00 2024-09-12T01:18:26.197 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: DB pointer 0x55f1410b0000 2024-09-12T01:18:26.197 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-12T01:18:26.198 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-12T01:18:26.198 INFO:journalctl@ceph.mon.a.smithi001.stdout: ** DB Stats ** 2024-09-12T01:18:26.198 INFO:journalctl@ceph.mon.a.smithi001.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T01:18:26.198 INFO:journalctl@ceph.mon.a.smithi001.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-12T01:18:26.198 INFO:journalctl@ceph.mon.a.smithi001.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T01:18:26.198 INFO:journalctl@ceph.mon.a.smithi001.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T01:18:26.198 INFO:journalctl@ceph.mon.a.smithi001.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-12T01:18:26.198 INFO:journalctl@ceph.mon.a.smithi001.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T01:18:26.198 INFO:journalctl@ceph.mon.a.smithi001.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T01:18:26.198 INFO:journalctl@ceph.mon.a.smithi001.stdout: 2024-09-12T01:18:26.198 INFO:journalctl@ceph.mon.a.smithi001.stdout: ** Compaction Stats [default] ** 2024-09-12T01:18:26.198 INFO:journalctl@ceph.mon.a.smithi001.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T01:18:26.198 INFO:journalctl@ceph.mon.a.smithi001.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-12T01:18:26.198 INFO:journalctl@ceph.mon.a.smithi001.stdout: L0 2/0 99.46 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 76.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T01:18:26.198 INFO:journalctl@ceph.mon.a.smithi001.stdout: Sum 2/0 99.46 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 76.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T01:18:26.199 INFO:journalctl@ceph.mon.a.smithi001.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 76.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T01:18:26.199 INFO:journalctl@ceph.mon.a.smithi001.stdout: 2024-09-12T01:18:26.199 INFO:journalctl@ceph.mon.a.smithi001.stdout: ** Compaction Stats [default] ** 2024-09-12T01:18:26.199 INFO:journalctl@ceph.mon.a.smithi001.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T01:18:26.199 INFO:journalctl@ceph.mon.a.smithi001.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-12T01:18:26.199 INFO:journalctl@ceph.mon.a.smithi001.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 76.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T01:18:26.199 INFO:journalctl@ceph.mon.a.smithi001.stdout: 2024-09-12T01:18:26.199 INFO:journalctl@ceph.mon.a.smithi001.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-12T01:18:26.199 INFO:journalctl@ceph.mon.a.smithi001.stdout: 2024-09-12T01:18:26.199 INFO:journalctl@ceph.mon.a.smithi001.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T01:18:26.199 INFO:journalctl@ceph.mon.a.smithi001.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-12T01:18:26.199 INFO:journalctl@ceph.mon.a.smithi001.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-12T01:18:26.199 INFO:journalctl@ceph.mon.a.smithi001.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-12T01:18:26.199 INFO:journalctl@ceph.mon.a.smithi001.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-12T01:18:26.199 INFO:journalctl@ceph.mon.a.smithi001.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-12T01:18:26.200 INFO:journalctl@ceph.mon.a.smithi001.stdout: Cumulative compaction: 0.00 GB write, 14.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T01:18:26.200 INFO:journalctl@ceph.mon.a.smithi001.stdout: Interval compaction: 0.00 GB write, 14.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T01:18:26.200 INFO:journalctl@ceph.mon.a.smithi001.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-12T01:18:26.200 INFO:journalctl@ceph.mon.a.smithi001.stdout: Block cache BinnedLRUCache@0x55f140f851f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-09-12T01:18:26.200 INFO:journalctl@ceph.mon.a.smithi001.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-12T01:18:26.200 INFO:journalctl@ceph.mon.a.smithi001.stdout: 2024-09-12T01:18:26.200 INFO:journalctl@ceph.mon.a.smithi001.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-12T01:18:26.200 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: starting mon.a rank 0 at public addrs [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] at bind addrs [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid c0e4f180-70a4-11ef-bcea-c7b262605968 2024-09-12T01:18:26.200 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: mon.a@-1(???) e1 preinit fsid c0e4f180-70a4-11ef-bcea-c7b262605968 2024-09-12T01:18:26.200 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: mon.a@-1(???).mds e1 new map 2024-09-12T01:18:26.200 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: mon.a@-1(???).mds e1 print_map 2024-09-12T01:18:26.200 INFO:journalctl@ceph.mon.a.smithi001.stdout: e1 2024-09-12T01:18:26.200 INFO:journalctl@ceph.mon.a.smithi001.stdout: btime 2024-09-12T01:18:19:337176+0000 2024-09-12T01:18:26.200 INFO:journalctl@ceph.mon.a.smithi001.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T01:18:26.201 INFO:journalctl@ceph.mon.a.smithi001.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T01:18:26.201 INFO:journalctl@ceph.mon.a.smithi001.stdout: legacy client fscid: -1 2024-09-12T01:18:26.201 INFO:journalctl@ceph.mon.a.smithi001.stdout: 2024-09-12T01:18:26.201 INFO:journalctl@ceph.mon.a.smithi001.stdout: No filesystems configured 2024-09-12T01:18:26.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-12T01:18:26.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T01:18:26.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T01:18:26.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T01:18:26.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-12T01:18:26.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-12T01:18:26.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: monmap epoch 1 2024-09-12T01:18:26.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: fsid c0e4f180-70a4-11ef-bcea-c7b262605968 2024-09-12T01:18:26.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: last_changed 2024-09-12T01:18:15.215734+0000 2024-09-12T01:18:26.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: created 2024-09-12T01:18:15.215734+0000 2024-09-12T01:18:26.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: min_mon_release 19 (squid) 2024-09-12T01:18:26.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: election_strategy: 1 2024-09-12T01:18:26.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-09-12T01:18:26.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: fsmap 2024-09-12T01:18:26.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: osdmap e1: 0 total, 0 up, 0 in 2024-09-12T01:18:26.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 ceph-mon[30387]: mgrmap e1: no daemons active 2024-09-12T01:18:26.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:25 smithi001 bash[30369]: 72a843457ce67b7e125e8b9fb9e94d917efcc71d6d6ce00727dbb3f1b9e267b2 2024-09-12T01:18:26.202 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:26 smithi001 systemd[1]: Started Ceph mon.a for c0e4f180-70a4-11ef-bcea-c7b262605968. 2024-09-12T01:18:27.284 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.487+0000 7fbeb8200640 1 Processor -- start 2024-09-12T01:18:27.289 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.487+0000 7fbeb8200640 1 -- start start 2024-09-12T01:18:27.289 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.488+0000 7fbeb8200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbeb0107600 0x7fbeb0107a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:18:27.289 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.488+0000 7fbeb8200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb0107f40 con 0x7fbeb0107600 2024-09-12T01:18:27.289 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.489+0000 7fbeb5800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbeb0107600 0x7fbeb0107a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:18:27.289 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.489+0000 7fbeb5800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbeb0107600 0x7fbeb0107a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54290/0 (socket says 172.21.15.1:54290) 2024-09-12T01:18:27.289 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.489+0000 7fbeb5800640 1 -- 172.21.15.1:0/1874146117 learned_addr learned my addr 172.21.15.1:0/1874146117 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:18:27.289 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.489+0000 7fbeb5800640 1 -- 172.21.15.1:0/1874146117 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb01080c0 con 0x7fbeb0107600 2024-09-12T01:18:27.289 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.492+0000 7fbeb5800640 1 --2- 172.21.15.1:0/1874146117 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbeb0107600 0x7fbeb0107a00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbea4009820 tx=0x7fbea402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c512ed66a6f76dc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:18:27.289 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.492+0000 7fbeafe00640 1 -- 172.21.15.1:0/1874146117 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbea4037070 con 0x7fbeb0107600 2024-09-12T01:18:27.289 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.492+0000 7fbeafe00640 1 -- 172.21.15.1:0/1874146117 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbea4032900 con 0x7fbeb0107600 2024-09-12T01:18:27.289 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.492+0000 7fbeafe00640 1 -- 172.21.15.1:0/1874146117 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbea4030850 con 0x7fbeb0107600 2024-09-12T01:18:27.290 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.493+0000 7fbeb8200640 1 -- 172.21.15.1:0/1874146117 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbeb0107600 msgr2=0x7fbeb0107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:18:27.290 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.493+0000 7fbeb8200640 1 --2- 172.21.15.1:0/1874146117 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbeb0107600 0x7fbeb0107a00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbea4009820 tx=0x7fbea402a270 comp rx=0 tx=0).stop 2024-09-12T01:18:27.290 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.493+0000 7fbeb8200640 1 -- 172.21.15.1:0/1874146117 shutdown_connections 2024-09-12T01:18:27.290 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.493+0000 7fbeb8200640 1 --2- 172.21.15.1:0/1874146117 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbeb0107600 0x7fbeb0107a00 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:27.290 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.493+0000 7fbeb8200640 1 -- 172.21.15.1:0/1874146117 >> 172.21.15.1:0/1874146117 conn(0x7fbeb0102e50 msgr2=0x7fbeb0105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:18:27.290 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.493+0000 7fbeb8200640 1 -- 172.21.15.1:0/1874146117 shutdown_connections 2024-09-12T01:18:27.290 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.493+0000 7fbeb8200640 1 -- 172.21.15.1:0/1874146117 wait complete. 2024-09-12T01:18:27.290 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.494+0000 7fbeb8200640 1 Processor -- start 2024-09-12T01:18:27.290 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.494+0000 7fbeb8200640 1 -- start start 2024-09-12T01:18:27.290 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.494+0000 7fbeb8200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbeb0107600 0x7fbeb007db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:18:27.290 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.494+0000 7fbeb8200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb010ba70 con 0x7fbeb0107600 2024-09-12T01:18:27.290 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.494+0000 7fbeb5800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbeb0107600 0x7fbeb007db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:18:27.290 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.494+0000 7fbeb5800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbeb0107600 0x7fbeb007db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54302/0 (socket says 172.21.15.1:54302) 2024-09-12T01:18:27.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.494+0000 7fbeb5800640 1 -- 172.21.15.1:0/2191082342 learned_addr learned my addr 172.21.15.1:0/2191082342 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:18:27.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.495+0000 7fbeb5800640 1 -- 172.21.15.1:0/2191082342 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb007e050 con 0x7fbeb0107600 2024-09-12T01:18:27.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.495+0000 7fbeb5800640 1 --2- 172.21.15.1:0/2191082342 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbeb0107600 0x7fbeb007db10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fbea40097f0 tx=0x7fbea4030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:18:27.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.495+0000 7fbeae000640 1 -- 172.21.15.1:0/2191082342 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbea4037040 con 0x7fbeb0107600 2024-09-12T01:18:27.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.495+0000 7fbeae000640 1 -- 172.21.15.1:0/2191082342 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbea4040070 con 0x7fbeb0107600 2024-09-12T01:18:27.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.495+0000 7fbeb8200640 1 -- 172.21.15.1:0/2191082342 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeb007c240 con 0x7fbeb0107600 2024-09-12T01:18:27.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.495+0000 7fbeae000640 1 -- 172.21.15.1:0/2191082342 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbea403a390 con 0x7fbeb0107600 2024-09-12T01:18:27.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.496+0000 7fbeb8200640 1 -- 172.21.15.1:0/2191082342 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeb007c720 con 0x7fbeb0107600 2024-09-12T01:18:27.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.496+0000 7fbeae000640 1 -- 172.21.15.1:0/2191082342 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbea403a630 con 0x7fbeb0107600 2024-09-12T01:18:27.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.496+0000 7fbeae000640 1 -- 172.21.15.1:0/2191082342 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbea40459a0 con 0x7fbeb0107600 2024-09-12T01:18:27.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.497+0000 7fbeb8200640 1 -- 172.21.15.1:0/2191082342 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeb0107a80 con 0x7fbeb0107600 2024-09-12T01:18:27.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.500+0000 7fbeae000640 1 -- 172.21.15.1:0/2191082342 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fbea4045460 con 0x7fbeb0107600 2024-09-12T01:18:27.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.562+0000 7fbeb8200640 1 -- 172.21.15.1:0/2191082342 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fbeb007cab0 con 0x7fbeb0107600 2024-09-12T01:18:27.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.564+0000 7fbeae000640 1 -- 172.21.15.1:0/2191082342 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fbea403b430 con 0x7fbeb0107600 2024-09-12T01:18:27.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.566+0000 7fbeb8200640 1 -- 172.21.15.1:0/2191082342 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbeb0107600 msgr2=0x7fbeb007db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:18:27.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.566+0000 7fbeb8200640 1 --2- 172.21.15.1:0/2191082342 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbeb0107600 0x7fbeb007db10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fbea40097f0 tx=0x7fbea4030d80 comp rx=0 tx=0).stop 2024-09-12T01:18:27.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.566+0000 7fbeb8200640 1 -- 172.21.15.1:0/2191082342 shutdown_connections 2024-09-12T01:18:27.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.566+0000 7fbeb8200640 1 --2- 172.21.15.1:0/2191082342 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbeb0107600 0x7fbeb007db10 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:27.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.566+0000 7fbeb8200640 1 -- 172.21.15.1:0/2191082342 >> 172.21.15.1:0/2191082342 conn(0x7fbeb0102e50 msgr2=0x7fbeb0103960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:18:27.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.566+0000 7fbeb8200640 1 -- 172.21.15.1:0/2191082342 shutdown_connections 2024-09-12T01:18:27.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:26.566+0000 7fbeb8200640 1 -- 172.21.15.1:0/2191082342 wait complete. 2024-09-12T01:18:27.292 INFO:teuthology.orchestra.run.smithi001.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-12T01:18:27.319 INFO:teuthology.orchestra.run.smithi001.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-12T01:18:27.319 INFO:teuthology.orchestra.run.smithi001.stdout:Creating mgr... 2024-09-12T01:18:27.319 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-12T01:18:27.320 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-12T01:18:27.684 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mgr.x 2024-09-12T01:18:27.684 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mgr.x.service: Unit ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mgr.x.service not loaded. 2024-09-12T01:18:27.917 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c0e4f180-70a4-11ef-bcea-c7b262605968.target.wants/ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@.service. 2024-09-12T01:18:27.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:27 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/2191082342' entity='client.admin' 2024-09-12T01:18:28.278 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:27 smithi001 systemd[1]: Starting Ceph mgr.x for c0e4f180-70a4-11ef-bcea-c7b262605968... 2024-09-12T01:18:28.616 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:28 smithi001 podman[30596]: 2024-09-12 01:18:28.183617265 +0000 UTC m=+0.041356695 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T01:18:28.616 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:28 smithi001 podman[30596]: 2024-09-12 01:18:28.298115144 +0000 UTC m=+0.155854578 container create 37d4931d977d617b131dfa8fdec6437092701d746dec92db2ab59df09b2d5822 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-12T01:18:28.616 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:28 smithi001 podman[30596]: 2024-09-12 01:18:28.492012813 +0000 UTC m=+0.349752240 container init 37d4931d977d617b131dfa8fdec6437092701d746dec92db2ab59df09b2d5822 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3) 2024-09-12T01:18:28.616 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:28 smithi001 podman[30596]: 2024-09-12 01:18:28.500464976 +0000 UTC m=+0.358204404 container start 37d4931d977d617b131dfa8fdec6437092701d746dec92db2ab59df09b2d5822 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True) 2024-09-12T01:18:28.616 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:28 smithi001 bash[30596]: 37d4931d977d617b131dfa8fdec6437092701d746dec92db2ab59df09b2d5822 2024-09-12T01:18:28.645 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T01:18:28.645 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout disabled 2024-09-12T01:18:28.659 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T01:18:28.659 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout inactive 2024-09-12T01:18:28.659 INFO:teuthology.orchestra.run.smithi001.stdout:firewalld.service is not enabled 2024-09-12T01:18:28.659 INFO:teuthology.orchestra.run.smithi001.stdout:Not possible to enable service . firewalld.service is not available 2024-09-12T01:18:28.678 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T01:18:28.678 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout disabled 2024-09-12T01:18:28.692 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T01:18:28.693 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout inactive 2024-09-12T01:18:28.693 INFO:teuthology.orchestra.run.smithi001.stdout:firewalld.service is not enabled 2024-09-12T01:18:28.693 INFO:teuthology.orchestra.run.smithi001.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-12T01:18:28.693 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for mgr to start... 2024-09-12T01:18:28.693 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for mgr... 2024-09-12T01:18:28.898 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:28 smithi001 systemd[1]: Started Ceph mgr.x for c0e4f180-70a4-11ef-bcea-c7b262605968. 2024-09-12T01:18:28.898 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:28 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:28.685+0000 7faa245da040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T01:18:28.898 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:28 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:28.763+0000 7faa245da040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T01:18:29.171 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:28 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:28.963+0000 7faa245da040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T01:18:29.171 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:29 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:29.050+0000 7faa245da040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T01:18:29.427 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:29 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:29.237+0000 7faa245da040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T01:18:29.427 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:29 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:29.419+0000 7faa245da040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T01:18:29.765 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:29 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/3543709457' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-12T01:18:30.107 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:29 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:29.823+0000 7faa245da040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T01:18:30.107 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:29 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:29.898+0000 7faa245da040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T01:18:30.115 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-09-12T01:18:30.117 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-09-12T01:18:30.118 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsid": "c0e4f180-70a4-11ef-bcea-c7b262605968", 2024-09-12T01:18:30.118 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "health": { 2024-09-12T01:18:30.118 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-12T01:18:30.118 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-12T01:18:30.118 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-12T01:18:30.118 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:18:30.118 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-12T01:18:30.118 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-12T01:18:30.118 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 0 2024-09-12T01:18:30.118 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-09-12T01:18:30.118 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-12T01:18:30.119 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "a" 2024-09-12T01:18:30.119 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-09-12T01:18:30.119 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-09-12T01:18:30.119 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-12T01:18:30.119 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T01:18:30.119 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-12T01:18:30.119 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-12T01:18:30.119 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:18:30.119 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-12T01:18:30.119 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T01:18:30.119 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-12T01:18:30.119 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-12T01:18:30.128 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-12T01:18:30.128 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-12T01:18:30.128 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-12T01:18:30.128 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-12T01:18:30.128 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:18:30.128 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-12T01:18:30.128 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-12T01:18:30.128 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-12T01:18:30.128 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-12T01:18:30.128 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-12T01:18:30.129 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-12T01:18:30.129 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-12T01:18:30.129 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-12T01:18:30.129 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-12T01:18:30.129 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:18:30.129 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-12T01:18:30.130 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T01:18:30.130 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "btime": "2024-09-12T01:18:19:337176+0000", 2024-09-12T01:18:30.130 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-12T01:18:30.130 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-12T01:18:30.130 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:18:30.130 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-12T01:18:30.130 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-12T01:18:30.131 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-12T01:18:30.131 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-12T01:18:30.131 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-12T01:18:30.131 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-12T01:18:30.131 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "restful" 2024-09-12T01:18:30.131 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-09-12T01:18:30.132 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T01:18:30.132 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:18:30.132 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-12T01:18:30.132 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T01:18:30.132 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modified": "2024-09-12T01:18:19.337976+0000", 2024-09-12T01:18:30.132 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T01:18:30.133 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:18:30.133 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-12T01:18:30.133 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-09-12T01:18:30.133 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.283+0000 7f0997e00640 1 Processor -- start 2024-09-12T01:18:30.133 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.284+0000 7f0997e00640 1 -- start start 2024-09-12T01:18:30.134 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.284+0000 7f0997e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09981062c0 0x7f09981066c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:18:30.134 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.284+0000 7f0997e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0998106c90 con 0x7f09981062c0 2024-09-12T01:18:30.134 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.285+0000 7f0996a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09981062c0 0x7f09981066c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:18:30.134 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.285+0000 7f0996a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09981062c0 0x7f09981066c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44464/0 (socket says 172.21.15.1:44464) 2024-09-12T01:18:30.134 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.285+0000 7f0996a00640 1 -- 172.21.15.1:0/4101534513 learned_addr learned my addr 172.21.15.1:0/4101534513 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:18:30.134 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.285+0000 7f0996a00640 1 -- 172.21.15.1:0/4101534513 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09981074c0 con 0x7f09981062c0 2024-09-12T01:18:30.135 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.287+0000 7f0996a00640 1 --2- 172.21.15.1:0/4101534513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09981062c0 0x7f09981066c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0988009820 tx=0x7f098802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=376c97cf17138afc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:18:30.135 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.287+0000 7f0995600640 1 -- 172.21.15.1:0/4101534513 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0988037070 con 0x7f09981062c0 2024-09-12T01:18:30.135 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.287+0000 7f0995600640 1 -- 172.21.15.1:0/4101534513 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0988032900 con 0x7f09981062c0 2024-09-12T01:18:30.135 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.287+0000 7f0995600640 1 -- 172.21.15.1:0/4101534513 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0988030850 con 0x7f09981062c0 2024-09-12T01:18:30.135 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.288+0000 7f0997e00640 1 -- 172.21.15.1:0/4101534513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09981062c0 msgr2=0x7f09981066c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:18:30.135 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.288+0000 7f0997e00640 1 --2- 172.21.15.1:0/4101534513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09981062c0 0x7f09981066c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0988009820 tx=0x7f098802a270 comp rx=0 tx=0).stop 2024-09-12T01:18:30.135 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.288+0000 7f0997e00640 1 -- 172.21.15.1:0/4101534513 shutdown_connections 2024-09-12T01:18:30.135 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.288+0000 7f0997e00640 1 --2- 172.21.15.1:0/4101534513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09981062c0 0x7f09981066c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:30.135 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.288+0000 7f0997e00640 1 -- 172.21.15.1:0/4101534513 >> 172.21.15.1:0/4101534513 conn(0x7f0998101a90 msgr2=0x7f0998103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:18:30.135 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.288+0000 7f0997e00640 1 -- 172.21.15.1:0/4101534513 shutdown_connections 2024-09-12T01:18:30.135 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.288+0000 7f0997e00640 1 -- 172.21.15.1:0/4101534513 wait complete. 2024-09-12T01:18:30.135 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.289+0000 7f0997e00640 1 Processor -- start 2024-09-12T01:18:30.136 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.289+0000 7f0997e00640 1 -- start start 2024-09-12T01:18:30.136 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.289+0000 7f0997e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09981062c0 0x7f09981914f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:18:30.136 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.289+0000 7f0997e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f099810a870 con 0x7f09981062c0 2024-09-12T01:18:30.136 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.290+0000 7f0996a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09981062c0 0x7f09981914f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:18:30.136 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.290+0000 7f0996a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09981062c0 0x7f09981914f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44476/0 (socket says 172.21.15.1:44476) 2024-09-12T01:18:30.136 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.290+0000 7f0996a00640 1 -- 172.21.15.1:0/3543709457 learned_addr learned my addr 172.21.15.1:0/3543709457 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:18:30.136 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.290+0000 7f0996a00640 1 -- 172.21.15.1:0/3543709457 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0998191a30 con 0x7f09981062c0 2024-09-12T01:18:30.136 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.291+0000 7f0996a00640 1 --2- 172.21.15.1:0/3543709457 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09981062c0 0x7f09981914f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0988009940 tx=0x7f0988032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:18:30.136 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.291+0000 7f099ca00640 1 -- 172.21.15.1:0/3543709457 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0988042070 con 0x7f09981062c0 2024-09-12T01:18:30.136 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.291+0000 7f099ca00640 1 -- 172.21.15.1:0/3543709457 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f09880308c0 con 0x7f09981062c0 2024-09-12T01:18:30.136 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.291+0000 7f099ca00640 1 -- 172.21.15.1:0/3543709457 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0988037040 con 0x7f09981062c0 2024-09-12T01:18:30.136 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.292+0000 7f0997e00640 1 -- 172.21.15.1:0/3543709457 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f099818fc20 con 0x7f09981062c0 2024-09-12T01:18:30.136 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.292+0000 7f0997e00640 1 -- 172.21.15.1:0/3543709457 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0998190160 con 0x7f09981062c0 2024-09-12T01:18:30.137 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.292+0000 7f099ca00640 1 -- 172.21.15.1:0/3543709457 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0988045070 con 0x7f09981062c0 2024-09-12T01:18:30.137 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.292+0000 7f099ca00640 1 -- 172.21.15.1:0/3543709457 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0988047440 con 0x7f09981062c0 2024-09-12T01:18:30.137 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.293+0000 7f098d600640 1 -- 172.21.15.1:0/3543709457 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0970005290 con 0x7f09981062c0 2024-09-12T01:18:30.137 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.298+0000 7f099ca00640 1 -- 172.21.15.1:0/3543709457 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f098803d430 con 0x7f09981062c0 2024-09-12T01:18:30.137 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.384+0000 7f098d600640 1 -- 172.21.15.1:0/3543709457 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f0970005850 con 0x7f09981062c0 2024-09-12T01:18:30.137 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.385+0000 7f099ca00640 1 -- 172.21.15.1:0/3543709457 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f098803c870 con 0x7f09981062c0 2024-09-12T01:18:30.137 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.386+0000 7f098d600640 1 -- 172.21.15.1:0/3543709457 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09981062c0 msgr2=0x7f09981914f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:18:30.137 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.386+0000 7f098d600640 1 --2- 172.21.15.1:0/3543709457 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09981062c0 0x7f09981914f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0988009940 tx=0x7f0988032ce0 comp rx=0 tx=0).stop 2024-09-12T01:18:30.137 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.386+0000 7f098d600640 1 -- 172.21.15.1:0/3543709457 shutdown_connections 2024-09-12T01:18:30.137 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.386+0000 7f098d600640 1 --2- 172.21.15.1:0/3543709457 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09981062c0 0x7f09981914f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:30.137 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.386+0000 7f098d600640 1 -- 172.21.15.1:0/3543709457 >> 172.21.15.1:0/3543709457 conn(0x7f0998101a90 msgr2=0x7f0998102990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:18:30.137 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.387+0000 7f098d600640 1 -- 172.21.15.1:0/3543709457 shutdown_connections 2024-09-12T01:18:30.137 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:29.387+0000 7f098d600640 1 -- 172.21.15.1:0/3543709457 wait complete. 2024-09-12T01:18:30.137 INFO:teuthology.orchestra.run.smithi001.stdout:mgr not available, waiting (1/15)... 2024-09-12T01:18:30.430 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:30 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:30.127+0000 7faa245da040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T01:18:31.677 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:31 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:31.274+0000 7faa245da040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T01:18:31.677 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:31 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:31.358+0000 7faa245da040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T01:18:32.635 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:32 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:32.351+0000 7faa245da040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T01:18:32.635 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:32 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:32.497+0000 7faa245da040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T01:18:32.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:32 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/2792709420' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-12T01:18:32.928 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:32 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:32.672+0000 7faa245da040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T01:18:32.928 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:32 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:32.750+0000 7faa245da040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T01:18:33.227 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:32 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:32.940+0000 7faa245da040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T01:18:33.227 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:33 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:33.027+0000 7faa245da040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T01:18:33.227 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:33 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:33.101+0000 7faa245da040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T01:18:33.485 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-09-12T01:18:33.487 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-09-12T01:18:33.487 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsid": "c0e4f180-70a4-11ef-bcea-c7b262605968", 2024-09-12T01:18:33.487 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "health": { 2024-09-12T01:18:33.487 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-12T01:18:33.487 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-12T01:18:33.487 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-12T01:18:33.488 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:18:33.488 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-12T01:18:33.488 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-12T01:18:33.488 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 0 2024-09-12T01:18:33.488 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-09-12T01:18:33.488 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-12T01:18:33.488 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "a" 2024-09-12T01:18:33.488 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-09-12T01:18:33.488 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-09-12T01:18:33.488 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-12T01:18:33.488 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T01:18:33.488 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-12T01:18:33.488 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-12T01:18:33.488 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:18:33.489 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-12T01:18:33.489 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T01:18:33.489 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-12T01:18:33.489 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-12T01:18:33.489 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-12T01:18:33.496 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-12T01:18:33.496 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-12T01:18:33.496 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-12T01:18:33.497 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:18:33.497 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-12T01:18:33.497 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-12T01:18:33.497 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-12T01:18:33.497 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-12T01:18:33.497 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-12T01:18:33.497 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-12T01:18:33.497 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-12T01:18:33.497 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-12T01:18:33.497 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-12T01:18:33.497 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:18:33.497 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-12T01:18:33.497 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T01:18:33.498 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "btime": "2024-09-12T01:18:19:337176+0000", 2024-09-12T01:18:33.498 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-12T01:18:33.498 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-12T01:18:33.498 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:18:33.498 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-12T01:18:33.498 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-12T01:18:33.498 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-12T01:18:33.498 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-12T01:18:33.498 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-12T01:18:33.498 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-12T01:18:33.498 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "restful" 2024-09-12T01:18:33.498 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-09-12T01:18:33.498 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T01:18:33.498 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:18:33.498 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-12T01:18:33.499 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T01:18:33.499 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modified": "2024-09-12T01:18:19.337976+0000", 2024-09-12T01:18:33.499 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T01:18:33.499 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:18:33.499 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-12T01:18:33.499 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-09-12T01:18:33.499 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.745+0000 7f54db600640 1 Processor -- start 2024-09-12T01:18:33.499 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.745+0000 7f54db600640 1 -- start start 2024-09-12T01:18:33.499 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.745+0000 7f54db600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54d41098c0 0x7f54d4109cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:18:33.499 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.745+0000 7f54db600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f54d410a290 con 0x7f54d41098c0 2024-09-12T01:18:33.499 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.746+0000 7f54d8c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54d41098c0 0x7f54d4109cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:18:33.499 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.746+0000 7f54d8c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54d41098c0 0x7f54d4109cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44492/0 (socket says 172.21.15.1:44492) 2024-09-12T01:18:33.499 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.746+0000 7f54d8c00640 1 -- 172.21.15.1:0/4053645674 learned_addr learned my addr 172.21.15.1:0/4053645674 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:18:33.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.746+0000 7f54d8c00640 1 -- 172.21.15.1:0/4053645674 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d410a9b0 con 0x7f54d41098c0 2024-09-12T01:18:33.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.747+0000 7f54d8c00640 1 --2- 172.21.15.1:0/4053645674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54d41098c0 0x7f54d4109cc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f54c8009820 tx=0x7f54c802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fac77a4e9156d6e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:18:33.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.747+0000 7f54d3400640 1 -- 172.21.15.1:0/4053645674 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54c8037070 con 0x7f54d41098c0 2024-09-12T01:18:33.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.748+0000 7f54d3400640 1 -- 172.21.15.1:0/4053645674 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f54c8032900 con 0x7f54d41098c0 2024-09-12T01:18:33.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.748+0000 7f54d3400640 1 -- 172.21.15.1:0/4053645674 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54c8030850 con 0x7f54d41098c0 2024-09-12T01:18:33.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.748+0000 7f54db600640 1 -- 172.21.15.1:0/4053645674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54d41098c0 msgr2=0x7f54d4109cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:18:33.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.748+0000 7f54db600640 1 --2- 172.21.15.1:0/4053645674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54d41098c0 0x7f54d4109cc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f54c8009820 tx=0x7f54c802a270 comp rx=0 tx=0).stop 2024-09-12T01:18:33.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.748+0000 7f54db600640 1 -- 172.21.15.1:0/4053645674 shutdown_connections 2024-09-12T01:18:33.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.748+0000 7f54db600640 1 --2- 172.21.15.1:0/4053645674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54d41098c0 0x7f54d4109cc0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:33.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.748+0000 7f54db600640 1 -- 172.21.15.1:0/4053645674 >> 172.21.15.1:0/4053645674 conn(0x7f54d407ce20 msgr2=0x7f54d407d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:18:33.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.748+0000 7f54db600640 1 -- 172.21.15.1:0/4053645674 shutdown_connections 2024-09-12T01:18:33.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.748+0000 7f54db600640 1 -- 172.21.15.1:0/4053645674 wait complete. 2024-09-12T01:18:33.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.749+0000 7f54db600640 1 Processor -- start 2024-09-12T01:18:33.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.750+0000 7f54db600640 1 -- start start 2024-09-12T01:18:33.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.750+0000 7f54db600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54d41098c0 0x7f54d419db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:18:33.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.750+0000 7f54db600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f54d410dd60 con 0x7f54d41098c0 2024-09-12T01:18:33.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.750+0000 7f54d8c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54d41098c0 0x7f54d419db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:18:33.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.750+0000 7f54d8c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54d41098c0 0x7f54d419db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44494/0 (socket says 172.21.15.1:44494) 2024-09-12T01:18:33.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.750+0000 7f54d8c00640 1 -- 172.21.15.1:0/2792709420 learned_addr learned my addr 172.21.15.1:0/2792709420 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:18:33.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.751+0000 7f54d8c00640 1 -- 172.21.15.1:0/2792709420 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d419e040 con 0x7f54d41098c0 2024-09-12T01:18:33.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.751+0000 7f54d8c00640 1 --2- 172.21.15.1:0/2792709420 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54d41098c0 0x7f54d419db00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f54c8009940 tx=0x7f54c80326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:18:33.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.751+0000 7f54d1600640 1 -- 172.21.15.1:0/2792709420 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54c8042070 con 0x7f54d41098c0 2024-09-12T01:18:33.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.751+0000 7f54d1600640 1 -- 172.21.15.1:0/2792709420 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f54c8032be0 con 0x7f54d41098c0 2024-09-12T01:18:33.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.751+0000 7f54db600640 1 -- 172.21.15.1:0/2792709420 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54d419e2d0 con 0x7f54d41098c0 2024-09-12T01:18:33.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.751+0000 7f54d1600640 1 -- 172.21.15.1:0/2792709420 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54c8037040 con 0x7f54d41098c0 2024-09-12T01:18:33.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.751+0000 7f54db600640 1 -- 172.21.15.1:0/2792709420 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54d41a0fc0 con 0x7f54d41098c0 2024-09-12T01:18:33.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.752+0000 7f54d1600640 1 -- 172.21.15.1:0/2792709420 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f54c8045070 con 0x7f54d41098c0 2024-09-12T01:18:33.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.752+0000 7f54d1600640 1 -- 172.21.15.1:0/2792709420 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f54c8047a80 con 0x7f54d41098c0 2024-09-12T01:18:33.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.753+0000 7f54db600640 1 -- 172.21.15.1:0/2792709420 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54d410dee0 con 0x7f54d41098c0 2024-09-12T01:18:33.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.755+0000 7f54d1600640 1 -- 172.21.15.1:0/2792709420 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f54c8032d90 con 0x7f54d41098c0 2024-09-12T01:18:33.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.821+0000 7f54db600640 1 -- 172.21.15.1:0/2792709420 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f54d4197650 con 0x7f54d41098c0 2024-09-12T01:18:33.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.821+0000 7f54d1600640 1 -- 172.21.15.1:0/2792709420 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f54c803d430 con 0x7f54d41098c0 2024-09-12T01:18:33.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.822+0000 7f54db600640 1 -- 172.21.15.1:0/2792709420 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54d41098c0 msgr2=0x7f54d419db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:18:33.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.822+0000 7f54db600640 1 --2- 172.21.15.1:0/2792709420 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54d41098c0 0x7f54d419db00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f54c8009940 tx=0x7f54c80326e0 comp rx=0 tx=0).stop 2024-09-12T01:18:33.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.822+0000 7f54db600640 1 -- 172.21.15.1:0/2792709420 shutdown_connections 2024-09-12T01:18:33.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.822+0000 7f54db600640 1 --2- 172.21.15.1:0/2792709420 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54d41098c0 0x7f54d419db00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:33.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.822+0000 7f54db600640 1 -- 172.21.15.1:0/2792709420 >> 172.21.15.1:0/2792709420 conn(0x7f54d407ce20 msgr2=0x7f54d41077d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:18:33.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.823+0000 7f54db600640 1 -- 172.21.15.1:0/2792709420 shutdown_connections 2024-09-12T01:18:33.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:32.823+0000 7f54db600640 1 -- 172.21.15.1:0/2792709420 wait complete. 2024-09-12T01:18:33.502 INFO:teuthology.orchestra.run.smithi001.stdout:mgr not available, waiting (2/15)... 2024-09-12T01:18:33.558 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:33 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:33.291+0000 7faa245da040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T01:18:33.558 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:33 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:33.373+0000 7faa245da040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T01:18:33.559 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:33 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:33.456+0000 7faa245da040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T01:18:33.559 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:33 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:33.559+0000 7faa245da040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T01:18:33.927 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:33 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:33.633+0000 7faa245da040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T01:18:33.927 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:33 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:33.706+0000 7faa245da040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T01:18:34.927 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:34 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:34.662+0000 7faa245da040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T01:18:34.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:34 smithi001 ceph-mon[30387]: Activating manager daemon x 2024-09-12T01:18:34.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:34 smithi001 ceph-mon[30387]: mgrmap e2: x(active, starting, since 0.00430769s) 2024-09-12T01:18:34.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:34 smithi001 ceph-mon[30387]: from='mgr.14100 172.21.15.1:0/1185519772' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T01:18:34.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:34 smithi001 ceph-mon[30387]: from='mgr.14100 172.21.15.1:0/1185519772' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T01:18:34.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:34 smithi001 ceph-mon[30387]: from='mgr.14100 172.21.15.1:0/1185519772' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T01:18:34.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:34 smithi001 ceph-mon[30387]: from='mgr.14100 172.21.15.1:0/1185519772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:18:34.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:34 smithi001 ceph-mon[30387]: from='mgr.14100 172.21.15.1:0/1185519772' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T01:18:34.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:34 smithi001 ceph-mon[30387]: Manager daemon x is now available 2024-09-12T01:18:34.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:34 smithi001 ceph-mon[30387]: from='mgr.14100 172.21.15.1:0/1185519772' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T01:18:34.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:34 smithi001 ceph-mon[30387]: from='mgr.14100 172.21.15.1:0/1185519772' entity='mgr.x' 2024-09-12T01:18:34.930 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:34 smithi001 ceph-mon[30387]: from='mgr.14100 172.21.15.1:0/1185519772' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T01:18:34.930 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:34 smithi001 ceph-mon[30387]: from='mgr.14100 172.21.15.1:0/1185519772' entity='mgr.x' 2024-09-12T01:18:34.930 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:34 smithi001 ceph-mon[30387]: from='mgr.14100 172.21.15.1:0/1185519772' entity='mgr.x' 2024-09-12T01:18:36.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:36 smithi001 ceph-mon[30387]: mgrmap e3: x(active, since 1.00846s) 2024-09-12T01:18:36.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:36 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/1695565537' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-12T01:18:36.951 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-09-12T01:18:36.951 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-09-12T01:18:36.951 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsid": "c0e4f180-70a4-11ef-bcea-c7b262605968", 2024-09-12T01:18:36.951 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "health": { 2024-09-12T01:18:36.951 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-12T01:18:36.951 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-12T01:18:36.951 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-12T01:18:36.951 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:18:36.951 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-12T01:18:36.951 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-12T01:18:36.951 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 0 2024-09-12T01:18:36.952 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-09-12T01:18:36.952 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-12T01:18:36.952 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "a" 2024-09-12T01:18:36.952 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-09-12T01:18:36.952 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-09-12T01:18:36.952 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-12T01:18:36.952 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T01:18:36.952 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-12T01:18:36.952 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-12T01:18:36.952 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:18:36.952 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-12T01:18:36.952 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T01:18:36.952 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-12T01:18:36.953 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-12T01:18:36.953 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-12T01:18:36.953 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-12T01:18:36.953 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-12T01:18:36.953 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-12T01:18:36.953 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:18:36.953 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-12T01:18:36.953 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-12T01:18:36.953 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-12T01:18:36.953 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-12T01:18:36.953 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-12T01:18:36.953 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-12T01:18:36.953 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-12T01:18:36.953 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-12T01:18:36.953 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-12T01:18:36.954 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:18:36.954 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-12T01:18:36.954 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T01:18:36.954 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "btime": "2024-09-12T01:18:19:337176+0000", 2024-09-12T01:18:36.954 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-12T01:18:36.954 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-12T01:18:36.954 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:18:36.954 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-12T01:18:36.954 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-12T01:18:36.954 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-12T01:18:36.954 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-12T01:18:36.954 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-12T01:18:36.954 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-12T01:18:36.954 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "restful" 2024-09-12T01:18:36.955 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-09-12T01:18:36.955 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T01:18:36.955 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:18:36.955 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-12T01:18:36.955 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T01:18:36.955 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modified": "2024-09-12T01:18:19.337976+0000", 2024-09-12T01:18:36.955 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T01:18:36.955 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:18:36.955 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-12T01:18:36.955 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-09-12T01:18:36.955 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:35.961+0000 7fd818e00640 1 Processor -- start 2024-09-12T01:18:36.955 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:35.962+0000 7fd818e00640 1 -- start start 2024-09-12T01:18:36.955 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:35.962+0000 7fd818e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd814109910 0x7fd814109d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:18:36.955 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:35.962+0000 7fd818e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd81410a2e0 con 0x7fd814109910 2024-09-12T01:18:36.956 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:35.962+0000 7fd813400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd814109910 0x7fd814109d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:18:36.956 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:35.962+0000 7fd813400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd814109910 0x7fd814109d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44558/0 (socket says 172.21.15.1:44558) 2024-09-12T01:18:36.956 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:35.962+0000 7fd813400640 1 -- 172.21.15.1:0/1912012012 learned_addr learned my addr 172.21.15.1:0/1912012012 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:18:36.956 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:35.963+0000 7fd813400640 1 -- 172.21.15.1:0/1912012012 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd81410a770 con 0x7fd814109910 2024-09-12T01:18:36.956 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:35.964+0000 7fd813400640 1 --2- 172.21.15.1:0/1912012012 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd814109910 0x7fd814109d10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd804009820 tx=0x7fd80402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e16faeb1d0b10d2f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:18:36.956 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:35.964+0000 7fd812000640 1 -- 172.21.15.1:0/1912012012 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd804037070 con 0x7fd814109910 2024-09-12T01:18:36.956 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:35.964+0000 7fd812000640 1 -- 172.21.15.1:0/1912012012 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd804032900 con 0x7fd814109910 2024-09-12T01:18:36.956 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:35.964+0000 7fd812000640 1 -- 172.21.15.1:0/1912012012 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd804030850 con 0x7fd814109910 2024-09-12T01:18:36.956 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:35.964+0000 7fd818e00640 1 -- 172.21.15.1:0/1912012012 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd814109910 msgr2=0x7fd814109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:18:36.956 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:35.964+0000 7fd818e00640 1 --2- 172.21.15.1:0/1912012012 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd814109910 0x7fd814109d10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd804009820 tx=0x7fd80402a270 comp rx=0 tx=0).stop 2024-09-12T01:18:36.956 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:35.965+0000 7fd818e00640 1 -- 172.21.15.1:0/1912012012 shutdown_connections 2024-09-12T01:18:36.956 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:35.965+0000 7fd818e00640 1 --2- 172.21.15.1:0/1912012012 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd814109910 0x7fd814109d10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:36.956 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:35.965+0000 7fd818e00640 1 -- 172.21.15.1:0/1912012012 >> 172.21.15.1:0/1912012012 conn(0x7fd81407cc40 msgr2=0x7fd81407d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:18:36.957 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:35.965+0000 7fd818e00640 1 -- 172.21.15.1:0/1912012012 shutdown_connections 2024-09-12T01:18:36.957 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:35.965+0000 7fd818e00640 1 -- 172.21.15.1:0/1912012012 wait complete. 2024-09-12T01:18:36.957 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:35.966+0000 7fd818e00640 1 Processor -- start 2024-09-12T01:18:36.957 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:35.966+0000 7fd818e00640 1 -- start start 2024-09-12T01:18:36.957 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:35.966+0000 7fd818e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd814109910 0x7fd81419d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:18:36.957 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:35.966+0000 7fd818e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd81410db20 con 0x7fd814109910 2024-09-12T01:18:36.957 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:35.966+0000 7fd813400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd814109910 0x7fd81419d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:18:36.957 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:35.966+0000 7fd813400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd814109910 0x7fd81419d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44574/0 (socket says 172.21.15.1:44574) 2024-09-12T01:18:36.957 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:35.966+0000 7fd813400640 1 -- 172.21.15.1:0/1695565537 learned_addr learned my addr 172.21.15.1:0/1695565537 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:18:36.957 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:35.967+0000 7fd813400640 1 -- 172.21.15.1:0/1695565537 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd81419db10 con 0x7fd814109910 2024-09-12T01:18:36.957 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:35.967+0000 7fd813400640 1 --2- 172.21.15.1:0/1695565537 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd814109910 0x7fd81419d5d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd804009940 tx=0x7fd804032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:18:36.957 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:35.967+0000 7fd80be00640 1 -- 172.21.15.1:0/1695565537 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd804042070 con 0x7fd814109910 2024-09-12T01:18:36.957 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:35.967+0000 7fd80be00640 1 -- 172.21.15.1:0/1695565537 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd8040308c0 con 0x7fd814109910 2024-09-12T01:18:36.957 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:35.967+0000 7fd818e00640 1 -- 172.21.15.1:0/1695565537 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd81419dda0 con 0x7fd814109910 2024-09-12T01:18:36.958 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:35.967+0000 7fd80be00640 1 -- 172.21.15.1:0/1695565537 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd804037040 con 0x7fd814109910 2024-09-12T01:18:36.958 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:35.967+0000 7fd818e00640 1 -- 172.21.15.1:0/1695565537 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8141a0a90 con 0x7fd814109910 2024-09-12T01:18:36.958 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:35.968+0000 7fd818e00640 1 -- 172.21.15.1:0/1695565537 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd814109d90 con 0x7fd814109910 2024-09-12T01:18:36.958 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:35.968+0000 7fd80be00640 1 -- 172.21.15.1:0/1695565537 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 3) ==== 46170+0+0 (secure 0 0 0) 0x7fd804045070 con 0x7fd814109910 2024-09-12T01:18:36.958 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:35.969+0000 7fd80be00640 1 --2- 172.21.15.1:0/1695565537 >> [v2:172.21.15.1:6800/2935968453,v1:172.21.15.1:6801/2935968453] conn(0x7fd7f8038e10 0x7fd7f803b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:18:36.958 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:35.969+0000 7fd80be00640 1 -- 172.21.15.1:0/1695565537 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd80406bda0 con 0x7fd814109910 2024-09-12T01:18:36.958 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:35.969+0000 7fd812a00640 1 --2- 172.21.15.1:0/1695565537 >> [v2:172.21.15.1:6800/2935968453,v1:172.21.15.1:6801/2935968453] conn(0x7fd7f8038e10 0x7fd7f803b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:18:36.958 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:35.970+0000 7fd812a00640 1 --2- 172.21.15.1:0/1695565537 >> [v2:172.21.15.1:6800/2935968453,v1:172.21.15.1:6801/2935968453] conn(0x7fd7f8038e10 0x7fd7f803b2d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd8000098c0 tx=0x7fd800006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:18:36.958 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:35.975+0000 7fd80be00640 1 -- 172.21.15.1:0/1695565537 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd80403cd90 con 0x7fd814109910 2024-09-12T01:18:36.958 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:36.231+0000 7fd818e00640 1 -- 172.21.15.1:0/1695565537 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd8141a1110 con 0x7fd814109910 2024-09-12T01:18:36.958 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:36.232+0000 7fd80be00640 1 -- 172.21.15.1:0/1695565537 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fd80402fd90 con 0x7fd814109910 2024-09-12T01:18:36.958 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:36.234+0000 7fd818e00640 1 -- 172.21.15.1:0/1695565537 >> [v2:172.21.15.1:6800/2935968453,v1:172.21.15.1:6801/2935968453] conn(0x7fd7f8038e10 msgr2=0x7fd7f803b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:18:36.958 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:36.234+0000 7fd818e00640 1 --2- 172.21.15.1:0/1695565537 >> [v2:172.21.15.1:6800/2935968453,v1:172.21.15.1:6801/2935968453] conn(0x7fd7f8038e10 0x7fd7f803b2d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd8000098c0 tx=0x7fd800006db0 comp rx=0 tx=0).stop 2024-09-12T01:18:36.959 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:36.234+0000 7fd818e00640 1 -- 172.21.15.1:0/1695565537 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd814109910 msgr2=0x7fd81419d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:18:36.959 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:36.234+0000 7fd818e00640 1 --2- 172.21.15.1:0/1695565537 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd814109910 0x7fd81419d5d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd804009940 tx=0x7fd804032ce0 comp rx=0 tx=0).stop 2024-09-12T01:18:36.959 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:36.235+0000 7fd818e00640 1 -- 172.21.15.1:0/1695565537 shutdown_connections 2024-09-12T01:18:36.959 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:36.235+0000 7fd818e00640 1 --2- 172.21.15.1:0/1695565537 >> [v2:172.21.15.1:6800/2935968453,v1:172.21.15.1:6801/2935968453] conn(0x7fd7f8038e10 0x7fd7f803b2d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:36.959 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:36.235+0000 7fd818e00640 1 --2- 172.21.15.1:0/1695565537 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd814109910 0x7fd81419d5d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:36.959 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:36.235+0000 7fd818e00640 1 -- 172.21.15.1:0/1695565537 >> 172.21.15.1:0/1695565537 conn(0x7fd81407cc40 msgr2=0x7fd814107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:18:36.959 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:36.235+0000 7fd818e00640 1 -- 172.21.15.1:0/1695565537 shutdown_connections 2024-09-12T01:18:36.959 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:36.235+0000 7fd818e00640 1 -- 172.21.15.1:0/1695565537 wait complete. 2024-09-12T01:18:36.959 INFO:teuthology.orchestra.run.smithi001.stdout:mgr is available 2024-09-12T01:18:37.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:37 smithi001 ceph-mon[30387]: mgrmap e4: x(active, since 2s) 2024-09-12T01:18:38.578 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-09-12T01:18:38.579 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout [global] 2024-09-12T01:18:38.579 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout fsid = c0e4f180-70a4-11ef-bcea-c7b262605968 2024-09-12T01:18:38.579 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.1:3300,v1:172.21.15.1:6789] 2024-09-12T01:18:38.579 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-12T01:18:38.579 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-12T01:18:38.579 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-12T01:18:38.580 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-12T01:18:38.580 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-09-12T01:18:38.580 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-12T01:18:38.580 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-12T01:18:38.580 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-09-12T01:18:38.580 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout [osd] 2024-09-12T01:18:38.580 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-12T01:18:38.580 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-12T01:18:38.580 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-12T01:18:38.580 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-12T01:18:38.580 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-12T01:18:38.580 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.735+0000 7f647d800640 1 Processor -- start 2024-09-12T01:18:38.580 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.735+0000 7f647d800640 1 -- start start 2024-09-12T01:18:38.581 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.735+0000 7f647d800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6478109620 0x7f6478109a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:18:38.581 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.735+0000 7f647d800640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6478109ff0 con 0x7f6478109620 2024-09-12T01:18:38.581 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.735+0000 7f6477e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6478109620 0x7f6478109a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:18:38.581 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.735+0000 7f6477e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6478109620 0x7f6478109a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44582/0 (socket says 172.21.15.1:44582) 2024-09-12T01:18:38.581 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.735+0000 7f6477e00640 1 -- 172.21.15.1:0/2165659298 learned_addr learned my addr 172.21.15.1:0/2165659298 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:18:38.581 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.736+0000 7f6477e00640 1 -- 172.21.15.1:0/2165659298 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f647810a7d0 con 0x7f6478109620 2024-09-12T01:18:38.581 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.737+0000 7f6477e00640 1 --2- 172.21.15.1:0/2165659298 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6478109620 0x7f6478109a20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6468009820 tx=0x7f646802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d681da7ef45b923b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:18:38.581 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.737+0000 7f6477400640 1 -- 172.21.15.1:0/2165659298 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6468037070 con 0x7f6478109620 2024-09-12T01:18:38.581 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.737+0000 7f6477400640 1 -- 172.21.15.1:0/2165659298 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6468032900 con 0x7f6478109620 2024-09-12T01:18:38.581 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.737+0000 7f6477400640 1 -- 172.21.15.1:0/2165659298 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6468030850 con 0x7f6478109620 2024-09-12T01:18:38.581 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.738+0000 7f647d800640 1 -- 172.21.15.1:0/2165659298 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6478109620 msgr2=0x7f6478109a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:18:38.581 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.738+0000 7f647d800640 1 --2- 172.21.15.1:0/2165659298 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6478109620 0x7f6478109a20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6468009820 tx=0x7f646802a270 comp rx=0 tx=0).stop 2024-09-12T01:18:38.581 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.738+0000 7f647d800640 1 -- 172.21.15.1:0/2165659298 shutdown_connections 2024-09-12T01:18:38.582 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.738+0000 7f647d800640 1 --2- 172.21.15.1:0/2165659298 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6478109620 0x7f6478109a20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:38.582 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.738+0000 7f647d800640 1 -- 172.21.15.1:0/2165659298 >> 172.21.15.1:0/2165659298 conn(0x7f647807cc70 msgr2=0x7f6478107360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:18:38.582 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.738+0000 7f647d800640 1 -- 172.21.15.1:0/2165659298 shutdown_connections 2024-09-12T01:18:38.582 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.738+0000 7f647d800640 1 -- 172.21.15.1:0/2165659298 wait complete. 2024-09-12T01:18:38.582 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.739+0000 7f647d800640 1 Processor -- start 2024-09-12T01:18:38.582 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.739+0000 7f647d800640 1 -- start start 2024-09-12T01:18:38.582 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.739+0000 7f647d800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6478109620 0x7f647819d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:18:38.582 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.739+0000 7f647d800640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f647810db80 con 0x7f6478109620 2024-09-12T01:18:38.582 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.740+0000 7f6477e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6478109620 0x7f647819d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:18:38.582 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.740+0000 7f6477e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6478109620 0x7f647819d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44590/0 (socket says 172.21.15.1:44590) 2024-09-12T01:18:38.582 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.740+0000 7f6477e00640 1 -- 172.21.15.1:0/158712481 learned_addr learned my addr 172.21.15.1:0/158712481 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:18:38.582 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.740+0000 7f6477e00640 1 -- 172.21.15.1:0/158712481 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f647819db10 con 0x7f6478109620 2024-09-12T01:18:38.582 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.740+0000 7f6477e00640 1 --2- 172.21.15.1:0/158712481 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6478109620 0x7f647819d5d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f64680097f0 tx=0x7f6468030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:18:38.582 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.740+0000 7f6475600640 1 -- 172.21.15.1:0/158712481 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6468037040 con 0x7f6478109620 2024-09-12T01:18:38.583 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.741+0000 7f6475600640 1 -- 172.21.15.1:0/158712481 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6468040070 con 0x7f6478109620 2024-09-12T01:18:38.583 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.741+0000 7f6475600640 1 -- 172.21.15.1:0/158712481 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f646803a390 con 0x7f6478109620 2024-09-12T01:18:38.583 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.741+0000 7f647d800640 1 -- 172.21.15.1:0/158712481 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f647819dda0 con 0x7f6478109620 2024-09-12T01:18:38.583 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.741+0000 7f647d800640 1 -- 172.21.15.1:0/158712481 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64781a0a90 con 0x7f6478109620 2024-09-12T01:18:38.583 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.742+0000 7f6475600640 1 -- 172.21.15.1:0/158712481 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f646803b430 con 0x7f6478109620 2024-09-12T01:18:38.583 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.742+0000 7f647d800640 1 -- 172.21.15.1:0/158712481 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6478109aa0 con 0x7f6478109620 2024-09-12T01:18:38.583 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.742+0000 7f6475600640 1 --2- 172.21.15.1:0/158712481 >> [v2:172.21.15.1:6800/2935968453,v1:172.21.15.1:6801/2935968453] conn(0x7f645c038f70 0x7f645c03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:18:38.583 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.742+0000 7f6475600640 1 -- 172.21.15.1:0/158712481 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f646806c0d0 con 0x7f6478109620 2024-09-12T01:18:38.583 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.743+0000 7f646fe00640 1 --2- 172.21.15.1:0/158712481 >> [v2:172.21.15.1:6800/2935968453,v1:172.21.15.1:6801/2935968453] conn(0x7f645c038f70 0x7f645c03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:18:38.583 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.743+0000 7f646fe00640 1 --2- 172.21.15.1:0/158712481 >> [v2:172.21.15.1:6800/2935968453,v1:172.21.15.1:6801/2935968453] conn(0x7f645c038f70 0x7f645c03b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f64600098c0 tx=0x7f6460006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:18:38.583 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.747+0000 7f6475600640 1 -- 172.21.15.1:0/158712481 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f64680371e0 con 0x7f6478109620 2024-09-12T01:18:38.583 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.933+0000 7f647d800640 1 -- 172.21.15.1:0/158712481 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f647819e220 con 0x7f6478109620 2024-09-12T01:18:38.583 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.934+0000 7f6475600640 1 -- 172.21.15.1:0/158712481 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+427 (secure 0 0 0) 0x7f6468032dd0 con 0x7f6478109620 2024-09-12T01:18:38.584 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.937+0000 7f647d800640 1 -- 172.21.15.1:0/158712481 >> [v2:172.21.15.1:6800/2935968453,v1:172.21.15.1:6801/2935968453] conn(0x7f645c038f70 msgr2=0x7f645c03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:18:38.584 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.937+0000 7f647d800640 1 --2- 172.21.15.1:0/158712481 >> [v2:172.21.15.1:6800/2935968453,v1:172.21.15.1:6801/2935968453] conn(0x7f645c038f70 0x7f645c03b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f64600098c0 tx=0x7f6460006db0 comp rx=0 tx=0).stop 2024-09-12T01:18:38.584 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.937+0000 7f647d800640 1 -- 172.21.15.1:0/158712481 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6478109620 msgr2=0x7f647819d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:18:38.584 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.937+0000 7f647d800640 1 --2- 172.21.15.1:0/158712481 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6478109620 0x7f647819d5d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f64680097f0 tx=0x7f6468030d80 comp rx=0 tx=0).stop 2024-09-12T01:18:38.584 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.937+0000 7f647d800640 1 -- 172.21.15.1:0/158712481 shutdown_connections 2024-09-12T01:18:38.584 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.937+0000 7f647d800640 1 --2- 172.21.15.1:0/158712481 >> [v2:172.21.15.1:6800/2935968453,v1:172.21.15.1:6801/2935968453] conn(0x7f645c038f70 0x7f645c03b430 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:38.584 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.937+0000 7f647d800640 1 --2- 172.21.15.1:0/158712481 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6478109620 0x7f647819d5d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:38.584 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.937+0000 7f647d800640 1 -- 172.21.15.1:0/158712481 >> 172.21.15.1:0/158712481 conn(0x7f647807cc70 msgr2=0x7f6478106b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:18:38.584 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.937+0000 7f647d800640 1 -- 172.21.15.1:0/158712481 shutdown_connections 2024-09-12T01:18:38.584 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:37.937+0000 7f647d800640 1 -- 172.21.15.1:0/158712481 wait complete. 2024-09-12T01:18:38.584 INFO:teuthology.orchestra.run.smithi001.stdout:Enabling cephadm module... 2024-09-12T01:18:38.865 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:38 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/158712481' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-12T01:18:39.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:39 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/2938420877' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-12T01:18:39.927 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:39 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: ignoring --setuser ceph since I am not root 2024-09-12T01:18:39.927 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:39 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: ignoring --setgroup ceph since I am not root 2024-09-12T01:18:40.220 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:39 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:39.968+0000 7fad105dc040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T01:18:40.221 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:40 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:40.042+0000 7fad105dc040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T01:18:40.500 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:40 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:40.237+0000 7fad105dc040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T01:18:40.500 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:40 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:40.321+0000 7fad105dc040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T01:18:40.544 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.122+0000 7f173c200640 1 Processor -- start 2024-09-12T01:18:40.544 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.122+0000 7f173c200640 1 -- start start 2024-09-12T01:18:40.544 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.122+0000 7f173c200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17341098c0 0x7f1734109cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:18:40.544 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.122+0000 7f173c200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f173410a290 con 0x7f17341098c0 2024-09-12T01:18:40.544 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.123+0000 7f1739800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17341098c0 0x7f1734109cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:18:40.545 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.123+0000 7f1739800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17341098c0 0x7f1734109cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40186/0 (socket says 172.21.15.1:40186) 2024-09-12T01:18:40.545 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.123+0000 7f1739800640 1 -- 172.21.15.1:0/2494874588 learned_addr learned my addr 172.21.15.1:0/2494874588 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:18:40.545 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.123+0000 7f1739800640 1 -- 172.21.15.1:0/2494874588 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f173410a9b0 con 0x7f17341098c0 2024-09-12T01:18:40.545 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.124+0000 7f1739800640 1 --2- 172.21.15.1:0/2494874588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17341098c0 0x7f1734109cc0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1728009820 tx=0x7f172802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fcedfd9ae59b5b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:18:40.545 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.124+0000 7f1733e00640 1 -- 172.21.15.1:0/2494874588 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1728037070 con 0x7f17341098c0 2024-09-12T01:18:40.545 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.124+0000 7f1733e00640 1 -- 172.21.15.1:0/2494874588 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1728032900 con 0x7f17341098c0 2024-09-12T01:18:40.545 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.124+0000 7f1733e00640 1 -- 172.21.15.1:0/2494874588 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1728030850 con 0x7f17341098c0 2024-09-12T01:18:40.545 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.125+0000 7f173c200640 1 -- 172.21.15.1:0/2494874588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17341098c0 msgr2=0x7f1734109cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:18:40.545 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.125+0000 7f173c200640 1 --2- 172.21.15.1:0/2494874588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17341098c0 0x7f1734109cc0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1728009820 tx=0x7f172802a270 comp rx=0 tx=0).stop 2024-09-12T01:18:40.545 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.125+0000 7f173c200640 1 -- 172.21.15.1:0/2494874588 shutdown_connections 2024-09-12T01:18:40.545 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.125+0000 7f173c200640 1 --2- 172.21.15.1:0/2494874588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17341098c0 0x7f1734109cc0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:40.545 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.125+0000 7f173c200640 1 -- 172.21.15.1:0/2494874588 >> 172.21.15.1:0/2494874588 conn(0x7f173407ce20 msgr2=0x7f173407d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:18:40.545 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.125+0000 7f173c200640 1 -- 172.21.15.1:0/2494874588 shutdown_connections 2024-09-12T01:18:40.546 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.125+0000 7f173c200640 1 -- 172.21.15.1:0/2494874588 wait complete. 2024-09-12T01:18:40.546 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.126+0000 7f173c200640 1 Processor -- start 2024-09-12T01:18:40.546 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.126+0000 7f173c200640 1 -- start start 2024-09-12T01:18:40.546 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.126+0000 7f173c200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17341098c0 0x7f173419db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:18:40.546 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.126+0000 7f173c200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f173410dd60 con 0x7f17341098c0 2024-09-12T01:18:40.546 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.127+0000 7f1739800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17341098c0 0x7f173419db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:18:40.546 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.127+0000 7f1739800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17341098c0 0x7f173419db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40202/0 (socket says 172.21.15.1:40202) 2024-09-12T01:18:40.546 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.127+0000 7f1739800640 1 -- 172.21.15.1:0/2938420877 learned_addr learned my addr 172.21.15.1:0/2938420877 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:18:40.546 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.127+0000 7f1739800640 1 -- 172.21.15.1:0/2938420877 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f173419e040 con 0x7f17341098c0 2024-09-12T01:18:40.546 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.127+0000 7f1739800640 1 --2- 172.21.15.1:0/2938420877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17341098c0 0x7f173419db00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1728009940 tx=0x7f17280326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:18:40.546 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.128+0000 7f1732000640 1 -- 172.21.15.1:0/2938420877 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1728042070 con 0x7f17341098c0 2024-09-12T01:18:40.546 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.128+0000 7f1732000640 1 -- 172.21.15.1:0/2938420877 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1728032be0 con 0x7f17341098c0 2024-09-12T01:18:40.546 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.128+0000 7f1732000640 1 -- 172.21.15.1:0/2938420877 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1728037040 con 0x7f17341098c0 2024-09-12T01:18:40.546 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.128+0000 7f173c200640 1 -- 172.21.15.1:0/2938420877 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f173419e2d0 con 0x7f17341098c0 2024-09-12T01:18:40.547 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.128+0000 7f173c200640 1 -- 172.21.15.1:0/2938420877 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17341a0fc0 con 0x7f17341098c0 2024-09-12T01:18:40.547 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.129+0000 7f1732000640 1 -- 172.21.15.1:0/2938420877 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f1728045070 con 0x7f17341098c0 2024-09-12T01:18:40.547 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.129+0000 7f173c200640 1 -- 172.21.15.1:0/2938420877 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f173410dee0 con 0x7f17341098c0 2024-09-12T01:18:40.547 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.130+0000 7f1732000640 1 --2- 172.21.15.1:0/2938420877 >> [v2:172.21.15.1:6800/2935968453,v1:172.21.15.1:6801/2935968453] conn(0x7f171c038f70 0x7f171c03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:18:40.547 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.130+0000 7f1732000640 1 -- 172.21.15.1:0/2938420877 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f172806be40 con 0x7f17341098c0 2024-09-12T01:18:40.547 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.130+0000 7f1738e00640 1 --2- 172.21.15.1:0/2938420877 >> [v2:172.21.15.1:6800/2935968453,v1:172.21.15.1:6801/2935968453] conn(0x7f171c038f70 0x7f171c03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:18:40.547 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.131+0000 7f1738e00640 1 --2- 172.21.15.1:0/2938420877 >> [v2:172.21.15.1:6800/2935968453,v1:172.21.15.1:6801/2935968453] conn(0x7f171c038f70 0x7f171c03b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f17200098c0 tx=0x7f1720006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:18:40.547 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.135+0000 7f1732000640 1 -- 172.21.15.1:0/2938420877 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f172804b170 con 0x7f17341098c0 2024-09-12T01:18:40.547 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.372+0000 7f173c200640 1 -- 172.21.15.1:0/2938420877 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f173402d7c0 con 0x7f17341098c0 2024-09-12T01:18:40.547 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.677+0000 7f1732000640 1 -- 172.21.15.1:0/2938420877 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f1728032d90 con 0x7f17341098c0 2024-09-12T01:18:40.547 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.679+0000 7f1732000640 1 -- 172.21.15.1:0/2938420877 <== mon.0 v2:172.21.15.1:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f172803d7b0 con 0x7f17341098c0 2024-09-12T01:18:40.547 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.681+0000 7f173c200640 1 -- 172.21.15.1:0/2938420877 >> [v2:172.21.15.1:6800/2935968453,v1:172.21.15.1:6801/2935968453] conn(0x7f171c038f70 msgr2=0x7f171c03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:18:40.547 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.681+0000 7f173c200640 1 --2- 172.21.15.1:0/2938420877 >> [v2:172.21.15.1:6800/2935968453,v1:172.21.15.1:6801/2935968453] conn(0x7f171c038f70 0x7f171c03b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f17200098c0 tx=0x7f1720006db0 comp rx=0 tx=0).stop 2024-09-12T01:18:40.547 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.681+0000 7f173c200640 1 -- 172.21.15.1:0/2938420877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17341098c0 msgr2=0x7f173419db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:18:40.548 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.681+0000 7f173c200640 1 --2- 172.21.15.1:0/2938420877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17341098c0 0x7f173419db00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1728009940 tx=0x7f17280326e0 comp rx=0 tx=0).stop 2024-09-12T01:18:40.548 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.681+0000 7f173c200640 1 -- 172.21.15.1:0/2938420877 shutdown_connections 2024-09-12T01:18:40.548 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.681+0000 7f173c200640 1 --2- 172.21.15.1:0/2938420877 >> [v2:172.21.15.1:6800/2935968453,v1:172.21.15.1:6801/2935968453] conn(0x7f171c038f70 0x7f171c03b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:40.548 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.681+0000 7f173c200640 1 --2- 172.21.15.1:0/2938420877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17341098c0 0x7f173419db00 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:40.548 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.681+0000 7f173c200640 1 -- 172.21.15.1:0/2938420877 >> 172.21.15.1:0/2938420877 conn(0x7f173407ce20 msgr2=0x7f17341077d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:18:40.548 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.681+0000 7f173c200640 1 -- 172.21.15.1:0/2938420877 shutdown_connections 2024-09-12T01:18:40.548 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:39.681+0000 7f173c200640 1 -- 172.21.15.1:0/2938420877 wait complete. 2024-09-12T01:18:40.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:40 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/2938420877' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-12T01:18:40.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:40 smithi001 ceph-mon[30387]: mgrmap e5: x(active, since 5s) 2024-09-12T01:18:40.928 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:40 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:40.499+0000 7fad105dc040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T01:18:40.928 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:40 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:40.679+0000 7fad105dc040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T01:18:41.351 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:41 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:41.067+0000 7fad105dc040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T01:18:41.351 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:41 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:41.145+0000 7fad105dc040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T01:18:41.617 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:41 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:41.371+0000 7fad105dc040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T01:18:41.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:41 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/323582216' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-12T01:18:42.345 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-09-12T01:18:42.345 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-12T01:18:42.345 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-12T01:18:42.345 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-12T01:18:42.345 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-12T01:18:42.345 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-09-12T01:18:42.345 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.369+0000 7f32b2a00640 1 Processor -- start 2024-09-12T01:18:42.345 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.369+0000 7f32b2a00640 1 -- start start 2024-09-12T01:18:42.345 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.369+0000 7f32b2a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32ac1098c0 0x7f32ac109cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:18:42.345 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.369+0000 7f32b2a00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac10a290 con 0x7f32ac1098c0 2024-09-12T01:18:42.345 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.370+0000 7f32abe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32ac1098c0 0x7f32ac109cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:18:42.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.370+0000 7f32abe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32ac1098c0 0x7f32ac109cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40228/0 (socket says 172.21.15.1:40228) 2024-09-12T01:18:42.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.370+0000 7f32abe00640 1 -- 172.21.15.1:0/1864217380 learned_addr learned my addr 172.21.15.1:0/1864217380 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:18:42.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.370+0000 7f32abe00640 1 -- 172.21.15.1:0/1864217380 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32ac10a9b0 con 0x7f32ac1098c0 2024-09-12T01:18:42.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.371+0000 7f32abe00640 1 --2- 172.21.15.1:0/1864217380 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32ac1098c0 0x7f32ac109cc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f329c009820 tx=0x7f329c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54c10f9ecace3e7f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:18:42.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.372+0000 7f32aaa00640 1 -- 172.21.15.1:0/1864217380 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f329c037070 con 0x7f32ac1098c0 2024-09-12T01:18:42.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.372+0000 7f32aaa00640 1 -- 172.21.15.1:0/1864217380 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f329c032900 con 0x7f32ac1098c0 2024-09-12T01:18:42.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.372+0000 7f32aaa00640 1 -- 172.21.15.1:0/1864217380 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f329c030850 con 0x7f32ac1098c0 2024-09-12T01:18:42.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.372+0000 7f32b2a00640 1 -- 172.21.15.1:0/1864217380 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32ac1098c0 msgr2=0x7f32ac109cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:18:42.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.372+0000 7f32b2a00640 1 --2- 172.21.15.1:0/1864217380 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32ac1098c0 0x7f32ac109cc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f329c009820 tx=0x7f329c02a270 comp rx=0 tx=0).stop 2024-09-12T01:18:42.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.372+0000 7f32b2a00640 1 -- 172.21.15.1:0/1864217380 shutdown_connections 2024-09-12T01:18:42.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.372+0000 7f32b2a00640 1 --2- 172.21.15.1:0/1864217380 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32ac1098c0 0x7f32ac109cc0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:42.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.372+0000 7f32b2a00640 1 -- 172.21.15.1:0/1864217380 >> 172.21.15.1:0/1864217380 conn(0x7f32ac07ce20 msgr2=0x7f32ac07d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:18:42.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.373+0000 7f32b2a00640 1 -- 172.21.15.1:0/1864217380 shutdown_connections 2024-09-12T01:18:42.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.373+0000 7f32b2a00640 1 -- 172.21.15.1:0/1864217380 wait complete. 2024-09-12T01:18:42.347 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.374+0000 7f32b2a00640 1 Processor -- start 2024-09-12T01:18:42.347 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.374+0000 7f32b2a00640 1 -- start start 2024-09-12T01:18:42.347 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.374+0000 7f32b2a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32ac1098c0 0x7f32ac19db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:18:42.347 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.374+0000 7f32b2a00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac10dd60 con 0x7f32ac1098c0 2024-09-12T01:18:42.347 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.374+0000 7f32abe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32ac1098c0 0x7f32ac19db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:18:42.347 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.374+0000 7f32abe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32ac1098c0 0x7f32ac19db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40238/0 (socket says 172.21.15.1:40238) 2024-09-12T01:18:42.347 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.374+0000 7f32abe00640 1 -- 172.21.15.1:0/323582216 learned_addr learned my addr 172.21.15.1:0/323582216 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:18:42.347 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.375+0000 7f32abe00640 1 -- 172.21.15.1:0/323582216 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32ac19e040 con 0x7f32ac1098c0 2024-09-12T01:18:42.347 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.375+0000 7f32abe00640 1 --2- 172.21.15.1:0/323582216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32ac1098c0 0x7f32ac19db00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f329c0097f0 tx=0x7f329c030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:18:42.347 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.375+0000 7f32a8c00640 1 -- 172.21.15.1:0/323582216 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f329c042070 con 0x7f32ac1098c0 2024-09-12T01:18:42.347 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.375+0000 7f32b2a00640 1 -- 172.21.15.1:0/323582216 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32ac19e2d0 con 0x7f32ac1098c0 2024-09-12T01:18:42.347 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.375+0000 7f32a8c00640 1 -- 172.21.15.1:0/323582216 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f329c032ca0 con 0x7f32ac1098c0 2024-09-12T01:18:42.347 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.375+0000 7f32a8c00640 1 -- 172.21.15.1:0/323582216 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f329c037040 con 0x7f32ac1098c0 2024-09-12T01:18:42.348 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.376+0000 7f32b2a00640 1 -- 172.21.15.1:0/323582216 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32ac1a0fc0 con 0x7f32ac1098c0 2024-09-12T01:18:42.348 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.376+0000 7f32a8c00640 1 -- 172.21.15.1:0/323582216 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f329c045070 con 0x7f32ac1098c0 2024-09-12T01:18:42.348 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.377+0000 7f32b2a00640 1 -- 172.21.15.1:0/323582216 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32ac10dee0 con 0x7f32ac1098c0 2024-09-12T01:18:42.348 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.377+0000 7f32a8c00640 1 --2- 172.21.15.1:0/323582216 >> [v2:172.21.15.1:6800/2935968453,v1:172.21.15.1:6801/2935968453] conn(0x7f3290038fc0 0x7f329003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:18:42.348 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.377+0000 7f32a8c00640 1 -- 172.21.15.1:0/323582216 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f329c06beb0 con 0x7f32ac1098c0 2024-09-12T01:18:42.348 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.377+0000 7f32ab400640 1 -- 172.21.15.1:0/323582216 >> [v2:172.21.15.1:6800/2935968453,v1:172.21.15.1:6801/2935968453] conn(0x7f3290038fc0 msgr2=0x7f329003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/2935968453 2024-09-12T01:18:42.348 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.377+0000 7f32ab400640 1 --2- 172.21.15.1:0/323582216 >> [v2:172.21.15.1:6800/2935968453,v1:172.21.15.1:6801/2935968453] conn(0x7f3290038fc0 0x7f329003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T01:18:42.348 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.382+0000 7f32a8c00640 1 -- 172.21.15.1:0/323582216 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f329c045360 con 0x7f32ac1098c0 2024-09-12T01:18:42.348 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.578+0000 7f32ab400640 1 -- 172.21.15.1:0/323582216 >> [v2:172.21.15.1:6800/2935968453,v1:172.21.15.1:6801/2935968453] conn(0x7f3290038fc0 msgr2=0x7f329003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/2935968453 2024-09-12T01:18:42.348 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.578+0000 7f32ab400640 1 --2- 172.21.15.1:0/323582216 >> [v2:172.21.15.1:6800/2935968453,v1:172.21.15.1:6801/2935968453] conn(0x7f3290038fc0 0x7f329003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T01:18:42.348 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.616+0000 7f32b2a00640 1 -- 172.21.15.1:0/323582216 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f32ac02d7c0 con 0x7f32ac1098c0 2024-09-12T01:18:42.348 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.617+0000 7f32a8c00640 1 -- 172.21.15.1:0/323582216 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f329c06b8d0 con 0x7f32ac1098c0 2024-09-12T01:18:42.348 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.620+0000 7f32b2a00640 1 -- 172.21.15.1:0/323582216 >> [v2:172.21.15.1:6800/2935968453,v1:172.21.15.1:6801/2935968453] conn(0x7f3290038fc0 msgr2=0x7f329003b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T01:18:42.349 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.620+0000 7f32b2a00640 1 --2- 172.21.15.1:0/323582216 >> [v2:172.21.15.1:6800/2935968453,v1:172.21.15.1:6801/2935968453] conn(0x7f3290038fc0 0x7f329003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:42.349 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.620+0000 7f32b2a00640 1 -- 172.21.15.1:0/323582216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32ac1098c0 msgr2=0x7f32ac19db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:18:42.349 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.620+0000 7f32b2a00640 1 --2- 172.21.15.1:0/323582216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32ac1098c0 0x7f32ac19db00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f329c0097f0 tx=0x7f329c030ec0 comp rx=0 tx=0).stop 2024-09-12T01:18:42.349 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.620+0000 7f32b2a00640 1 -- 172.21.15.1:0/323582216 shutdown_connections 2024-09-12T01:18:42.349 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.620+0000 7f32b2a00640 1 --2- 172.21.15.1:0/323582216 >> [v2:172.21.15.1:6800/2935968453,v1:172.21.15.1:6801/2935968453] conn(0x7f3290038fc0 0x7f329003b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:42.349 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.620+0000 7f32b2a00640 1 --2- 172.21.15.1:0/323582216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32ac1098c0 0x7f32ac19db00 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:42.349 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.620+0000 7f32b2a00640 1 -- 172.21.15.1:0/323582216 >> 172.21.15.1:0/323582216 conn(0x7f32ac07ce20 msgr2=0x7f32ac1077d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:18:42.349 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.620+0000 7f32b2a00640 1 -- 172.21.15.1:0/323582216 shutdown_connections 2024-09-12T01:18:42.349 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:41.620+0000 7f32b2a00640 1 -- 172.21.15.1:0/323582216 wait complete. 2024-09-12T01:18:42.349 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for the mgr to restart... 2024-09-12T01:18:42.349 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for mgr epoch 5... 2024-09-12T01:18:42.808 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:42 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:42.554+0000 7fad105dc040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T01:18:42.809 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:42 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:42.640+0000 7fad105dc040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T01:18:43.921 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:43 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:43.608+0000 7fad105dc040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T01:18:43.921 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:43 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:43.750+0000 7fad105dc040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T01:18:43.921 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:43 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:43.920+0000 7fad105dc040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T01:18:44.173 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:43 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:43.995+0000 7fad105dc040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T01:18:44.173 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:44 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:44.172+0000 7fad105dc040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T01:18:44.427 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:44 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:44.257+0000 7fad105dc040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T01:18:44.428 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:44 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:44.332+0000 7fad105dc040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T01:18:44.784 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:44 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:44.520+0000 7fad105dc040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T01:18:44.784 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:44 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:44.600+0000 7fad105dc040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T01:18:44.784 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:44 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:44.683+0000 7fad105dc040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T01:18:44.784 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:44 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:44.783+0000 7fad105dc040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T01:18:45.177 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:44 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:44.857+0000 7fad105dc040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T01:18:45.177 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:44 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:44.930+0000 7fad105dc040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T01:18:46.177 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:45 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:45.879+0000 7fad105dc040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T01:18:46.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:45 smithi001 ceph-mon[30387]: Active manager daemon x restarted 2024-09-12T01:18:46.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:45 smithi001 ceph-mon[30387]: Activating manager daemon x 2024-09-12T01:18:46.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:45 smithi001 ceph-mon[30387]: osdmap e2: 0 total, 0 up, 0 in 2024-09-12T01:18:46.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:45 smithi001 ceph-mon[30387]: mgrmap e6: x(active, starting, since 0.00364158s) 2024-09-12T01:18:46.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:45 smithi001 ceph-mon[30387]: from='mgr.14118 172.21.15.1:0/1617053689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:18:46.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:45 smithi001 ceph-mon[30387]: from='mgr.14118 172.21.15.1:0/1617053689' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T01:18:46.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:45 smithi001 ceph-mon[30387]: from='mgr.14118 172.21.15.1:0/1617053689' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T01:18:46.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:45 smithi001 ceph-mon[30387]: from='mgr.14118 172.21.15.1:0/1617053689' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T01:18:46.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:45 smithi001 ceph-mon[30387]: from='mgr.14118 172.21.15.1:0/1617053689' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T01:18:46.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:45 smithi001 ceph-mon[30387]: Manager daemon x is now available 2024-09-12T01:18:46.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:45 smithi001 ceph-mon[30387]: from='mgr.14118 172.21.15.1:0/1617053689' entity='mgr.x' 2024-09-12T01:18:46.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:45 smithi001 ceph-mon[30387]: from='mgr.14118 172.21.15.1:0/1617053689' entity='mgr.x' 2024-09-12T01:18:46.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:45 smithi001 ceph-mon[30387]: from='mgr.14118 172.21.15.1:0/1617053689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:18:46.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:45 smithi001 ceph-mon[30387]: from='mgr.14118 172.21.15.1:0/1617053689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:18:47.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:46 smithi001 ceph-mon[30387]: Found migration_current of "None". Setting to last migration. 2024-09-12T01:18:47.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:46 smithi001 ceph-mon[30387]: from='mgr.14118 172.21.15.1:0/1617053689' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T01:18:47.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:46 smithi001 ceph-mon[30387]: from='mgr.14118 172.21.15.1:0/1617053689' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T01:18:47.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:46 smithi001 ceph-mon[30387]: from='mgr.14118 172.21.15.1:0/1617053689' entity='mgr.x' 2024-09-12T01:18:47.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:46 smithi001 ceph-mon[30387]: from='mgr.14118 172.21.15.1:0/1617053689' entity='mgr.x' 2024-09-12T01:18:47.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:46 smithi001 ceph-mon[30387]: mgrmap e7: x(active, since 1.0068s) 2024-09-12T01:18:47.593 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-09-12T01:18:47.600 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-12T01:18:47.600 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-12T01:18:47.600 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-09-12T01:18:47.600 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:42.920+0000 7fc44f200640 1 Processor -- start 2024-09-12T01:18:47.600 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:42.921+0000 7fc44f200640 1 -- start start 2024-09-12T01:18:47.601 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:42.921+0000 7fc44f200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4501062c0 0x7fc4501066c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:18:47.601 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:42.921+0000 7fc44f200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc450106c90 con 0x7fc4501062c0 2024-09-12T01:18:47.601 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:42.922+0000 7fc44de00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4501062c0 0x7fc4501066c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:18:47.601 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:42.922+0000 7fc44de00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4501062c0 0x7fc4501066c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40254/0 (socket says 172.21.15.1:40254) 2024-09-12T01:18:47.601 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:42.922+0000 7fc44de00640 1 -- 172.21.15.1:0/155305720 learned_addr learned my addr 172.21.15.1:0/155305720 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:18:47.601 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:42.922+0000 7fc44de00640 1 -- 172.21.15.1:0/155305720 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4501074c0 con 0x7fc4501062c0 2024-09-12T01:18:47.601 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:42.923+0000 7fc44de00640 1 --2- 172.21.15.1:0/155305720 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4501062c0 0x7fc4501066c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc440009820 tx=0x7fc44002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a6514ddfae7578d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:18:47.601 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:42.923+0000 7fc44ca00640 1 -- 172.21.15.1:0/155305720 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc440037070 con 0x7fc4501062c0 2024-09-12T01:18:47.601 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:42.923+0000 7fc44ca00640 1 -- 172.21.15.1:0/155305720 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc440032900 con 0x7fc4501062c0 2024-09-12T01:18:47.601 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:42.923+0000 7fc44ca00640 1 -- 172.21.15.1:0/155305720 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc440030850 con 0x7fc4501062c0 2024-09-12T01:18:47.601 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:42.924+0000 7fc44f200640 1 -- 172.21.15.1:0/155305720 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4501062c0 msgr2=0x7fc4501066c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:18:47.601 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:42.924+0000 7fc44f200640 1 --2- 172.21.15.1:0/155305720 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4501062c0 0x7fc4501066c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc440009820 tx=0x7fc44002a270 comp rx=0 tx=0).stop 2024-09-12T01:18:47.601 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:42.924+0000 7fc44f200640 1 -- 172.21.15.1:0/155305720 shutdown_connections 2024-09-12T01:18:47.602 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:42.924+0000 7fc44f200640 1 --2- 172.21.15.1:0/155305720 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4501062c0 0x7fc4501066c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:47.602 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:42.924+0000 7fc44f200640 1 -- 172.21.15.1:0/155305720 >> 172.21.15.1:0/155305720 conn(0x7fc450101a90 msgr2=0x7fc450103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:18:47.602 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:42.924+0000 7fc44f200640 1 -- 172.21.15.1:0/155305720 shutdown_connections 2024-09-12T01:18:47.602 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:42.925+0000 7fc44f200640 1 -- 172.21.15.1:0/155305720 wait complete. 2024-09-12T01:18:47.602 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:42.925+0000 7fc44f200640 1 Processor -- start 2024-09-12T01:18:47.602 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:42.926+0000 7fc44f200640 1 -- start start 2024-09-12T01:18:47.602 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:42.926+0000 7fc44f200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4501062c0 0x7fc45019d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:18:47.602 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:42.926+0000 7fc44f200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc45010a870 con 0x7fc4501062c0 2024-09-12T01:18:47.602 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:42.926+0000 7fc44de00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4501062c0 0x7fc45019d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:18:47.602 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:42.926+0000 7fc44de00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4501062c0 0x7fc45019d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40266/0 (socket says 172.21.15.1:40266) 2024-09-12T01:18:47.602 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:42.926+0000 7fc44de00640 1 -- 172.21.15.1:0/1811888424 learned_addr learned my addr 172.21.15.1:0/1811888424 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:18:47.602 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:42.927+0000 7fc44de00640 1 -- 172.21.15.1:0/1811888424 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc45019db60 con 0x7fc4501062c0 2024-09-12T01:18:47.602 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:42.927+0000 7fc44de00640 1 --2- 172.21.15.1:0/1811888424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4501062c0 0x7fc45019d620 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc440009940 tx=0x7fc440032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:18:47.602 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:42.927+0000 7fc446a00640 1 -- 172.21.15.1:0/1811888424 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc440042070 con 0x7fc4501062c0 2024-09-12T01:18:47.603 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:42.927+0000 7fc446a00640 1 -- 172.21.15.1:0/1811888424 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc4400308c0 con 0x7fc4501062c0 2024-09-12T01:18:47.603 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:42.927+0000 7fc44f200640 1 -- 172.21.15.1:0/1811888424 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc45019ddf0 con 0x7fc4501062c0 2024-09-12T01:18:47.603 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:42.927+0000 7fc446a00640 1 -- 172.21.15.1:0/1811888424 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc440037040 con 0x7fc4501062c0 2024-09-12T01:18:47.603 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:42.927+0000 7fc44f200640 1 -- 172.21.15.1:0/1811888424 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4501a0ae0 con 0x7fc4501062c0 2024-09-12T01:18:47.603 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:42.929+0000 7fc446a00640 1 -- 172.21.15.1:0/1811888424 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fc440045070 con 0x7fc4501062c0 2024-09-12T01:18:47.603 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:42.929+0000 7fc446a00640 1 --2- 172.21.15.1:0/1811888424 >> [v2:172.21.15.1:6800/2935968453,v1:172.21.15.1:6801/2935968453] conn(0x7fc434038fc0 0x7fc43403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:18:47.603 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:42.929+0000 7fc446a00640 1 -- 172.21.15.1:0/1811888424 --> [v2:172.21.15.1:6800/2935968453,v1:172.21.15.1:6801/2935968453] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc43403bb90 con 0x7fc434038fc0 2024-09-12T01:18:47.603 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:42.929+0000 7fc446a00640 1 -- 172.21.15.1:0/1811888424 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc44006c220 con 0x7fc4501062c0 2024-09-12T01:18:47.603 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:42.929+0000 7fc44d400640 1 -- 172.21.15.1:0/1811888424 >> [v2:172.21.15.1:6800/2935968453,v1:172.21.15.1:6801/2935968453] conn(0x7fc434038fc0 msgr2=0x7fc43403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/2935968453 2024-09-12T01:18:47.603 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:42.929+0000 7fc44d400640 1 --2- 172.21.15.1:0/1811888424 >> [v2:172.21.15.1:6800/2935968453,v1:172.21.15.1:6801/2935968453] conn(0x7fc434038fc0 0x7fc43403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T01:18:47.603 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:43.130+0000 7fc44d400640 1 -- 172.21.15.1:0/1811888424 >> [v2:172.21.15.1:6800/2935968453,v1:172.21.15.1:6801/2935968453] conn(0x7fc434038fc0 msgr2=0x7fc43403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/2935968453 2024-09-12T01:18:47.603 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:43.130+0000 7fc44d400640 1 --2- 172.21.15.1:0/1811888424 >> [v2:172.21.15.1:6800/2935968453,v1:172.21.15.1:6801/2935968453] conn(0x7fc434038fc0 0x7fc43403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T01:18:47.603 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:43.531+0000 7fc44d400640 1 -- 172.21.15.1:0/1811888424 >> [v2:172.21.15.1:6800/2935968453,v1:172.21.15.1:6801/2935968453] conn(0x7fc434038fc0 msgr2=0x7fc43403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/2935968453 2024-09-12T01:18:47.604 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:43.531+0000 7fc44d400640 1 --2- 172.21.15.1:0/1811888424 >> [v2:172.21.15.1:6800/2935968453,v1:172.21.15.1:6801/2935968453] conn(0x7fc434038fc0 0x7fc43403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-12T01:18:47.604 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:44.332+0000 7fc44d400640 1 -- 172.21.15.1:0/1811888424 >> [v2:172.21.15.1:6800/2935968453,v1:172.21.15.1:6801/2935968453] conn(0x7fc434038fc0 msgr2=0x7fc43403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/2935968453 2024-09-12T01:18:47.604 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:44.332+0000 7fc44d400640 1 --2- 172.21.15.1:0/1811888424 >> [v2:172.21.15.1:6800/2935968453,v1:172.21.15.1:6801/2935968453] conn(0x7fc434038fc0 0x7fc43403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-12T01:18:47.604 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:45.883+0000 7fc446a00640 1 -- 172.21.15.1:0/1811888424 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7fc44006ad40 con 0x7fc4501062c0 2024-09-12T01:18:47.604 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:45.883+0000 7fc446a00640 1 -- 172.21.15.1:0/1811888424 >> [v2:172.21.15.1:6800/2935968453,v1:172.21.15.1:6801/2935968453] conn(0x7fc434038fc0 msgr2=0x7fc43403b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T01:18:47.604 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:45.883+0000 7fc446a00640 1 --2- 172.21.15.1:0/1811888424 >> [v2:172.21.15.1:6800/2935968453,v1:172.21.15.1:6801/2935968453] conn(0x7fc434038fc0 0x7fc43403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:47.604 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:46.886+0000 7fc446a00640 1 -- 172.21.15.1:0/1811888424 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7fc440042d00 con 0x7fc4501062c0 2024-09-12T01:18:47.604 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:46.886+0000 7fc446a00640 1 --2- 172.21.15.1:0/1811888424 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7fc43403c8f0 0x7fc43403ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:18:47.604 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:46.886+0000 7fc446a00640 1 -- 172.21.15.1:0/1811888424 --> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc44006b990 con 0x7fc43403c8f0 2024-09-12T01:18:47.604 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:46.887+0000 7fc44d400640 1 --2- 172.21.15.1:0/1811888424 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7fc43403c8f0 0x7fc43403ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:18:47.604 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:46.888+0000 7fc44d400640 1 --2- 172.21.15.1:0/1811888424 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7fc43403c8f0 0x7fc43403ece0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc43c003980 tx=0x7fc43c0092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:18:47.604 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:46.888+0000 7fc446a00640 1 -- 172.21.15.1:0/1811888424 <== mgr.14118 v2:172.21.15.1:6800/4101281615 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fc44006b990 con 0x7fc43403c8f0 2024-09-12T01:18:47.604 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:46.896+0000 7fc43be00640 1 -- 172.21.15.1:0/1811888424 --> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fc428002570 con 0x7fc43403c8f0 2024-09-12T01:18:47.604 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:46.896+0000 7fc446a00640 1 -- 172.21.15.1:0/1811888424 <== mgr.14118 v2:172.21.15.1:6800/4101281615 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fc428002570 con 0x7fc43403c8f0 2024-09-12T01:18:47.605 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:46.897+0000 7fc43be00640 1 -- 172.21.15.1:0/1811888424 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7fc43403c8f0 msgr2=0x7fc43403ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:18:47.605 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:46.897+0000 7fc43be00640 1 --2- 172.21.15.1:0/1811888424 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7fc43403c8f0 0x7fc43403ece0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc43c003980 tx=0x7fc43c0092b0 comp rx=0 tx=0).stop 2024-09-12T01:18:47.605 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:46.897+0000 7fc43be00640 1 -- 172.21.15.1:0/1811888424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4501062c0 msgr2=0x7fc45019d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:18:47.605 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:46.897+0000 7fc43be00640 1 --2- 172.21.15.1:0/1811888424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4501062c0 0x7fc45019d620 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc440009940 tx=0x7fc440032ce0 comp rx=0 tx=0).stop 2024-09-12T01:18:47.605 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:46.897+0000 7fc43be00640 1 -- 172.21.15.1:0/1811888424 shutdown_connections 2024-09-12T01:18:47.605 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:46.897+0000 7fc43be00640 1 --2- 172.21.15.1:0/1811888424 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7fc43403c8f0 0x7fc43403ece0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:47.605 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:46.897+0000 7fc43be00640 1 --2- 172.21.15.1:0/1811888424 >> [v2:172.21.15.1:6800/2935968453,v1:172.21.15.1:6801/2935968453] conn(0x7fc434038fc0 0x7fc43403b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:47.605 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:46.897+0000 7fc43be00640 1 --2- 172.21.15.1:0/1811888424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4501062c0 0x7fc45019d620 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:47.605 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:46.897+0000 7fc43be00640 1 -- 172.21.15.1:0/1811888424 >> 172.21.15.1:0/1811888424 conn(0x7fc450101a90 msgr2=0x7fc4501029b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:18:47.605 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:46.897+0000 7fc43be00640 1 -- 172.21.15.1:0/1811888424 shutdown_connections 2024-09-12T01:18:47.605 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:46.897+0000 7fc43be00640 1 -- 172.21.15.1:0/1811888424 wait complete. 2024-09-12T01:18:47.605 INFO:teuthology.orchestra.run.smithi001.stdout:mgr epoch 5 is available 2024-09-12T01:18:47.605 INFO:teuthology.orchestra.run.smithi001.stdout:Setting orchestrator backend to cephadm... 2024-09-12T01:18:48.100 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:47 smithi001 ceph-mon[30387]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-12T01:18:48.100 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:47 smithi001 ceph-mon[30387]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-12T01:18:48.928 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:48 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:48.494+0000 7fac67e00640 -1 log_channel(cephadm) log [ERR] : [12/Sep/2024:01:18:48] ENGINE Error in HTTPServer.serve 2024-09-12T01:18:48.928 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:48 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: Traceback (most recent call last): 2024-09-12T01:18:48.928 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:48 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T01:18:48.928 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:48 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: self._connections.run(self.expiration_interval) 2024-09-12T01:18:48.928 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:48 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T01:18:48.928 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:48 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: self._run(expiration_interval) 2024-09-12T01:18:48.928 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:48 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T01:18:48.928 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:48 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T01:18:48.928 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:48 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T01:18:48.928 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:48 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T01:18:48.929 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:48 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T01:18:48.929 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:48 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: s = self.context.wrap_socket( 2024-09-12T01:18:48.929 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:48 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T01:18:48.929 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:48 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: return self.sslsocket_class._create( 2024-09-12T01:18:48.929 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:48 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T01:18:48.929 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:48 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: self.do_handshake() 2024-09-12T01:18:48.929 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:48 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T01:18:48.929 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:48 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: self._sslobj.do_handshake() 2024-09-12T01:18:48.929 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:48 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T01:18:48.929 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:48 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:18:48.989 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.100+0000 7f6e7f800640 1 Processor -- start 2024-09-12T01:18:48.989 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.100+0000 7f6e7f800640 1 -- start start 2024-09-12T01:18:48.989 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.101+0000 7f6e7f800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e781098c0 0x7f6e78109cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:18:48.989 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.101+0000 7f6e7f800640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6e7810a290 con 0x7f6e781098c0 2024-09-12T01:18:48.989 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.101+0000 7f6e7ce00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e781098c0 0x7f6e78109cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:18:48.989 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.101+0000 7f6e7ce00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e781098c0 0x7f6e78109cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40316/0 (socket says 172.21.15.1:40316) 2024-09-12T01:18:48.989 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.101+0000 7f6e7ce00640 1 -- 172.21.15.1:0/3802427264 learned_addr learned my addr 172.21.15.1:0/3802427264 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:18:48.990 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.101+0000 7f6e7ce00640 1 -- 172.21.15.1:0/3802427264 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e7810a9b0 con 0x7f6e781098c0 2024-09-12T01:18:48.990 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.103+0000 7f6e7ce00640 1 --2- 172.21.15.1:0/3802427264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e781098c0 0x7f6e78109cc0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c009820 tx=0x7f6e6c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcdb2335ba60a51d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:18:48.990 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.103+0000 7f6e77400640 1 -- 172.21.15.1:0/3802427264 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e6c037070 con 0x7f6e781098c0 2024-09-12T01:18:48.990 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.103+0000 7f6e77400640 1 -- 172.21.15.1:0/3802427264 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6e6c032900 con 0x7f6e781098c0 2024-09-12T01:18:48.990 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.103+0000 7f6e77400640 1 -- 172.21.15.1:0/3802427264 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e6c030850 con 0x7f6e781098c0 2024-09-12T01:18:48.990 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.104+0000 7f6e7f800640 1 -- 172.21.15.1:0/3802427264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e781098c0 msgr2=0x7f6e78109cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:18:48.990 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.104+0000 7f6e7f800640 1 --2- 172.21.15.1:0/3802427264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e781098c0 0x7f6e78109cc0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c009820 tx=0x7f6e6c02a270 comp rx=0 tx=0).stop 2024-09-12T01:18:48.990 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.104+0000 7f6e7f800640 1 -- 172.21.15.1:0/3802427264 shutdown_connections 2024-09-12T01:18:48.990 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.104+0000 7f6e7f800640 1 --2- 172.21.15.1:0/3802427264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e781098c0 0x7f6e78109cc0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:48.990 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.104+0000 7f6e7f800640 1 -- 172.21.15.1:0/3802427264 >> 172.21.15.1:0/3802427264 conn(0x7f6e7807ce20 msgr2=0x7f6e7807d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:18:48.990 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.104+0000 7f6e7f800640 1 -- 172.21.15.1:0/3802427264 shutdown_connections 2024-09-12T01:18:48.990 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.104+0000 7f6e7f800640 1 -- 172.21.15.1:0/3802427264 wait complete. 2024-09-12T01:18:48.990 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.105+0000 7f6e7f800640 1 Processor -- start 2024-09-12T01:18:48.990 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.105+0000 7f6e7f800640 1 -- start start 2024-09-12T01:18:48.991 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.106+0000 7f6e7f800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e781098c0 0x7f6e7819db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:18:48.991 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.106+0000 7f6e7f800640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6e7810dd60 con 0x7f6e781098c0 2024-09-12T01:18:48.991 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.106+0000 7f6e7ce00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e781098c0 0x7f6e7819db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:18:48.991 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.106+0000 7f6e7ce00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e781098c0 0x7f6e7819db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40322/0 (socket says 172.21.15.1:40322) 2024-09-12T01:18:48.991 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.106+0000 7f6e7ce00640 1 -- 172.21.15.1:0/2124003513 learned_addr learned my addr 172.21.15.1:0/2124003513 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:18:48.991 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.106+0000 7f6e7ce00640 1 -- 172.21.15.1:0/2124003513 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e7819e040 con 0x7f6e781098c0 2024-09-12T01:18:48.991 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.107+0000 7f6e7ce00640 1 --2- 172.21.15.1:0/2124003513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e781098c0 0x7f6e7819db00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c0097f0 tx=0x7f6e6c030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:18:48.991 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.107+0000 7f6e75600640 1 -- 172.21.15.1:0/2124003513 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e6c042070 con 0x7f6e781098c0 2024-09-12T01:18:48.991 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.107+0000 7f6e75600640 1 -- 172.21.15.1:0/2124003513 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6e6c032ca0 con 0x7f6e781098c0 2024-09-12T01:18:48.991 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.107+0000 7f6e75600640 1 -- 172.21.15.1:0/2124003513 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e6c037040 con 0x7f6e781098c0 2024-09-12T01:18:48.991 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.107+0000 7f6e7f800640 1 -- 172.21.15.1:0/2124003513 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e7819e2d0 con 0x7f6e781098c0 2024-09-12T01:18:48.991 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.107+0000 7f6e7f800640 1 -- 172.21.15.1:0/2124003513 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e781a0fc0 con 0x7f6e781098c0 2024-09-12T01:18:48.991 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.108+0000 7f6e75600640 1 -- 172.21.15.1:0/2124003513 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f6e6c045070 con 0x7f6e781098c0 2024-09-12T01:18:48.991 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.108+0000 7f6e7f800640 1 -- 172.21.15.1:0/2124003513 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e7810dee0 con 0x7f6e781098c0 2024-09-12T01:18:48.992 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.109+0000 7f6e75600640 1 --2- 172.21.15.1:0/2124003513 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f6e60038fc0 0x7f6e6003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:18:48.992 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.109+0000 7f6e75600640 1 -- 172.21.15.1:0/2124003513 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6e6c06bf40 con 0x7f6e781098c0 2024-09-12T01:18:48.992 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.109+0000 7f6e77e00640 1 --2- 172.21.15.1:0/2124003513 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f6e60038fc0 0x7f6e6003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:18:48.992 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.110+0000 7f6e77e00640 1 --2- 172.21.15.1:0/2124003513 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f6e60038fc0 0x7f6e6003b480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6e640098c0 tx=0x7f6e64006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:18:48.992 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.114+0000 7f6e75600640 1 -- 172.21.15.1:0/2124003513 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6e6c04b170 con 0x7f6e781098c0 2024-09-12T01:18:48.992 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.318+0000 7f6e7f800640 1 -- 172.21.15.1:0/2124003513 --> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f6e78062630 con 0x7f6e60038fc0 2024-09-12T01:18:48.992 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.324+0000 7f6e75600640 1 -- 172.21.15.1:0/2124003513 <== mgr.14118 v2:172.21.15.1:6800/4101281615 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6e78062630 con 0x7f6e60038fc0 2024-09-12T01:18:48.992 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.326+0000 7f6e7f800640 1 -- 172.21.15.1:0/2124003513 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f6e60038fc0 msgr2=0x7f6e6003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:18:48.992 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.326+0000 7f6e7f800640 1 --2- 172.21.15.1:0/2124003513 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f6e60038fc0 0x7f6e6003b480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6e640098c0 tx=0x7f6e64006db0 comp rx=0 tx=0).stop 2024-09-12T01:18:48.992 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.326+0000 7f6e7f800640 1 -- 172.21.15.1:0/2124003513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e781098c0 msgr2=0x7f6e7819db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:18:48.992 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.326+0000 7f6e7f800640 1 --2- 172.21.15.1:0/2124003513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e781098c0 0x7f6e7819db00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c0097f0 tx=0x7f6e6c030ec0 comp rx=0 tx=0).stop 2024-09-12T01:18:48.992 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.326+0000 7f6e7f800640 1 -- 172.21.15.1:0/2124003513 shutdown_connections 2024-09-12T01:18:48.992 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.326+0000 7f6e7f800640 1 --2- 172.21.15.1:0/2124003513 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f6e60038fc0 0x7f6e6003b480 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:48.992 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.326+0000 7f6e7f800640 1 --2- 172.21.15.1:0/2124003513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e781098c0 0x7f6e7819db00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:48.993 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.326+0000 7f6e7f800640 1 -- 172.21.15.1:0/2124003513 >> 172.21.15.1:0/2124003513 conn(0x7f6e7807ce20 msgr2=0x7f6e781077d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:18:48.993 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.327+0000 7f6e7f800640 1 -- 172.21.15.1:0/2124003513 shutdown_connections 2024-09-12T01:18:48.993 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:48.327+0000 7f6e7f800640 1 -- 172.21.15.1:0/2124003513 wait complete. 2024-09-12T01:18:49.263 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:48 smithi001 ceph-mon[30387]: mgrmap e8: x(active, since 2s) 2024-09-12T01:18:49.264 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:48 smithi001 ceph-mon[30387]: [12/Sep/2024:01:18:48] ENGINE Bus STARTING 2024-09-12T01:18:49.264 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:48 smithi001 ceph-mon[30387]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:18:49.264 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:48 smithi001 ceph-mon[30387]: from='mgr.14118 172.21.15.1:0/1617053689' entity='mgr.x' 2024-09-12T01:18:49.264 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:48 smithi001 ceph-mon[30387]: from='mgr.14118 172.21.15.1:0/1617053689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:18:49.264 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:48 smithi001 ceph-mon[30387]: [12/Sep/2024:01:18:48] ENGINE Serving on http://172.21.15.1:8765 2024-09-12T01:18:49.264 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:48 smithi001 ceph-mon[30387]: [12/Sep/2024:01:18:48] ENGINE Serving on https://172.21.15.1:7150 2024-09-12T01:18:49.264 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:48 smithi001 ceph-mon[30387]: [12/Sep/2024:01:18:48] ENGINE Bus STARTED 2024-09-12T01:18:49.264 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:48 smithi001 ceph-mon[30387]: from='mgr.14118 172.21.15.1:0/1617053689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:18:49.264 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:48 smithi001 ceph-mon[30387]: [12/Sep/2024:01:18:48] ENGINE Error in HTTPServer.serve 2024-09-12T01:18:49.264 INFO:journalctl@ceph.mon.a.smithi001.stdout: Traceback (most recent call last): 2024-09-12T01:18:49.264 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T01:18:49.264 INFO:journalctl@ceph.mon.a.smithi001.stdout: self._connections.run(self.expiration_interval) 2024-09-12T01:18:49.265 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T01:18:49.265 INFO:journalctl@ceph.mon.a.smithi001.stdout: self._run(expiration_interval) 2024-09-12T01:18:49.265 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T01:18:49.265 INFO:journalctl@ceph.mon.a.smithi001.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T01:18:49.265 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T01:18:49.265 INFO:journalctl@ceph.mon.a.smithi001.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T01:18:49.265 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T01:18:49.265 INFO:journalctl@ceph.mon.a.smithi001.stdout: s = self.context.wrap_socket( 2024-09-12T01:18:49.265 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T01:18:49.265 INFO:journalctl@ceph.mon.a.smithi001.stdout: return self.sslsocket_class._create( 2024-09-12T01:18:49.265 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T01:18:49.265 INFO:journalctl@ceph.mon.a.smithi001.stdout: self.do_handshake() 2024-09-12T01:18:49.265 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T01:18:49.265 INFO:journalctl@ceph.mon.a.smithi001.stdout: self._sslobj.do_handshake() 2024-09-12T01:18:49.266 INFO:journalctl@ceph.mon.a.smithi001.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T01:18:50.382 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:49 smithi001 ceph-mon[30387]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:18:50.393 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-12T01:18:50.393 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.479+0000 7f1712e00640 1 Processor -- start 2024-09-12T01:18:50.393 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.479+0000 7f1712e00640 1 -- start start 2024-09-12T01:18:50.393 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.480+0000 7f1712e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f170c107600 0x7f170c107a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:18:50.394 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.480+0000 7f1712e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f170c107f40 con 0x7f170c107600 2024-09-12T01:18:50.394 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.480+0000 7f170be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f170c107600 0x7f170c107a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:18:50.394 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.480+0000 7f170be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f170c107600 0x7f170c107a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55150/0 (socket says 172.21.15.1:55150) 2024-09-12T01:18:50.394 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.480+0000 7f170be00640 1 -- 172.21.15.1:0/3502541564 learned_addr learned my addr 172.21.15.1:0/3502541564 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:18:50.394 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.481+0000 7f170be00640 1 -- 172.21.15.1:0/3502541564 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f170c1080c0 con 0x7f170c107600 2024-09-12T01:18:50.394 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.482+0000 7f170be00640 1 --2- 172.21.15.1:0/3502541564 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f170c107600 0x7f170c107a00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f16fc009820 tx=0x7f16fc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c9eb2f7db76ff79 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:18:50.395 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.482+0000 7f170aa00640 1 -- 172.21.15.1:0/3502541564 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16fc037070 con 0x7f170c107600 2024-09-12T01:18:50.395 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.482+0000 7f170aa00640 1 -- 172.21.15.1:0/3502541564 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f16fc032900 con 0x7f170c107600 2024-09-12T01:18:50.395 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.482+0000 7f170aa00640 1 -- 172.21.15.1:0/3502541564 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16fc030850 con 0x7f170c107600 2024-09-12T01:18:50.395 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.482+0000 7f1712e00640 1 -- 172.21.15.1:0/3502541564 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f170c107600 msgr2=0x7f170c107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:18:50.395 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.482+0000 7f1712e00640 1 --2- 172.21.15.1:0/3502541564 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f170c107600 0x7f170c107a00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f16fc009820 tx=0x7f16fc02a270 comp rx=0 tx=0).stop 2024-09-12T01:18:50.396 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.483+0000 7f1712e00640 1 -- 172.21.15.1:0/3502541564 shutdown_connections 2024-09-12T01:18:50.396 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.483+0000 7f1712e00640 1 --2- 172.21.15.1:0/3502541564 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f170c107600 0x7f170c107a00 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:50.396 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.483+0000 7f1712e00640 1 -- 172.21.15.1:0/3502541564 >> 172.21.15.1:0/3502541564 conn(0x7f170c102e50 msgr2=0x7f170c105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:18:50.396 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.483+0000 7f1712e00640 1 -- 172.21.15.1:0/3502541564 shutdown_connections 2024-09-12T01:18:50.396 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.483+0000 7f1712e00640 1 -- 172.21.15.1:0/3502541564 wait complete. 2024-09-12T01:18:50.396 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.484+0000 7f1712e00640 1 Processor -- start 2024-09-12T01:18:50.396 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.484+0000 7f1712e00640 1 -- start start 2024-09-12T01:18:50.397 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.484+0000 7f1712e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f170c107600 0x7f170c19d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:18:50.397 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.484+0000 7f1712e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f170c10ba70 con 0x7f170c107600 2024-09-12T01:18:50.397 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.484+0000 7f170be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f170c107600 0x7f170c19d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:18:50.397 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.484+0000 7f170be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f170c107600 0x7f170c19d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55166/0 (socket says 172.21.15.1:55166) 2024-09-12T01:18:50.397 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.484+0000 7f170be00640 1 -- 172.21.15.1:0/3556355445 learned_addr learned my addr 172.21.15.1:0/3556355445 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:18:50.397 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.485+0000 7f170be00640 1 -- 172.21.15.1:0/3556355445 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f170c19dce0 con 0x7f170c107600 2024-09-12T01:18:50.397 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.485+0000 7f170be00640 1 --2- 172.21.15.1:0/3556355445 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f170c107600 0x7f170c19d7a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0097f0 tx=0x7f16fc030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:18:50.398 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.485+0000 7f1708c00640 1 -- 172.21.15.1:0/3556355445 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16fc037040 con 0x7f170c107600 2024-09-12T01:18:50.398 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.485+0000 7f1708c00640 1 -- 172.21.15.1:0/3556355445 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f16fc040070 con 0x7f170c107600 2024-09-12T01:18:50.398 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.485+0000 7f1712e00640 1 -- 172.21.15.1:0/3556355445 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f170c19df70 con 0x7f170c107600 2024-09-12T01:18:50.398 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.485+0000 7f1708c00640 1 -- 172.21.15.1:0/3556355445 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16fc03a390 con 0x7f170c107600 2024-09-12T01:18:50.398 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.485+0000 7f1712e00640 1 -- 172.21.15.1:0/3556355445 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f170c1a0c60 con 0x7f170c107600 2024-09-12T01:18:50.398 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.487+0000 7f1712e00640 1 -- 172.21.15.1:0/3556355445 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f170c107a80 con 0x7f170c107600 2024-09-12T01:18:50.399 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.487+0000 7f1708c00640 1 -- 172.21.15.1:0/3556355445 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f16fc032900 con 0x7f170c107600 2024-09-12T01:18:50.399 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.487+0000 7f1708c00640 1 --2- 172.21.15.1:0/3556355445 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f16f0038fc0 0x7f16f003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:18:50.399 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.487+0000 7f1708c00640 1 -- 172.21.15.1:0/3556355445 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f16fc06c520 con 0x7f170c107600 2024-09-12T01:18:50.399 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.488+0000 7f170b400640 1 --2- 172.21.15.1:0/3556355445 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f16f0038fc0 0x7f16f003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:18:50.399 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.489+0000 7f170b400640 1 --2- 172.21.15.1:0/3556355445 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f16f0038fc0 0x7f16f003b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f16f80098c0 tx=0x7f16f8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:18:50.400 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.492+0000 7f1708c00640 1 -- 172.21.15.1:0/3556355445 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f16fc06be70 con 0x7f170c107600 2024-09-12T01:18:50.400 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.683+0000 7f1712e00640 1 -- 172.21.15.1:0/3556355445 --> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f170c103fd0 con 0x7f16f0038fc0 2024-09-12T01:18:50.400 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.684+0000 7f1708c00640 1 -- 172.21.15.1:0/3556355445 <== mgr.14118 v2:172.21.15.1:6800/4101281615 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f170c103fd0 con 0x7f16f0038fc0 2024-09-12T01:18:50.400 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.686+0000 7f1712e00640 1 -- 172.21.15.1:0/3556355445 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f16f0038fc0 msgr2=0x7f16f003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:18:50.400 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.686+0000 7f1712e00640 1 --2- 172.21.15.1:0/3556355445 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f16f0038fc0 0x7f16f003b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f16f80098c0 tx=0x7f16f8006db0 comp rx=0 tx=0).stop 2024-09-12T01:18:50.401 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.686+0000 7f1712e00640 1 -- 172.21.15.1:0/3556355445 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f170c107600 msgr2=0x7f170c19d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:18:50.401 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.686+0000 7f1712e00640 1 --2- 172.21.15.1:0/3556355445 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f170c107600 0x7f170c19d7a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0097f0 tx=0x7f16fc030d80 comp rx=0 tx=0).stop 2024-09-12T01:18:50.401 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.686+0000 7f1712e00640 1 -- 172.21.15.1:0/3556355445 shutdown_connections 2024-09-12T01:18:50.401 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.686+0000 7f1712e00640 1 --2- 172.21.15.1:0/3556355445 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f16f0038fc0 0x7f16f003b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:50.401 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.686+0000 7f1712e00640 1 --2- 172.21.15.1:0/3556355445 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f170c107600 0x7f170c19d7a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:50.401 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.686+0000 7f1712e00640 1 -- 172.21.15.1:0/3556355445 >> 172.21.15.1:0/3556355445 conn(0x7f170c102e50 msgr2=0x7f170c1038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:18:50.401 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.686+0000 7f1712e00640 1 -- 172.21.15.1:0/3556355445 shutdown_connections 2024-09-12T01:18:50.401 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:49.686+0000 7f1712e00640 1 -- 172.21.15.1:0/3556355445 wait complete. 2024-09-12T01:18:50.401 INFO:teuthology.orchestra.run.smithi001.stdout:Generating ssh key... 2024-09-12T01:18:51.928 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:51 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: Generating public/private rsa key pair. 2024-09-12T01:18:51.928 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:51 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: Your identification has been saved in /tmp/tmpn67m_5hk/key 2024-09-12T01:18:51.928 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:51 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: Your public key has been saved in /tmp/tmpn67m_5hk/key.pub 2024-09-12T01:18:51.928 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:51 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: The key fingerprint is: 2024-09-12T01:18:51.928 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:51 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: SHA256:uC4/pYCfe1noiQGLd4TnKAtXPSPWyulHRP93Uzygvmk ceph-c0e4f180-70a4-11ef-bcea-c7b262605968 2024-09-12T01:18:51.928 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:51 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: The key's randomart image is: 2024-09-12T01:18:51.929 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:51 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: +---[RSA 3072]----+ 2024-09-12T01:18:51.929 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:51 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: | | 2024-09-12T01:18:51.929 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:51 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: | . . | 2024-09-12T01:18:51.929 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:51 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: | . + . . .. | 2024-09-12T01:18:51.929 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:51 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: | o * *.. . .o| 2024-09-12T01:18:51.929 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:51 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: | . % =.+S.. ..| 2024-09-12T01:18:51.929 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:51 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: |+ * O o.o ... o | 2024-09-12T01:18:51.929 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:51 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: |.= + B.* .o. . | 2024-09-12T01:18:51.929 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:51 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: |. *.O E | 2024-09-12T01:18:51.929 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:51 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: | .*o. . | 2024-09-12T01:18:51.929 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:18:51 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: +----[SHA256]-----+ 2024-09-12T01:18:52.248 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:50.902+0000 7f5de7000640 1 Processor -- start 2024-09-12T01:18:52.249 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:50.902+0000 7f5de7000640 1 -- start start 2024-09-12T01:18:52.249 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:50.902+0000 7f5de7000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5de0109910 0x7f5de0109d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:18:52.249 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:50.902+0000 7f5de7000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5de010a2e0 con 0x7f5de0109910 2024-09-12T01:18:52.249 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:50.902+0000 7f5de5c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5de0109910 0x7f5de0109d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:18:52.249 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:50.902+0000 7f5de5c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5de0109910 0x7f5de0109d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55170/0 (socket says 172.21.15.1:55170) 2024-09-12T01:18:52.249 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:50.902+0000 7f5de5c00640 1 -- 172.21.15.1:0/337628230 learned_addr learned my addr 172.21.15.1:0/337628230 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:18:52.249 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:50.903+0000 7f5de5c00640 1 -- 172.21.15.1:0/337628230 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de010a770 con 0x7f5de0109910 2024-09-12T01:18:52.249 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:50.904+0000 7f5de5c00640 1 --2- 172.21.15.1:0/337628230 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5de0109910 0x7f5de0109d10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4009e80 tx=0x7f5dd402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7889fc097410c4e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:18:52.249 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:50.904+0000 7f5ddfe00640 1 -- 172.21.15.1:0/337628230 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5dd4036070 con 0x7f5de0109910 2024-09-12T01:18:52.249 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:50.904+0000 7f5ddfe00640 1 -- 172.21.15.1:0/337628230 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5dd40318c0 con 0x7f5de0109910 2024-09-12T01:18:52.249 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:50.904+0000 7f5ddfe00640 1 -- 172.21.15.1:0/337628230 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5dd4007ab0 con 0x7f5de0109910 2024-09-12T01:18:52.249 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:50.905+0000 7f5de7000640 1 -- 172.21.15.1:0/337628230 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5de0109910 msgr2=0x7f5de0109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:18:52.249 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:50.905+0000 7f5de7000640 1 --2- 172.21.15.1:0/337628230 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5de0109910 0x7f5de0109d10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4009e80 tx=0x7f5dd402a7a0 comp rx=0 tx=0).stop 2024-09-12T01:18:52.249 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:50.905+0000 7f5de7000640 1 -- 172.21.15.1:0/337628230 shutdown_connections 2024-09-12T01:18:52.250 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:50.905+0000 7f5de7000640 1 --2- 172.21.15.1:0/337628230 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5de0109910 0x7f5de0109d10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:52.250 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:50.905+0000 7f5de7000640 1 -- 172.21.15.1:0/337628230 >> 172.21.15.1:0/337628230 conn(0x7f5de007cc40 msgr2=0x7f5de007d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:18:52.250 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:50.905+0000 7f5de7000640 1 -- 172.21.15.1:0/337628230 shutdown_connections 2024-09-12T01:18:52.250 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:50.905+0000 7f5de7000640 1 -- 172.21.15.1:0/337628230 wait complete. 2024-09-12T01:18:52.250 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:50.906+0000 7f5de7000640 1 Processor -- start 2024-09-12T01:18:52.250 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:50.906+0000 7f5de7000640 1 -- start start 2024-09-12T01:18:52.250 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:50.906+0000 7f5de7000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5de0109910 0x7f5de019d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:18:52.250 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:50.906+0000 7f5de7000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5de010db20 con 0x7f5de0109910 2024-09-12T01:18:52.250 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:50.906+0000 7f5de5c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5de0109910 0x7f5de019d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:18:52.250 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:50.906+0000 7f5de5c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5de0109910 0x7f5de019d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55172/0 (socket says 172.21.15.1:55172) 2024-09-12T01:18:52.250 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:50.906+0000 7f5de5c00640 1 -- 172.21.15.1:0/2099502104 learned_addr learned my addr 172.21.15.1:0/2099502104 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:18:52.250 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:50.907+0000 7f5de5c00640 1 -- 172.21.15.1:0/2099502104 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de019db10 con 0x7f5de0109910 2024-09-12T01:18:52.250 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:50.907+0000 7f5de5c00640 1 --2- 172.21.15.1:0/2099502104 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5de0109910 0x7f5de019d5d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5dd402acc0 tx=0x7f5dd4031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:18:52.250 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:50.907+0000 7f5dde000640 1 -- 172.21.15.1:0/2099502104 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5dd4036050 con 0x7f5de0109910 2024-09-12T01:18:52.251 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:50.907+0000 7f5dde000640 1 -- 172.21.15.1:0/2099502104 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5dd4037040 con 0x7f5de0109910 2024-09-12T01:18:52.251 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:50.907+0000 7f5dde000640 1 -- 172.21.15.1:0/2099502104 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5dd403d8c0 con 0x7f5de0109910 2024-09-12T01:18:52.251 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:50.907+0000 7f5de7000640 1 -- 172.21.15.1:0/2099502104 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5de019dda0 con 0x7f5de0109910 2024-09-12T01:18:52.251 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:50.908+0000 7f5de7000640 1 -- 172.21.15.1:0/2099502104 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5de01a0a90 con 0x7f5de0109910 2024-09-12T01:18:52.251 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:50.909+0000 7f5dde000640 1 -- 172.21.15.1:0/2099502104 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f5dd4044020 con 0x7f5de0109910 2024-09-12T01:18:52.251 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:50.909+0000 7f5de7000640 1 -- 172.21.15.1:0/2099502104 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5de0109d90 con 0x7f5de0109910 2024-09-12T01:18:52.251 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:50.909+0000 7f5dde000640 1 --2- 172.21.15.1:0/2099502104 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f5dc8038fc0 0x7f5dc803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:18:52.251 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:50.909+0000 7f5dde000640 1 -- 172.21.15.1:0/2099502104 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5dd4042020 con 0x7f5de0109910 2024-09-12T01:18:52.251 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:50.909+0000 7f5de5200640 1 --2- 172.21.15.1:0/2099502104 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f5dc8038fc0 0x7f5dc803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:18:52.251 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:50.910+0000 7f5de5200640 1 --2- 172.21.15.1:0/2099502104 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f5dc8038fc0 0x7f5dc803b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc0098c0 tx=0x7f5dcc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:18:52.251 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:50.914+0000 7f5dde000640 1 -- 172.21.15.1:0/2099502104 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5dd404b020 con 0x7f5de0109910 2024-09-12T01:18:52.251 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:51.102+0000 7f5de7000640 1 -- 172.21.15.1:0/2099502104 --> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f5de0107c70 con 0x7f5dc8038fc0 2024-09-12T01:18:52.251 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:51.579+0000 7f5dde000640 1 -- 172.21.15.1:0/2099502104 <== mgr.14118 v2:172.21.15.1:6800/4101281615 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5de0107c70 con 0x7f5dc8038fc0 2024-09-12T01:18:52.251 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:51.581+0000 7f5de7000640 1 -- 172.21.15.1:0/2099502104 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f5dc8038fc0 msgr2=0x7f5dc803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:18:52.252 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:51.581+0000 7f5de7000640 1 --2- 172.21.15.1:0/2099502104 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f5dc8038fc0 0x7f5dc803b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc0098c0 tx=0x7f5dcc006db0 comp rx=0 tx=0).stop 2024-09-12T01:18:52.252 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:51.581+0000 7f5de7000640 1 -- 172.21.15.1:0/2099502104 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5de0109910 msgr2=0x7f5de019d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:18:52.252 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:51.581+0000 7f5de7000640 1 --2- 172.21.15.1:0/2099502104 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5de0109910 0x7f5de019d5d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5dd402acc0 tx=0x7f5dd4031ca0 comp rx=0 tx=0).stop 2024-09-12T01:18:52.252 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:51.582+0000 7f5de7000640 1 -- 172.21.15.1:0/2099502104 shutdown_connections 2024-09-12T01:18:52.252 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:51.582+0000 7f5de7000640 1 --2- 172.21.15.1:0/2099502104 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f5dc8038fc0 0x7f5dc803b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:52.252 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:51.582+0000 7f5de7000640 1 --2- 172.21.15.1:0/2099502104 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5de0109910 0x7f5de019d5d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:52.253 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:51.582+0000 7f5de7000640 1 -- 172.21.15.1:0/2099502104 >> 172.21.15.1:0/2099502104 conn(0x7f5de007cc40 msgr2=0x7f5de0107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:18:52.253 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:51.582+0000 7f5de7000640 1 -- 172.21.15.1:0/2099502104 shutdown_connections 2024-09-12T01:18:52.253 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:51.582+0000 7f5de7000640 1 -- 172.21.15.1:0/2099502104 wait complete. 2024-09-12T01:18:52.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:52 smithi001 ceph-mon[30387]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:18:52.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:52 smithi001 ceph-mon[30387]: Generating ssh key... 2024-09-12T01:18:52.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:52 smithi001 ceph-mon[30387]: from='mgr.14118 172.21.15.1:0/1617053689' entity='mgr.x' 2024-09-12T01:18:52.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:52 smithi001 ceph-mon[30387]: from='mgr.14118 172.21.15.1:0/1617053689' entity='mgr.x' 2024-09-12T01:18:53.643 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9pYVDjjOEWSD7atjnaT7qHWC2nRy+lN8m2hT/GIIj2Wi1T+/jPeK2b3kcs/bC6X+8dz0nGj3hl0Ie1C1eYofICplHdYkquBSGRawo0m33jI2miveIeZqDuUR9b8yoEzBcrTSCfpQ146ZQRRAXKrcTeoDN9sfogLGpG/gM/l+0iSZrYsxaeBaD9VaMZGP+as8mlvU7BPBDpkn3UJIX2G/isEObv5DcQoB1+fVyFftMwLo5K5S7TisCux74Z3BY3NMJpK7y55PNfTtMJ0NjQralxPhXGebAaGXakChKa6GVnZRJKg0LX/tomFmQb2hbxIjAqQDmxGpnEbCx50sfQtYg2fWuEF2FlXH/dqemBzdIJ00XSDAlEGX2Ld3ZsL306HU8SU/Anx29Hi/sU7Fe4GbS7lr5CrorKPZ7QktEUDHZGSYI1lKMuTidi2+NOiCTEMOW7jcy9+CKH+LHm3zRslC2B5JvoG3XCKg6DJsCUdXR1f9g/bDanCa4f8v/r2uJ4iE= ceph-c0e4f180-70a4-11ef-bcea-c7b262605968 2024-09-12T01:18:53.643 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.734+0000 7f7aaf200640 1 Processor -- start 2024-09-12T01:18:53.643 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.735+0000 7f7aaf200640 1 -- start start 2024-09-12T01:18:53.643 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.735+0000 7f7aaf200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ab0107460 0x7f7ab0107860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:18:53.643 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.735+0000 7f7aaf200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab0107e30 con 0x7f7ab0107460 2024-09-12T01:18:53.643 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.735+0000 7f7aade00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ab0107460 0x7f7ab0107860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:18:53.644 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.735+0000 7f7aade00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ab0107460 0x7f7ab0107860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55178/0 (socket says 172.21.15.1:55178) 2024-09-12T01:18:53.644 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.735+0000 7f7aade00640 1 -- 172.21.15.1:0/3394653521 learned_addr learned my addr 172.21.15.1:0/3394653521 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:18:53.644 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.736+0000 7f7aade00640 1 -- 172.21.15.1:0/3394653521 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab01085c0 con 0x7f7ab0107460 2024-09-12T01:18:53.644 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.737+0000 7f7aade00640 1 --2- 172.21.15.1:0/3394653521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ab0107460 0x7f7ab0107860 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0009820 tx=0x7f7aa002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cab405c236ff507d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:18:53.644 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.737+0000 7f7aaca00640 1 -- 172.21.15.1:0/3394653521 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7aa0037070 con 0x7f7ab0107460 2024-09-12T01:18:53.644 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.737+0000 7f7aaca00640 1 -- 172.21.15.1:0/3394653521 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7aa0032900 con 0x7f7ab0107460 2024-09-12T01:18:53.644 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.737+0000 7f7aaca00640 1 -- 172.21.15.1:0/3394653521 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7aa0030850 con 0x7f7ab0107460 2024-09-12T01:18:53.644 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.738+0000 7f7aaf200640 1 -- 172.21.15.1:0/3394653521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ab0107460 msgr2=0x7f7ab0107860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:18:53.644 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.738+0000 7f7aaf200640 1 --2- 172.21.15.1:0/3394653521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ab0107460 0x7f7ab0107860 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0009820 tx=0x7f7aa002a270 comp rx=0 tx=0).stop 2024-09-12T01:18:53.644 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.738+0000 7f7aaf200640 1 -- 172.21.15.1:0/3394653521 shutdown_connections 2024-09-12T01:18:53.644 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.738+0000 7f7aaf200640 1 --2- 172.21.15.1:0/3394653521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ab0107460 0x7f7ab0107860 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:53.644 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.738+0000 7f7aaf200640 1 -- 172.21.15.1:0/3394653521 >> 172.21.15.1:0/3394653521 conn(0x7f7ab0102c30 msgr2=0x7f7ab0105050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:18:53.644 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.738+0000 7f7aaf200640 1 -- 172.21.15.1:0/3394653521 shutdown_connections 2024-09-12T01:18:53.644 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.738+0000 7f7aaf200640 1 -- 172.21.15.1:0/3394653521 wait complete. 2024-09-12T01:18:53.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.739+0000 7f7aaf200640 1 Processor -- start 2024-09-12T01:18:53.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.739+0000 7f7aaf200640 1 -- start start 2024-09-12T01:18:53.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.739+0000 7f7aaf200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ab0107460 0x7f7ab019d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:18:53.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.739+0000 7f7aaf200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab010b970 con 0x7f7ab0107460 2024-09-12T01:18:53.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.739+0000 7f7aade00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ab0107460 0x7f7ab019d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:18:53.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.739+0000 7f7aade00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ab0107460 0x7f7ab019d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55188/0 (socket says 172.21.15.1:55188) 2024-09-12T01:18:53.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.739+0000 7f7aade00640 1 -- 172.21.15.1:0/352295659 learned_addr learned my addr 172.21.15.1:0/352295659 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:18:53.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.740+0000 7f7aade00640 1 -- 172.21.15.1:0/352295659 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab019db60 con 0x7f7ab0107460 2024-09-12T01:18:53.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.740+0000 7f7aade00640 1 --2- 172.21.15.1:0/352295659 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ab0107460 0x7f7ab019d620 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7aa00097f0 tx=0x7f7aa0030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:18:53.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.740+0000 7f7aa6a00640 1 -- 172.21.15.1:0/352295659 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7aa0037040 con 0x7f7ab0107460 2024-09-12T01:18:53.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.740+0000 7f7aa6a00640 1 -- 172.21.15.1:0/352295659 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7aa0040070 con 0x7f7ab0107460 2024-09-12T01:18:53.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.740+0000 7f7aaf200640 1 -- 172.21.15.1:0/352295659 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ab019ddf0 con 0x7f7ab0107460 2024-09-12T01:18:53.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.740+0000 7f7aa6a00640 1 -- 172.21.15.1:0/352295659 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7aa003a390 con 0x7f7ab0107460 2024-09-12T01:18:53.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.740+0000 7f7aaf200640 1 -- 172.21.15.1:0/352295659 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ab01a0ae0 con 0x7f7ab0107460 2024-09-12T01:18:53.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.741+0000 7f7aa6a00640 1 -- 172.21.15.1:0/352295659 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f7aa003b430 con 0x7f7ab0107460 2024-09-12T01:18:53.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.741+0000 7f7aaf200640 1 -- 172.21.15.1:0/352295659 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ab01078e0 con 0x7f7ab0107460 2024-09-12T01:18:53.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.742+0000 7f7aa6a00640 1 --2- 172.21.15.1:0/352295659 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f7a94038f70 0x7f7a9403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:18:53.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.742+0000 7f7aa6a00640 1 -- 172.21.15.1:0/352295659 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7aa006c570 con 0x7f7ab0107460 2024-09-12T01:18:53.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.742+0000 7f7aad400640 1 --2- 172.21.15.1:0/352295659 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f7a94038f70 0x7f7a9403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:18:53.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.743+0000 7f7aad400640 1 --2- 172.21.15.1:0/352295659 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f7a94038f70 0x7f7a9403b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c0098c0 tx=0x7f7a9c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:18:53.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.747+0000 7f7aa6a00640 1 -- 172.21.15.1:0/352295659 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7aa006bec0 con 0x7f7ab0107460 2024-09-12T01:18:53.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.933+0000 7f7aaf200640 1 -- 172.21.15.1:0/352295659 --> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f7ab0103e70 con 0x7f7a94038f70 2024-09-12T01:18:53.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.933+0000 7f7aa6a00640 1 -- 172.21.15.1:0/352295659 <== mgr.14118 v2:172.21.15.1:6800/4101281615 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f7ab0103e70 con 0x7f7a94038f70 2024-09-12T01:18:53.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.936+0000 7f7aaf200640 1 -- 172.21.15.1:0/352295659 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f7a94038f70 msgr2=0x7f7a9403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:18:53.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.936+0000 7f7aaf200640 1 --2- 172.21.15.1:0/352295659 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f7a94038f70 0x7f7a9403b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c0098c0 tx=0x7f7a9c006db0 comp rx=0 tx=0).stop 2024-09-12T01:18:53.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.936+0000 7f7aaf200640 1 -- 172.21.15.1:0/352295659 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ab0107460 msgr2=0x7f7ab019d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:18:53.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.936+0000 7f7aaf200640 1 --2- 172.21.15.1:0/352295659 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ab0107460 0x7f7ab019d620 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7aa00097f0 tx=0x7f7aa0030d80 comp rx=0 tx=0).stop 2024-09-12T01:18:53.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.936+0000 7f7aaf200640 1 -- 172.21.15.1:0/352295659 shutdown_connections 2024-09-12T01:18:53.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.936+0000 7f7aaf200640 1 --2- 172.21.15.1:0/352295659 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f7a94038f70 0x7f7a9403b430 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:53.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.936+0000 7f7aaf200640 1 --2- 172.21.15.1:0/352295659 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ab0107460 0x7f7ab019d620 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:53.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.936+0000 7f7aaf200640 1 -- 172.21.15.1:0/352295659 >> 172.21.15.1:0/352295659 conn(0x7f7ab0102c30 msgr2=0x7f7ab0103760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:18:53.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.936+0000 7f7aaf200640 1 -- 172.21.15.1:0/352295659 shutdown_connections 2024-09-12T01:18:53.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:52.936+0000 7f7aaf200640 1 -- 172.21.15.1:0/352295659 wait complete. 2024-09-12T01:18:53.647 INFO:teuthology.orchestra.run.smithi001.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-12T01:18:53.647 INFO:teuthology.orchestra.run.smithi001.stdout:Adding key to root@localhost authorized_keys... 2024-09-12T01:18:53.648 INFO:teuthology.orchestra.run.smithi001.stdout:Adding host smithi001... 2024-09-12T01:18:54.137 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:53 smithi001 ceph-mon[30387]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:18:55.151 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:54 smithi001 ceph-mon[30387]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi001", "addr": "172.21.15.1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:18:56.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:55 smithi001 ceph-mon[30387]: Deploying cephadm binary to smithi001 2024-09-12T01:18:59.623 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout Added host 'smithi001' with addr '172.21.15.1' 2024-09-12T01:18:59.623 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:54.137+0000 7fe88be00640 1 Processor -- start 2024-09-12T01:18:59.623 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:54.138+0000 7fe88be00640 1 -- start start 2024-09-12T01:18:59.624 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:54.138+0000 7fe88be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe88c109910 0x7fe88c109d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:18:59.624 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:54.138+0000 7fe88be00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c10a2e0 con 0x7fe88c109910 2024-09-12T01:18:59.624 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:54.138+0000 7fe88aa00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe88c109910 0x7fe88c109d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:18:59.624 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:54.138+0000 7fe88aa00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe88c109910 0x7fe88c109d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55190/0 (socket says 172.21.15.1:55190) 2024-09-12T01:18:59.624 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:54.138+0000 7fe88aa00640 1 -- 172.21.15.1:0/1244053877 learned_addr learned my addr 172.21.15.1:0/1244053877 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:18:59.624 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:54.139+0000 7fe88aa00640 1 -- 172.21.15.1:0/1244053877 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe88c10a770 con 0x7fe88c109910 2024-09-12T01:18:59.624 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:54.140+0000 7fe88aa00640 1 --2- 172.21.15.1:0/1244053877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe88c109910 0x7fe88c109d10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe87c009820 tx=0x7fe87c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b542c47b9023dc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:18:59.624 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:54.140+0000 7fe88a000640 1 -- 172.21.15.1:0/1244053877 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe87c037070 con 0x7fe88c109910 2024-09-12T01:18:59.624 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:54.140+0000 7fe88a000640 1 -- 172.21.15.1:0/1244053877 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe87c032900 con 0x7fe88c109910 2024-09-12T01:18:59.624 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:54.140+0000 7fe88a000640 1 -- 172.21.15.1:0/1244053877 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe87c030850 con 0x7fe88c109910 2024-09-12T01:18:59.624 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:54.141+0000 7fe88be00640 1 -- 172.21.15.1:0/1244053877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe88c109910 msgr2=0x7fe88c109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:18:59.624 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:54.141+0000 7fe88be00640 1 --2- 172.21.15.1:0/1244053877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe88c109910 0x7fe88c109d10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe87c009820 tx=0x7fe87c02a270 comp rx=0 tx=0).stop 2024-09-12T01:18:59.625 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:54.141+0000 7fe88be00640 1 -- 172.21.15.1:0/1244053877 shutdown_connections 2024-09-12T01:18:59.625 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:54.141+0000 7fe88be00640 1 --2- 172.21.15.1:0/1244053877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe88c109910 0x7fe88c109d10 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:59.625 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:54.141+0000 7fe88be00640 1 -- 172.21.15.1:0/1244053877 >> 172.21.15.1:0/1244053877 conn(0x7fe88c07cc40 msgr2=0x7fe88c07d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:18:59.625 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:54.141+0000 7fe88be00640 1 -- 172.21.15.1:0/1244053877 shutdown_connections 2024-09-12T01:18:59.625 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:54.141+0000 7fe88be00640 1 -- 172.21.15.1:0/1244053877 wait complete. 2024-09-12T01:18:59.625 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:54.142+0000 7fe88be00640 1 Processor -- start 2024-09-12T01:18:59.625 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:54.142+0000 7fe88be00640 1 -- start start 2024-09-12T01:18:59.625 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:54.142+0000 7fe88be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe88c109910 0x7fe88c19d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:18:59.625 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:54.142+0000 7fe88be00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c10db20 con 0x7fe88c109910 2024-09-12T01:18:59.625 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:54.142+0000 7fe88aa00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe88c109910 0x7fe88c19d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:18:59.625 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:54.142+0000 7fe88aa00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe88c109910 0x7fe88c19d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55202/0 (socket says 172.21.15.1:55202) 2024-09-12T01:18:59.625 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:54.142+0000 7fe88aa00640 1 -- 172.21.15.1:0/1724343299 learned_addr learned my addr 172.21.15.1:0/1724343299 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:18:59.625 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:54.143+0000 7fe88aa00640 1 -- 172.21.15.1:0/1724343299 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe88c19db10 con 0x7fe88c109910 2024-09-12T01:18:59.625 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:54.143+0000 7fe88aa00640 1 --2- 172.21.15.1:0/1724343299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe88c109910 0x7fe88c19d5d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe87c009940 tx=0x7fe87c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:18:59.626 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:54.143+0000 7fe890a00640 1 -- 172.21.15.1:0/1724343299 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe87c042070 con 0x7fe88c109910 2024-09-12T01:18:59.626 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:54.143+0000 7fe890a00640 1 -- 172.21.15.1:0/1724343299 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe87c0308c0 con 0x7fe88c109910 2024-09-12T01:18:59.626 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:54.143+0000 7fe890a00640 1 -- 172.21.15.1:0/1724343299 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe87c037040 con 0x7fe88c109910 2024-09-12T01:18:59.626 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:54.143+0000 7fe88be00640 1 -- 172.21.15.1:0/1724343299 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe88c19dda0 con 0x7fe88c109910 2024-09-12T01:18:59.626 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:54.143+0000 7fe88be00640 1 -- 172.21.15.1:0/1724343299 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe88c1a0a90 con 0x7fe88c109910 2024-09-12T01:18:59.626 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:54.145+0000 7fe890a00640 1 -- 172.21.15.1:0/1724343299 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fe87c045070 con 0x7fe88c109910 2024-09-12T01:18:59.626 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:54.145+0000 7fe88be00640 1 -- 172.21.15.1:0/1724343299 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe88c109d90 con 0x7fe88c109910 2024-09-12T01:18:59.626 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:54.145+0000 7fe890a00640 1 --2- 172.21.15.1:0/1724343299 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7fe870038fc0 0x7fe87003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:18:59.626 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:54.145+0000 7fe890a00640 1 -- 172.21.15.1:0/1724343299 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe87c06c030 con 0x7fe88c109910 2024-09-12T01:18:59.626 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:54.145+0000 7fe883e00640 1 --2- 172.21.15.1:0/1724343299 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7fe870038fc0 0x7fe87003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:18:59.626 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:54.146+0000 7fe883e00640 1 --2- 172.21.15.1:0/1724343299 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7fe870038fc0 0x7fe87003b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe8740098c0 tx=0x7fe874006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:18:59.626 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:54.150+0000 7fe890a00640 1 -- 172.21.15.1:0/1724343299 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe87c04b170 con 0x7fe88c109910 2024-09-12T01:18:59.626 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:54.341+0000 7fe88be00640 1 -- 172.21.15.1:0/1724343299 --> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi001", "addr": "172.21.15.1", "target": ["mon-mgr", ""]}) -- 0x7fe88c107c70 con 0x7fe870038fc0 2024-09-12T01:18:59.627 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:58.901+0000 7fe890a00640 1 -- 172.21.15.1:0/1724343299 <== mgr.14118 v2:172.21.15.1:6800/4101281615 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7fe88c107c70 con 0x7fe870038fc0 2024-09-12T01:18:59.627 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:58.904+0000 7fe88be00640 1 -- 172.21.15.1:0/1724343299 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7fe870038fc0 msgr2=0x7fe87003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:18:59.627 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:58.904+0000 7fe88be00640 1 --2- 172.21.15.1:0/1724343299 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7fe870038fc0 0x7fe87003b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe8740098c0 tx=0x7fe874006db0 comp rx=0 tx=0).stop 2024-09-12T01:18:59.627 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:58.904+0000 7fe88be00640 1 -- 172.21.15.1:0/1724343299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe88c109910 msgr2=0x7fe88c19d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:18:59.627 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:58.904+0000 7fe88be00640 1 --2- 172.21.15.1:0/1724343299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe88c109910 0x7fe88c19d5d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe87c009940 tx=0x7fe87c032ce0 comp rx=0 tx=0).stop 2024-09-12T01:18:59.627 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:58.904+0000 7fe88be00640 1 -- 172.21.15.1:0/1724343299 shutdown_connections 2024-09-12T01:18:59.627 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:58.904+0000 7fe88be00640 1 --2- 172.21.15.1:0/1724343299 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7fe870038fc0 0x7fe87003b480 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:59.627 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:58.904+0000 7fe88be00640 1 --2- 172.21.15.1:0/1724343299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe88c109910 0x7fe88c19d5d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:18:59.627 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:58.904+0000 7fe88be00640 1 -- 172.21.15.1:0/1724343299 >> 172.21.15.1:0/1724343299 conn(0x7fe88c07cc40 msgr2=0x7fe88c107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:18:59.627 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:58.905+0000 7fe88be00640 1 -- 172.21.15.1:0/1724343299 shutdown_connections 2024-09-12T01:18:59.627 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:18:58.905+0000 7fe88be00640 1 -- 172.21.15.1:0/1724343299 wait complete. 2024-09-12T01:18:59.627 INFO:teuthology.orchestra.run.smithi001.stdout:Deploying unmanaged mon service... 2024-09-12T01:18:59.900 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:59 smithi001 ceph-mon[30387]: from='mgr.14118 172.21.15.1:0/1617053689' entity='mgr.x' 2024-09-12T01:18:59.900 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:59 smithi001 ceph-mon[30387]: Added host smithi001 2024-09-12T01:18:59.900 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:18:59 smithi001 ceph-mon[30387]: from='mgr.14118 172.21.15.1:0/1617053689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:19:01.284 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-12T01:19:01.285 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.146+0000 7fdc93600640 1 Processor -- start 2024-09-12T01:19:01.285 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.146+0000 7fdc93600640 1 -- start start 2024-09-12T01:19:01.285 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.147+0000 7fdc93600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc8c07dd30 0x7fdc8c07c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:01.285 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.147+0000 7fdc93600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8c07c7e0 con 0x7fdc8c07dd30 2024-09-12T01:19:01.285 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.147+0000 7fdc90c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc8c07dd30 0x7fdc8c07c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:01.285 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.147+0000 7fdc90c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc8c07dd30 0x7fdc8c07c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59752/0 (socket says 172.21.15.1:59752) 2024-09-12T01:19:01.285 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.147+0000 7fdc90c00640 1 -- 172.21.15.1:0/3300752173 learned_addr learned my addr 172.21.15.1:0/3300752173 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:19:01.285 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.148+0000 7fdc90c00640 1 -- 172.21.15.1:0/3300752173 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc8c07c960 con 0x7fdc8c07dd30 2024-09-12T01:19:01.285 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.149+0000 7fdc90c00640 1 --2- 172.21.15.1:0/3300752173 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc8c07dd30 0x7fdc8c07c2a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c009820 tx=0x7fdc7c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9adbc8593b4a155 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:01.285 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.149+0000 7fdc8be00640 1 -- 172.21.15.1:0/3300752173 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc7c037070 con 0x7fdc8c07dd30 2024-09-12T01:19:01.286 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.149+0000 7fdc8be00640 1 -- 172.21.15.1:0/3300752173 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdc7c032900 con 0x7fdc8c07dd30 2024-09-12T01:19:01.286 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.149+0000 7fdc8be00640 1 -- 172.21.15.1:0/3300752173 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc7c030850 con 0x7fdc8c07dd30 2024-09-12T01:19:01.286 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.149+0000 7fdc93600640 1 -- 172.21.15.1:0/3300752173 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc8c07dd30 msgr2=0x7fdc8c07c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:01.286 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.149+0000 7fdc93600640 1 --2- 172.21.15.1:0/3300752173 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc8c07dd30 0x7fdc8c07c2a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c009820 tx=0x7fdc7c02a270 comp rx=0 tx=0).stop 2024-09-12T01:19:01.286 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.150+0000 7fdc93600640 1 -- 172.21.15.1:0/3300752173 shutdown_connections 2024-09-12T01:19:01.286 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.150+0000 7fdc93600640 1 --2- 172.21.15.1:0/3300752173 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc8c07dd30 0x7fdc8c07c2a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:01.286 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.150+0000 7fdc93600640 1 -- 172.21.15.1:0/3300752173 >> 172.21.15.1:0/3300752173 conn(0x7fdc8c102e50 msgr2=0x7fdc8c105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:19:01.286 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.150+0000 7fdc93600640 1 -- 172.21.15.1:0/3300752173 shutdown_connections 2024-09-12T01:19:01.286 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.150+0000 7fdc93600640 1 -- 172.21.15.1:0/3300752173 wait complete. 2024-09-12T01:19:01.286 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.151+0000 7fdc93600640 1 Processor -- start 2024-09-12T01:19:01.286 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.151+0000 7fdc93600640 1 -- start start 2024-09-12T01:19:01.286 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.151+0000 7fdc93600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc8c07dd30 0x7fdc8c19d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:01.286 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.151+0000 7fdc93600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8c10db80 con 0x7fdc8c07dd30 2024-09-12T01:19:01.287 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.152+0000 7fdc90c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc8c07dd30 0x7fdc8c19d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:01.287 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.152+0000 7fdc90c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc8c07dd30 0x7fdc8c19d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59764/0 (socket says 172.21.15.1:59764) 2024-09-12T01:19:01.287 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.152+0000 7fdc90c00640 1 -- 172.21.15.1:0/3408371264 learned_addr learned my addr 172.21.15.1:0/3408371264 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:19:01.287 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.152+0000 7fdc90c00640 1 -- 172.21.15.1:0/3408371264 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc8c19dd00 con 0x7fdc8c07dd30 2024-09-12T01:19:01.287 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.152+0000 7fdc90c00640 1 --2- 172.21.15.1:0/3408371264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc8c07dd30 0x7fdc8c19d7c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c0097f0 tx=0x7fdc7c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:01.287 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.153+0000 7fdc8a000640 1 -- 172.21.15.1:0/3408371264 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc7c037040 con 0x7fdc8c07dd30 2024-09-12T01:19:01.287 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.153+0000 7fdc8a000640 1 -- 172.21.15.1:0/3408371264 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdc7c040070 con 0x7fdc8c07dd30 2024-09-12T01:19:01.287 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.153+0000 7fdc8a000640 1 -- 172.21.15.1:0/3408371264 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc7c03a390 con 0x7fdc8c07dd30 2024-09-12T01:19:01.287 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.153+0000 7fdc93600640 1 -- 172.21.15.1:0/3408371264 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc8c19df90 con 0x7fdc8c07dd30 2024-09-12T01:19:01.287 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.153+0000 7fdc93600640 1 -- 172.21.15.1:0/3408371264 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc8c1a0c80 con 0x7fdc8c07dd30 2024-09-12T01:19:01.287 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.154+0000 7fdc8a000640 1 -- 172.21.15.1:0/3408371264 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fdc7c03b430 con 0x7fdc8c07dd30 2024-09-12T01:19:01.287 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.154+0000 7fdc93600640 1 -- 172.21.15.1:0/3408371264 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc8c07c320 con 0x7fdc8c07dd30 2024-09-12T01:19:01.287 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.155+0000 7fdc8a000640 1 --2- 172.21.15.1:0/3408371264 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7fdc70038fc0 0x7fdc7003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:01.288 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.155+0000 7fdc8a000640 1 -- 172.21.15.1:0/3408371264 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdc7c06c4d0 con 0x7fdc8c07dd30 2024-09-12T01:19:01.288 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.155+0000 7fdc83e00640 1 --2- 172.21.15.1:0/3408371264 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7fdc70038fc0 0x7fdc7003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:01.288 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.156+0000 7fdc83e00640 1 --2- 172.21.15.1:0/3408371264 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7fdc70038fc0 0x7fdc7003b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdc740098c0 tx=0x7fdc74006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:01.288 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.160+0000 7fdc8a000640 1 -- 172.21.15.1:0/3408371264 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdc7c032dd0 con 0x7fdc8c07dd30 2024-09-12T01:19:01.288 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.360+0000 7fdc93600640 1 -- 172.21.15.1:0/3408371264 --> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fdc8c104090 con 0x7fdc70038fc0 2024-09-12T01:19:01.288 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.365+0000 7fdc8a000640 1 -- 172.21.15.1:0/3408371264 <== mgr.14118 v2:172.21.15.1:6800/4101281615 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fdc8c104090 con 0x7fdc70038fc0 2024-09-12T01:19:01.288 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.368+0000 7fdc93600640 1 -- 172.21.15.1:0/3408371264 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7fdc70038fc0 msgr2=0x7fdc7003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:01.288 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.368+0000 7fdc93600640 1 --2- 172.21.15.1:0/3408371264 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7fdc70038fc0 0x7fdc7003b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdc740098c0 tx=0x7fdc74006db0 comp rx=0 tx=0).stop 2024-09-12T01:19:01.288 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.368+0000 7fdc93600640 1 -- 172.21.15.1:0/3408371264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc8c07dd30 msgr2=0x7fdc8c19d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:01.288 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.368+0000 7fdc93600640 1 --2- 172.21.15.1:0/3408371264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc8c07dd30 0x7fdc8c19d7c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c0097f0 tx=0x7fdc7c030d80 comp rx=0 tx=0).stop 2024-09-12T01:19:01.288 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.368+0000 7fdc93600640 1 -- 172.21.15.1:0/3408371264 shutdown_connections 2024-09-12T01:19:01.289 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.368+0000 7fdc93600640 1 --2- 172.21.15.1:0/3408371264 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7fdc70038fc0 0x7fdc7003b480 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:01.289 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.368+0000 7fdc93600640 1 --2- 172.21.15.1:0/3408371264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc8c07dd30 0x7fdc8c19d7c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:01.289 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.368+0000 7fdc93600640 1 -- 172.21.15.1:0/3408371264 >> 172.21.15.1:0/3408371264 conn(0x7fdc8c102e50 msgr2=0x7fdc8c103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:19:01.289 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.368+0000 7fdc93600640 1 -- 172.21.15.1:0/3408371264 shutdown_connections 2024-09-12T01:19:01.289 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:00.368+0000 7fdc93600640 1 -- 172.21.15.1:0/3408371264 wait complete. 2024-09-12T01:19:01.289 INFO:teuthology.orchestra.run.smithi001.stdout:Deploying unmanaged mgr service... 2024-09-12T01:19:01.586 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:01 smithi001 ceph-mon[30387]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:19:01.586 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:01 smithi001 ceph-mon[30387]: Saving service mon spec with placement count:5 2024-09-12T01:19:01.587 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:01 smithi001 ceph-mon[30387]: from='mgr.14118 172.21.15.1:0/1617053689' entity='mgr.x' 2024-09-12T01:19:03.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:03 smithi001 ceph-mon[30387]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:19:03.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:03 smithi001 ceph-mon[30387]: Saving service mgr spec with placement count:2 2024-09-12T01:19:03.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:03 smithi001 ceph-mon[30387]: from='mgr.14118 172.21.15.1:0/1617053689' entity='mgr.x' 2024-09-12T01:19:03.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:03 smithi001 ceph-mon[30387]: from='mgr.14118 172.21.15.1:0/1617053689' entity='mgr.x' 2024-09-12T01:19:03.562 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-12T01:19:03.562 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.066+0000 7f6c3f600640 1 Processor -- start 2024-09-12T01:19:03.562 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.066+0000 7f6c3f600640 1 -- start start 2024-09-12T01:19:03.563 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.067+0000 7f6c3f600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c381098c0 0x7f6c38109cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:03.563 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.067+0000 7f6c3f600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3810a290 con 0x7f6c381098c0 2024-09-12T01:19:03.563 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.067+0000 7f6c3cc00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c381098c0 0x7f6c38109cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:03.563 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.067+0000 7f6c3cc00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c381098c0 0x7f6c38109cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59780/0 (socket says 172.21.15.1:59780) 2024-09-12T01:19:03.563 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.067+0000 7f6c3cc00640 1 -- 172.21.15.1:0/548820995 learned_addr learned my addr 172.21.15.1:0/548820995 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:19:03.563 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.068+0000 7f6c3cc00640 1 -- 172.21.15.1:0/548820995 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c3810a9b0 con 0x7f6c381098c0 2024-09-12T01:19:03.563 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.069+0000 7f6c3cc00640 1 --2- 172.21.15.1:0/548820995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c381098c0 0x7f6c38109cc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c009820 tx=0x7f6c2c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a6c848e6b657a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:03.563 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.069+0000 7f6c37400640 1 -- 172.21.15.1:0/548820995 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c2c037070 con 0x7f6c381098c0 2024-09-12T01:19:03.563 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.069+0000 7f6c37400640 1 -- 172.21.15.1:0/548820995 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6c2c032900 con 0x7f6c381098c0 2024-09-12T01:19:03.563 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.069+0000 7f6c37400640 1 -- 172.21.15.1:0/548820995 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c2c030850 con 0x7f6c381098c0 2024-09-12T01:19:03.563 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.069+0000 7f6c3f600640 1 -- 172.21.15.1:0/548820995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c381098c0 msgr2=0x7f6c38109cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:03.563 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.069+0000 7f6c3f600640 1 --2- 172.21.15.1:0/548820995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c381098c0 0x7f6c38109cc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c009820 tx=0x7f6c2c02a270 comp rx=0 tx=0).stop 2024-09-12T01:19:03.563 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.070+0000 7f6c3f600640 1 -- 172.21.15.1:0/548820995 shutdown_connections 2024-09-12T01:19:03.563 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.070+0000 7f6c3f600640 1 --2- 172.21.15.1:0/548820995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c381098c0 0x7f6c38109cc0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:03.564 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.070+0000 7f6c3f600640 1 -- 172.21.15.1:0/548820995 >> 172.21.15.1:0/548820995 conn(0x7f6c3807ce20 msgr2=0x7f6c3807d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:19:03.564 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.070+0000 7f6c3f600640 1 -- 172.21.15.1:0/548820995 shutdown_connections 2024-09-12T01:19:03.564 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.070+0000 7f6c3f600640 1 -- 172.21.15.1:0/548820995 wait complete. 2024-09-12T01:19:03.564 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.071+0000 7f6c3f600640 1 Processor -- start 2024-09-12T01:19:03.564 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.071+0000 7f6c3f600640 1 -- start start 2024-09-12T01:19:03.564 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.072+0000 7f6c3f600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c381098c0 0x7f6c3819db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:03.564 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.072+0000 7f6c3f600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3810dd60 con 0x7f6c381098c0 2024-09-12T01:19:03.564 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.072+0000 7f6c3cc00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c381098c0 0x7f6c3819db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:03.564 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.072+0000 7f6c3cc00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c381098c0 0x7f6c3819db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59794/0 (socket says 172.21.15.1:59794) 2024-09-12T01:19:03.564 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.072+0000 7f6c3cc00640 1 -- 172.21.15.1:0/2573373679 learned_addr learned my addr 172.21.15.1:0/2573373679 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:19:03.564 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.072+0000 7f6c3cc00640 1 -- 172.21.15.1:0/2573373679 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c3819e040 con 0x7f6c381098c0 2024-09-12T01:19:03.564 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.073+0000 7f6c3cc00640 1 --2- 172.21.15.1:0/2573373679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c381098c0 0x7f6c3819db00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c009940 tx=0x7f6c2c0326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:03.564 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.073+0000 7f6c35600640 1 -- 172.21.15.1:0/2573373679 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c2c042070 con 0x7f6c381098c0 2024-09-12T01:19:03.564 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.073+0000 7f6c35600640 1 -- 172.21.15.1:0/2573373679 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6c2c032be0 con 0x7f6c381098c0 2024-09-12T01:19:03.565 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.073+0000 7f6c3f600640 1 -- 172.21.15.1:0/2573373679 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c3819e2d0 con 0x7f6c381098c0 2024-09-12T01:19:03.565 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.073+0000 7f6c35600640 1 -- 172.21.15.1:0/2573373679 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c2c037040 con 0x7f6c381098c0 2024-09-12T01:19:03.565 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.073+0000 7f6c3f600640 1 -- 172.21.15.1:0/2573373679 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c381a0fc0 con 0x7f6c381098c0 2024-09-12T01:19:03.565 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.074+0000 7f6c35600640 1 -- 172.21.15.1:0/2573373679 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f6c2c045070 con 0x7f6c381098c0 2024-09-12T01:19:03.565 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.074+0000 7f6c3f600640 1 -- 172.21.15.1:0/2573373679 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c3810dee0 con 0x7f6c381098c0 2024-09-12T01:19:03.565 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.075+0000 7f6c35600640 1 --2- 172.21.15.1:0/2573373679 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f6c20038fc0 0x7f6c2003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:03.565 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.075+0000 7f6c35600640 1 -- 172.21.15.1:0/2573373679 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6c2c06bf50 con 0x7f6c381098c0 2024-09-12T01:19:03.565 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.075+0000 7f6c37e00640 1 --2- 172.21.15.1:0/2573373679 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f6c20038fc0 0x7f6c2003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:03.565 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.076+0000 7f6c37e00640 1 --2- 172.21.15.1:0/2573373679 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f6c20038fc0 0x7f6c2003b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6c240098c0 tx=0x7f6c24006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:03.565 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.080+0000 7f6c35600640 1 -- 172.21.15.1:0/2573373679 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6c2c04b170 con 0x7f6c381098c0 2024-09-12T01:19:03.565 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.283+0000 7f6c3f600640 1 -- 172.21.15.1:0/2573373679 --> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f6c38062630 con 0x7f6c20038fc0 2024-09-12T01:19:03.565 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.288+0000 7f6c35600640 1 -- 172.21.15.1:0/2573373679 <== mgr.14118 v2:172.21.15.1:6800/4101281615 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6c38062630 con 0x7f6c20038fc0 2024-09-12T01:19:03.565 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.290+0000 7f6c3f600640 1 -- 172.21.15.1:0/2573373679 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f6c20038fc0 msgr2=0x7f6c2003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:03.566 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.290+0000 7f6c3f600640 1 --2- 172.21.15.1:0/2573373679 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f6c20038fc0 0x7f6c2003b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6c240098c0 tx=0x7f6c24006db0 comp rx=0 tx=0).stop 2024-09-12T01:19:03.566 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.290+0000 7f6c3f600640 1 -- 172.21.15.1:0/2573373679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c381098c0 msgr2=0x7f6c3819db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:03.566 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.290+0000 7f6c3f600640 1 --2- 172.21.15.1:0/2573373679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c381098c0 0x7f6c3819db00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c009940 tx=0x7f6c2c0326e0 comp rx=0 tx=0).stop 2024-09-12T01:19:03.566 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.290+0000 7f6c3f600640 1 -- 172.21.15.1:0/2573373679 shutdown_connections 2024-09-12T01:19:03.566 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.290+0000 7f6c3f600640 1 --2- 172.21.15.1:0/2573373679 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f6c20038fc0 0x7f6c2003b480 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:03.566 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.290+0000 7f6c3f600640 1 --2- 172.21.15.1:0/2573373679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c381098c0 0x7f6c3819db00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:03.566 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.290+0000 7f6c3f600640 1 -- 172.21.15.1:0/2573373679 >> 172.21.15.1:0/2573373679 conn(0x7f6c3807ce20 msgr2=0x7f6c381077d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:19:03.566 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.291+0000 7f6c3f600640 1 -- 172.21.15.1:0/2573373679 shutdown_connections 2024-09-12T01:19:03.566 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:02.291+0000 7f6c3f600640 1 -- 172.21.15.1:0/2573373679 wait complete. 2024-09-12T01:19:04.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:04 smithi001 ceph-mon[30387]: from='mgr.14118 172.21.15.1:0/1617053689' entity='mgr.x' 2024-09-12T01:19:04.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:04 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/4239155818' entity='client.admin' 2024-09-12T01:19:05.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.123+0000 7f154b200640 1 Processor -- start 2024-09-12T01:19:05.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.123+0000 7f154b200640 1 -- start start 2024-09-12T01:19:05.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.123+0000 7f154b200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15441098c0 0x7f1544109cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:05.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.123+0000 7f154b200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f154410a290 con 0x7f15441098c0 2024-09-12T01:19:05.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.123+0000 7f1543e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15441098c0 0x7f1544109cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:05.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.123+0000 7f1543e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15441098c0 0x7f1544109cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59800/0 (socket says 172.21.15.1:59800) 2024-09-12T01:19:05.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.123+0000 7f1543e00640 1 -- 172.21.15.1:0/817382136 learned_addr learned my addr 172.21.15.1:0/817382136 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:19:05.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.124+0000 7f1543e00640 1 -- 172.21.15.1:0/817382136 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f154410a9b0 con 0x7f15441098c0 2024-09-12T01:19:05.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.125+0000 7f1543e00640 1 --2- 172.21.15.1:0/817382136 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15441098c0 0x7f1544109cc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1534009820 tx=0x7f153402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c0fd7fff82109b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:05.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.125+0000 7f1542a00640 1 -- 172.21.15.1:0/817382136 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1534037070 con 0x7f15441098c0 2024-09-12T01:19:05.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.125+0000 7f1542a00640 1 -- 172.21.15.1:0/817382136 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1534032920 con 0x7f15441098c0 2024-09-12T01:19:05.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.125+0000 7f1542a00640 1 -- 172.21.15.1:0/817382136 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1534030810 con 0x7f15441098c0 2024-09-12T01:19:05.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.126+0000 7f154b200640 1 -- 172.21.15.1:0/817382136 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15441098c0 msgr2=0x7f1544109cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:05.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.126+0000 7f154b200640 1 --2- 172.21.15.1:0/817382136 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15441098c0 0x7f1544109cc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1534009820 tx=0x7f153402a270 comp rx=0 tx=0).stop 2024-09-12T01:19:05.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.126+0000 7f154b200640 1 -- 172.21.15.1:0/817382136 shutdown_connections 2024-09-12T01:19:05.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.126+0000 7f154b200640 1 --2- 172.21.15.1:0/817382136 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15441098c0 0x7f1544109cc0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:05.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.126+0000 7f154b200640 1 -- 172.21.15.1:0/817382136 >> 172.21.15.1:0/817382136 conn(0x7f154407ce20 msgr2=0x7f154407d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:19:05.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.126+0000 7f154b200640 1 -- 172.21.15.1:0/817382136 shutdown_connections 2024-09-12T01:19:05.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.126+0000 7f154b200640 1 -- 172.21.15.1:0/817382136 wait complete. 2024-09-12T01:19:05.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.127+0000 7f154b200640 1 Processor -- start 2024-09-12T01:19:05.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.127+0000 7f154b200640 1 -- start start 2024-09-12T01:19:05.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.127+0000 7f154b200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15441098c0 0x7f154419db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:05.648 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.127+0000 7f154b200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f154410dd60 con 0x7f15441098c0 2024-09-12T01:19:05.648 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.128+0000 7f1543e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15441098c0 0x7f154419db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:05.648 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.128+0000 7f1543e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15441098c0 0x7f154419db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59802/0 (socket says 172.21.15.1:59802) 2024-09-12T01:19:05.648 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.128+0000 7f1543e00640 1 -- 172.21.15.1:0/4239155818 learned_addr learned my addr 172.21.15.1:0/4239155818 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:19:05.648 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.128+0000 7f1543e00640 1 -- 172.21.15.1:0/4239155818 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f154419e0b0 con 0x7f15441098c0 2024-09-12T01:19:05.648 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.128+0000 7f1543e00640 1 --2- 172.21.15.1:0/4239155818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15441098c0 0x7f154419db70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1534030560 tx=0x7f1534032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:05.648 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.129+0000 7f1540c00640 1 -- 172.21.15.1:0/4239155818 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1534037040 con 0x7f15441098c0 2024-09-12T01:19:05.648 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.129+0000 7f1540c00640 1 -- 172.21.15.1:0/4239155818 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1534040070 con 0x7f15441098c0 2024-09-12T01:19:05.648 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.129+0000 7f1540c00640 1 -- 172.21.15.1:0/4239155818 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f153403a6d0 con 0x7f15441098c0 2024-09-12T01:19:05.648 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.129+0000 7f154b200640 1 -- 172.21.15.1:0/4239155818 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f154419e340 con 0x7f15441098c0 2024-09-12T01:19:05.648 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.129+0000 7f154b200640 1 -- 172.21.15.1:0/4239155818 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15441a1030 con 0x7f15441098c0 2024-09-12T01:19:05.648 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.130+0000 7f1540c00640 1 -- 172.21.15.1:0/4239155818 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f153403b420 con 0x7f15441098c0 2024-09-12T01:19:05.648 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.130+0000 7f154b200640 1 -- 172.21.15.1:0/4239155818 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f154410dee0 con 0x7f15441098c0 2024-09-12T01:19:05.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.130+0000 7f1540c00640 1 --2- 172.21.15.1:0/4239155818 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f1528038fc0 0x7f152803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:05.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.130+0000 7f1540c00640 1 -- 172.21.15.1:0/4239155818 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f153406c630 con 0x7f15441098c0 2024-09-12T01:19:05.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.131+0000 7f1543400640 1 --2- 172.21.15.1:0/4239155818 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f1528038fc0 0x7f152803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:05.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.131+0000 7f1543400640 1 --2- 172.21.15.1:0/4239155818 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f1528038fc0 0x7f152803b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f15300098c0 tx=0x7f1530006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:05.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.136+0000 7f1540c00640 1 -- 172.21.15.1:0/4239155818 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1534043360 con 0x7f15441098c0 2024-09-12T01:19:05.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.346+0000 7f154b200640 1 -- 172.21.15.1:0/4239155818 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f1544109d40 con 0x7f15441098c0 2024-09-12T01:19:05.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.349+0000 7f1540c00640 1 -- 172.21.15.1:0/4239155818 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f1534041270 con 0x7f15441098c0 2024-09-12T01:19:05.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.351+0000 7f154b200640 1 -- 172.21.15.1:0/4239155818 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f1528038fc0 msgr2=0x7f152803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:05.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.351+0000 7f154b200640 1 --2- 172.21.15.1:0/4239155818 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f1528038fc0 0x7f152803b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f15300098c0 tx=0x7f1530006db0 comp rx=0 tx=0).stop 2024-09-12T01:19:05.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.351+0000 7f154b200640 1 -- 172.21.15.1:0/4239155818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15441098c0 msgr2=0x7f154419db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:05.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.351+0000 7f154b200640 1 --2- 172.21.15.1:0/4239155818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15441098c0 0x7f154419db70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1534030560 tx=0x7f1534032670 comp rx=0 tx=0).stop 2024-09-12T01:19:05.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.352+0000 7f154b200640 1 -- 172.21.15.1:0/4239155818 shutdown_connections 2024-09-12T01:19:05.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.352+0000 7f154b200640 1 --2- 172.21.15.1:0/4239155818 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f1528038fc0 0x7f152803b480 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:05.650 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.352+0000 7f154b200640 1 --2- 172.21.15.1:0/4239155818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15441098c0 0x7f154419db70 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:05.650 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.352+0000 7f154b200640 1 -- 172.21.15.1:0/4239155818 >> 172.21.15.1:0/4239155818 conn(0x7f154407ce20 msgr2=0x7f1544107780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:19:05.650 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.352+0000 7f154b200640 1 -- 172.21.15.1:0/4239155818 shutdown_connections 2024-09-12T01:19:05.650 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:04.352+0000 7f154b200640 1 -- 172.21.15.1:0/4239155818 wait complete. 2024-09-12T01:19:06.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:06 smithi001 ceph-mon[30387]: from='mgr.14118 172.21.15.1:0/1617053689' entity='mgr.x' 2024-09-12T01:19:07.088 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.197+0000 7fd568000640 1 Processor -- start 2024-09-12T01:19:07.088 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.197+0000 7fd568000640 1 -- start start 2024-09-12T01:19:07.088 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.198+0000 7fd568000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd560107600 0x7fd560107a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:07.088 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.198+0000 7fd568000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd560107f40 con 0x7fd560107600 2024-09-12T01:19:07.088 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.198+0000 7fd565600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd560107600 0x7fd560107a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:07.088 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.198+0000 7fd565600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd560107600 0x7fd560107a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59814/0 (socket says 172.21.15.1:59814) 2024-09-12T01:19:07.088 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.198+0000 7fd565600640 1 -- 172.21.15.1:0/4143552888 learned_addr learned my addr 172.21.15.1:0/4143552888 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:19:07.088 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.199+0000 7fd565600640 1 -- 172.21.15.1:0/4143552888 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5601080c0 con 0x7fd560107600 2024-09-12T01:19:07.089 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.200+0000 7fd565600640 1 --2- 172.21.15.1:0/4143552888 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd560107600 0x7fd560107a00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd554009820 tx=0x7fd55402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3365ce29f4185fd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:07.089 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.200+0000 7fd55fe00640 1 -- 172.21.15.1:0/4143552888 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd554037070 con 0x7fd560107600 2024-09-12T01:19:07.089 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.200+0000 7fd55fe00640 1 -- 172.21.15.1:0/4143552888 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd554032920 con 0x7fd560107600 2024-09-12T01:19:07.089 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.200+0000 7fd55fe00640 1 -- 172.21.15.1:0/4143552888 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd554030810 con 0x7fd560107600 2024-09-12T01:19:07.089 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.200+0000 7fd568000640 1 -- 172.21.15.1:0/4143552888 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd560107600 msgr2=0x7fd560107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:07.089 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.200+0000 7fd568000640 1 --2- 172.21.15.1:0/4143552888 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd560107600 0x7fd560107a00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd554009820 tx=0x7fd55402a270 comp rx=0 tx=0).stop 2024-09-12T01:19:07.089 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.201+0000 7fd568000640 1 -- 172.21.15.1:0/4143552888 shutdown_connections 2024-09-12T01:19:07.089 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.201+0000 7fd568000640 1 --2- 172.21.15.1:0/4143552888 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd560107600 0x7fd560107a00 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:07.089 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.201+0000 7fd568000640 1 -- 172.21.15.1:0/4143552888 >> 172.21.15.1:0/4143552888 conn(0x7fd560102e50 msgr2=0x7fd560105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:19:07.089 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.201+0000 7fd568000640 1 -- 172.21.15.1:0/4143552888 shutdown_connections 2024-09-12T01:19:07.089 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.201+0000 7fd568000640 1 -- 172.21.15.1:0/4143552888 wait complete. 2024-09-12T01:19:07.089 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.202+0000 7fd568000640 1 Processor -- start 2024-09-12T01:19:07.089 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.202+0000 7fd568000640 1 -- start start 2024-09-12T01:19:07.089 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.202+0000 7fd568000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd560107600 0x7fd56007db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:07.090 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.203+0000 7fd568000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd56010ba70 con 0x7fd560107600 2024-09-12T01:19:07.090 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.203+0000 7fd565600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd560107600 0x7fd56007db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:07.090 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.203+0000 7fd565600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd560107600 0x7fd56007db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59822/0 (socket says 172.21.15.1:59822) 2024-09-12T01:19:07.090 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.203+0000 7fd565600640 1 -- 172.21.15.1:0/3608122949 learned_addr learned my addr 172.21.15.1:0/3608122949 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:19:07.090 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.203+0000 7fd565600640 1 -- 172.21.15.1:0/3608122949 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd56007e050 con 0x7fd560107600 2024-09-12T01:19:07.090 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.203+0000 7fd565600640 1 --2- 172.21.15.1:0/3608122949 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd560107600 0x7fd56007db10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd554030380 tx=0x7fd554032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:07.090 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.204+0000 7fd55e000640 1 -- 172.21.15.1:0/3608122949 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd554037040 con 0x7fd560107600 2024-09-12T01:19:07.090 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.204+0000 7fd55e000640 1 -- 172.21.15.1:0/3608122949 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd554040070 con 0x7fd560107600 2024-09-12T01:19:07.090 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.204+0000 7fd568000640 1 -- 172.21.15.1:0/3608122949 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd56007c240 con 0x7fd560107600 2024-09-12T01:19:07.090 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.204+0000 7fd55e000640 1 -- 172.21.15.1:0/3608122949 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd55403a7b0 con 0x7fd560107600 2024-09-12T01:19:07.090 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.204+0000 7fd568000640 1 -- 172.21.15.1:0/3608122949 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd56007c780 con 0x7fd560107600 2024-09-12T01:19:07.090 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.205+0000 7fd55e000640 1 -- 172.21.15.1:0/3608122949 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fd55403b420 con 0x7fd560107600 2024-09-12T01:19:07.090 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.205+0000 7fd568000640 1 -- 172.21.15.1:0/3608122949 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd560107a80 con 0x7fd560107600 2024-09-12T01:19:07.090 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.206+0000 7fd55e000640 1 --2- 172.21.15.1:0/3608122949 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7fd548038fc0 0x7fd54803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:07.091 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.206+0000 7fd55e000640 1 -- 172.21.15.1:0/3608122949 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd55406c5f0 con 0x7fd560107600 2024-09-12T01:19:07.091 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.206+0000 7fd564c00640 1 --2- 172.21.15.1:0/3608122949 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7fd548038fc0 0x7fd54803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:07.091 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.207+0000 7fd564c00640 1 --2- 172.21.15.1:0/3608122949 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7fd548038fc0 0x7fd54803b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd54c0098c0 tx=0x7fd54c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:07.091 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.211+0000 7fd55e000640 1 -- 172.21.15.1:0/3608122949 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd55404a200 con 0x7fd560107600 2024-09-12T01:19:07.091 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.398+0000 7fd568000640 1 -- 172.21.15.1:0/3608122949 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fd56007d0d0 con 0x7fd560107600 2024-09-12T01:19:07.091 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.400+0000 7fd55e000640 1 -- 172.21.15.1:0/3608122949 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fd56007d0d0 con 0x7fd560107600 2024-09-12T01:19:07.091 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.403+0000 7fd568000640 1 -- 172.21.15.1:0/3608122949 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7fd548038fc0 msgr2=0x7fd54803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:07.091 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.403+0000 7fd568000640 1 --2- 172.21.15.1:0/3608122949 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7fd548038fc0 0x7fd54803b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd54c0098c0 tx=0x7fd54c006db0 comp rx=0 tx=0).stop 2024-09-12T01:19:07.091 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.403+0000 7fd568000640 1 -- 172.21.15.1:0/3608122949 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd560107600 msgr2=0x7fd56007db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:07.091 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.403+0000 7fd568000640 1 --2- 172.21.15.1:0/3608122949 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd560107600 0x7fd56007db10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd554030380 tx=0x7fd554032670 comp rx=0 tx=0).stop 2024-09-12T01:19:07.091 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.403+0000 7fd568000640 1 -- 172.21.15.1:0/3608122949 shutdown_connections 2024-09-12T01:19:07.091 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.403+0000 7fd568000640 1 --2- 172.21.15.1:0/3608122949 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7fd548038fc0 0x7fd54803b480 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:07.091 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.403+0000 7fd568000640 1 --2- 172.21.15.1:0/3608122949 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd560107600 0x7fd56007db10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:07.091 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.403+0000 7fd568000640 1 -- 172.21.15.1:0/3608122949 >> 172.21.15.1:0/3608122949 conn(0x7fd560102e50 msgr2=0x7fd560103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:19:07.092 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.403+0000 7fd568000640 1 -- 172.21.15.1:0/3608122949 shutdown_connections 2024-09-12T01:19:07.092 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:06.403+0000 7fd568000640 1 -- 172.21.15.1:0/3608122949 wait complete. 2024-09-12T01:19:07.092 INFO:teuthology.orchestra.run.smithi001.stdout:Enabling the dashboard module... 2024-09-12T01:19:07.644 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:07 smithi001 ceph-mon[30387]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:19:07.644 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:07 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/3608122949' entity='client.admin' 2024-09-12T01:19:07.644 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:07 smithi001 ceph-mon[30387]: from='mgr.14118 172.21.15.1:0/1617053689' entity='mgr.x' 2024-09-12T01:19:08.518 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:08 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/2302131339' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-12T01:19:08.888 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:08 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: ignoring --setuser ceph since I am not root 2024-09-12T01:19:08.889 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:08 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: ignoring --setgroup ceph since I am not root 2024-09-12T01:19:08.889 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:08 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:19:08.651+0000 7feb9fe36040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T01:19:08.889 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:08 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:19:08.723+0000 7feb9fe36040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T01:19:09.177 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:08 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:19:08.927+0000 7feb9fe36040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T01:19:09.177 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:09 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:19:09.009+0000 7feb9fe36040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T01:19:09.238 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:07.644+0000 7f6552c00640 1 Processor -- start 2024-09-12T01:19:09.238 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:07.644+0000 7f6552c00640 1 -- start start 2024-09-12T01:19:09.238 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:07.645+0000 7f6552c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f654c1098c0 0x7f654c109cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:09.238 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:07.645+0000 7f6552c00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f654c10a290 con 0x7f654c1098c0 2024-09-12T01:19:09.238 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:07.645+0000 7f654be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f654c1098c0 0x7f654c109cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:09.238 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:07.645+0000 7f654be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f654c1098c0 0x7f654c109cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59826/0 (socket says 172.21.15.1:59826) 2024-09-12T01:19:09.238 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:07.645+0000 7f654be00640 1 -- 172.21.15.1:0/2308575606 learned_addr learned my addr 172.21.15.1:0/2308575606 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:19:09.238 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:07.646+0000 7f654be00640 1 -- 172.21.15.1:0/2308575606 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f654c10a9b0 con 0x7f654c1098c0 2024-09-12T01:19:09.239 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:07.647+0000 7f654be00640 1 --2- 172.21.15.1:0/2308575606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f654c1098c0 0x7f654c109cc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f653c009820 tx=0x7f653c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f0af8dbb684f580 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:09.239 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:07.648+0000 7f654b400640 1 -- 172.21.15.1:0/2308575606 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f653c037070 con 0x7f654c1098c0 2024-09-12T01:19:09.239 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:07.648+0000 7f654b400640 1 -- 172.21.15.1:0/2308575606 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f653c032920 con 0x7f654c1098c0 2024-09-12T01:19:09.239 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:07.648+0000 7f654b400640 1 -- 172.21.15.1:0/2308575606 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f653c030810 con 0x7f654c1098c0 2024-09-12T01:19:09.239 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:07.648+0000 7f6552c00640 1 -- 172.21.15.1:0/2308575606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f654c1098c0 msgr2=0x7f654c109cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:09.239 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:07.649+0000 7f6552c00640 1 --2- 172.21.15.1:0/2308575606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f654c1098c0 0x7f654c109cc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f653c009820 tx=0x7f653c02a270 comp rx=0 tx=0).stop 2024-09-12T01:19:09.239 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:07.649+0000 7f6552c00640 1 -- 172.21.15.1:0/2308575606 shutdown_connections 2024-09-12T01:19:09.239 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:07.649+0000 7f6552c00640 1 --2- 172.21.15.1:0/2308575606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f654c1098c0 0x7f654c109cc0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:09.239 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:07.649+0000 7f6552c00640 1 -- 172.21.15.1:0/2308575606 >> 172.21.15.1:0/2308575606 conn(0x7f654c07ce20 msgr2=0x7f654c07d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:19:09.239 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:07.649+0000 7f6552c00640 1 -- 172.21.15.1:0/2308575606 shutdown_connections 2024-09-12T01:19:09.239 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:07.649+0000 7f6552c00640 1 -- 172.21.15.1:0/2308575606 wait complete. 2024-09-12T01:19:09.239 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:07.650+0000 7f6552c00640 1 Processor -- start 2024-09-12T01:19:09.239 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:07.650+0000 7f6552c00640 1 -- start start 2024-09-12T01:19:09.239 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:07.650+0000 7f6552c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f654c1098c0 0x7f654c19db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:09.240 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:07.650+0000 7f6552c00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f654c10dd60 con 0x7f654c1098c0 2024-09-12T01:19:09.240 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:07.651+0000 7f654be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f654c1098c0 0x7f654c19db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:09.240 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:07.651+0000 7f654be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f654c1098c0 0x7f654c19db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59828/0 (socket says 172.21.15.1:59828) 2024-09-12T01:19:09.240 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:07.651+0000 7f654be00640 1 -- 172.21.15.1:0/2302131339 learned_addr learned my addr 172.21.15.1:0/2302131339 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:19:09.240 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:07.651+0000 7f654be00640 1 -- 172.21.15.1:0/2302131339 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f654c19e090 con 0x7f654c1098c0 2024-09-12T01:19:09.240 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:07.651+0000 7f654be00640 1 --2- 172.21.15.1:0/2302131339 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f654c1098c0 0x7f654c19db50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f653c0097f0 tx=0x7f653c030d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:09.240 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:07.652+0000 7f6549600640 1 -- 172.21.15.1:0/2302131339 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f653c037040 con 0x7f654c1098c0 2024-09-12T01:19:09.240 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:07.652+0000 7f6552c00640 1 -- 172.21.15.1:0/2302131339 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f654c19e320 con 0x7f654c1098c0 2024-09-12T01:19:09.240 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:07.652+0000 7f6549600640 1 -- 172.21.15.1:0/2302131339 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f653c040070 con 0x7f654c1098c0 2024-09-12T01:19:09.240 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:07.652+0000 7f6549600640 1 -- 172.21.15.1:0/2302131339 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f653c03a610 con 0x7f654c1098c0 2024-09-12T01:19:09.240 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:07.652+0000 7f6552c00640 1 -- 172.21.15.1:0/2302131339 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f654c1a1010 con 0x7f654c1098c0 2024-09-12T01:19:09.240 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:07.653+0000 7f6552c00640 1 -- 172.21.15.1:0/2302131339 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f654c10dee0 con 0x7f654c1098c0 2024-09-12T01:19:09.240 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:07.654+0000 7f6549600640 1 -- 172.21.15.1:0/2302131339 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f653c0327b0 con 0x7f654c1098c0 2024-09-12T01:19:09.240 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:07.654+0000 7f6549600640 1 --2- 172.21.15.1:0/2302131339 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f6530038f70 0x7f653003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:09.241 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:07.654+0000 7f6549600640 1 -- 172.21.15.1:0/2302131339 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f653c06c560 con 0x7f654c1098c0 2024-09-12T01:19:09.241 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:07.654+0000 7f6543e00640 1 --2- 172.21.15.1:0/2302131339 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f6530038f70 0x7f653003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:09.241 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:07.655+0000 7f6543e00640 1 --2- 172.21.15.1:0/2302131339 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f6530038f70 0x7f653003b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f65340098c0 tx=0x7f6534006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:09.241 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:07.662+0000 7f6549600640 1 -- 172.21.15.1:0/2302131339 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f653c04a200 con 0x7f654c1098c0 2024-09-12T01:19:09.241 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:07.899+0000 7f6552c00640 1 -- 172.21.15.1:0/2302131339 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f654c02d7c0 con 0x7f654c1098c0 2024-09-12T01:19:09.241 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:08.403+0000 7f6549600640 1 -- 172.21.15.1:0/2302131339 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f653c031e80 con 0x7f654c1098c0 2024-09-12T01:19:09.241 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:08.405+0000 7f6549600640 1 -- 172.21.15.1:0/2302131339 <== mon.0 v2:172.21.15.1:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f653c03b420 con 0x7f654c1098c0 2024-09-12T01:19:09.241 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:08.407+0000 7f6552c00640 1 -- 172.21.15.1:0/2302131339 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f6530038f70 msgr2=0x7f653003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:09.241 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:08.407+0000 7f6552c00640 1 --2- 172.21.15.1:0/2302131339 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f6530038f70 0x7f653003b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f65340098c0 tx=0x7f6534006db0 comp rx=0 tx=0).stop 2024-09-12T01:19:09.241 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:08.407+0000 7f6552c00640 1 -- 172.21.15.1:0/2302131339 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f654c1098c0 msgr2=0x7f654c19db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:09.241 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:08.407+0000 7f6552c00640 1 --2- 172.21.15.1:0/2302131339 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f654c1098c0 0x7f654c19db50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f653c0097f0 tx=0x7f653c030d40 comp rx=0 tx=0).stop 2024-09-12T01:19:09.241 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:08.407+0000 7f6552c00640 1 -- 172.21.15.1:0/2302131339 shutdown_connections 2024-09-12T01:19:09.241 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:08.407+0000 7f6552c00640 1 --2- 172.21.15.1:0/2302131339 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f6530038f70 0x7f653003b430 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:09.241 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:08.407+0000 7f6552c00640 1 --2- 172.21.15.1:0/2302131339 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f654c1098c0 0x7f654c19db50 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:09.242 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:08.407+0000 7f6552c00640 1 -- 172.21.15.1:0/2302131339 >> 172.21.15.1:0/2302131339 conn(0x7f654c07ce20 msgr2=0x7f654c1077d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:19:09.242 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:08.407+0000 7f6552c00640 1 -- 172.21.15.1:0/2302131339 shutdown_connections 2024-09-12T01:19:09.242 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:08.407+0000 7f6552c00640 1 -- 172.21.15.1:0/2302131339 wait complete. 2024-09-12T01:19:09.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:09 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/2302131339' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-12T01:19:09.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:09 smithi001 ceph-mon[30387]: mgrmap e9: x(active, since 22s) 2024-09-12T01:19:09.632 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:09 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:19:09.186+0000 7feb9fe36040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T01:19:09.632 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:09 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:19:09.366+0000 7feb9fe36040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T01:19:09.920 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:09 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:19:09.751+0000 7feb9fe36040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T01:19:09.920 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:09 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:19:09.826+0000 7feb9fe36040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T01:19:10.177 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:10 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:19:10.050+0000 7feb9fe36040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T01:19:10.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:10 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/3260218829' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-12T01:19:11.067 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-09-12T01:19:11.067 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-12T01:19:11.067 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-12T01:19:11.067 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-12T01:19:11.067 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-12T01:19:11.067 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-09-12T01:19:11.067 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.152+0000 7ff283000640 1 Processor -- start 2024-09-12T01:19:11.067 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.152+0000 7ff283000640 1 -- start start 2024-09-12T01:19:11.067 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.153+0000 7ff283000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff27c07cca0 0x7ff27c07b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:11.067 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.153+0000 7ff283000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff27c07b690 con 0x7ff27c07cca0 2024-09-12T01:19:11.067 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.153+0000 7ff27be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff27c07cca0 0x7ff27c07b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:11.067 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.153+0000 7ff27be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff27c07cca0 0x7ff27c07b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39596/0 (socket says 172.21.15.1:39596) 2024-09-12T01:19:11.068 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.153+0000 7ff27be00640 1 -- 172.21.15.1:0/2205741774 learned_addr learned my addr 172.21.15.1:0/2205741774 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:19:11.068 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.153+0000 7ff27be00640 1 -- 172.21.15.1:0/2205741774 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff27c07b810 con 0x7ff27c07cca0 2024-09-12T01:19:11.068 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.154+0000 7ff27be00640 1 --2- 172.21.15.1:0/2205741774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff27c07cca0 0x7ff27c07b150 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff26c009820 tx=0x7ff26c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e736a97ace8a6c36 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:11.068 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.155+0000 7ff27aa00640 1 -- 172.21.15.1:0/2205741774 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff26c037070 con 0x7ff27c07cca0 2024-09-12T01:19:11.068 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.155+0000 7ff27aa00640 1 -- 172.21.15.1:0/2205741774 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff26c032920 con 0x7ff27c07cca0 2024-09-12T01:19:11.068 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.155+0000 7ff27aa00640 1 -- 172.21.15.1:0/2205741774 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff26c030810 con 0x7ff27c07cca0 2024-09-12T01:19:11.068 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.155+0000 7ff283000640 1 -- 172.21.15.1:0/2205741774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff27c07cca0 msgr2=0x7ff27c07b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:11.068 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.155+0000 7ff283000640 1 --2- 172.21.15.1:0/2205741774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff27c07cca0 0x7ff27c07b150 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff26c009820 tx=0x7ff26c02a270 comp rx=0 tx=0).stop 2024-09-12T01:19:11.069 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.156+0000 7ff283000640 1 -- 172.21.15.1:0/2205741774 shutdown_connections 2024-09-12T01:19:11.069 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.156+0000 7ff283000640 1 --2- 172.21.15.1:0/2205741774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff27c07cca0 0x7ff27c07b150 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:11.069 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.156+0000 7ff283000640 1 -- 172.21.15.1:0/2205741774 >> 172.21.15.1:0/2205741774 conn(0x7ff27c102d20 msgr2=0x7ff27c105180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:19:11.069 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.156+0000 7ff283000640 1 -- 172.21.15.1:0/2205741774 shutdown_connections 2024-09-12T01:19:11.069 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.156+0000 7ff283000640 1 -- 172.21.15.1:0/2205741774 wait complete. 2024-09-12T01:19:11.069 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.157+0000 7ff283000640 1 Processor -- start 2024-09-12T01:19:11.069 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.157+0000 7ff283000640 1 -- start start 2024-09-12T01:19:11.069 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.158+0000 7ff283000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff27c07cca0 0x7ff27c19d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:11.069 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.158+0000 7ff283000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff27c10dc00 con 0x7ff27c07cca0 2024-09-12T01:19:11.069 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.158+0000 7ff27be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff27c07cca0 0x7ff27c19d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:11.069 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.158+0000 7ff27be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff27c07cca0 0x7ff27c19d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39608/0 (socket says 172.21.15.1:39608) 2024-09-12T01:19:11.069 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.158+0000 7ff27be00640 1 -- 172.21.15.1:0/3260218829 learned_addr learned my addr 172.21.15.1:0/3260218829 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:19:11.069 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.158+0000 7ff27be00640 1 -- 172.21.15.1:0/3260218829 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff27c19dc80 con 0x7ff27c07cca0 2024-09-12T01:19:11.070 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.159+0000 7ff27be00640 1 --2- 172.21.15.1:0/3260218829 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff27c07cca0 0x7ff27c19d740 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff26c0097f0 tx=0x7ff26c0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:11.070 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.159+0000 7ff278c00640 1 -- 172.21.15.1:0/3260218829 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff26c037040 con 0x7ff27c07cca0 2024-09-12T01:19:11.070 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.159+0000 7ff278c00640 1 -- 172.21.15.1:0/3260218829 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff26c030cc0 con 0x7ff27c07cca0 2024-09-12T01:19:11.070 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.159+0000 7ff278c00640 1 -- 172.21.15.1:0/3260218829 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff26c03a6b0 con 0x7ff27c07cca0 2024-09-12T01:19:11.070 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.159+0000 7ff283000640 1 -- 172.21.15.1:0/3260218829 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff27c19df10 con 0x7ff27c07cca0 2024-09-12T01:19:11.070 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.159+0000 7ff283000640 1 -- 172.21.15.1:0/3260218829 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff27c1a0c00 con 0x7ff27c07cca0 2024-09-12T01:19:11.070 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.160+0000 7ff278c00640 1 -- 172.21.15.1:0/3260218829 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7ff26c03b3f0 con 0x7ff27c07cca0 2024-09-12T01:19:11.070 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.160+0000 7ff283000640 1 -- 172.21.15.1:0/3260218829 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff27c07b1d0 con 0x7ff27c07cca0 2024-09-12T01:19:11.070 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.161+0000 7ff278c00640 1 --2- 172.21.15.1:0/3260218829 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7ff260039020 0x7ff26003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:11.070 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.161+0000 7ff278c00640 1 -- 172.21.15.1:0/3260218829 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff26c06c5a0 con 0x7ff27c07cca0 2024-09-12T01:19:11.070 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.161+0000 7ff27b400640 1 -- 172.21.15.1:0/3260218829 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7ff260039020 msgr2=0x7ff26003b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/4101281615 2024-09-12T01:19:11.070 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.161+0000 7ff27b400640 1 --2- 172.21.15.1:0/3260218829 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7ff260039020 0x7ff26003b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T01:19:11.070 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.166+0000 7ff278c00640 1 -- 172.21.15.1:0/3260218829 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff26c06bef0 con 0x7ff27c07cca0 2024-09-12T01:19:11.071 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.362+0000 7ff27b400640 1 -- 172.21.15.1:0/3260218829 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7ff260039020 msgr2=0x7ff26003b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/4101281615 2024-09-12T01:19:11.071 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.362+0000 7ff27b400640 1 --2- 172.21.15.1:0/3260218829 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7ff260039020 0x7ff26003b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T01:19:11.071 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.400+0000 7ff283000640 1 -- 172.21.15.1:0/3260218829 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7ff27c1a11f0 con 0x7ff27c07cca0 2024-09-12T01:19:11.071 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.401+0000 7ff278c00640 1 -- 172.21.15.1:0/3260218829 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7ff26c03a850 con 0x7ff27c07cca0 2024-09-12T01:19:11.071 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.403+0000 7ff283000640 1 -- 172.21.15.1:0/3260218829 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7ff260039020 msgr2=0x7ff26003b4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T01:19:11.071 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.403+0000 7ff283000640 1 --2- 172.21.15.1:0/3260218829 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7ff260039020 0x7ff26003b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:11.071 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.403+0000 7ff283000640 1 -- 172.21.15.1:0/3260218829 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff27c07cca0 msgr2=0x7ff27c19d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:11.071 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.403+0000 7ff283000640 1 --2- 172.21.15.1:0/3260218829 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff27c07cca0 0x7ff27c19d740 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff26c0097f0 tx=0x7ff26c0327b0 comp rx=0 tx=0).stop 2024-09-12T01:19:11.071 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.403+0000 7ff283000640 1 -- 172.21.15.1:0/3260218829 shutdown_connections 2024-09-12T01:19:11.071 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.403+0000 7ff283000640 1 --2- 172.21.15.1:0/3260218829 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7ff260039020 0x7ff26003b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:11.071 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.403+0000 7ff283000640 1 --2- 172.21.15.1:0/3260218829 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff27c07cca0 0x7ff27c19d740 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:11.071 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.403+0000 7ff283000640 1 -- 172.21.15.1:0/3260218829 >> 172.21.15.1:0/3260218829 conn(0x7ff27c102d20 msgr2=0x7ff27c103940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:19:11.071 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.403+0000 7ff283000640 1 -- 172.21.15.1:0/3260218829 shutdown_connections 2024-09-12T01:19:11.071 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:10.403+0000 7ff283000640 1 -- 172.21.15.1:0/3260218829 wait complete. 2024-09-12T01:19:11.072 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for the mgr to restart... 2024-09-12T01:19:11.072 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for mgr epoch 9... 2024-09-12T01:19:11.353 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:11 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:19:11.211+0000 7feb9fe36040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T01:19:11.353 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:11 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:19:11.297+0000 7feb9fe36040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T01:19:12.362 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:12 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:19:12.281+0000 7feb9fe36040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T01:19:12.677 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:12 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:19:12.427+0000 7feb9fe36040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T01:19:12.677 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:12 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:19:12.601+0000 7feb9fe36040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T01:19:12.946 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:12 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:19:12.678+0000 7feb9fe36040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T01:19:12.946 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:12 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:19:12.860+0000 7feb9fe36040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T01:19:12.946 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:12 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:19:12.946+0000 7feb9fe36040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T01:19:13.211 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:13 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:19:13.023+0000 7feb9fe36040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T01:19:13.211 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:13 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:19:13.212+0000 7feb9fe36040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T01:19:13.476 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:13 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:19:13.293+0000 7feb9fe36040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T01:19:13.476 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:13 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:19:13.375+0000 7feb9fe36040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T01:19:13.476 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:13 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:19:13.476+0000 7feb9fe36040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T01:19:13.927 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:13 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:19:13.552+0000 7feb9fe36040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T01:19:13.927 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:13 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:19:13.625+0000 7feb9fe36040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T01:19:14.849 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:14 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:19:14.578+0000 7feb9fe36040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T01:19:14.849 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:14 smithi001 ceph-mon[30387]: Active manager daemon x restarted 2024-09-12T01:19:14.850 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:14 smithi001 ceph-mon[30387]: Activating manager daemon x 2024-09-12T01:19:14.850 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:14 smithi001 ceph-mon[30387]: osdmap e3: 0 total, 0 up, 0 in 2024-09-12T01:19:14.850 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:14 smithi001 ceph-mon[30387]: mgrmap e10: x(active, starting, since 0.00376059s) 2024-09-12T01:19:14.850 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:14 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:19:14.850 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:14 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T01:19:14.850 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:14 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T01:19:14.850 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:14 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T01:19:14.850 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:14 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T01:19:14.850 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:14 smithi001 ceph-mon[30387]: Manager daemon x is now available 2024-09-12T01:19:14.850 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:14 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:19:15.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:15 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T01:19:15.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:15 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T01:19:15.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:15 smithi001 ceph-mon[30387]: mgrmap e11: x(active, since 1.00872s) 2024-09-12T01:19:16.657 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:16 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:19:16.311+0000 7feaf6400640 -1 log_channel(cephadm) log [ERR] : [12/Sep/2024:01:19:16] ENGINE Error in HTTPServer.serve 2024-09-12T01:19:16.657 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:16 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: Traceback (most recent call last): 2024-09-12T01:19:16.657 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:16 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T01:19:16.657 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:16 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: self._connections.run(self.expiration_interval) 2024-09-12T01:19:16.657 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:16 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T01:19:16.657 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:16 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: self._run(expiration_interval) 2024-09-12T01:19:16.657 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:16 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T01:19:16.658 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:16 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T01:19:16.658 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:16 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T01:19:16.658 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:16 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T01:19:16.658 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:16 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T01:19:16.658 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:16 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: s = self.context.wrap_socket( 2024-09-12T01:19:16.658 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:16 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T01:19:16.659 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:16 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: return self.sslsocket_class._create( 2024-09-12T01:19:16.659 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:16 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T01:19:16.659 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:16 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: self.do_handshake() 2024-09-12T01:19:16.659 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:16 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T01:19:16.659 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:16 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: self._sslobj.do_handshake() 2024-09-12T01:19:16.659 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:16 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T01:19:16.659 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:19:16 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:19:17.001 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-09-12T01:19:17.001 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-12T01:19:17.001 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-12T01:19:17.001 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-09-12T01:19:17.001 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:11.570+0000 7f6562000640 1 Processor -- start 2024-09-12T01:19:17.001 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:11.570+0000 7f6562000640 1 -- start start 2024-09-12T01:19:17.002 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:11.570+0000 7f6562000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f655c109810 0x7f655c109c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:17.002 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:11.570+0000 7f6562000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f655c10a1e0 con 0x7f655c109810 2024-09-12T01:19:17.002 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:11.571+0000 7f655b400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f655c109810 0x7f655c109c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:17.002 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:11.571+0000 7f655b400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f655c109810 0x7f655c109c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39614/0 (socket says 172.21.15.1:39614) 2024-09-12T01:19:17.002 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:11.571+0000 7f655b400640 1 -- 172.21.15.1:0/3308855780 learned_addr learned my addr 172.21.15.1:0/3308855780 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:19:17.002 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:11.571+0000 7f655b400640 1 -- 172.21.15.1:0/3308855780 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f655c10a9c0 con 0x7f655c109810 2024-09-12T01:19:17.002 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:11.572+0000 7f655b400640 1 --2- 172.21.15.1:0/3308855780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f655c109810 0x7f655c109c10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f654c009820 tx=0x7f654c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5e20737fd0b2a11 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:17.002 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:11.572+0000 7f655a000640 1 -- 172.21.15.1:0/3308855780 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f654c037070 con 0x7f655c109810 2024-09-12T01:19:17.002 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:11.572+0000 7f655a000640 1 -- 172.21.15.1:0/3308855780 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f654c032920 con 0x7f655c109810 2024-09-12T01:19:17.002 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:11.572+0000 7f655a000640 1 -- 172.21.15.1:0/3308855780 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f654c030810 con 0x7f655c109810 2024-09-12T01:19:17.002 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:11.573+0000 7f6562000640 1 -- 172.21.15.1:0/3308855780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f655c109810 msgr2=0x7f655c109c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:17.003 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:11.573+0000 7f6562000640 1 --2- 172.21.15.1:0/3308855780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f655c109810 0x7f655c109c10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f654c009820 tx=0x7f654c02a270 comp rx=0 tx=0).stop 2024-09-12T01:19:17.003 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:11.573+0000 7f6562000640 1 -- 172.21.15.1:0/3308855780 shutdown_connections 2024-09-12T01:19:17.003 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:11.573+0000 7f6562000640 1 --2- 172.21.15.1:0/3308855780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f655c109810 0x7f655c109c10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:17.003 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:11.573+0000 7f6562000640 1 -- 172.21.15.1:0/3308855780 >> 172.21.15.1:0/3308855780 conn(0x7f655c07ce80 msgr2=0x7f655c07d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:19:17.003 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:11.573+0000 7f6562000640 1 -- 172.21.15.1:0/3308855780 shutdown_connections 2024-09-12T01:19:17.003 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:11.573+0000 7f6562000640 1 -- 172.21.15.1:0/3308855780 wait complete. 2024-09-12T01:19:17.003 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:11.574+0000 7f6562000640 1 Processor -- start 2024-09-12T01:19:17.003 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:11.574+0000 7f6562000640 1 -- start start 2024-09-12T01:19:17.003 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:11.575+0000 7f6562000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f655c109810 0x7f655c19d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:17.003 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:11.575+0000 7f6562000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f655c10dd70 con 0x7f655c109810 2024-09-12T01:19:17.003 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:11.575+0000 7f655b400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f655c109810 0x7f655c19d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:17.003 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:11.575+0000 7f655b400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f655c109810 0x7f655c19d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39618/0 (socket says 172.21.15.1:39618) 2024-09-12T01:19:17.004 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:11.575+0000 7f655b400640 1 -- 172.21.15.1:0/1130586326 learned_addr learned my addr 172.21.15.1:0/1130586326 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:19:17.004 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:11.575+0000 7f655b400640 1 -- 172.21.15.1:0/1130586326 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f655c19dcc0 con 0x7f655c109810 2024-09-12T01:19:17.004 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:11.576+0000 7f655b400640 1 --2- 172.21.15.1:0/1130586326 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f655c109810 0x7f655c19d780 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f654c0097f0 tx=0x7f654c0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:17.004 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:11.576+0000 7f6553e00640 1 -- 172.21.15.1:0/1130586326 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f654c037040 con 0x7f655c109810 2024-09-12T01:19:17.004 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:11.576+0000 7f6562000640 1 -- 172.21.15.1:0/1130586326 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f655c19df50 con 0x7f655c109810 2024-09-12T01:19:17.004 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:11.576+0000 7f6553e00640 1 -- 172.21.15.1:0/1130586326 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f654c030cc0 con 0x7f655c109810 2024-09-12T01:19:17.004 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:11.576+0000 7f6553e00640 1 -- 172.21.15.1:0/1130586326 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f654c03a6b0 con 0x7f655c109810 2024-09-12T01:19:17.004 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:11.576+0000 7f6562000640 1 -- 172.21.15.1:0/1130586326 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f655c1a0c40 con 0x7f655c109810 2024-09-12T01:19:17.004 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:11.577+0000 7f6553e00640 1 -- 172.21.15.1:0/1130586326 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f654c03baf0 con 0x7f655c109810 2024-09-12T01:19:17.004 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:11.578+0000 7f6553e00640 1 --2- 172.21.15.1:0/1130586326 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f6540038fc0 0x7f654003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:17.004 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:11.578+0000 7f6553e00640 1 -- 172.21.15.1:0/1130586326 --> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f654003bb90 con 0x7f6540038fc0 2024-09-12T01:19:17.004 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:11.578+0000 7f6553e00640 1 -- 172.21.15.1:0/1130586326 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f654c06c610 con 0x7f655c109810 2024-09-12T01:19:17.004 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:11.578+0000 7f655aa00640 1 -- 172.21.15.1:0/1130586326 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f6540038fc0 msgr2=0x7f654003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/4101281615 2024-09-12T01:19:17.005 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:11.578+0000 7f655aa00640 1 --2- 172.21.15.1:0/1130586326 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f6540038fc0 0x7f654003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T01:19:17.005 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:11.779+0000 7f655aa00640 1 -- 172.21.15.1:0/1130586326 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f6540038fc0 msgr2=0x7f654003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/4101281615 2024-09-12T01:19:17.005 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:11.779+0000 7f655aa00640 1 --2- 172.21.15.1:0/1130586326 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f6540038fc0 0x7f654003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T01:19:17.005 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:12.180+0000 7f655aa00640 1 -- 172.21.15.1:0/1130586326 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f6540038fc0 msgr2=0x7f654003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/4101281615 2024-09-12T01:19:17.005 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:12.180+0000 7f655aa00640 1 --2- 172.21.15.1:0/1130586326 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f6540038fc0 0x7f654003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-12T01:19:17.005 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:12.981+0000 7f655aa00640 1 -- 172.21.15.1:0/1130586326 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f6540038fc0 msgr2=0x7f654003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/4101281615 2024-09-12T01:19:17.005 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:12.981+0000 7f655aa00640 1 --2- 172.21.15.1:0/1130586326 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f6540038fc0 0x7f654003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-12T01:19:17.005 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:14.582+0000 7f6553e00640 1 -- 172.21.15.1:0/1130586326 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f654c03b3f0 con 0x7f655c109810 2024-09-12T01:19:17.005 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:14.582+0000 7f6553e00640 1 -- 172.21.15.1:0/1130586326 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f6540038fc0 msgr2=0x7f654003b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T01:19:17.005 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:14.582+0000 7f6553e00640 1 --2- 172.21.15.1:0/1130586326 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f6540038fc0 0x7f654003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:17.005 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:15.585+0000 7f6553e00640 1 -- 172.21.15.1:0/1130586326 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f654c0405a0 con 0x7f655c109810 2024-09-12T01:19:17.005 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:15.585+0000 7f6553e00640 1 --2- 172.21.15.1:0/1130586326 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f654003ca80 0x7f654003ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:17.005 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:15.585+0000 7f6553e00640 1 -- 172.21.15.1:0/1130586326 --> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f654003bb90 con 0x7f654003ca80 2024-09-12T01:19:17.006 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:15.586+0000 7f655aa00640 1 --2- 172.21.15.1:0/1130586326 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f654003ca80 0x7f654003ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:17.006 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:15.588+0000 7f655aa00640 1 --2- 172.21.15.1:0/1130586326 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f654003ca80 0x7f654003ee70 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f6548003980 tx=0x7f6548006e10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:17.006 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:15.589+0000 7f6553e00640 1 -- 172.21.15.1:0/1130586326 <== mgr.14150 v2:172.21.15.1:6800/1928662456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f654003bb90 con 0x7f654003ca80 2024-09-12T01:19:17.006 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:15.596+0000 7f6562000640 1 -- 172.21.15.1:0/1130586326 --> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f655c109c90 con 0x7f654003ca80 2024-09-12T01:19:17.006 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:15.597+0000 7f6553e00640 1 -- 172.21.15.1:0/1130586326 <== mgr.14150 v2:172.21.15.1:6800/1928662456 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f655c109c90 con 0x7f654003ca80 2024-09-12T01:19:17.006 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:15.597+0000 7f6562000640 1 -- 172.21.15.1:0/1130586326 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f654003ca80 msgr2=0x7f654003ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:17.006 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:15.597+0000 7f6562000640 1 --2- 172.21.15.1:0/1130586326 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f654003ca80 0x7f654003ee70 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f6548003980 tx=0x7f6548006e10 comp rx=0 tx=0).stop 2024-09-12T01:19:17.006 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:15.597+0000 7f6562000640 1 -- 172.21.15.1:0/1130586326 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f655c109810 msgr2=0x7f655c19d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:17.006 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:15.597+0000 7f6562000640 1 --2- 172.21.15.1:0/1130586326 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f655c109810 0x7f655c19d780 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f654c0097f0 tx=0x7f654c0327b0 comp rx=0 tx=0).stop 2024-09-12T01:19:17.006 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:15.597+0000 7f6562000640 1 -- 172.21.15.1:0/1130586326 shutdown_connections 2024-09-12T01:19:17.006 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:15.597+0000 7f6562000640 1 --2- 172.21.15.1:0/1130586326 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f654003ca80 0x7f654003ee70 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:17.006 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:15.597+0000 7f6562000640 1 --2- 172.21.15.1:0/1130586326 >> [v2:172.21.15.1:6800/4101281615,v1:172.21.15.1:6801/4101281615] conn(0x7f6540038fc0 0x7f654003b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:17.007 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:15.597+0000 7f6562000640 1 --2- 172.21.15.1:0/1130586326 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f655c109810 0x7f655c19d780 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:17.007 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:15.597+0000 7f6562000640 1 -- 172.21.15.1:0/1130586326 >> 172.21.15.1:0/1130586326 conn(0x7f655c07ce80 msgr2=0x7f655c107460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:19:17.007 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:15.597+0000 7f6562000640 1 -- 172.21.15.1:0/1130586326 shutdown_connections 2024-09-12T01:19:17.007 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:15.598+0000 7f6562000640 1 -- 172.21.15.1:0/1130586326 wait complete. 2024-09-12T01:19:17.007 INFO:teuthology.orchestra.run.smithi001.stdout:mgr epoch 9 is available 2024-09-12T01:19:17.007 INFO:teuthology.orchestra.run.smithi001.stdout:Generating a dashboard self-signed certificate... 2024-09-12T01:19:17.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:17 smithi001 ceph-mon[30387]: [12/Sep/2024:01:19:16] ENGINE Bus STARTING 2024-09-12T01:19:17.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:17 smithi001 ceph-mon[30387]: [12/Sep/2024:01:19:16] ENGINE Serving on http://172.21.15.1:8765 2024-09-12T01:19:17.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:17 smithi001 ceph-mon[30387]: [12/Sep/2024:01:19:16] ENGINE Serving on https://172.21.15.1:7150 2024-09-12T01:19:17.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:17 smithi001 ceph-mon[30387]: [12/Sep/2024:01:19:16] ENGINE Bus STARTED 2024-09-12T01:19:17.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:17 smithi001 ceph-mon[30387]: [12/Sep/2024:01:19:16] ENGINE Error in HTTPServer.serve 2024-09-12T01:19:17.678 INFO:journalctl@ceph.mon.a.smithi001.stdout: Traceback (most recent call last): 2024-09-12T01:19:17.678 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T01:19:17.678 INFO:journalctl@ceph.mon.a.smithi001.stdout: self._connections.run(self.expiration_interval) 2024-09-12T01:19:17.678 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T01:19:17.678 INFO:journalctl@ceph.mon.a.smithi001.stdout: self._run(expiration_interval) 2024-09-12T01:19:17.678 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T01:19:17.679 INFO:journalctl@ceph.mon.a.smithi001.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T01:19:17.679 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T01:19:17.679 INFO:journalctl@ceph.mon.a.smithi001.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T01:19:17.679 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T01:19:17.679 INFO:journalctl@ceph.mon.a.smithi001.stdout: s = self.context.wrap_socket( 2024-09-12T01:19:17.679 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T01:19:17.679 INFO:journalctl@ceph.mon.a.smithi001.stdout: return self.sslsocket_class._create( 2024-09-12T01:19:17.679 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T01:19:17.679 INFO:journalctl@ceph.mon.a.smithi001.stdout: self.do_handshake() 2024-09-12T01:19:17.679 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T01:19:17.679 INFO:journalctl@ceph.mon.a.smithi001.stdout: self._sslobj.do_handshake() 2024-09-12T01:19:17.679 INFO:journalctl@ceph.mon.a.smithi001.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T01:19:17.679 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:17 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:18.521 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:18 smithi001 ceph-mon[30387]: mgrmap e12: x(active, since 2s) 2024-09-12T01:19:18.521 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:18 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:18.521 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:18 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:18.521 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:18 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:18.534 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-12T01:19:18.534 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.508+0000 7f420ea00640 1 Processor -- start 2024-09-12T01:19:18.534 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.509+0000 7f420ea00640 1 -- start start 2024-09-12T01:19:18.535 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.509+0000 7f420ea00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4208107600 0x7f4208107a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:18.535 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.509+0000 7f420ea00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4208107f40 con 0x7f4208107600 2024-09-12T01:19:18.535 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.510+0000 7f4207e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4208107600 0x7f4208107a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:18.535 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.510+0000 7f4207e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4208107600 0x7f4208107a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39684/0 (socket says 172.21.15.1:39684) 2024-09-12T01:19:18.535 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.510+0000 7f4207e00640 1 -- 172.21.15.1:0/4162076049 learned_addr learned my addr 172.21.15.1:0/4162076049 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:19:18.535 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.510+0000 7f4207e00640 1 -- 172.21.15.1:0/4162076049 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42081080c0 con 0x7f4208107600 2024-09-12T01:19:18.535 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.511+0000 7f4207e00640 1 --2- 172.21.15.1:0/4162076049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4208107600 0x7f4208107a00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f41f8009820 tx=0x7f41f802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a3d8542f0fb211e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:18.535 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.511+0000 7f4207400640 1 -- 172.21.15.1:0/4162076049 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41f8037070 con 0x7f4208107600 2024-09-12T01:19:18.535 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.511+0000 7f4207400640 1 -- 172.21.15.1:0/4162076049 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41f8032920 con 0x7f4208107600 2024-09-12T01:19:18.536 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.511+0000 7f4207400640 1 -- 172.21.15.1:0/4162076049 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41f8030810 con 0x7f4208107600 2024-09-12T01:19:18.536 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.512+0000 7f420ea00640 1 -- 172.21.15.1:0/4162076049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4208107600 msgr2=0x7f4208107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:18.536 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.512+0000 7f420ea00640 1 --2- 172.21.15.1:0/4162076049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4208107600 0x7f4208107a00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f41f8009820 tx=0x7f41f802a270 comp rx=0 tx=0).stop 2024-09-12T01:19:18.536 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.512+0000 7f420ea00640 1 -- 172.21.15.1:0/4162076049 shutdown_connections 2024-09-12T01:19:18.536 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.512+0000 7f420ea00640 1 --2- 172.21.15.1:0/4162076049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4208107600 0x7f4208107a00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:18.536 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.512+0000 7f420ea00640 1 -- 172.21.15.1:0/4162076049 >> 172.21.15.1:0/4162076049 conn(0x7f4208102e50 msgr2=0x7f4208105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:19:18.536 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.512+0000 7f420ea00640 1 -- 172.21.15.1:0/4162076049 shutdown_connections 2024-09-12T01:19:18.536 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.512+0000 7f420ea00640 1 -- 172.21.15.1:0/4162076049 wait complete. 2024-09-12T01:19:18.536 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.513+0000 7f420ea00640 1 Processor -- start 2024-09-12T01:19:18.536 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.513+0000 7f420ea00640 1 -- start start 2024-09-12T01:19:18.536 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.513+0000 7f420ea00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4208107600 0x7f420807db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:18.536 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.513+0000 7f420ea00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f420810ba70 con 0x7f4208107600 2024-09-12T01:19:18.537 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.514+0000 7f4207e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4208107600 0x7f420807db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:18.537 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.514+0000 7f4207e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4208107600 0x7f420807db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39700/0 (socket says 172.21.15.1:39700) 2024-09-12T01:19:18.537 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.514+0000 7f4207e00640 1 -- 172.21.15.1:0/4112840813 learned_addr learned my addr 172.21.15.1:0/4112840813 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:19:18.537 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.514+0000 7f4207e00640 1 -- 172.21.15.1:0/4112840813 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f420807e050 con 0x7f4208107600 2024-09-12T01:19:18.537 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.514+0000 7f4207e00640 1 --2- 172.21.15.1:0/4112840813 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4208107600 0x7f420807db10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f41f80097f0 tx=0x7f41f80327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:18.537 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.515+0000 7f4205600640 1 -- 172.21.15.1:0/4112840813 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41f8037040 con 0x7f4208107600 2024-09-12T01:19:18.537 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.515+0000 7f4205600640 1 -- 172.21.15.1:0/4112840813 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41f8030cc0 con 0x7f4208107600 2024-09-12T01:19:18.537 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.515+0000 7f4205600640 1 -- 172.21.15.1:0/4112840813 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41f803a6b0 con 0x7f4208107600 2024-09-12T01:19:18.537 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.515+0000 7f420ea00640 1 -- 172.21.15.1:0/4112840813 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f420807c240 con 0x7f4208107600 2024-09-12T01:19:18.537 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.515+0000 7f420ea00640 1 -- 172.21.15.1:0/4112840813 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f420807c780 con 0x7f4208107600 2024-09-12T01:19:18.537 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.516+0000 7f4205600640 1 -- 172.21.15.1:0/4112840813 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f41f803baf0 con 0x7f4208107600 2024-09-12T01:19:18.537 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.516+0000 7f420ea00640 1 -- 172.21.15.1:0/4112840813 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4208107a80 con 0x7f4208107600 2024-09-12T01:19:18.537 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.517+0000 7f4205600640 1 --2- 172.21.15.1:0/4112840813 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f41ec039020 0x7f41ec03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:18.538 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.517+0000 7f4205600640 1 -- 172.21.15.1:0/4112840813 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f41f806c6b0 con 0x7f4208107600 2024-09-12T01:19:18.538 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.517+0000 7f41ffe00640 1 --2- 172.21.15.1:0/4112840813 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f41ec039020 0x7f41ec03b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:18.538 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.518+0000 7f41ffe00640 1 --2- 172.21.15.1:0/4112840813 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f41ec039020 0x7f41ec03b4e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f41f00098c0 tx=0x7f41f0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:18.538 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.522+0000 7f4205600640 1 -- 172.21.15.1:0/4112840813 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f41f80371e0 con 0x7f4208107600 2024-09-12T01:19:18.538 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.716+0000 7f420ea00640 1 -- 172.21.15.1:0/4112840813 --> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f420807d0d0 con 0x7f41ec039020 2024-09-12T01:19:18.538 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.814+0000 7f4205600640 1 -- 172.21.15.1:0/4112840813 <== mgr.14150 v2:172.21.15.1:6800/1928662456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f420807d0d0 con 0x7f41ec039020 2024-09-12T01:19:18.538 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.817+0000 7f420ea00640 1 -- 172.21.15.1:0/4112840813 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f41ec039020 msgr2=0x7f41ec03b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:18.538 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.817+0000 7f420ea00640 1 --2- 172.21.15.1:0/4112840813 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f41ec039020 0x7f41ec03b4e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f41f00098c0 tx=0x7f41f0006db0 comp rx=0 tx=0).stop 2024-09-12T01:19:18.538 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.817+0000 7f420ea00640 1 -- 172.21.15.1:0/4112840813 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4208107600 msgr2=0x7f420807db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:18.538 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.817+0000 7f420ea00640 1 --2- 172.21.15.1:0/4112840813 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4208107600 0x7f420807db10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f41f80097f0 tx=0x7f41f80327b0 comp rx=0 tx=0).stop 2024-09-12T01:19:18.539 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.817+0000 7f420ea00640 1 -- 172.21.15.1:0/4112840813 shutdown_connections 2024-09-12T01:19:18.539 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.817+0000 7f420ea00640 1 --2- 172.21.15.1:0/4112840813 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f41ec039020 0x7f41ec03b4e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:18.539 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.817+0000 7f420ea00640 1 --2- 172.21.15.1:0/4112840813 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4208107600 0x7f420807db10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:18.539 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.817+0000 7f420ea00640 1 -- 172.21.15.1:0/4112840813 >> 172.21.15.1:0/4112840813 conn(0x7f4208102e50 msgr2=0x7f4208103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:19:18.539 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.817+0000 7f420ea00640 1 -- 172.21.15.1:0/4112840813 shutdown_connections 2024-09-12T01:19:18.539 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:17.817+0000 7f420ea00640 1 -- 172.21.15.1:0/4112840813 wait complete. 2024-09-12T01:19:18.539 INFO:teuthology.orchestra.run.smithi001.stdout:Creating initial admin user... 2024-09-12T01:19:19.582 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:19 smithi001 ceph-mon[30387]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:19:20.668 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:20 smithi001 ceph-mon[30387]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:19:20.668 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:20 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:21.108 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$dDOHLA6u1j2.knElM1olrOmjR/Yv8n3JmWmv9eLabpDYUmX4IZsma", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726103959, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-12T01:19:21.108 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.039+0000 7f3edc000640 1 Processor -- start 2024-09-12T01:19:21.109 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.039+0000 7f3edc000640 1 -- start start 2024-09-12T01:19:21.109 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.040+0000 7f3edc000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ed407cd20 0x7f3ed407b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:21.109 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.040+0000 7f3edc000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed407b7a0 con 0x7f3ed407cd20 2024-09-12T01:19:21.109 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.040+0000 7f3ed9600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ed407cd20 0x7f3ed407b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:21.109 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.040+0000 7f3ed9600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ed407cd20 0x7f3ed407b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51620/0 (socket says 172.21.15.1:51620) 2024-09-12T01:19:21.109 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.040+0000 7f3ed9600640 1 -- 172.21.15.1:0/1951126856 learned_addr learned my addr 172.21.15.1:0/1951126856 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:19:21.113 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.041+0000 7f3ed9600640 1 -- 172.21.15.1:0/1951126856 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed407b920 con 0x7f3ed407cd20 2024-09-12T01:19:21.113 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.042+0000 7f3ed9600640 1 --2- 172.21.15.1:0/1951126856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ed407cd20 0x7f3ed407b1d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8009820 tx=0x7f3ec802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d362c7b2a6c6dd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:21.113 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.042+0000 7f3ed3e00640 1 -- 172.21.15.1:0/1951126856 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ec8037070 con 0x7f3ed407cd20 2024-09-12T01:19:21.114 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.042+0000 7f3ed3e00640 1 -- 172.21.15.1:0/1951126856 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ec8032920 con 0x7f3ed407cd20 2024-09-12T01:19:21.114 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.042+0000 7f3ed3e00640 1 -- 172.21.15.1:0/1951126856 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ec8030810 con 0x7f3ed407cd20 2024-09-12T01:19:21.114 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.043+0000 7f3edc000640 1 -- 172.21.15.1:0/1951126856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ed407cd20 msgr2=0x7f3ed407b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:21.114 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.043+0000 7f3edc000640 1 --2- 172.21.15.1:0/1951126856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ed407cd20 0x7f3ed407b1d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8009820 tx=0x7f3ec802a270 comp rx=0 tx=0).stop 2024-09-12T01:19:21.114 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.043+0000 7f3edc000640 1 -- 172.21.15.1:0/1951126856 shutdown_connections 2024-09-12T01:19:21.114 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.043+0000 7f3edc000640 1 --2- 172.21.15.1:0/1951126856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ed407cd20 0x7f3ed407b1d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:21.114 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.043+0000 7f3edc000640 1 -- 172.21.15.1:0/1951126856 >> 172.21.15.1:0/1951126856 conn(0x7f3ed4102de0 msgr2=0x7f3ed4105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:19:21.114 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.043+0000 7f3edc000640 1 -- 172.21.15.1:0/1951126856 shutdown_connections 2024-09-12T01:19:21.114 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.043+0000 7f3edc000640 1 -- 172.21.15.1:0/1951126856 wait complete. 2024-09-12T01:19:21.114 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.044+0000 7f3edc000640 1 Processor -- start 2024-09-12T01:19:21.114 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.044+0000 7f3edc000640 1 -- start start 2024-09-12T01:19:21.114 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.044+0000 7f3edc000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ed407cd20 0x7f3ed41a1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:21.114 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.044+0000 7f3edc000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed4109830 con 0x7f3ed407cd20 2024-09-12T01:19:21.114 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.045+0000 7f3ed9600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ed407cd20 0x7f3ed41a1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:21.115 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.045+0000 7f3ed9600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ed407cd20 0x7f3ed41a1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51636/0 (socket says 172.21.15.1:51636) 2024-09-12T01:19:21.115 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.045+0000 7f3ed9600640 1 -- 172.21.15.1:0/3964849309 learned_addr learned my addr 172.21.15.1:0/3964849309 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:19:21.115 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.045+0000 7f3ed9600640 1 -- 172.21.15.1:0/3964849309 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed41a20a0 con 0x7f3ed407cd20 2024-09-12T01:19:21.115 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.045+0000 7f3ed9600640 1 --2- 172.21.15.1:0/3964849309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ed407cd20 0x7f3ed41a1b60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3ec80097f0 tx=0x7f3ec80327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:21.115 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.046+0000 7f3ed2000640 1 -- 172.21.15.1:0/3964849309 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ec8037040 con 0x7f3ed407cd20 2024-09-12T01:19:21.115 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.046+0000 7f3ed2000640 1 -- 172.21.15.1:0/3964849309 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ec8030cc0 con 0x7f3ed407cd20 2024-09-12T01:19:21.115 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.046+0000 7f3ed2000640 1 -- 172.21.15.1:0/3964849309 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ec803a6b0 con 0x7f3ed407cd20 2024-09-12T01:19:21.115 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.046+0000 7f3edc000640 1 -- 172.21.15.1:0/3964849309 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ed41a2330 con 0x7f3ed407cd20 2024-09-12T01:19:21.115 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.046+0000 7f3edc000640 1 -- 172.21.15.1:0/3964849309 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ed41a5020 con 0x7f3ed407cd20 2024-09-12T01:19:21.115 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.047+0000 7f3ed2000640 1 -- 172.21.15.1:0/3964849309 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f3ec803baf0 con 0x7f3ed407cd20 2024-09-12T01:19:21.115 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.047+0000 7f3edc000640 1 -- 172.21.15.1:0/3964849309 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ed407b250 con 0x7f3ed407cd20 2024-09-12T01:19:21.115 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.048+0000 7f3ed2000640 1 --2- 172.21.15.1:0/3964849309 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f3ebc039020 0x7f3ebc03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:21.115 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.048+0000 7f3ed2000640 1 -- 172.21.15.1:0/3964849309 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3ec806c5c0 con 0x7f3ed407cd20 2024-09-12T01:19:21.115 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.048+0000 7f3ed8c00640 1 --2- 172.21.15.1:0/3964849309 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f3ebc039020 0x7f3ebc03b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:21.116 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.049+0000 7f3ed8c00640 1 --2- 172.21.15.1:0/3964849309 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f3ebc039020 0x7f3ebc03b4e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3ec00098c0 tx=0x7f3ec0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:21.116 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.053+0000 7f3ed2000640 1 -- 172.21.15.1:0/3964849309 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3ec8042e10 con 0x7f3ed407cd20 2024-09-12T01:19:21.116 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.260+0000 7f3edc000640 1 -- 172.21.15.1:0/3964849309 --> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f3ed402d9b0 con 0x7f3ebc039020 2024-09-12T01:19:21.116 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.511+0000 7f3ed2000640 1 -- 172.21.15.1:0/3964849309 <== mgr.14150 v2:172.21.15.1:6800/1928662456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f3ed402d9b0 con 0x7f3ebc039020 2024-09-12T01:19:21.116 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.514+0000 7f3edc000640 1 -- 172.21.15.1:0/3964849309 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f3ebc039020 msgr2=0x7f3ebc03b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:21.116 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.514+0000 7f3edc000640 1 --2- 172.21.15.1:0/3964849309 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f3ebc039020 0x7f3ebc03b4e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3ec00098c0 tx=0x7f3ec0006db0 comp rx=0 tx=0).stop 2024-09-12T01:19:21.116 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.514+0000 7f3edc000640 1 -- 172.21.15.1:0/3964849309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ed407cd20 msgr2=0x7f3ed41a1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:21.116 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.514+0000 7f3edc000640 1 --2- 172.21.15.1:0/3964849309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ed407cd20 0x7f3ed41a1b60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3ec80097f0 tx=0x7f3ec80327b0 comp rx=0 tx=0).stop 2024-09-12T01:19:21.116 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.514+0000 7f3edc000640 1 -- 172.21.15.1:0/3964849309 shutdown_connections 2024-09-12T01:19:21.116 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.514+0000 7f3edc000640 1 --2- 172.21.15.1:0/3964849309 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f3ebc039020 0x7f3ebc03b4e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:21.116 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.514+0000 7f3edc000640 1 --2- 172.21.15.1:0/3964849309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ed407cd20 0x7f3ed41a1b60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:21.116 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.514+0000 7f3edc000640 1 -- 172.21.15.1:0/3964849309 >> 172.21.15.1:0/3964849309 conn(0x7f3ed4102de0 msgr2=0x7f3ed41051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:19:21.116 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.514+0000 7f3edc000640 1 -- 172.21.15.1:0/3964849309 shutdown_connections 2024-09-12T01:19:21.117 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:19.514+0000 7f3edc000640 1 -- 172.21.15.1:0/3964849309 wait complete. 2024-09-12T01:19:21.117 INFO:teuthology.orchestra.run.smithi001.stdout:Fetching dashboard port number... 2024-09-12T01:19:22.518 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 8443 2024-09-12T01:19:22.519 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.655+0000 7f67f8e00640 1 Processor -- start 2024-09-12T01:19:22.519 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.655+0000 7f67f8e00640 1 -- start start 2024-09-12T01:19:22.519 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.656+0000 7f67f8e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67f4107600 0x7f67f4107a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:22.519 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.656+0000 7f67f8e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f67f4107f40 con 0x7f67f4107600 2024-09-12T01:19:22.519 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.656+0000 7f67f1e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67f4107600 0x7f67f4107a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:22.520 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.656+0000 7f67f1e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67f4107600 0x7f67f4107a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51650/0 (socket says 172.21.15.1:51650) 2024-09-12T01:19:22.520 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.656+0000 7f67f1e00640 1 -- 172.21.15.1:0/4164566233 learned_addr learned my addr 172.21.15.1:0/4164566233 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:19:22.520 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.656+0000 7f67f1e00640 1 -- 172.21.15.1:0/4164566233 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67f41080c0 con 0x7f67f4107600 2024-09-12T01:19:22.520 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.657+0000 7f67f1e00640 1 --2- 172.21.15.1:0/4164566233 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67f4107600 0x7f67f4107a00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f67e4009820 tx=0x7f67e402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10efa1d3851454a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:22.520 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.658+0000 7f67f1400640 1 -- 172.21.15.1:0/4164566233 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67e4037070 con 0x7f67f4107600 2024-09-12T01:19:22.520 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.658+0000 7f67f1400640 1 -- 172.21.15.1:0/4164566233 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67e4032920 con 0x7f67f4107600 2024-09-12T01:19:22.521 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.658+0000 7f67f1400640 1 -- 172.21.15.1:0/4164566233 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67e4030810 con 0x7f67f4107600 2024-09-12T01:19:22.521 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.658+0000 7f67f8e00640 1 -- 172.21.15.1:0/4164566233 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67f4107600 msgr2=0x7f67f4107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:22.521 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.658+0000 7f67f8e00640 1 --2- 172.21.15.1:0/4164566233 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67f4107600 0x7f67f4107a00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f67e4009820 tx=0x7f67e402a270 comp rx=0 tx=0).stop 2024-09-12T01:19:22.521 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.658+0000 7f67f8e00640 1 -- 172.21.15.1:0/4164566233 shutdown_connections 2024-09-12T01:19:22.521 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.658+0000 7f67f8e00640 1 --2- 172.21.15.1:0/4164566233 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67f4107600 0x7f67f4107a00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:22.521 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.658+0000 7f67f8e00640 1 -- 172.21.15.1:0/4164566233 >> 172.21.15.1:0/4164566233 conn(0x7f67f4102e50 msgr2=0x7f67f4105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:19:22.521 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.658+0000 7f67f8e00640 1 -- 172.21.15.1:0/4164566233 shutdown_connections 2024-09-12T01:19:22.521 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.658+0000 7f67f8e00640 1 -- 172.21.15.1:0/4164566233 wait complete. 2024-09-12T01:19:22.521 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.659+0000 7f67f8e00640 1 Processor -- start 2024-09-12T01:19:22.521 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.660+0000 7f67f8e00640 1 -- start start 2024-09-12T01:19:22.521 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.660+0000 7f67f8e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67f4107600 0x7f67f419d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:22.521 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.660+0000 7f67f8e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f67f410ba70 con 0x7f67f4107600 2024-09-12T01:19:22.521 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.660+0000 7f67f1e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67f4107600 0x7f67f419d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:22.522 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.660+0000 7f67f1e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67f4107600 0x7f67f419d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51662/0 (socket says 172.21.15.1:51662) 2024-09-12T01:19:22.522 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.660+0000 7f67f1e00640 1 -- 172.21.15.1:0/2097795762 learned_addr learned my addr 172.21.15.1:0/2097795762 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:19:22.522 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.660+0000 7f67f1e00640 1 -- 172.21.15.1:0/2097795762 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67f419dd00 con 0x7f67f4107600 2024-09-12T01:19:22.522 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.661+0000 7f67f1e00640 1 --2- 172.21.15.1:0/2097795762 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67f4107600 0x7f67f419d7c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f67e4030380 tx=0x7f67e4032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:22.522 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.661+0000 7f67eaa00640 1 -- 172.21.15.1:0/2097795762 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67e4037040 con 0x7f67f4107600 2024-09-12T01:19:22.522 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.661+0000 7f67eaa00640 1 -- 172.21.15.1:0/2097795762 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67e4040070 con 0x7f67f4107600 2024-09-12T01:19:22.522 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.661+0000 7f67eaa00640 1 -- 172.21.15.1:0/2097795762 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67e403a7b0 con 0x7f67f4107600 2024-09-12T01:19:22.522 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.661+0000 7f67f8e00640 1 -- 172.21.15.1:0/2097795762 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67f419df90 con 0x7f67f4107600 2024-09-12T01:19:22.522 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.661+0000 7f67f8e00640 1 -- 172.21.15.1:0/2097795762 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67f41a0c80 con 0x7f67f4107600 2024-09-12T01:19:22.522 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.662+0000 7f67f8e00640 1 -- 172.21.15.1:0/2097795762 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67f4107a80 con 0x7f67f4107600 2024-09-12T01:19:22.523 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.663+0000 7f67eaa00640 1 -- 172.21.15.1:0/2097795762 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f67e403bc00 con 0x7f67f4107600 2024-09-12T01:19:22.523 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.663+0000 7f67eaa00640 1 --2- 172.21.15.1:0/2097795762 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f67d80390c0 0x7f67d803b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:22.523 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.663+0000 7f67eaa00640 1 -- 172.21.15.1:0/2097795762 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f67e406c850 con 0x7f67f4107600 2024-09-12T01:19:22.523 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.663+0000 7f67ebe00640 1 --2- 172.21.15.1:0/2097795762 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f67d80390c0 0x7f67d803b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:22.523 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.664+0000 7f67ebe00640 1 --2- 172.21.15.1:0/2097795762 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f67d80390c0 0x7f67d803b580 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f67dc0098c0 tx=0x7f67dc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:22.523 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.669+0000 7f67eaa00640 1 -- 172.21.15.1:0/2097795762 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f67e4043360 con 0x7f67f4107600 2024-09-12T01:19:22.523 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.854+0000 7f67f8e00640 1 -- 172.21.15.1:0/2097795762 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f67f41a1270 con 0x7f67f4107600 2024-09-12T01:19:22.523 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.855+0000 7f67eaa00640 1 -- 172.21.15.1:0/2097795762 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f67e4040220 con 0x7f67f4107600 2024-09-12T01:19:22.523 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.857+0000 7f67f8e00640 1 -- 172.21.15.1:0/2097795762 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f67d80390c0 msgr2=0x7f67d803b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:22.523 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.857+0000 7f67f8e00640 1 --2- 172.21.15.1:0/2097795762 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f67d80390c0 0x7f67d803b580 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f67dc0098c0 tx=0x7f67dc006db0 comp rx=0 tx=0).stop 2024-09-12T01:19:22.524 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.857+0000 7f67f8e00640 1 -- 172.21.15.1:0/2097795762 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67f4107600 msgr2=0x7f67f419d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:22.524 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.857+0000 7f67f8e00640 1 --2- 172.21.15.1:0/2097795762 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67f4107600 0x7f67f419d7c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f67e4030380 tx=0x7f67e4032670 comp rx=0 tx=0).stop 2024-09-12T01:19:22.524 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.857+0000 7f67f8e00640 1 -- 172.21.15.1:0/2097795762 shutdown_connections 2024-09-12T01:19:22.524 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.857+0000 7f67f8e00640 1 --2- 172.21.15.1:0/2097795762 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f67d80390c0 0x7f67d803b580 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:22.524 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.857+0000 7f67f8e00640 1 --2- 172.21.15.1:0/2097795762 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67f4107600 0x7f67f419d7c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:22.524 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.857+0000 7f67f8e00640 1 -- 172.21.15.1:0/2097795762 >> 172.21.15.1:0/2097795762 conn(0x7f67f4102e50 msgr2=0x7f67f4103910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:19:22.524 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.857+0000 7f67f8e00640 1 -- 172.21.15.1:0/2097795762 shutdown_connections 2024-09-12T01:19:22.524 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:21.858+0000 7f67f8e00640 1 -- 172.21.15.1:0/2097795762 wait complete. 2024-09-12T01:19:22.534 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T01:19:22.534 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout disabled 2024-09-12T01:19:22.547 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T01:19:22.548 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout inactive 2024-09-12T01:19:22.548 INFO:teuthology.orchestra.run.smithi001.stdout:firewalld.service is not enabled 2024-09-12T01:19:22.548 INFO:teuthology.orchestra.run.smithi001.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-12T01:19:22.550 INFO:teuthology.orchestra.run.smithi001.stdout:Ceph Dashboard is now available at: 2024-09-12T01:19:22.550 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:19:22.550 INFO:teuthology.orchestra.run.smithi001.stdout: URL: https://smithi001.front.sepia.ceph.com:8443/ 2024-09-12T01:19:22.550 INFO:teuthology.orchestra.run.smithi001.stdout: User: admin 2024-09-12T01:19:22.550 INFO:teuthology.orchestra.run.smithi001.stdout: Password: surnr3gwr0 2024-09-12T01:19:22.550 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:19:22.551 INFO:teuthology.orchestra.run.smithi001.stdout:Saving cluster configuration to /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/config directory 2024-09-12T01:19:22.821 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:22 smithi001 ceph-mon[30387]: mgrmap e13: x(active, since 6s) 2024-09-12T01:19:22.822 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:22 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/2097795762' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-12T01:19:24.024 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.088+0000 7f150e400640 1 Processor -- start 2024-09-12T01:19:24.024 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.088+0000 7f150e400640 1 -- start start 2024-09-12T01:19:24.024 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.088+0000 7f150e400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f150807dd30 0x7f150807c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:24.025 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.089+0000 7f150e400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f150807c7e0 con 0x7f150807dd30 2024-09-12T01:19:24.025 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.089+0000 7f1507400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f150807dd30 0x7f150807c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:24.025 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.089+0000 7f1507400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f150807dd30 0x7f150807c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51674/0 (socket says 172.21.15.1:51674) 2024-09-12T01:19:24.025 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.089+0000 7f1507400640 1 -- 172.21.15.1:0/387408716 learned_addr learned my addr 172.21.15.1:0/387408716 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:19:24.025 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.089+0000 7f1507400640 1 -- 172.21.15.1:0/387408716 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f150807c960 con 0x7f150807dd30 2024-09-12T01:19:24.025 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.090+0000 7f1507400640 1 --2- 172.21.15.1:0/387408716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f150807dd30 0x7f150807c2a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f14f8009820 tx=0x7f14f802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=822183eec4357f77 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:24.025 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.091+0000 7f1506a00640 1 -- 172.21.15.1:0/387408716 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14f8037070 con 0x7f150807dd30 2024-09-12T01:19:24.025 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.091+0000 7f1506a00640 1 -- 172.21.15.1:0/387408716 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14f8032920 con 0x7f150807dd30 2024-09-12T01:19:24.025 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.091+0000 7f1506a00640 1 -- 172.21.15.1:0/387408716 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14f8030810 con 0x7f150807dd30 2024-09-12T01:19:24.025 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.091+0000 7f150e400640 1 -- 172.21.15.1:0/387408716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f150807dd30 msgr2=0x7f150807c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:24.026 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.091+0000 7f150e400640 1 --2- 172.21.15.1:0/387408716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f150807dd30 0x7f150807c2a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f14f8009820 tx=0x7f14f802a270 comp rx=0 tx=0).stop 2024-09-12T01:19:24.026 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.091+0000 7f150e400640 1 -- 172.21.15.1:0/387408716 shutdown_connections 2024-09-12T01:19:24.026 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.091+0000 7f150e400640 1 --2- 172.21.15.1:0/387408716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f150807dd30 0x7f150807c2a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:24.026 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.091+0000 7f150e400640 1 -- 172.21.15.1:0/387408716 >> 172.21.15.1:0/387408716 conn(0x7f1508102e50 msgr2=0x7f1508105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:19:24.026 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.092+0000 7f150e400640 1 -- 172.21.15.1:0/387408716 shutdown_connections 2024-09-12T01:19:24.026 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.092+0000 7f150e400640 1 -- 172.21.15.1:0/387408716 wait complete. 2024-09-12T01:19:24.026 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.092+0000 7f150e400640 1 Processor -- start 2024-09-12T01:19:24.026 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.093+0000 7f150e400640 1 -- start start 2024-09-12T01:19:24.026 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.093+0000 7f150e400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f150807dd30 0x7f150819d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:24.026 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.093+0000 7f150e400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f150810db80 con 0x7f150807dd30 2024-09-12T01:19:24.026 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.093+0000 7f1507400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f150807dd30 0x7f150819d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:24.026 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.093+0000 7f1507400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f150807dd30 0x7f150819d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51680/0 (socket says 172.21.15.1:51680) 2024-09-12T01:19:24.027 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.093+0000 7f1507400640 1 -- 172.21.15.1:0/3976012150 learned_addr learned my addr 172.21.15.1:0/3976012150 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:19:24.027 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.093+0000 7f1507400640 1 -- 172.21.15.1:0/3976012150 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f150819dce0 con 0x7f150807dd30 2024-09-12T01:19:24.027 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.094+0000 7f1507400640 1 --2- 172.21.15.1:0/3976012150 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f150807dd30 0x7f150819d7a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f14f80097f0 tx=0x7f14f80327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:24.027 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.094+0000 7f1504c00640 1 -- 172.21.15.1:0/3976012150 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14f8037040 con 0x7f150807dd30 2024-09-12T01:19:24.027 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.094+0000 7f1504c00640 1 -- 172.21.15.1:0/3976012150 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14f8030cc0 con 0x7f150807dd30 2024-09-12T01:19:24.027 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.094+0000 7f1504c00640 1 -- 172.21.15.1:0/3976012150 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14f803a6b0 con 0x7f150807dd30 2024-09-12T01:19:24.027 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.094+0000 7f150e400640 1 -- 172.21.15.1:0/3976012150 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f150819df70 con 0x7f150807dd30 2024-09-12T01:19:24.027 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.094+0000 7f150e400640 1 -- 172.21.15.1:0/3976012150 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15081a0c60 con 0x7f150807dd30 2024-09-12T01:19:24.027 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.095+0000 7f1504c00640 1 -- 172.21.15.1:0/3976012150 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f14f803b3f0 con 0x7f150807dd30 2024-09-12T01:19:24.027 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.095+0000 7f150e400640 1 -- 172.21.15.1:0/3976012150 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f150807c320 con 0x7f150807dd30 2024-09-12T01:19:24.027 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.096+0000 7f1504c00640 1 --2- 172.21.15.1:0/3976012150 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f14ec0390c0 0x7f14ec03b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:24.027 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.096+0000 7f1504c00640 1 -- 172.21.15.1:0/3976012150 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f14f806c680 con 0x7f150807dd30 2024-09-12T01:19:24.028 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.096+0000 7f14ffe00640 1 --2- 172.21.15.1:0/3976012150 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f14ec0390c0 0x7f14ec03b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:24.028 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.097+0000 7f14ffe00640 1 --2- 172.21.15.1:0/3976012150 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f14ec0390c0 0x7f14ec03b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f14f00098c0 tx=0x7f14f0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:24.028 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.101+0000 7f1504c00640 1 -- 172.21.15.1:0/3976012150 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f14f803a850 con 0x7f150807dd30 2024-09-12T01:19:24.028 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.383+0000 7f150e400640 1 -- 172.21.15.1:0/3976012150 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f15081a1250 con 0x7f150807dd30 2024-09-12T01:19:24.028 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.386+0000 7f1504c00640 1 -- 172.21.15.1:0/3976012150 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f14f8044c50 con 0x7f150807dd30 2024-09-12T01:19:24.028 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-12T01:19:24.028 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.388+0000 7f150e400640 1 -- 172.21.15.1:0/3976012150 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f14ec0390c0 msgr2=0x7f14ec03b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:24.028 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.388+0000 7f150e400640 1 --2- 172.21.15.1:0/3976012150 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f14ec0390c0 0x7f14ec03b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f14f00098c0 tx=0x7f14f0006db0 comp rx=0 tx=0).stop 2024-09-12T01:19:24.028 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.388+0000 7f150e400640 1 -- 172.21.15.1:0/3976012150 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f150807dd30 msgr2=0x7f150819d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:24.028 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.388+0000 7f150e400640 1 --2- 172.21.15.1:0/3976012150 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f150807dd30 0x7f150819d7a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f14f80097f0 tx=0x7f14f80327b0 comp rx=0 tx=0).stop 2024-09-12T01:19:24.028 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.388+0000 7f150e400640 1 -- 172.21.15.1:0/3976012150 shutdown_connections 2024-09-12T01:19:24.029 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.388+0000 7f150e400640 1 --2- 172.21.15.1:0/3976012150 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f14ec0390c0 0x7f14ec03b580 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:24.029 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.388+0000 7f150e400640 1 --2- 172.21.15.1:0/3976012150 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f150807dd30 0x7f150819d7a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:24.029 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.388+0000 7f150e400640 1 -- 172.21.15.1:0/3976012150 >> 172.21.15.1:0/3976012150 conn(0x7f1508102e50 msgr2=0x7f1508103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:19:24.029 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.388+0000 7f150e400640 1 -- 172.21.15.1:0/3976012150 shutdown_connections 2024-09-12T01:19:24.029 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-12T01:19:23.388+0000 7f150e400640 1 -- 172.21.15.1:0/3976012150 wait complete. 2024-09-12T01:19:24.029 INFO:teuthology.orchestra.run.smithi001.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-12T01:19:24.029 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:19:24.029 INFO:teuthology.orchestra.run.smithi001.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-12T01:19:24.029 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:19:24.029 INFO:teuthology.orchestra.run.smithi001.stdout:Or, if you are only running a single cluster on this host: 2024-09-12T01:19:24.029 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:19:24.029 INFO:teuthology.orchestra.run.smithi001.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-12T01:19:24.029 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:19:24.030 INFO:teuthology.orchestra.run.smithi001.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-12T01:19:24.030 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:19:24.030 INFO:teuthology.orchestra.run.smithi001.stdout: ceph telemetry on 2024-09-12T01:19:24.030 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:19:24.030 INFO:teuthology.orchestra.run.smithi001.stdout:For more information see: 2024-09-12T01:19:24.030 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:19:24.030 INFO:teuthology.orchestra.run.smithi001.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-12T01:19:24.030 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:19:24.030 INFO:teuthology.orchestra.run.smithi001.stdout:Bootstrap complete. 2024-09-12T01:19:24.080 INFO:tasks.cephadm:Fetching config... 2024-09-12T01:19:24.080 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-12T01:19:24.080 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-12T01:19:24.117 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-12T01:19:24.117 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-12T01:19:24.117 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-12T01:19:24.194 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-12T01:19:24.194 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-12T01:19:24.195 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-12T01:19:24.291 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-12T01:19:24.291 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-12T01:19:24.291 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-12T01:19:24.367 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-12T01:19:24.368 DEBUG:teuthology.orchestra.run.smithi001:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9pYVDjjOEWSD7atjnaT7qHWC2nRy+lN8m2hT/GIIj2Wi1T+/jPeK2b3kcs/bC6X+8dz0nGj3hl0Ie1C1eYofICplHdYkquBSGRawo0m33jI2miveIeZqDuUR9b8yoEzBcrTSCfpQ146ZQRRAXKrcTeoDN9sfogLGpG/gM/l+0iSZrYsxaeBaD9VaMZGP+as8mlvU7BPBDpkn3UJIX2G/isEObv5DcQoB1+fVyFftMwLo5K5S7TisCux74Z3BY3NMJpK7y55PNfTtMJ0NjQralxPhXGebAaGXakChKa6GVnZRJKg0LX/tomFmQb2hbxIjAqQDmxGpnEbCx50sfQtYg2fWuEF2FlXH/dqemBzdIJ00XSDAlEGX2Ld3ZsL306HU8SU/Anx29Hi/sU7Fe4GbS7lr5CrorKPZ7QktEUDHZGSYI1lKMuTidi2+NOiCTEMOW7jcy9+CKH+LHm3zRslC2B5JvoG3XCKg6DJsCUdXR1f9g/bDanCa4f8v/r2uJ4iE= ceph-c0e4f180-70a4-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-12T01:19:24.493 INFO:teuthology.orchestra.run.smithi001.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9pYVDjjOEWSD7atjnaT7qHWC2nRy+lN8m2hT/GIIj2Wi1T+/jPeK2b3kcs/bC6X+8dz0nGj3hl0Ie1C1eYofICplHdYkquBSGRawo0m33jI2miveIeZqDuUR9b8yoEzBcrTSCfpQ146ZQRRAXKrcTeoDN9sfogLGpG/gM/l+0iSZrYsxaeBaD9VaMZGP+as8mlvU7BPBDpkn3UJIX2G/isEObv5DcQoB1+fVyFftMwLo5K5S7TisCux74Z3BY3NMJpK7y55PNfTtMJ0NjQralxPhXGebAaGXakChKa6GVnZRJKg0LX/tomFmQb2hbxIjAqQDmxGpnEbCx50sfQtYg2fWuEF2FlXH/dqemBzdIJ00XSDAlEGX2Ld3ZsL306HU8SU/Anx29Hi/sU7Fe4GbS7lr5CrorKPZ7QktEUDHZGSYI1lKMuTidi2+NOiCTEMOW7jcy9+CKH+LHm3zRslC2B5JvoG3XCKg6DJsCUdXR1f9g/bDanCa4f8v/r2uJ4iE= ceph-c0e4f180-70a4-11ef-bcea-c7b262605968 2024-09-12T01:19:24.518 DEBUG:teuthology.orchestra.run.smithi053:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9pYVDjjOEWSD7atjnaT7qHWC2nRy+lN8m2hT/GIIj2Wi1T+/jPeK2b3kcs/bC6X+8dz0nGj3hl0Ie1C1eYofICplHdYkquBSGRawo0m33jI2miveIeZqDuUR9b8yoEzBcrTSCfpQ146ZQRRAXKrcTeoDN9sfogLGpG/gM/l+0iSZrYsxaeBaD9VaMZGP+as8mlvU7BPBDpkn3UJIX2G/isEObv5DcQoB1+fVyFftMwLo5K5S7TisCux74Z3BY3NMJpK7y55PNfTtMJ0NjQralxPhXGebAaGXakChKa6GVnZRJKg0LX/tomFmQb2hbxIjAqQDmxGpnEbCx50sfQtYg2fWuEF2FlXH/dqemBzdIJ00XSDAlEGX2Ld3ZsL306HU8SU/Anx29Hi/sU7Fe4GbS7lr5CrorKPZ7QktEUDHZGSYI1lKMuTidi2+NOiCTEMOW7jcy9+CKH+LHm3zRslC2B5JvoG3XCKg6DJsCUdXR1f9g/bDanCa4f8v/r2uJ4iE= ceph-c0e4f180-70a4-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-12T01:19:24.559 INFO:teuthology.orchestra.run.smithi053.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9pYVDjjOEWSD7atjnaT7qHWC2nRy+lN8m2hT/GIIj2Wi1T+/jPeK2b3kcs/bC6X+8dz0nGj3hl0Ie1C1eYofICplHdYkquBSGRawo0m33jI2miveIeZqDuUR9b8yoEzBcrTSCfpQ146ZQRRAXKrcTeoDN9sfogLGpG/gM/l+0iSZrYsxaeBaD9VaMZGP+as8mlvU7BPBDpkn3UJIX2G/isEObv5DcQoB1+fVyFftMwLo5K5S7TisCux74Z3BY3NMJpK7y55PNfTtMJ0NjQralxPhXGebAaGXakChKa6GVnZRJKg0LX/tomFmQb2hbxIjAqQDmxGpnEbCx50sfQtYg2fWuEF2FlXH/dqemBzdIJ00XSDAlEGX2Ld3ZsL306HU8SU/Anx29Hi/sU7Fe4GbS7lr5CrorKPZ7QktEUDHZGSYI1lKMuTidi2+NOiCTEMOW7jcy9+CKH+LHm3zRslC2B5JvoG3XCKg6DJsCUdXR1f9g/bDanCa4f8v/r2uJ4iE= ceph-c0e4f180-70a4-11ef-bcea-c7b262605968 2024-09-12T01:19:24.572 DEBUG:teuthology.orchestra.run.smithi104:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9pYVDjjOEWSD7atjnaT7qHWC2nRy+lN8m2hT/GIIj2Wi1T+/jPeK2b3kcs/bC6X+8dz0nGj3hl0Ie1C1eYofICplHdYkquBSGRawo0m33jI2miveIeZqDuUR9b8yoEzBcrTSCfpQ146ZQRRAXKrcTeoDN9sfogLGpG/gM/l+0iSZrYsxaeBaD9VaMZGP+as8mlvU7BPBDpkn3UJIX2G/isEObv5DcQoB1+fVyFftMwLo5K5S7TisCux74Z3BY3NMJpK7y55PNfTtMJ0NjQralxPhXGebAaGXakChKa6GVnZRJKg0LX/tomFmQb2hbxIjAqQDmxGpnEbCx50sfQtYg2fWuEF2FlXH/dqemBzdIJ00XSDAlEGX2Ld3ZsL306HU8SU/Anx29Hi/sU7Fe4GbS7lr5CrorKPZ7QktEUDHZGSYI1lKMuTidi2+NOiCTEMOW7jcy9+CKH+LHm3zRslC2B5JvoG3XCKg6DJsCUdXR1f9g/bDanCa4f8v/r2uJ4iE= ceph-c0e4f180-70a4-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-12T01:19:24.597 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:24 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/3976012150' entity='client.admin' 2024-09-12T01:19:24.613 INFO:teuthology.orchestra.run.smithi104.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9pYVDjjOEWSD7atjnaT7qHWC2nRy+lN8m2hT/GIIj2Wi1T+/jPeK2b3kcs/bC6X+8dz0nGj3hl0Ie1C1eYofICplHdYkquBSGRawo0m33jI2miveIeZqDuUR9b8yoEzBcrTSCfpQ146ZQRRAXKrcTeoDN9sfogLGpG/gM/l+0iSZrYsxaeBaD9VaMZGP+as8mlvU7BPBDpkn3UJIX2G/isEObv5DcQoB1+fVyFftMwLo5K5S7TisCux74Z3BY3NMJpK7y55PNfTtMJ0NjQralxPhXGebAaGXakChKa6GVnZRJKg0LX/tomFmQb2hbxIjAqQDmxGpnEbCx50sfQtYg2fWuEF2FlXH/dqemBzdIJ00XSDAlEGX2Ld3ZsL306HU8SU/Anx29Hi/sU7Fe4GbS7lr5CrorKPZ7QktEUDHZGSYI1lKMuTidi2+NOiCTEMOW7jcy9+CKH+LHm3zRslC2B5JvoG3XCKg6DJsCUdXR1f9g/bDanCa4f8v/r2uJ4iE= ceph-c0e4f180-70a4-11ef-bcea-c7b262605968 2024-09-12T01:19:24.626 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-12T01:19:24.994 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:19:25.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:25.555+0000 7f01d3c00640 1 -- 172.21.15.1:0/1244502865 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f01cc1037f0 msgr2=0x7f01cc103bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:25.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:25.555+0000 7f01d3c00640 1 --2- 172.21.15.1:0/1244502865 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f01cc1037f0 0x7f01cc103bf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f01bc009880 tx=0x7f01bc02a5d0 comp rx=0 tx=0).stop 2024-09-12T01:19:25.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:25.556+0000 7f01d3c00640 1 -- 172.21.15.1:0/1244502865 shutdown_connections 2024-09-12T01:19:25.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:25.556+0000 7f01d3c00640 1 --2- 172.21.15.1:0/1244502865 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f01cc1037f0 0x7f01cc103bf0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:25.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:25.556+0000 7f01d3c00640 1 -- 172.21.15.1:0/1244502865 >> 172.21.15.1:0/1244502865 conn(0x7f01cc0fefc0 msgr2=0x7f01cc101400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:19:25.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:25.556+0000 7f01d3c00640 1 -- 172.21.15.1:0/1244502865 shutdown_connections 2024-09-12T01:19:25.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:25.556+0000 7f01d3c00640 1 -- 172.21.15.1:0/1244502865 wait complete. 2024-09-12T01:19:25.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:25.556+0000 7f01d3c00640 1 Processor -- start 2024-09-12T01:19:25.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:25.557+0000 7f01d3c00640 1 -- start start 2024-09-12T01:19:25.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:25.557+0000 7f01d3c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f01cc1037f0 0x7f01cc198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:25.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:25.557+0000 7f01d3c00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f01cc109900 con 0x7f01cc1037f0 2024-09-12T01:19:25.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:25.557+0000 7f01d1200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f01cc1037f0 0x7f01cc198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:25.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:25.557+0000 7f01d1200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f01cc1037f0 0x7f01cc198f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51698/0 (socket says 172.21.15.1:51698) 2024-09-12T01:19:25.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:25.557+0000 7f01d1200640 1 -- 172.21.15.1:0/3800522711 learned_addr learned my addr 172.21.15.1:0/3800522711 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:19:25.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:25.558+0000 7f01d1200640 1 -- 172.21.15.1:0/3800522711 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01cc1994c0 con 0x7f01cc1037f0 2024-09-12T01:19:25.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:25.558+0000 7f01d1200640 1 --2- 172.21.15.1:0/3800522711 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f01cc1037f0 0x7f01cc198f80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f01bc004f60 tx=0x7f01bc0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:25.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:25.558+0000 7f01ca000640 1 -- 172.21.15.1:0/3800522711 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01bc03f070 con 0x7f01cc1037f0 2024-09-12T01:19:25.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:25.558+0000 7f01ca000640 1 -- 172.21.15.1:0/3800522711 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01bc004d40 con 0x7f01cc1037f0 2024-09-12T01:19:25.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:25.558+0000 7f01ca000640 1 -- 172.21.15.1:0/3800522711 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01bc035030 con 0x7f01cc1037f0 2024-09-12T01:19:25.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:25.558+0000 7f01d3c00640 1 -- 172.21.15.1:0/3800522711 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01cc199750 con 0x7f01cc1037f0 2024-09-12T01:19:25.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:25.559+0000 7f01d3c00640 1 -- 172.21.15.1:0/3800522711 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01cc19c440 con 0x7f01cc1037f0 2024-09-12T01:19:25.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:25.560+0000 7f01ca000640 1 -- 172.21.15.1:0/3800522711 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f01bc042070 con 0x7f01cc1037f0 2024-09-12T01:19:25.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:25.560+0000 7f01d3c00640 1 -- 172.21.15.1:0/3800522711 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01cc10a080 con 0x7f01cc1037f0 2024-09-12T01:19:25.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:25.560+0000 7f01ca000640 1 --2- 172.21.15.1:0/3800522711 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f01b00390c0 0x7f01b003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:25.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:25.560+0000 7f01ca000640 1 -- 172.21.15.1:0/3800522711 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f01bc06cd80 con 0x7f01cc1037f0 2024-09-12T01:19:25.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:25.561+0000 7f01c3e00640 1 --2- 172.21.15.1:0/3800522711 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f01b00390c0 0x7f01b003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:25.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:25.561+0000 7f01c3e00640 1 --2- 172.21.15.1:0/3800522711 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f01b00390c0 0x7f01b003b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f01b40098c0 tx=0x7f01b4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:25.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:25.568+0000 7f01ca000640 1 -- 172.21.15.1:0/3800522711 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f01bc04a260 con 0x7f01cc1037f0 2024-09-12T01:19:25.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:25.763+0000 7f01d3c00640 1 -- 172.21.15.1:0/3800522711 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f01cc02d830 con 0x7f01cc1037f0 2024-09-12T01:19:25.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:25.766+0000 7f01ca000640 1 -- 172.21.15.1:0/3800522711 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f01bc0035f0 con 0x7f01cc1037f0 2024-09-12T01:19:25.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:25.768+0000 7f01d3c00640 1 -- 172.21.15.1:0/3800522711 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f01b00390c0 msgr2=0x7f01b003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:25.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:25.768+0000 7f01d3c00640 1 --2- 172.21.15.1:0/3800522711 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f01b00390c0 0x7f01b003b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f01b40098c0 tx=0x7f01b4006db0 comp rx=0 tx=0).stop 2024-09-12T01:19:25.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:25.768+0000 7f01d3c00640 1 -- 172.21.15.1:0/3800522711 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f01cc1037f0 msgr2=0x7f01cc198f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:25.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:25.768+0000 7f01d3c00640 1 --2- 172.21.15.1:0/3800522711 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f01cc1037f0 0x7f01cc198f80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f01bc004f60 tx=0x7f01bc0048f0 comp rx=0 tx=0).stop 2024-09-12T01:19:25.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:25.768+0000 7f01d3c00640 1 -- 172.21.15.1:0/3800522711 shutdown_connections 2024-09-12T01:19:25.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:25.768+0000 7f01d3c00640 1 --2- 172.21.15.1:0/3800522711 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f01b00390c0 0x7f01b003b580 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:25.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:25.768+0000 7f01d3c00640 1 --2- 172.21.15.1:0/3800522711 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f01cc1037f0 0x7f01cc198f80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:25.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:25.768+0000 7f01d3c00640 1 -- 172.21.15.1:0/3800522711 >> 172.21.15.1:0/3800522711 conn(0x7f01cc0fefc0 msgr2=0x7f01cc0ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:19:25.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:25.769+0000 7f01d3c00640 1 -- 172.21.15.1:0/3800522711 shutdown_connections 2024-09-12T01:19:25.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:25.769+0000 7f01d3c00640 1 -- 172.21.15.1:0/3800522711 wait complete. 2024-09-12T01:19:26.973 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:26 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/3800522711' entity='client.admin' 2024-09-12T01:19:27.410 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-12T01:19:27.410 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-12T01:19:27.959 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:19:28.098 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:27 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:28.098 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:27 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:28.099 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:27 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi001", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:19:28.099 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:27 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:28.099 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:27 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:19:28.099 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:27 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:28.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:28.478+0000 7f1701800640 1 -- 172.21.15.1:0/3928073641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16fc101430 msgr2=0x7f16fc101830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:28.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:28.478+0000 7f1701800640 1 --2- 172.21.15.1:0/3928073641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16fc101430 0x7f16fc101830 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f16ec009880 tx=0x7f16ec02a5d0 comp rx=0 tx=0).stop 2024-09-12T01:19:28.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:28.478+0000 7f1701800640 1 -- 172.21.15.1:0/3928073641 shutdown_connections 2024-09-12T01:19:28.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:28.478+0000 7f1701800640 1 --2- 172.21.15.1:0/3928073641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16fc101430 0x7f16fc101830 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:28.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:28.478+0000 7f1701800640 1 -- 172.21.15.1:0/3928073641 >> 172.21.15.1:0/3928073641 conn(0x7f16fc0fcbc0 msgr2=0x7f16fc0ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:19:28.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:28.478+0000 7f1701800640 1 -- 172.21.15.1:0/3928073641 shutdown_connections 2024-09-12T01:19:28.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:28.478+0000 7f1701800640 1 -- 172.21.15.1:0/3928073641 wait complete. 2024-09-12T01:19:28.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:28.479+0000 7f1701800640 1 Processor -- start 2024-09-12T01:19:28.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:28.479+0000 7f1701800640 1 -- start start 2024-09-12T01:19:28.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:28.480+0000 7f1701800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16fc101430 0x7f16fc075eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:28.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:28.480+0000 7f1701800640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f16fc107540 con 0x7f16fc101430 2024-09-12T01:19:28.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:28.480+0000 7f16fbe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16fc101430 0x7f16fc075eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:28.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:28.480+0000 7f16fbe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16fc101430 0x7f16fc075eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51704/0 (socket says 172.21.15.1:51704) 2024-09-12T01:19:28.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:28.480+0000 7f16fbe00640 1 -- 172.21.15.1:0/3917837024 learned_addr learned my addr 172.21.15.1:0/3917837024 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:19:28.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:28.480+0000 7f16fbe00640 1 -- 172.21.15.1:0/3917837024 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16fc0763f0 con 0x7f16fc101430 2024-09-12T01:19:28.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:28.480+0000 7f16fbe00640 1 --2- 172.21.15.1:0/3917837024 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16fc101430 0x7f16fc075eb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f16ec004f60 tx=0x7f16ec0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:28.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:28.481+0000 7f16f9600640 1 -- 172.21.15.1:0/3917837024 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16ec03f070 con 0x7f16fc101430 2024-09-12T01:19:28.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:28.481+0000 7f16f9600640 1 -- 172.21.15.1:0/3917837024 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16ec004d40 con 0x7f16fc101430 2024-09-12T01:19:28.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:28.481+0000 7f1701800640 1 -- 172.21.15.1:0/3917837024 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16fc0745e0 con 0x7f16fc101430 2024-09-12T01:19:28.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:28.481+0000 7f16f9600640 1 -- 172.21.15.1:0/3917837024 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16ec035030 con 0x7f16fc101430 2024-09-12T01:19:28.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:28.481+0000 7f1701800640 1 -- 172.21.15.1:0/3917837024 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16fc074b20 con 0x7f16fc101430 2024-09-12T01:19:28.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:28.482+0000 7f1701800640 1 -- 172.21.15.1:0/3917837024 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16fc107cc0 con 0x7f16fc101430 2024-09-12T01:19:28.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:28.482+0000 7f16f9600640 1 -- 172.21.15.1:0/3917837024 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f16ec042070 con 0x7f16fc101430 2024-09-12T01:19:28.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:28.483+0000 7f16f9600640 1 --2- 172.21.15.1:0/3917837024 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f16e0039060 0x7f16e003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:28.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:28.483+0000 7f16f9600640 1 -- 172.21.15.1:0/3917837024 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f16ec06cd00 con 0x7f16fc101430 2024-09-12T01:19:28.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:28.483+0000 7f16f3e00640 1 --2- 172.21.15.1:0/3917837024 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f16e0039060 0x7f16e003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:28.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:28.484+0000 7f16f3e00640 1 --2- 172.21.15.1:0/3917837024 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f16e0039060 0x7f16e003b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f16e40098c0 tx=0x7f16e4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:28.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:28.488+0000 7f16f9600640 1 -- 172.21.15.1:0/3917837024 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f16ec04a260 con 0x7f16fc101430 2024-09-12T01:19:28.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:28.733+0000 7f1701800640 1 -- 172.21.15.1:0/3917837024 --> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f16fc190000 con 0x7f16e0039060 2024-09-12T01:19:28.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:28.736+0000 7f16f9600640 1 -- 172.21.15.1:0/3917837024 <== mgr.14150 v2:172.21.15.1:6800/1928662456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f16fc190000 con 0x7f16e0039060 2024-09-12T01:19:28.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:28.739+0000 7f1701800640 1 -- 172.21.15.1:0/3917837024 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f16e0039060 msgr2=0x7f16e003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:28.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:28.739+0000 7f1701800640 1 --2- 172.21.15.1:0/3917837024 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f16e0039060 0x7f16e003b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f16e40098c0 tx=0x7f16e4006db0 comp rx=0 tx=0).stop 2024-09-12T01:19:28.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:28.739+0000 7f1701800640 1 -- 172.21.15.1:0/3917837024 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16fc101430 msgr2=0x7f16fc075eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:28.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:28.739+0000 7f1701800640 1 --2- 172.21.15.1:0/3917837024 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16fc101430 0x7f16fc075eb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f16ec004f60 tx=0x7f16ec0048f0 comp rx=0 tx=0).stop 2024-09-12T01:19:28.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:28.739+0000 7f1701800640 1 -- 172.21.15.1:0/3917837024 shutdown_connections 2024-09-12T01:19:28.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:28.739+0000 7f1701800640 1 --2- 172.21.15.1:0/3917837024 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f16e0039060 0x7f16e003b520 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:28.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:28.739+0000 7f1701800640 1 --2- 172.21.15.1:0/3917837024 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16fc101430 0x7f16fc075eb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:28.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:28.739+0000 7f1701800640 1 -- 172.21.15.1:0/3917837024 >> 172.21.15.1:0/3917837024 conn(0x7f16fc0fcbc0 msgr2=0x7f16fc0fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:19:28.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:28.740+0000 7f1701800640 1 -- 172.21.15.1:0/3917837024 shutdown_connections 2024-09-12T01:19:28.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:28.740+0000 7f1701800640 1 -- 172.21.15.1:0/3917837024 wait complete. 2024-09-12T01:19:29.978 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:29 smithi001 ceph-mon[30387]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:19:29.978 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:29 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:29.978 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:29 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:19:29.978 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:29 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:19:29.978 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:29 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:19:29.979 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:29 smithi001 ceph-mon[30387]: Updating smithi001:/etc/ceph/ceph.conf 2024-09-12T01:19:29.979 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:29 smithi001 ceph-mon[30387]: Updating smithi001:/var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:19:30.381 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi053 2024-09-12T01:19:30.381 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-12T01:19:30.381 DEBUG:teuthology.orchestra.run.smithi053:> dd of=/etc/ceph/ceph.conf 2024-09-12T01:19:30.401 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-12T01:19:30.401 DEBUG:teuthology.orchestra.run.smithi053:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T01:19:30.459 INFO:tasks.cephadm:Adding host smithi053 to orchestrator... 2024-09-12T01:19:30.459 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph orch host add smithi053 2024-09-12T01:19:30.893 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:19:31.032 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:30 smithi001 ceph-mon[30387]: Updating smithi001:/etc/ceph/ceph.client.admin.keyring 2024-09-12T01:19:31.032 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:30 smithi001 ceph-mon[30387]: Updating smithi001:/var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T01:19:31.032 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:30 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:31.032 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:30 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:31.032 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:30 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:31.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:31.422+0000 7f183d400640 1 -- 172.21.15.1:0/1884668614 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1838107660 msgr2=0x7f1838107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:31.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:31.422+0000 7f183d400640 1 --2- 172.21.15.1:0/1884668614 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1838107660 0x7f1838107a40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1828009880 tx=0x7f182802a5d0 comp rx=0 tx=0).stop 2024-09-12T01:19:31.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:31.422+0000 7f183d400640 1 -- 172.21.15.1:0/1884668614 shutdown_connections 2024-09-12T01:19:31.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:31.422+0000 7f183d400640 1 --2- 172.21.15.1:0/1884668614 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1838107660 0x7f1838107a40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:31.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:31.422+0000 7f183d400640 1 -- 172.21.15.1:0/1884668614 >> 172.21.15.1:0/1884668614 conn(0x7f18380fd290 msgr2=0x7f18380ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:19:31.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:31.422+0000 7f183d400640 1 -- 172.21.15.1:0/1884668614 shutdown_connections 2024-09-12T01:19:31.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:31.422+0000 7f183d400640 1 -- 172.21.15.1:0/1884668614 wait complete. 2024-09-12T01:19:31.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:31.423+0000 7f183d400640 1 Processor -- start 2024-09-12T01:19:31.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:31.424+0000 7f183d400640 1 -- start start 2024-09-12T01:19:31.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:31.424+0000 7f183d400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1838107660 0x7f1838199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:31.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:31.424+0000 7f183d400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f183810bc90 con 0x7f1838107660 2024-09-12T01:19:31.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:31.424+0000 7f1836800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1838107660 0x7f1838199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:31.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:31.424+0000 7f1836800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1838107660 0x7f1838199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51220/0 (socket says 172.21.15.1:51220) 2024-09-12T01:19:31.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:31.424+0000 7f1836800640 1 -- 172.21.15.1:0/357923972 learned_addr learned my addr 172.21.15.1:0/357923972 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:19:31.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:31.425+0000 7f1836800640 1 -- 172.21.15.1:0/357923972 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1838199860 con 0x7f1838107660 2024-09-12T01:19:31.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:31.425+0000 7f1836800640 1 --2- 172.21.15.1:0/357923972 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1838107660 0x7f1838199320 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1828009230 tx=0x7f1828003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:31.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:31.425+0000 7f182f400640 1 -- 172.21.15.1:0/357923972 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f182803f070 con 0x7f1838107660 2024-09-12T01:19:31.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:31.425+0000 7f182f400640 1 -- 172.21.15.1:0/357923972 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f182803bd40 con 0x7f1838107660 2024-09-12T01:19:31.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:31.425+0000 7f183d400640 1 -- 172.21.15.1:0/357923972 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1838199af0 con 0x7f1838107660 2024-09-12T01:19:31.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:31.425+0000 7f182f400640 1 -- 172.21.15.1:0/357923972 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1828035030 con 0x7f1838107660 2024-09-12T01:19:31.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:31.425+0000 7f183d400640 1 -- 172.21.15.1:0/357923972 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f183819dbf0 con 0x7f1838107660 2024-09-12T01:19:31.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:31.426+0000 7f183d400640 1 -- 172.21.15.1:0/357923972 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1838102cd0 con 0x7f1838107660 2024-09-12T01:19:31.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:31.427+0000 7f182f400640 1 -- 172.21.15.1:0/357923972 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f1828042070 con 0x7f1838107660 2024-09-12T01:19:31.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:31.427+0000 7f182f400640 1 --2- 172.21.15.1:0/357923972 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f181c039060 0x7f181c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:31.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:31.427+0000 7f182f400640 1 -- 172.21.15.1:0/357923972 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f182806cc60 con 0x7f1838107660 2024-09-12T01:19:31.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:31.428+0000 7f1835e00640 1 --2- 172.21.15.1:0/357923972 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f181c039060 0x7f181c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:31.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:31.428+0000 7f1835e00640 1 --2- 172.21.15.1:0/357923972 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f181c039060 0x7f181c03b520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f18240098c0 tx=0x7f1824006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:31.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:31.432+0000 7f182f400640 1 -- 172.21.15.1:0/357923972 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1828030d60 con 0x7f1838107660 2024-09-12T01:19:31.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:31.623+0000 7f183d400640 1 -- 172.21.15.1:0/357923972 --> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi053", "target": ["mon-mgr", ""]}) -- 0x7f183819a680 con 0x7f181c039060 2024-09-12T01:19:32.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:32 smithi001 ceph-mon[30387]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi053", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:19:33.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:33 smithi001 ceph-mon[30387]: Deploying cephadm binary to smithi053 2024-09-12T01:19:35.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:35.592+0000 7f182f400640 1 -- 172.21.15.1:0/357923972 <== mgr.14150 v2:172.21.15.1:6800/1928662456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f183819a680 con 0x7f181c039060 2024-09-12T01:19:35.592 INFO:teuthology.orchestra.run.smithi001.stdout:Added host 'smithi053' with addr '172.21.15.53' 2024-09-12T01:19:35.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:35.595+0000 7f183d400640 1 -- 172.21.15.1:0/357923972 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f181c039060 msgr2=0x7f181c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:35.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:35.595+0000 7f183d400640 1 --2- 172.21.15.1:0/357923972 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f181c039060 0x7f181c03b520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f18240098c0 tx=0x7f1824006db0 comp rx=0 tx=0).stop 2024-09-12T01:19:35.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:35.595+0000 7f183d400640 1 -- 172.21.15.1:0/357923972 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1838107660 msgr2=0x7f1838199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:35.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:35.595+0000 7f183d400640 1 --2- 172.21.15.1:0/357923972 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1838107660 0x7f1838199320 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1828009230 tx=0x7f1828003d80 comp rx=0 tx=0).stop 2024-09-12T01:19:35.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:35.595+0000 7f183d400640 1 -- 172.21.15.1:0/357923972 shutdown_connections 2024-09-12T01:19:35.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:35.595+0000 7f183d400640 1 --2- 172.21.15.1:0/357923972 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f181c039060 0x7f181c03b520 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:35.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:35.595+0000 7f183d400640 1 --2- 172.21.15.1:0/357923972 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1838107660 0x7f1838199320 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:35.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:35.595+0000 7f183d400640 1 -- 172.21.15.1:0/357923972 >> 172.21.15.1:0/357923972 conn(0x7f18380fd290 msgr2=0x7f1838075e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:19:35.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:35.595+0000 7f183d400640 1 -- 172.21.15.1:0/357923972 shutdown_connections 2024-09-12T01:19:35.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:35.595+0000 7f183d400640 1 -- 172.21.15.1:0/357923972 wait complete. 2024-09-12T01:19:35.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:35 smithi001 ceph-mon[30387]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:19:35.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:35 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:35.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:35 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:19:36.339 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-12T01:19:36.708 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:19:37.123 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:36 smithi001 ceph-mon[30387]: Added host smithi053 2024-09-12T01:19:37.123 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:36 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:37.123 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:36 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:37.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:37.239+0000 7f27a0e00640 1 -- 172.21.15.1:0/4034266427 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f279c107640 msgr2=0x7f279c107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:37.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:37.239+0000 7f27a0e00640 1 --2- 172.21.15.1:0/4034266427 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f279c107640 0x7f279c107a20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f278c009880 tx=0x7f278c02a5d0 comp rx=0 tx=0).stop 2024-09-12T01:19:37.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:37.239+0000 7f27a0e00640 1 -- 172.21.15.1:0/4034266427 shutdown_connections 2024-09-12T01:19:37.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:37.239+0000 7f27a0e00640 1 --2- 172.21.15.1:0/4034266427 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f279c107640 0x7f279c107a20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:37.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:37.239+0000 7f27a0e00640 1 -- 172.21.15.1:0/4034266427 >> 172.21.15.1:0/4034266427 conn(0x7f279c0fd230 msgr2=0x7f279c0ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:19:37.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:37.239+0000 7f27a0e00640 1 -- 172.21.15.1:0/4034266427 shutdown_connections 2024-09-12T01:19:37.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:37.239+0000 7f27a0e00640 1 -- 172.21.15.1:0/4034266427 wait complete. 2024-09-12T01:19:37.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:37.240+0000 7f27a0e00640 1 Processor -- start 2024-09-12T01:19:37.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:37.240+0000 7f27a0e00640 1 -- start start 2024-09-12T01:19:37.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:37.240+0000 7f27a0e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f279c107640 0x7f279c199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:37.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:37.240+0000 7f27a0e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f279c10bc70 con 0x7f279c107640 2024-09-12T01:19:37.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:37.241+0000 7f2799e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f279c107640 0x7f279c199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:37.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:37.241+0000 7f2799e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f279c107640 0x7f279c199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51240/0 (socket says 172.21.15.1:51240) 2024-09-12T01:19:37.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:37.241+0000 7f2799e00640 1 -- 172.21.15.1:0/2372332147 learned_addr learned my addr 172.21.15.1:0/2372332147 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:19:37.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:37.241+0000 7f2799e00640 1 -- 172.21.15.1:0/2372332147 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f279c199870 con 0x7f279c107640 2024-09-12T01:19:37.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:37.241+0000 7f2799e00640 1 --2- 172.21.15.1:0/2372332147 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f279c107640 0x7f279c199330 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f278c009230 tx=0x7f278c003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:37.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:37.242+0000 7f2792a00640 1 -- 172.21.15.1:0/2372332147 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f278c03f070 con 0x7f279c107640 2024-09-12T01:19:37.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:37.242+0000 7f2792a00640 1 -- 172.21.15.1:0/2372332147 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f278c03bd40 con 0x7f279c107640 2024-09-12T01:19:37.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:37.242+0000 7f2792a00640 1 -- 172.21.15.1:0/2372332147 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f278c035030 con 0x7f279c107640 2024-09-12T01:19:37.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:37.242+0000 7f27a0e00640 1 -- 172.21.15.1:0/2372332147 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f279c199b00 con 0x7f279c107640 2024-09-12T01:19:37.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:37.242+0000 7f27a0e00640 1 -- 172.21.15.1:0/2372332147 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f279c19dc00 con 0x7f279c107640 2024-09-12T01:19:37.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:37.243+0000 7f2792a00640 1 -- 172.21.15.1:0/2372332147 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f278c042070 con 0x7f279c107640 2024-09-12T01:19:37.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:37.243+0000 7f27a0e00640 1 -- 172.21.15.1:0/2372332147 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f279c102cb0 con 0x7f279c107640 2024-09-12T01:19:37.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:37.243+0000 7f2792a00640 1 --2- 172.21.15.1:0/2372332147 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f2780039060 0x7f278003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:37.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:37.243+0000 7f2792a00640 1 -- 172.21.15.1:0/2372332147 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f278c06cce0 con 0x7f279c107640 2024-09-12T01:19:37.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:37.244+0000 7f2799400640 1 --2- 172.21.15.1:0/2372332147 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f2780039060 0x7f278003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:37.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:37.244+0000 7f2799400640 1 --2- 172.21.15.1:0/2372332147 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f2780039060 0x7f278003b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f27880098c0 tx=0x7f2788006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:37.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:37.248+0000 7f2792a00640 1 -- 172.21.15.1:0/2372332147 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f278c030d60 con 0x7f279c107640 2024-09-12T01:19:37.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:37.438+0000 7f27a0e00640 1 -- 172.21.15.1:0/2372332147 --> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f279c19a690 con 0x7f2780039060 2024-09-12T01:19:37.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:37.439+0000 7f2792a00640 1 -- 172.21.15.1:0/2372332147 <== mgr.14150 v2:172.21.15.1:6800/1928662456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+158 (secure 0 0 0) 0x7f279c19a690 con 0x7f2780039060 2024-09-12T01:19:37.438 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:19:37.439 INFO:teuthology.orchestra.run.smithi001.stdout:[{"addr": "172.21.15.1", "hostname": "smithi001", "labels": [], "status": ""}, {"addr": "172.21.15.53", "hostname": "smithi053", "labels": [], "status": ""}] 2024-09-12T01:19:37.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:37.441+0000 7f27a0e00640 1 -- 172.21.15.1:0/2372332147 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f2780039060 msgr2=0x7f278003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:37.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:37.441+0000 7f27a0e00640 1 --2- 172.21.15.1:0/2372332147 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f2780039060 0x7f278003b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f27880098c0 tx=0x7f2788006db0 comp rx=0 tx=0).stop 2024-09-12T01:19:37.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:37.441+0000 7f27a0e00640 1 -- 172.21.15.1:0/2372332147 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f279c107640 msgr2=0x7f279c199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:37.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:37.441+0000 7f27a0e00640 1 --2- 172.21.15.1:0/2372332147 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f279c107640 0x7f279c199330 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f278c009230 tx=0x7f278c003d80 comp rx=0 tx=0).stop 2024-09-12T01:19:37.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:37.441+0000 7f27a0e00640 1 -- 172.21.15.1:0/2372332147 shutdown_connections 2024-09-12T01:19:37.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:37.441+0000 7f27a0e00640 1 --2- 172.21.15.1:0/2372332147 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f2780039060 0x7f278003b520 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:37.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:37.441+0000 7f27a0e00640 1 --2- 172.21.15.1:0/2372332147 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f279c107640 0x7f279c199330 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:37.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:37.442+0000 7f27a0e00640 1 -- 172.21.15.1:0/2372332147 >> 172.21.15.1:0/2372332147 conn(0x7f279c0fd230 msgr2=0x7f279c075e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:19:37.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:37.442+0000 7f27a0e00640 1 -- 172.21.15.1:0/2372332147 shutdown_connections 2024-09-12T01:19:37.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:37.442+0000 7f27a0e00640 1 -- 172.21.15.1:0/2372332147 wait complete. 2024-09-12T01:19:37.996 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:37 smithi001 ceph-mon[30387]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:19:37.997 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:37 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:37.997 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:37 smithi001 ceph-mon[30387]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:19:38.070 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi104 2024-09-12T01:19:38.071 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-12T01:19:38.071 DEBUG:teuthology.orchestra.run.smithi104:> dd of=/etc/ceph/ceph.conf 2024-09-12T01:19:38.090 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-12T01:19:38.091 DEBUG:teuthology.orchestra.run.smithi104:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T01:19:38.150 INFO:tasks.cephadm:Adding host smithi104 to orchestrator... 2024-09-12T01:19:38.150 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph orch host add smithi104 2024-09-12T01:19:38.513 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:19:39.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:39.068+0000 7f58f2600640 1 -- 172.21.15.1:0/2033704641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58ec107640 msgr2=0x7f58ec107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:39.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:39.068+0000 7f58f2600640 1 --2- 172.21.15.1:0/2033704641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58ec107640 0x7f58ec107a20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f58dc009880 tx=0x7f58dc02a5d0 comp rx=0 tx=0).stop 2024-09-12T01:19:39.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:39.068+0000 7f58f2600640 1 -- 172.21.15.1:0/2033704641 shutdown_connections 2024-09-12T01:19:39.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:39.068+0000 7f58f2600640 1 --2- 172.21.15.1:0/2033704641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58ec107640 0x7f58ec107a20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:39.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:39.068+0000 7f58f2600640 1 -- 172.21.15.1:0/2033704641 >> 172.21.15.1:0/2033704641 conn(0x7f58ec0fd230 msgr2=0x7f58ec0ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:19:39.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:39.068+0000 7f58f2600640 1 -- 172.21.15.1:0/2033704641 shutdown_connections 2024-09-12T01:19:39.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:39.068+0000 7f58f2600640 1 -- 172.21.15.1:0/2033704641 wait complete. 2024-09-12T01:19:39.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:39.069+0000 7f58f2600640 1 Processor -- start 2024-09-12T01:19:39.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:39.070+0000 7f58f2600640 1 -- start start 2024-09-12T01:19:39.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:39.070+0000 7f58f2600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58ec107640 0x7f58ec199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:39.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:39.070+0000 7f58f2600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f58ec10bc70 con 0x7f58ec107640 2024-09-12T01:19:39.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:39.070+0000 7f58eb400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58ec107640 0x7f58ec199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:39.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:39.070+0000 7f58eb400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58ec107640 0x7f58ec199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37432/0 (socket says 172.21.15.1:37432) 2024-09-12T01:19:39.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:39.070+0000 7f58eb400640 1 -- 172.21.15.1:0/4249241895 learned_addr learned my addr 172.21.15.1:0/4249241895 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:19:39.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:39.071+0000 7f58eb400640 1 -- 172.21.15.1:0/4249241895 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58ec199870 con 0x7f58ec107640 2024-09-12T01:19:39.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:39.071+0000 7f58eb400640 1 --2- 172.21.15.1:0/4249241895 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58ec107640 0x7f58ec199330 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f58dc009230 tx=0x7f58dc003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:39.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:39.071+0000 7f58e3e00640 1 -- 172.21.15.1:0/4249241895 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58dc03f070 con 0x7f58ec107640 2024-09-12T01:19:39.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:39.071+0000 7f58e3e00640 1 -- 172.21.15.1:0/4249241895 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58dc03bc80 con 0x7f58ec107640 2024-09-12T01:19:39.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:39.071+0000 7f58f2600640 1 -- 172.21.15.1:0/4249241895 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58ec199b00 con 0x7f58ec107640 2024-09-12T01:19:39.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:39.071+0000 7f58e3e00640 1 -- 172.21.15.1:0/4249241895 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58dc035030 con 0x7f58ec107640 2024-09-12T01:19:39.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:39.071+0000 7f58f2600640 1 -- 172.21.15.1:0/4249241895 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58ec19dc00 con 0x7f58ec107640 2024-09-12T01:19:39.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:39.072+0000 7f58e3e00640 1 -- 172.21.15.1:0/4249241895 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f58dc042070 con 0x7f58ec107640 2024-09-12T01:19:39.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:39.072+0000 7f58f2600640 1 -- 172.21.15.1:0/4249241895 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58ec102cb0 con 0x7f58ec107640 2024-09-12T01:19:39.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:39.073+0000 7f58e3e00640 1 --2- 172.21.15.1:0/4249241895 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f58d0039060 0x7f58d003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:39.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:39.073+0000 7f58e3e00640 1 -- 172.21.15.1:0/4249241895 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f58dc06ca40 con 0x7f58ec107640 2024-09-12T01:19:39.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:39.073+0000 7f58eaa00640 1 --2- 172.21.15.1:0/4249241895 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f58d0039060 0x7f58d003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:39.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:39.074+0000 7f58eaa00640 1 --2- 172.21.15.1:0/4249241895 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f58d0039060 0x7f58d003b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f58d80098c0 tx=0x7f58d8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:39.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:39.078+0000 7f58e3e00640 1 -- 172.21.15.1:0/4249241895 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f58dc031e20 con 0x7f58ec107640 2024-09-12T01:19:39.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:39.266+0000 7f58f2600640 1 -- 172.21.15.1:0/4249241895 --> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi104", "target": ["mon-mgr", ""]}) -- 0x7f58ec19a690 con 0x7f58d0039060 2024-09-12T01:19:40.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:39 smithi001 ceph-mon[30387]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:19:40.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:39 smithi001 ceph-mon[30387]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi104", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:19:40.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:39 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:40.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:39 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:40.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:39 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:40.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:39 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:40.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:39 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi053", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:19:40.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:39 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:19:40.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:39 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:19:41.329 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:40 smithi001 ceph-mon[30387]: Updating smithi053:/etc/ceph/ceph.conf 2024-09-12T01:19:41.330 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:40 smithi001 ceph-mon[30387]: Updating smithi053:/var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:19:41.330 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:40 smithi001 ceph-mon[30387]: Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2024-09-12T01:19:41.330 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:40 smithi001 ceph-mon[30387]: Updating smithi053:/var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T01:19:41.330 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:40 smithi001 ceph-mon[30387]: Deploying cephadm binary to smithi104 2024-09-12T01:19:41.330 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:40 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:41.330 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:40 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:41.330 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:40 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:42.220 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:41 smithi001 ceph-mon[30387]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:19:43.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:43.180+0000 7f58e3e00640 1 -- 172.21.15.1:0/4249241895 <== mgr.14150 v2:172.21.15.1:6800/1928662456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f58ec19a690 con 0x7f58d0039060 2024-09-12T01:19:43.181 INFO:teuthology.orchestra.run.smithi001.stdout:Added host 'smithi104' with addr '172.21.15.104' 2024-09-12T01:19:43.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:43.182+0000 7f58f2600640 1 -- 172.21.15.1:0/4249241895 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f58d0039060 msgr2=0x7f58d003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:43.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:43.182+0000 7f58f2600640 1 --2- 172.21.15.1:0/4249241895 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f58d0039060 0x7f58d003b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f58d80098c0 tx=0x7f58d8006db0 comp rx=0 tx=0).stop 2024-09-12T01:19:43.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:43.182+0000 7f58f2600640 1 -- 172.21.15.1:0/4249241895 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58ec107640 msgr2=0x7f58ec199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:43.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:43.182+0000 7f58f2600640 1 --2- 172.21.15.1:0/4249241895 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58ec107640 0x7f58ec199330 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f58dc009230 tx=0x7f58dc003d40 comp rx=0 tx=0).stop 2024-09-12T01:19:43.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:43.183+0000 7f58f2600640 1 -- 172.21.15.1:0/4249241895 shutdown_connections 2024-09-12T01:19:43.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:43.183+0000 7f58f2600640 1 --2- 172.21.15.1:0/4249241895 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f58d0039060 0x7f58d003b520 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:43.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:43.183+0000 7f58f2600640 1 --2- 172.21.15.1:0/4249241895 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58ec107640 0x7f58ec199330 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:43.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:43.183+0000 7f58f2600640 1 -- 172.21.15.1:0/4249241895 >> 172.21.15.1:0/4249241895 conn(0x7f58ec0fd230 msgr2=0x7f58ec075e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:19:43.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:43.183+0000 7f58f2600640 1 -- 172.21.15.1:0/4249241895 shutdown_connections 2024-09-12T01:19:43.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:43.183+0000 7f58f2600640 1 -- 172.21.15.1:0/4249241895 wait complete. 2024-09-12T01:19:43.929 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-12T01:19:44.302 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:19:44.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:44 smithi001 ceph-mon[30387]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:19:44.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:44 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:44.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:44 smithi001 ceph-mon[30387]: Added host smithi104 2024-09-12T01:19:44.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:44 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:19:44.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:44 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:44.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:44 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:44.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:44.874+0000 7f372ca00640 1 -- 172.21.15.1:0/3590312529 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3728109670 msgr2=0x7f3728109a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:44.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:44.875+0000 7f372ca00640 1 --2- 172.21.15.1:0/3590312529 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3728109670 0x7f3728109a50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3718009880 tx=0x7f371802a5d0 comp rx=0 tx=0).stop 2024-09-12T01:19:44.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:44.875+0000 7f372ca00640 1 -- 172.21.15.1:0/3590312529 shutdown_connections 2024-09-12T01:19:44.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:44.875+0000 7f372ca00640 1 --2- 172.21.15.1:0/3590312529 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3728109670 0x7f3728109a50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:44.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:44.875+0000 7f372ca00640 1 -- 172.21.15.1:0/3590312529 >> 172.21.15.1:0/3590312529 conn(0x7f37280ff260 msgr2=0x7f3728101680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:19:44.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:44.875+0000 7f372ca00640 1 -- 172.21.15.1:0/3590312529 shutdown_connections 2024-09-12T01:19:44.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:44.875+0000 7f372ca00640 1 -- 172.21.15.1:0/3590312529 wait complete. 2024-09-12T01:19:44.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:44.876+0000 7f372ca00640 1 Processor -- start 2024-09-12T01:19:44.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:44.876+0000 7f372ca00640 1 -- start start 2024-09-12T01:19:44.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:44.876+0000 7f372ca00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3728109670 0x7f3728199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:44.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:44.876+0000 7f372ca00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f372810dca0 con 0x7f3728109670 2024-09-12T01:19:44.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:44.876+0000 7f3727400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3728109670 0x7f3728199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:44.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:44.876+0000 7f3727400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3728109670 0x7f3728199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37446/0 (socket says 172.21.15.1:37446) 2024-09-12T01:19:44.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:44.876+0000 7f3727400640 1 -- 172.21.15.1:0/4223396665 learned_addr learned my addr 172.21.15.1:0/4223396665 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:19:44.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:44.877+0000 7f3727400640 1 -- 172.21.15.1:0/4223396665 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3728199640 con 0x7f3728109670 2024-09-12T01:19:44.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:44.877+0000 7f3727400640 1 --2- 172.21.15.1:0/4223396665 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3728109670 0x7f3728199100 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3718009230 tx=0x7f3718003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:44.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:44.877+0000 7f371fe00640 1 -- 172.21.15.1:0/4223396665 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f371803f070 con 0x7f3728109670 2024-09-12T01:19:44.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:44.877+0000 7f372ca00640 1 -- 172.21.15.1:0/4223396665 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37281998d0 con 0x7f3728109670 2024-09-12T01:19:44.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:44.877+0000 7f371fe00640 1 -- 172.21.15.1:0/4223396665 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f371803bd40 con 0x7f3728109670 2024-09-12T01:19:44.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:44.877+0000 7f371fe00640 1 -- 172.21.15.1:0/4223396665 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3718035030 con 0x7f3728109670 2024-09-12T01:19:44.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:44.877+0000 7f372ca00640 1 -- 172.21.15.1:0/4223396665 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f372819d9d0 con 0x7f3728109670 2024-09-12T01:19:44.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:44.878+0000 7f371fe00640 1 -- 172.21.15.1:0/4223396665 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f3718042070 con 0x7f3728109670 2024-09-12T01:19:44.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:44.878+0000 7f372ca00640 1 -- 172.21.15.1:0/4223396665 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3728104ce0 con 0x7f3728109670 2024-09-12T01:19:44.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:44.879+0000 7f371fe00640 1 --2- 172.21.15.1:0/4223396665 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f370c0390c0 0x7f370c03b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:44.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:44.879+0000 7f371fe00640 1 -- 172.21.15.1:0/4223396665 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f371806cc90 con 0x7f3728109670 2024-09-12T01:19:44.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:44.879+0000 7f3726a00640 1 --2- 172.21.15.1:0/4223396665 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f370c0390c0 0x7f370c03b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:44.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:44.880+0000 7f3726a00640 1 --2- 172.21.15.1:0/4223396665 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f370c0390c0 0x7f370c03b580 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f37140098c0 tx=0x7f3714006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:44.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:44.884+0000 7f371fe00640 1 -- 172.21.15.1:0/4223396665 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3718036df0 con 0x7f3728109670 2024-09-12T01:19:45.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:45.072+0000 7f372ca00640 1 -- 172.21.15.1:0/4223396665 --> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f372819a460 con 0x7f370c0390c0 2024-09-12T01:19:45.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:45.073+0000 7f371fe00640 1 -- 172.21.15.1:0/4223396665 <== mgr.14150 v2:172.21.15.1:6800/1928662456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+238 (secure 0 0 0) 0x7f372819a460 con 0x7f370c0390c0 2024-09-12T01:19:45.073 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:19:45.073 INFO:teuthology.orchestra.run.smithi001.stdout:[{"addr": "172.21.15.1", "hostname": "smithi001", "labels": [], "status": ""}, {"addr": "172.21.15.53", "hostname": "smithi053", "labels": [], "status": ""}, {"addr": "172.21.15.104", "hostname": "smithi104", "labels": [], "status": ""}] 2024-09-12T01:19:45.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:45.075+0000 7f372ca00640 1 -- 172.21.15.1:0/4223396665 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f370c0390c0 msgr2=0x7f370c03b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:45.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:45.075+0000 7f372ca00640 1 --2- 172.21.15.1:0/4223396665 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f370c0390c0 0x7f370c03b580 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f37140098c0 tx=0x7f3714006db0 comp rx=0 tx=0).stop 2024-09-12T01:19:45.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:45.075+0000 7f372ca00640 1 -- 172.21.15.1:0/4223396665 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3728109670 msgr2=0x7f3728199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:45.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:45.075+0000 7f372ca00640 1 --2- 172.21.15.1:0/4223396665 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3728109670 0x7f3728199100 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3718009230 tx=0x7f3718003d80 comp rx=0 tx=0).stop 2024-09-12T01:19:45.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:45.076+0000 7f372ca00640 1 -- 172.21.15.1:0/4223396665 shutdown_connections 2024-09-12T01:19:45.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:45.076+0000 7f372ca00640 1 --2- 172.21.15.1:0/4223396665 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f370c0390c0 0x7f370c03b580 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:45.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:45.076+0000 7f372ca00640 1 --2- 172.21.15.1:0/4223396665 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3728109670 0x7f3728199100 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:45.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:45.076+0000 7f372ca00640 1 -- 172.21.15.1:0/4223396665 >> 172.21.15.1:0/4223396665 conn(0x7f37280ff260 msgr2=0x7f3728100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:19:45.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:45.076+0000 7f372ca00640 1 -- 172.21.15.1:0/4223396665 shutdown_connections 2024-09-12T01:19:45.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:45.076+0000 7f372ca00640 1 -- 172.21.15.1:0/4223396665 wait complete. 2024-09-12T01:19:45.730 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:45 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:45.766 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-12T01:19:45.766 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-12T01:19:46.131 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:19:46.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:46.691+0000 7fd3ff000640 1 -- 172.21.15.1:0/259930605 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3f8076720 msgr2=0x7fd3f8076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:46.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:46.691+0000 7fd3ff000640 1 --2- 172.21.15.1:0/259930605 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3f8076720 0x7fd3f8076b00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec009880 tx=0x7fd3ec02a5d0 comp rx=0 tx=0).stop 2024-09-12T01:19:46.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:46.691+0000 7fd3ff000640 1 -- 172.21.15.1:0/259930605 shutdown_connections 2024-09-12T01:19:46.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:46.691+0000 7fd3ff000640 1 --2- 172.21.15.1:0/259930605 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3f8076720 0x7fd3f8076b00 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:46.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:46.691+0000 7fd3ff000640 1 -- 172.21.15.1:0/259930605 >> 172.21.15.1:0/259930605 conn(0x7fd3f80ff2c0 msgr2=0x7fd3f81016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:19:46.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:46.691+0000 7fd3ff000640 1 -- 172.21.15.1:0/259930605 shutdown_connections 2024-09-12T01:19:46.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:46.691+0000 7fd3ff000640 1 -- 172.21.15.1:0/259930605 wait complete. 2024-09-12T01:19:46.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:46.692+0000 7fd3ff000640 1 Processor -- start 2024-09-12T01:19:46.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:46.692+0000 7fd3ff000640 1 -- start start 2024-09-12T01:19:46.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:46.692+0000 7fd3ff000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3f8076720 0x7fd3f819d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:46.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:46.692+0000 7fd3ff000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f81121d0 con 0x7fd3f8076720 2024-09-12T01:19:46.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:46.693+0000 7fd3fdc00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3f8076720 0x7fd3f819d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:46.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:46.693+0000 7fd3fdc00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3f8076720 0x7fd3f819d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37454/0 (socket says 172.21.15.1:37454) 2024-09-12T01:19:46.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:46.693+0000 7fd3fdc00640 1 -- 172.21.15.1:0/2207965967 learned_addr learned my addr 172.21.15.1:0/2207965967 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:19:46.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:46.693+0000 7fd3fdc00640 1 -- 172.21.15.1:0/2207965967 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3f819dbf0 con 0x7fd3f8076720 2024-09-12T01:19:46.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:46.693+0000 7fd3fdc00640 1 --2- 172.21.15.1:0/2207965967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3f8076720 0x7fd3f819d6b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec009230 tx=0x7fd3ec003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:46.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:46.693+0000 7fd3f6000640 1 -- 172.21.15.1:0/2207965967 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3ec03f070 con 0x7fd3f8076720 2024-09-12T01:19:46.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:46.693+0000 7fd3f6000640 1 -- 172.21.15.1:0/2207965967 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3ec03bd40 con 0x7fd3f8076720 2024-09-12T01:19:46.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:46.693+0000 7fd3f6000640 1 -- 172.21.15.1:0/2207965967 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3ec035030 con 0x7fd3f8076720 2024-09-12T01:19:46.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:46.693+0000 7fd3ff000640 1 -- 172.21.15.1:0/2207965967 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3f819de80 con 0x7fd3f8076720 2024-09-12T01:19:46.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:46.694+0000 7fd3ff000640 1 -- 172.21.15.1:0/2207965967 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3f819e2e0 con 0x7fd3f8076720 2024-09-12T01:19:46.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:46.695+0000 7fd3f6000640 1 -- 172.21.15.1:0/2207965967 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fd3ec042070 con 0x7fd3f8076720 2024-09-12T01:19:46.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:46.695+0000 7fd3ff000640 1 -- 172.21.15.1:0/2207965967 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3f8077e60 con 0x7fd3f8076720 2024-09-12T01:19:46.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:46.695+0000 7fd3f6000640 1 --2- 172.21.15.1:0/2207965967 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fd3e00390c0 0x7fd3e003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:46.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:46.695+0000 7fd3f6000640 1 -- 172.21.15.1:0/2207965967 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd3ec06ccd0 con 0x7fd3f8076720 2024-09-12T01:19:46.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:46.695+0000 7fd3fd200640 1 --2- 172.21.15.1:0/2207965967 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fd3e00390c0 0x7fd3e003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:46.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:46.696+0000 7fd3fd200640 1 --2- 172.21.15.1:0/2207965967 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fd3e00390c0 0x7fd3e003b580 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd3e40098c0 tx=0x7fd3e4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:46.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:46.701+0000 7fd3f6000640 1 -- 172.21.15.1:0/2207965967 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd3ec0412c0 con 0x7fd3f8076720 2024-09-12T01:19:46.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:46.889+0000 7fd3ff000640 1 -- 172.21.15.1:0/2207965967 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fd3f819ea10 con 0x7fd3f8076720 2024-09-12T01:19:46.889 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:46 smithi001 ceph-mon[30387]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:19:46.889 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:46 smithi001 ceph-mon[30387]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:19:47.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:47.530+0000 7fd3f6000640 1 -- 172.21.15.1:0/2207965967 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fd3ec03b860 con 0x7fd3f8076720 2024-09-12T01:19:47.530 INFO:teuthology.orchestra.run.smithi001.stderr:adjusted tunables profile to default 2024-09-12T01:19:47.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:47.533+0000 7fd3ff000640 1 -- 172.21.15.1:0/2207965967 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fd3e00390c0 msgr2=0x7fd3e003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:47.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:47.533+0000 7fd3ff000640 1 --2- 172.21.15.1:0/2207965967 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fd3e00390c0 0x7fd3e003b580 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd3e40098c0 tx=0x7fd3e4006db0 comp rx=0 tx=0).stop 2024-09-12T01:19:47.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:47.533+0000 7fd3ff000640 1 -- 172.21.15.1:0/2207965967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3f8076720 msgr2=0x7fd3f819d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:47.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:47.533+0000 7fd3ff000640 1 --2- 172.21.15.1:0/2207965967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3f8076720 0x7fd3f819d6b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec009230 tx=0x7fd3ec003d80 comp rx=0 tx=0).stop 2024-09-12T01:19:47.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:47.533+0000 7fd3ff000640 1 -- 172.21.15.1:0/2207965967 shutdown_connections 2024-09-12T01:19:47.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:47.533+0000 7fd3ff000640 1 --2- 172.21.15.1:0/2207965967 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fd3e00390c0 0x7fd3e003b580 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:47.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:47.533+0000 7fd3ff000640 1 --2- 172.21.15.1:0/2207965967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3f8076720 0x7fd3f819d6b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:47.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:47.533+0000 7fd3ff000640 1 -- 172.21.15.1:0/2207965967 >> 172.21.15.1:0/2207965967 conn(0x7fd3f80ff2c0 msgr2=0x7fd3f810dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:19:47.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:47.533+0000 7fd3ff000640 1 -- 172.21.15.1:0/2207965967 shutdown_connections 2024-09-12T01:19:47.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:19:47.533+0000 7fd3ff000640 1 -- 172.21.15.1:0/2207965967 wait complete. 2024-09-12T01:19:47.913 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:47 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/2207965967' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-12T01:19:47.913 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:47 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:47.913 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:47 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:47.913 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:47 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:47.913 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:47 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:47.913 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:47 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi104", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:19:47.914 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:47 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:19:47.914 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:47 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:19:48.294 INFO:tasks.cephadm:Adding mon.a on smithi001 2024-09-12T01:19:48.294 INFO:tasks.cephadm:Adding mon.b on smithi053 2024-09-12T01:19:48.294 INFO:tasks.cephadm:Adding mon.c on smithi104 2024-09-12T01:19:48.294 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi001:172.21.15.1=a;smithi053:172.21.15.53=b;smithi104:172.21.15.104=c' 2024-09-12T01:19:48.527 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:19:48.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:48 smithi001 ceph-mon[30387]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:19:48.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:48 smithi001 ceph-mon[30387]: Updating smithi104:/etc/ceph/ceph.conf 2024-09-12T01:19:48.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:48 smithi001 ceph-mon[30387]: Updating smithi104:/var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:19:48.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:48 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/2207965967' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-12T01:19:48.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:48 smithi001 ceph-mon[30387]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T01:19:48.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:48 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:48.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:48 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:48.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:48 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:48.929 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:48.929+0000 7f1b36e31640 1 -- 172.21.15.104:0/2230353613 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b301075e0 msgr2=0x7f1b301079c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:48.929 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:48.929+0000 7f1b36e31640 1 --2- 172.21.15.104:0/2230353613 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b301075e0 0x7f1b301079c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c009880 tx=0x7f1b2c02a5d0 comp rx=0 tx=0).stop 2024-09-12T01:19:48.929 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:48.929+0000 7f1b36e31640 1 -- 172.21.15.104:0/2230353613 shutdown_connections 2024-09-12T01:19:48.929 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:48.929+0000 7f1b36e31640 1 --2- 172.21.15.104:0/2230353613 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b301075e0 0x7f1b301079c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:48.929 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:48.929+0000 7f1b36e31640 1 -- 172.21.15.104:0/2230353613 >> 172.21.15.104:0/2230353613 conn(0x7f1b300fb8c0 msgr2=0x7f1b300fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:19:48.929 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:48.929+0000 7f1b36e31640 1 -- 172.21.15.104:0/2230353613 shutdown_connections 2024-09-12T01:19:48.929 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:48.929+0000 7f1b36e31640 1 -- 172.21.15.104:0/2230353613 wait complete. 2024-09-12T01:19:48.930 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:48.930+0000 7f1b36e31640 1 Processor -- start 2024-09-12T01:19:48.930 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:48.930+0000 7f1b36e31640 1 -- start start 2024-09-12T01:19:48.930 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:48.930+0000 7f1b36e31640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b301075e0 0x7f1b30197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:48.930 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:48.930+0000 7f1b36e31640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3010bb90 con 0x7f1b301075e0 2024-09-12T01:19:48.930 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:48.930+0000 7f1b34bcd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b301075e0 0x7f1b30197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:48.930 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:48.930+0000 7f1b34bcd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b301075e0 0x7f1b30197090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.104:34526/0 (socket says 172.21.15.104:34526) 2024-09-12T01:19:48.930 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:48.930+0000 7f1b34bcd640 1 -- 172.21.15.104:0/2688560793 learned_addr learned my addr 172.21.15.104:0/2688560793 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-09-12T01:19:48.931 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:48.931+0000 7f1b34bcd640 1 -- 172.21.15.104:0/2688560793 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b301975d0 con 0x7f1b301075e0 2024-09-12T01:19:48.931 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:48.931+0000 7f1b34bcd640 1 --2- 172.21.15.104:0/2688560793 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b301075e0 0x7f1b30197090 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c009230 tx=0x7f1b2c003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:48.932 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:48.931+0000 7f1b25ffb640 1 -- 172.21.15.104:0/2688560793 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b2c03f070 con 0x7f1b301075e0 2024-09-12T01:19:48.932 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:48.931+0000 7f1b36e31640 1 -- 172.21.15.104:0/2688560793 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b301978c0 con 0x7f1b301075e0 2024-09-12T01:19:48.932 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:48.931+0000 7f1b25ffb640 1 -- 172.21.15.104:0/2688560793 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b2c03bd40 con 0x7f1b301075e0 2024-09-12T01:19:48.932 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:48.931+0000 7f1b25ffb640 1 -- 172.21.15.104:0/2688560793 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b2c035030 con 0x7f1b301075e0 2024-09-12T01:19:48.932 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:48.931+0000 7f1b36e31640 1 -- 172.21.15.104:0/2688560793 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b3019b960 con 0x7f1b301075e0 2024-09-12T01:19:48.932 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:48.932+0000 7f1b25ffb640 1 -- 172.21.15.104:0/2688560793 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f1b2c042070 con 0x7f1b301075e0 2024-09-12T01:19:48.932 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:48.932+0000 7f1b36e31640 1 -- 172.21.15.104:0/2688560793 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b30102d90 con 0x7f1b301075e0 2024-09-12T01:19:48.932 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:48.932+0000 7f1b25ffb640 1 --2- 172.21.15.104:0/2688560793 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1b18039060 0x7f1b1803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:48.932 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:48.932+0000 7f1b25ffb640 1 -- 172.21.15.104:0/2688560793 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1b2c06cd90 con 0x7f1b301075e0 2024-09-12T01:19:48.932 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:48.933+0000 7f1b27fff640 1 --2- 172.21.15.104:0/2688560793 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1b18039060 0x7f1b1803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:48.933 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:48.933+0000 7f1b27fff640 1 --2- 172.21.15.104:0/2688560793 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1b18039060 0x7f1b1803b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1b200098c0 tx=0x7f1b20006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:48.938 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:48.939+0000 7f1b25ffb640 1 -- 172.21.15.104:0/2688560793 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1b2c03a890 con 0x7f1b301075e0 2024-09-12T01:19:49.122 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:49.123+0000 7f1b36e31640 1 -- 172.21.15.104:0/2688560793 --> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi001:172.21.15.1=a;smithi053:172.21.15.53=b;smithi104:172.21.15.104=c", "target": ["mon-mgr", ""]}) -- 0x7f1b301983f0 con 0x7f1b18039060 2024-09-12T01:19:49.129 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:49.130+0000 7f1b25ffb640 1 -- 172.21.15.104:0/2688560793 <== mgr.14150 v2:172.21.15.1:6800/1928662456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f1b301983f0 con 0x7f1b18039060 2024-09-12T01:19:49.129 INFO:teuthology.orchestra.run.smithi104.stdout:Scheduled mon update... 2024-09-12T01:19:49.131 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:49.132+0000 7f1b36e31640 1 -- 172.21.15.104:0/2688560793 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1b18039060 msgr2=0x7f1b1803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:49.131 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:49.132+0000 7f1b36e31640 1 --2- 172.21.15.104:0/2688560793 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1b18039060 0x7f1b1803b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1b200098c0 tx=0x7f1b20006db0 comp rx=0 tx=0).stop 2024-09-12T01:19:49.132 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:49.132+0000 7f1b36e31640 1 -- 172.21.15.104:0/2688560793 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b301075e0 msgr2=0x7f1b30197090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:49.132 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:49.132+0000 7f1b36e31640 1 --2- 172.21.15.104:0/2688560793 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b301075e0 0x7f1b30197090 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c009230 tx=0x7f1b2c003d80 comp rx=0 tx=0).stop 2024-09-12T01:19:49.132 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:49.132+0000 7f1b36e31640 1 -- 172.21.15.104:0/2688560793 shutdown_connections 2024-09-12T01:19:49.132 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:49.132+0000 7f1b36e31640 1 --2- 172.21.15.104:0/2688560793 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1b18039060 0x7f1b1803b520 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:49.132 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:49.132+0000 7f1b36e31640 1 --2- 172.21.15.104:0/2688560793 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b301075e0 0x7f1b30197090 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:49.132 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:49.132+0000 7f1b36e31640 1 -- 172.21.15.104:0/2688560793 >> 172.21.15.104:0/2688560793 conn(0x7f1b300fb8c0 msgr2=0x7f1b300fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:19:49.132 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:49.132+0000 7f1b36e31640 1 -- 172.21.15.104:0/2688560793 shutdown_connections 2024-09-12T01:19:49.132 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:49.132+0000 7f1b36e31640 1 -- 172.21.15.104:0/2688560793 wait complete. 2024-09-12T01:19:49.696 DEBUG:teuthology.orchestra.run.smithi053:mon.b> sudo journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mon.b.service 2024-09-12T01:19:49.699 DEBUG:teuthology.orchestra.run.smithi104:mon.c> sudo journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mon.c.service 2024-09-12T01:19:49.701 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T01:19:49.701 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T01:19:49.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:49 smithi001 ceph-mon[30387]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-09-12T01:19:49.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:49 smithi001 ceph-mon[30387]: Updating smithi104:/var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T01:19:49.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:49 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:49.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:49 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:19:49.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:49 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:19:49.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:49 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:19:49.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:49 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:49.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:49 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T01:19:49.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:49 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:19:50.504 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:19:50.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:50 smithi001 ceph-mon[30387]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:19:50.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:50 smithi001 ceph-mon[30387]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi001:172.21.15.1=a;smithi053:172.21.15.53=b;smithi104:172.21.15.104=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:19:50.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:50 smithi001 ceph-mon[30387]: Saving service mon spec with placement smithi001:172.21.15.1=a;smithi053:172.21.15.53=b;smithi104:172.21.15.104=c;count:3 2024-09-12T01:19:50.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:50 smithi001 ceph-mon[30387]: Deploying daemon mon.c on smithi104 2024-09-12T01:19:51.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:51 smithi001 ceph-mon[30387]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:19:52.174 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:52.172+0000 7fc1e0c06640 1 -- 172.21.15.104:0/394057819 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1dc102730 msgr2=0x7fc1dc102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:52.174 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:52.172+0000 7fc1e0c06640 1 --2- 172.21.15.104:0/394057819 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1dc102730 0x7fc1dc102b10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc009810 tx=0x7fc1cc02a540 comp rx=0 tx=0).stop 2024-09-12T01:19:52.174 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:52.172+0000 7fc1e0c06640 1 -- 172.21.15.104:0/394057819 shutdown_connections 2024-09-12T01:19:52.174 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:52.172+0000 7fc1e0c06640 1 --2- 172.21.15.104:0/394057819 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1dc102730 0x7fc1dc102b10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:52.174 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:52.172+0000 7fc1e0c06640 1 -- 172.21.15.104:0/394057819 >> 172.21.15.104:0/394057819 conn(0x7fc1dc077310 msgr2=0x7fc1dc0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:19:52.174 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:52.172+0000 7fc1e0c06640 1 -- 172.21.15.104:0/394057819 shutdown_connections 2024-09-12T01:19:52.174 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:52.172+0000 7fc1e0c06640 1 -- 172.21.15.104:0/394057819 wait complete. 2024-09-12T01:19:52.174 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:52.173+0000 7fc1e0c06640 1 Processor -- start 2024-09-12T01:19:52.174 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:52.173+0000 7fc1e0c06640 1 -- start start 2024-09-12T01:19:52.174 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:52.173+0000 7fc1e0c06640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1dc102730 0x7fc1dc1971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:52.174 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:52.173+0000 7fc1e0c06640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc1dc10bc80 con 0x7fc1dc102730 2024-09-12T01:19:52.175 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:52.174+0000 7fc1da59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1dc102730 0x7fc1dc1971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:52.175 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:52.174+0000 7fc1da59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1dc102730 0x7fc1dc1971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.104:34552/0 (socket says 172.21.15.104:34552) 2024-09-12T01:19:52.175 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:52.174+0000 7fc1da59c640 1 -- 172.21.15.104:0/2146903470 learned_addr learned my addr 172.21.15.104:0/2146903470 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-09-12T01:19:52.175 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:52.174+0000 7fc1da59c640 1 -- 172.21.15.104:0/2146903470 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1dc1976f0 con 0x7fc1dc102730 2024-09-12T01:19:52.175 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:52.174+0000 7fc1da59c640 1 --2- 172.21.15.104:0/2146903470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1dc102730 0x7fc1dc1971b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc0045e0 tx=0x7fc1cc004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:52.175 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:52.174+0000 7fc1cb7fe640 1 -- 172.21.15.104:0/2146903470 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc1cc030e70 con 0x7fc1dc102730 2024-09-12T01:19:52.175 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:52.174+0000 7fc1e0c06640 1 -- 172.21.15.104:0/2146903470 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1dc197980 con 0x7fc1dc102730 2024-09-12T01:19:52.175 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:52.174+0000 7fc1cb7fe640 1 -- 172.21.15.104:0/2146903470 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1cc0318e0 con 0x7fc1dc102730 2024-09-12T01:19:52.175 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:52.174+0000 7fc1cb7fe640 1 -- 172.21.15.104:0/2146903470 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc1cc0049c0 con 0x7fc1dc102730 2024-09-12T01:19:52.175 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:52.174+0000 7fc1e0c06640 1 -- 172.21.15.104:0/2146903470 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1dc197de0 con 0x7fc1dc102730 2024-09-12T01:19:52.176 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:52.175+0000 7fc1cb7fe640 1 -- 172.21.15.104:0/2146903470 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fc1cc039040 con 0x7fc1dc102730 2024-09-12T01:19:52.176 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:52.176+0000 7fc1cb7fe640 1 --2- 172.21.15.104:0/2146903470 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fc1c4039060 0x7fc1c403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:52.176 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:52.176+0000 7fc1cb7fe640 1 -- 172.21.15.104:0/2146903470 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc1cc06d350 con 0x7fc1dc102730 2024-09-12T01:19:52.176 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:52.176+0000 7fc1e0c06640 1 -- 172.21.15.104:0/2146903470 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1dc069f90 con 0x7fc1dc102730 2024-09-12T01:19:52.176 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:52.176+0000 7fc1d9d9b640 1 --2- 172.21.15.104:0/2146903470 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fc1c4039060 0x7fc1c403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:52.176 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:52.177+0000 7fc1d9d9b640 1 --2- 172.21.15.104:0/2146903470 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fc1c4039060 0x7fc1c403b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc1d00098c0 tx=0x7fc1d0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:52.181 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:52.182+0000 7fc1cb7fe640 1 -- 172.21.15.104:0/2146903470 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc1cc040030 con 0x7fc1dc102730 2024-09-12T01:19:52.436 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:52.437+0000 7fc1e0c06640 1 -- 172.21.15.104:0/2146903470 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fc1dc198540 con 0x7fc1dc102730 2024-09-12T01:19:52.437 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:52.438+0000 7fc1cb7fe640 1 -- 172.21.15.104:0/2146903470 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+741 (secure 0 0 0) 0x7fc1cc043a00 con 0x7fc1dc102730 2024-09-12T01:19:52.437 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-12T01:19:52.437 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":1,"fsid":"c0e4f180-70a4-11ef-bcea-c7b262605968","modified":"2024-09-12T01:18:15.215734Z","created":"2024-09-12T01:18:15.215734Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-12T01:19:52.438 INFO:teuthology.orchestra.run.smithi104.stderr:dumped monmap epoch 1 2024-09-12T01:19:52.439 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:52.440+0000 7fc1e0c06640 1 -- 172.21.15.104:0/2146903470 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fc1c4039060 msgr2=0x7fc1c403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:52.439 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:52.440+0000 7fc1e0c06640 1 --2- 172.21.15.104:0/2146903470 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fc1c4039060 0x7fc1c403b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc1d00098c0 tx=0x7fc1d0006db0 comp rx=0 tx=0).stop 2024-09-12T01:19:52.439 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:52.440+0000 7fc1e0c06640 1 -- 172.21.15.104:0/2146903470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1dc102730 msgr2=0x7fc1dc1971b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:52.440 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:52.440+0000 7fc1e0c06640 1 --2- 172.21.15.104:0/2146903470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1dc102730 0x7fc1dc1971b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc0045e0 tx=0x7fc1cc004610 comp rx=0 tx=0).stop 2024-09-12T01:19:52.440 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:52.440+0000 7fc1e0c06640 1 -- 172.21.15.104:0/2146903470 shutdown_connections 2024-09-12T01:19:52.440 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:52.440+0000 7fc1e0c06640 1 --2- 172.21.15.104:0/2146903470 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fc1c4039060 0x7fc1c403b520 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:52.440 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:52.440+0000 7fc1e0c06640 1 --2- 172.21.15.104:0/2146903470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1dc102730 0x7fc1dc1971b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:52.440 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:52.440+0000 7fc1e0c06640 1 -- 172.21.15.104:0/2146903470 >> 172.21.15.104:0/2146903470 conn(0x7fc1dc077310 msgr2=0x7fc1dc1054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:19:52.440 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:52.440+0000 7fc1e0c06640 1 -- 172.21.15.104:0/2146903470 shutdown_connections 2024-09-12T01:19:52.440 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:52.440+0000 7fc1e0c06640 1 -- 172.21.15.104:0/2146903470 wait complete. 2024-09-12T01:19:52.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:52 smithi001 ceph-mon[30387]: from='client.? 172.21.15.104:0/2146903470' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T01:19:53.641 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: mon.c@-1(???) e0 preinit fsid c0e4f180-70a4-11ef-bcea-c7b262605968 2024-09-12T01:19:53.641 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: mon.c@-1(synchronizing).mds e1 new map 2024-09-12T01:19:53.641 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-12T01:19:53.642 INFO:journalctl@ceph.mon.c.smithi104.stdout: e1 2024-09-12T01:19:53.642 INFO:journalctl@ceph.mon.c.smithi104.stdout: btime 2024-09-12T01:18:19:337176+0000 2024-09-12T01:19:53.642 INFO:journalctl@ceph.mon.c.smithi104.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T01:19:53.642 INFO:journalctl@ceph.mon.c.smithi104.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T01:19:53.642 INFO:journalctl@ceph.mon.c.smithi104.stdout: legacy client fscid: -1 2024-09-12T01:19:53.642 INFO:journalctl@ceph.mon.c.smithi104.stdout: 2024-09-12T01:19:53.642 INFO:journalctl@ceph.mon.c.smithi104.stdout: No filesystems configured 2024-09-12T01:19:53.642 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-12T01:19:53.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:53 smithi001 ceph-mon[30387]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:19:54.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-12T01:19:54.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-12T01:19:54.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-12T01:19:54.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-12T01:19:54.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-12T01:19:54.076 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-12T01:19:54.076 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T01:19:54.076 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T01:19:54.076 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T01:19:54.076 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:19:54.076 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi104", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:19:54.076 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:54.076 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:54.076 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:54.076 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:54.076 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi053", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:19:54.076 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:19:54.077 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:19:54.077 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: Updating smithi053:/etc/ceph/ceph.conf 2024-09-12T01:19:54.077 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: Updating smithi053:/var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:19:54.077 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2024-09-12T01:19:54.077 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: Updating smithi053:/var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T01:19:54.077 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: Deploying cephadm binary to smithi104 2024-09-12T01:19:54.077 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:54.077 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:54.077 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:54.077 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:19:54.077 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:19:54.077 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:54.077 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: Added host smithi104 2024-09-12T01:19:54.077 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:19:54.078 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:54.078 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:54.078 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:54.078 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:19:54.078 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:19:54.078 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/2207965967' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-12T01:19:54.078 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:54.078 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:54.078 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:54.078 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:54.078 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi104", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:19:54.078 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:19:54.078 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:19:54.079 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:19:54.079 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: Updating smithi104:/etc/ceph/ceph.conf 2024-09-12T01:19:54.079 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: Updating smithi104:/var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:19:54.079 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/2207965967' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-12T01:19:54.079 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T01:19:54.079 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:54.079 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:54.079 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:54.079 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-09-12T01:19:54.079 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: Updating smithi104:/var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T01:19:54.079 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:54.079 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:19:54.079 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:19:54.079 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:19:54.080 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:54.080 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T01:19:54.080 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:19:54.080 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:19:54.080 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi001:172.21.15.1=a;smithi053:172.21.15.53=b;smithi104:172.21.15.104=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:19:54.080 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: Saving service mon spec with placement smithi001:172.21.15.1=a;smithi053:172.21.15.53=b;smithi104:172.21.15.104=c;count:3 2024-09-12T01:19:54.080 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: Deploying daemon mon.c on smithi104 2024-09-12T01:19:54.080 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:19:54.080 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: from='client.? 172.21.15.104:0/2146903470' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T01:19:54.080 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:19:54.080 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 ceph-mon[35219]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-12T01:19:54.080 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 bash[35200]: b8b8b09583ac35a38f447711ee391c25f3ed43b218ee97ece37d4dbfba0ce28c 2024-09-12T01:19:54.080 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:53 smithi104 systemd[1]: Started Ceph mon.c for c0e4f180-70a4-11ef-bcea-c7b262605968. 2024-09-12T01:19:54.228 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T01:19:54.228 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T01:19:54.477 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:19:59.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:58 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:19:59.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:58 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:19:59.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:58 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:19:59.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:58 smithi104 ceph-mon[35219]: mon.a calling monitor election 2024-09-12T01:19:59.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:58 smithi104 ceph-mon[35219]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:19:59.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:58 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:19:59.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:58 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:19:59.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:58 smithi104 ceph-mon[35219]: mon.c calling monitor election 2024-09-12T01:19:59.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:58 smithi104 ceph-mon[35219]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:19:59.076 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:58 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:19:59.076 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:58 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:19:59.076 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:58 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:19:59.076 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:58 smithi104 ceph-mon[35219]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T01:19:59.076 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:58 smithi104 ceph-mon[35219]: monmap epoch 2 2024-09-12T01:19:59.076 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:58 smithi104 ceph-mon[35219]: fsid c0e4f180-70a4-11ef-bcea-c7b262605968 2024-09-12T01:19:59.076 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:58 smithi104 ceph-mon[35219]: last_changed 2024-09-12T01:19:53.648242+0000 2024-09-12T01:19:59.076 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:58 smithi104 ceph-mon[35219]: created 2024-09-12T01:18:15.215734+0000 2024-09-12T01:19:59.076 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:58 smithi104 ceph-mon[35219]: min_mon_release 19 (squid) 2024-09-12T01:19:59.076 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:58 smithi104 ceph-mon[35219]: election_strategy: 1 2024-09-12T01:19:59.076 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:58 smithi104 ceph-mon[35219]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-09-12T01:19:59.076 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:58 smithi104 ceph-mon[35219]: 1: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.c 2024-09-12T01:19:59.076 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:58 smithi104 ceph-mon[35219]: fsmap 2024-09-12T01:19:59.077 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:58 smithi104 ceph-mon[35219]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T01:19:59.077 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:58 smithi104 ceph-mon[35219]: mgrmap e13: x(active, since 44s) 2024-09-12T01:19:59.077 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:58 smithi104 ceph-mon[35219]: overall HEALTH_OK 2024-09-12T01:19:59.077 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:58 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:59.077 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:58 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:59.077 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:58 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:59.077 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:58 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T01:19:59.077 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:58 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:19:59.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:58 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:19:59.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:58 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:19:59.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:58 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:19:59.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:58 smithi001 ceph-mon[30387]: mon.a calling monitor election 2024-09-12T01:19:59.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:58 smithi001 ceph-mon[30387]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:19:59.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:58 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:19:59.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:58 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:19:59.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:58 smithi001 ceph-mon[30387]: mon.c calling monitor election 2024-09-12T01:19:59.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:58 smithi001 ceph-mon[30387]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:19:59.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:58 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:19:59.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:58 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:19:59.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:58 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:19:59.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:58 smithi001 ceph-mon[30387]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T01:19:59.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:58 smithi001 ceph-mon[30387]: monmap epoch 2 2024-09-12T01:19:59.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:58 smithi001 ceph-mon[30387]: fsid c0e4f180-70a4-11ef-bcea-c7b262605968 2024-09-12T01:19:59.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:58 smithi001 ceph-mon[30387]: last_changed 2024-09-12T01:19:53.648242+0000 2024-09-12T01:19:59.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:58 smithi001 ceph-mon[30387]: created 2024-09-12T01:18:15.215734+0000 2024-09-12T01:19:59.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:58 smithi001 ceph-mon[30387]: min_mon_release 19 (squid) 2024-09-12T01:19:59.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:58 smithi001 ceph-mon[30387]: election_strategy: 1 2024-09-12T01:19:59.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:58 smithi001 ceph-mon[30387]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-09-12T01:19:59.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:58 smithi001 ceph-mon[30387]: 1: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.c 2024-09-12T01:19:59.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:58 smithi001 ceph-mon[30387]: fsmap 2024-09-12T01:19:59.180 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:58 smithi001 ceph-mon[30387]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T01:19:59.180 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:58 smithi001 ceph-mon[30387]: mgrmap e13: x(active, since 44s) 2024-09-12T01:19:59.180 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:58 smithi001 ceph-mon[30387]: overall HEALTH_OK 2024-09-12T01:19:59.180 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:58 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:59.180 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:58 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:59.180 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:58 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:19:59.180 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:58 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T01:19:59.180 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:58 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:19:59.362 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.362+0000 7f9085377640 1 -- 172.21.15.104:0/3198550030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9068003530 msgr2=0x7f90680059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:59.363 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.362+0000 7f9085377640 1 --2- 172.21.15.104:0/3198550030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9068003530 0x7f90680059c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f90801016a0 tx=0x7f907002c490 comp rx=0 tx=0).stop 2024-09-12T01:19:59.363 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.363+0000 7f9085377640 1 -- 172.21.15.104:0/3198550030 shutdown_connections 2024-09-12T01:19:59.363 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.363+0000 7f9085377640 1 --2- 172.21.15.104:0/3198550030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9068003530 0x7f90680059c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:59.363 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.363+0000 7f9085377640 1 -- 172.21.15.104:0/3198550030 >> 172.21.15.104:0/3198550030 conn(0x7f90800fc220 msgr2=0x7f90800fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:19:59.363 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.363+0000 7f9085377640 1 -- 172.21.15.104:0/3198550030 shutdown_connections 2024-09-12T01:19:59.363 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.363+0000 7f9085377640 1 -- 172.21.15.104:0/3198550030 wait complete. 2024-09-12T01:19:59.363 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.363+0000 7f9085377640 1 Processor -- start 2024-09-12T01:19:59.365 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.364+0000 7f9085377640 1 -- start start 2024-09-12T01:19:59.365 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.364+0000 7f9085377640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9068003530 0x7f9080199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:59.365 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.364+0000 7f9085377640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90801999c0 0x7f908019dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:59.365 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.364+0000 7f9085377640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9080108670 con 0x7f90801999c0 2024-09-12T01:19:59.365 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.364+0000 7f9085377640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f90801084f0 con 0x7f9068003530 2024-09-12T01:19:59.365 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.364+0000 7f907effd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9068003530 0x7f9080199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:59.365 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.364+0000 7f907effd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9068003530 0x7f9080199480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:40924/0 (socket says 172.21.15.104:40924) 2024-09-12T01:19:59.365 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.364+0000 7f907effd640 1 -- 172.21.15.104:0/3620326590 learned_addr learned my addr 172.21.15.104:0/3620326590 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-09-12T01:19:59.366 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.364+0000 7f907effd640 1 -- 172.21.15.104:0/3620326590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9068003530 msgr2=0x7f9080199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-12T01:19:59.366 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.364+0000 7f907effd640 1 -- 172.21.15.104:0/3620326590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9068003530 msgr2=0x7f9080199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T01:19:59.366 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.365+0000 7f907effd640 1 --2- 172.21.15.104:0/3620326590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9068003530 0x7f9080199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T01:19:59.366 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.365+0000 7f907effd640 1 --2- 172.21.15.104:0/3620326590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9068003530 0x7f9080199480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T01:19:59.366 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.365+0000 7f907e7fc640 1 --2- 172.21.15.104:0/3620326590 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90801999c0 0x7f908019dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:59.366 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.365+0000 7f907e7fc640 1 -- 172.21.15.104:0/3620326590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9068003530 msgr2=0x7f9080199480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T01:19:59.366 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.365+0000 7f907e7fc640 1 --2- 172.21.15.104:0/3620326590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9068003530 0x7f9080199480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:59.366 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.365+0000 7f907e7fc640 1 -- 172.21.15.104:0/3620326590 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9080199fe0 con 0x7f90801999c0 2024-09-12T01:19:59.366 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.365+0000 7f907e7fc640 1 --2- 172.21.15.104:0/3620326590 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90801999c0 0x7f908019dd50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f907400b3d0 tx=0x7f907400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:59.366 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.365+0000 7f9067fff640 1 -- 172.21.15.104:0/3620326590 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f9074011630 con 0x7f90801999c0 2024-09-12T01:19:59.366 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.365+0000 7f9085377640 1 -- 172.21.15.104:0/3620326590 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f908019e290 con 0x7f90801999c0 2024-09-12T01:19:59.366 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.366+0000 7f9067fff640 1 -- 172.21.15.104:0/3620326590 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9074011cb0 con 0x7f90801999c0 2024-09-12T01:19:59.366 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.366+0000 7f9067fff640 1 -- 172.21.15.104:0/3620326590 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f9074010df0 con 0x7f90801999c0 2024-09-12T01:19:59.367 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.366+0000 7f9085377640 1 -- 172.21.15.104:0/3620326590 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f908019e850 con 0x7f90801999c0 2024-09-12T01:19:59.367 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.367+0000 7f9085377640 1 -- 172.21.15.104:0/3620326590 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f908019eb60 con 0x7f90801999c0 2024-09-12T01:19:59.367 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.367+0000 7f9067fff640 1 -- 172.21.15.104:0/3620326590 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f9074018020 con 0x7f90801999c0 2024-09-12T01:19:59.367 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.367+0000 7f9067fff640 1 --2- 172.21.15.104:0/3620326590 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f906c038e00 0x7f906c03b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:19:59.368 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.367+0000 7f9067fff640 1 -- 172.21.15.104:0/3620326590 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f907404bc00 con 0x7f90801999c0 2024-09-12T01:19:59.368 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.367+0000 7f907effd640 1 --2- 172.21.15.104:0/3620326590 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f906c038e00 0x7f906c03b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:19:59.368 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.368+0000 7f907effd640 1 --2- 172.21.15.104:0/3620326590 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f906c038e00 0x7f906c03b2c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9070012c10 tx=0x7f90700323a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:19:59.371 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.372+0000 7f9067fff640 1 -- 172.21.15.104:0/3620326590 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9074052080 con 0x7f90801999c0 2024-09-12T01:19:59.604 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.605+0000 7f9085377640 1 -- 172.21.15.104:0/3620326590 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f908019a840 con 0x7f90801999c0 2024-09-12T01:19:59.605 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.606+0000 7f9067fff640 1 -- 172.21.15.104:0/3620326590 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7f907401f020 con 0x7f90801999c0 2024-09-12T01:19:59.605 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-12T01:19:59.605 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":2,"fsid":"c0e4f180-70a4-11ef-bcea-c7b262605968","modified":"2024-09-12T01:19:53.648242Z","created":"2024-09-12T01:18:15.215734Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:3300","nonce":0},{"type":"v1","addr":"172.21.15.104:6789","nonce":0}]},"addr":"172.21.15.104:6789/0","public_addr":"172.21.15.104:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-12T01:19:59.606 INFO:teuthology.orchestra.run.smithi104.stderr:dumped monmap epoch 2 2024-09-12T01:19:59.608 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.608+0000 7f9085377640 1 -- 172.21.15.104:0/3620326590 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f906c038e00 msgr2=0x7f906c03b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:59.608 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.608+0000 7f9085377640 1 --2- 172.21.15.104:0/3620326590 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f906c038e00 0x7f906c03b2c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9070012c10 tx=0x7f90700323a0 comp rx=0 tx=0).stop 2024-09-12T01:19:59.608 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.608+0000 7f9085377640 1 -- 172.21.15.104:0/3620326590 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90801999c0 msgr2=0x7f908019dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:19:59.608 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.608+0000 7f9085377640 1 --2- 172.21.15.104:0/3620326590 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90801999c0 0x7f908019dd50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f907400b3d0 tx=0x7f907400b890 comp rx=0 tx=0).stop 2024-09-12T01:19:59.608 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.608+0000 7f9085377640 1 -- 172.21.15.104:0/3620326590 shutdown_connections 2024-09-12T01:19:59.609 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.608+0000 7f9085377640 1 --2- 172.21.15.104:0/3620326590 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90801999c0 0x7f908019dd50 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:59.609 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.608+0000 7f9085377640 1 --2- 172.21.15.104:0/3620326590 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f906c038e00 0x7f906c03b2c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:59.609 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.608+0000 7f9085377640 1 --2- 172.21.15.104:0/3620326590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9068003530 0x7f9080199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:19:59.609 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.608+0000 7f9085377640 1 -- 172.21.15.104:0/3620326590 >> 172.21.15.104:0/3620326590 conn(0x7f90800fc220 msgr2=0x7f908006a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:19:59.609 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.608+0000 7f9085377640 1 -- 172.21.15.104:0/3620326590 shutdown_connections 2024-09-12T01:19:59.609 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:19:59.608+0000 7f9085377640 1 -- 172.21.15.104:0/3620326590 wait complete. 2024-09-12T01:19:59.961 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:59 smithi104 ceph-mon[35219]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:19:59.961 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:59 smithi104 ceph-mon[35219]: Deploying daemon mon.b on smithi053 2024-09-12T01:19:59.961 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:59 smithi104 ceph-mon[35219]: from='client.? 172.21.15.104:0/3620326590' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T01:19:59.961 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:19:59 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:20:00.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:59 smithi001 ceph-mon[30387]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:00.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:59 smithi001 ceph-mon[30387]: Deploying daemon mon.b on smithi053 2024-09-12T01:20:00.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:59 smithi001 ceph-mon[30387]: from='client.? 172.21.15.104:0/3620326590' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T01:20:00.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:19:59 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:20:01.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:00 smithi104 ceph-mon[35219]: overall HEALTH_OK 2024-09-12T01:20:01.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:00 smithi001 ceph-mon[30387]: overall HEALTH_OK 2024-09-12T01:20:01.301 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T01:20:01.301 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T01:20:01.551 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:20:01.955 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:01 smithi104 ceph-mon[35219]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:01.962 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:01.962+0000 7f60d61db640 1 -- 172.21.15.104:0/1667590387 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60d0107670 msgr2=0x7f60d0107a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:20:01.962 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:01.962+0000 7f60d61db640 1 --2- 172.21.15.104:0/1667590387 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60d0107670 0x7f60d0107a50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f60c0009c20 tx=0x7f60c002aa90 comp rx=0 tx=0).stop 2024-09-12T01:20:01.962 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:01.963+0000 7f60d61db640 1 -- 172.21.15.104:0/1667590387 shutdown_connections 2024-09-12T01:20:01.962 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:01.963+0000 7f60d61db640 1 --2- 172.21.15.104:0/1667590387 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60d0107670 0x7f60d0107a50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:20:01.962 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:01.963+0000 7f60d61db640 1 -- 172.21.15.104:0/1667590387 >> 172.21.15.104:0/1667590387 conn(0x7f60d00fd330 msgr2=0x7f60d00ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:20:01.962 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:01.963+0000 7f60d61db640 1 -- 172.21.15.104:0/1667590387 shutdown_connections 2024-09-12T01:20:01.962 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:01.963+0000 7f60d61db640 1 -- 172.21.15.104:0/1667590387 wait complete. 2024-09-12T01:20:01.963 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:01.963+0000 7f60d61db640 1 Processor -- start 2024-09-12T01:20:01.964 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:01.963+0000 7f60d61db640 1 -- start start 2024-09-12T01:20:01.965 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:01.964+0000 7f60d61db640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60d0107670 0x7f60d01973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:20:01.965 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:01.964+0000 7f60d61db640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60d01978f0 0x7f60d019bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:20:01.965 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:01.964+0000 7f60d61db640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f60d010bd30 con 0x7f60d0107670 2024-09-12T01:20:01.965 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:01.964+0000 7f60d61db640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f60d010bbb0 con 0x7f60d01978f0 2024-09-12T01:20:01.965 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:01.964+0000 7f60ceffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60d01978f0 0x7f60d019bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:20:01.965 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:01.964+0000 7f60ceffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60d01978f0 0x7f60d019bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:40930/0 (socket says 172.21.15.104:40930) 2024-09-12T01:20:01.965 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:01.964+0000 7f60ceffd640 1 -- 172.21.15.104:0/744782997 learned_addr learned my addr 172.21.15.104:0/744782997 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-09-12T01:20:01.965 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:01.964+0000 7f60ceffd640 1 -- 172.21.15.104:0/744782997 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60d01978f0 msgr2=0x7f60d019bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-12T01:20:01.965 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:01.964+0000 7f60ceffd640 1 -- 172.21.15.104:0/744782997 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60d01978f0 msgr2=0x7f60d019bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T01:20:01.966 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:01.964+0000 7f60ceffd640 1 --2- 172.21.15.104:0/744782997 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60d01978f0 0x7f60d019bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T01:20:01.966 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:01.964+0000 7f60ceffd640 1 --2- 172.21.15.104:0/744782997 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60d01978f0 0x7f60d019bc80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T01:20:01.966 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:01.964+0000 7f60cf7fe640 1 --2- 172.21.15.104:0/744782997 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60d0107670 0x7f60d01973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:20:01.966 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:01.964+0000 7f60cf7fe640 1 -- 172.21.15.104:0/744782997 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60d01978f0 msgr2=0x7f60d019bc80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T01:20:01.966 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:01.964+0000 7f60cf7fe640 1 --2- 172.21.15.104:0/744782997 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60d01978f0 0x7f60d019bc80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:20:01.966 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:01.964+0000 7f60cf7fe640 1 -- 172.21.15.104:0/744782997 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d0197f10 con 0x7f60d0107670 2024-09-12T01:20:01.966 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:01.965+0000 7f60cf7fe640 1 --2- 172.21.15.104:0/744782997 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60d0107670 0x7f60d01973b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f60c0009c20 tx=0x7f60c002e5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:20:01.966 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:01.965+0000 7f60ccff9640 1 -- 172.21.15.104:0/744782997 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f60c002e970 con 0x7f60d0107670 2024-09-12T01:20:01.966 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:01.965+0000 7f60ccff9640 1 -- 172.21.15.104:0/744782997 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60c0038930 con 0x7f60d0107670 2024-09-12T01:20:01.966 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:01.965+0000 7f60d61db640 1 -- 172.21.15.104:0/744782997 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60d019c1c0 con 0x7f60d0107670 2024-09-12T01:20:01.966 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:01.965+0000 7f60ccff9640 1 -- 172.21.15.104:0/744782997 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f60c0043980 con 0x7f60d0107670 2024-09-12T01:20:01.966 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:01.965+0000 7f60d61db640 1 -- 172.21.15.104:0/744782997 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60d019c700 con 0x7f60d0107670 2024-09-12T01:20:01.967 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:01.966+0000 7f60ccff9640 1 -- 172.21.15.104:0/744782997 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f60c0036070 con 0x7f60d0107670 2024-09-12T01:20:01.967 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:01.966+0000 7f60d61db640 1 -- 172.21.15.104:0/744782997 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60d0102db0 con 0x7f60d0107670 2024-09-12T01:20:01.967 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:01.966+0000 7f60ccff9640 1 --2- 172.21.15.104:0/744782997 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f60bc038e00 0x7f60bc03b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:20:01.967 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:01.966+0000 7f60ccff9640 1 -- 172.21.15.104:0/744782997 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f60c006d440 con 0x7f60d0107670 2024-09-12T01:20:01.967 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:01.967+0000 7f60ceffd640 1 --2- 172.21.15.104:0/744782997 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f60bc038e00 0x7f60bc03b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:20:01.968 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:01.967+0000 7f60ceffd640 1 --2- 172.21.15.104:0/744782997 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f60bc038e00 0x7f60bc03b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f60b8009100 tx=0x7f60b8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:20:01.971 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:01.972+0000 7f60ccff9640 1 -- 172.21.15.104:0/744782997 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f60c003e070 con 0x7f60d0107670 2024-09-12T01:20:02.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:01 smithi001 ceph-mon[30387]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:02.202 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:02.203+0000 7f60d61db640 1 -- 172.21.15.104:0/744782997 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f60d0107a50 con 0x7f60d0107670 2024-09-12T01:20:02.203 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:02.204+0000 7f60ccff9640 1 -- 172.21.15.104:0/744782997 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7f60c004bc40 con 0x7f60d0107670 2024-09-12T01:20:02.203 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-12T01:20:02.203 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":2,"fsid":"c0e4f180-70a4-11ef-bcea-c7b262605968","modified":"2024-09-12T01:19:53.648242Z","created":"2024-09-12T01:18:15.215734Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:3300","nonce":0},{"type":"v1","addr":"172.21.15.104:6789","nonce":0}]},"addr":"172.21.15.104:6789/0","public_addr":"172.21.15.104:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-12T01:20:02.203 INFO:teuthology.orchestra.run.smithi104.stderr:dumped monmap epoch 2 2024-09-12T01:20:02.205 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:02.206+0000 7f60d61db640 1 -- 172.21.15.104:0/744782997 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f60bc038e00 msgr2=0x7f60bc03b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:20:02.205 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:02.206+0000 7f60d61db640 1 --2- 172.21.15.104:0/744782997 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f60bc038e00 0x7f60bc03b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f60b8009100 tx=0x7f60b8008040 comp rx=0 tx=0).stop 2024-09-12T01:20:02.205 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:02.206+0000 7f60d61db640 1 -- 172.21.15.104:0/744782997 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60d0107670 msgr2=0x7f60d01973b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:20:02.205 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:02.206+0000 7f60d61db640 1 --2- 172.21.15.104:0/744782997 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60d0107670 0x7f60d01973b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f60c0009c20 tx=0x7f60c002e5f0 comp rx=0 tx=0).stop 2024-09-12T01:20:02.205 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:02.206+0000 7f60d61db640 1 -- 172.21.15.104:0/744782997 shutdown_connections 2024-09-12T01:20:02.205 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:02.206+0000 7f60d61db640 1 --2- 172.21.15.104:0/744782997 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f60bc038e00 0x7f60bc03b2c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:20:02.205 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:02.206+0000 7f60d61db640 1 --2- 172.21.15.104:0/744782997 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60d01978f0 0x7f60d019bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:20:02.205 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:02.206+0000 7f60d61db640 1 --2- 172.21.15.104:0/744782997 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60d0107670 0x7f60d01973b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:20:02.206 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:02.206+0000 7f60d61db640 1 -- 172.21.15.104:0/744782997 >> 172.21.15.104:0/744782997 conn(0x7f60d00fd330 msgr2=0x7f60d00fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:20:02.206 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:02.206+0000 7f60d61db640 1 -- 172.21.15.104:0/744782997 shutdown_connections 2024-09-12T01:20:02.206 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:02.206+0000 7f60d61db640 1 -- 172.21.15.104:0/744782997 wait complete. 2024-09-12T01:20:02.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 podman[35139]: 2024-09-12 01:20:02.152497347 +0000 UTC m=+0.121239957 container create a344fc8ebfb66e31006082877ac60329fe62e38c30afe02d745d460e84c957cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mon-b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2) 2024-09-12T01:20:02.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 podman[35139]: 2024-09-12 01:20:02.363070133 +0000 UTC m=+0.331812742 container init a344fc8ebfb66e31006082877ac60329fe62e38c30afe02d745d460e84c957cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mon-b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-12T01:20:02.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 podman[35139]: 2024-09-12 01:20:02.36633904 +0000 UTC m=+0.335081649 container start a344fc8ebfb66e31006082877ac60329fe62e38c30afe02d745d460e84c957cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mon-b, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-12T01:20:02.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: set uid:gid to 167:167 (ceph:ceph) 2024-09-12T01:20:02.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable), process ceph-mon, pid 2 2024-09-12T01:20:02.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: pidfile_write: ignore empty --pid-file 2024-09-12T01:20:02.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: load: jerasure load: lrc 2024-09-12T01:20:02.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: RocksDB version: 7.9.2 2024-09-12T01:20:02.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Git sha 0 2024-09-12T01:20:02.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Compile date 2024-09-09 16:23:28 2024-09-12T01:20:02.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: DB SUMMARY 2024-09-12T01:20:02.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: DB Session ID: 8ZVL14HYXKZJO4B9ZU26 2024-09-12T01:20:02.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: CURRENT file: CURRENT 2024-09-12T01:20:02.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: IDENTITY file: IDENTITY 2024-09-12T01:20:02.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-12T01:20:02.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-12T01:20:02.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-09-12T01:20:02.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.error_if_exists: 0 2024-09-12T01:20:02.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.create_if_missing: 0 2024-09-12T01:20:02.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.paranoid_checks: 1 2024-09-12T01:20:02.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-12T01:20:02.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-12T01:20:02.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-12T01:20:02.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.env: 0x555ebb0cb940 2024-09-12T01:20:02.742 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.fs: PosixFileSystem 2024-09-12T01:20:02.742 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.info_log: 0x555ebc6036e0 2024-09-12T01:20:02.742 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.max_file_opening_threads: 16 2024-09-12T01:20:02.743 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.statistics: (nil) 2024-09-12T01:20:02.744 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.use_fsync: 0 2024-09-12T01:20:02.744 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.max_log_file_size: 0 2024-09-12T01:20:02.744 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-12T01:20:02.744 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-12T01:20:02.744 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.keep_log_file_num: 1000 2024-09-12T01:20:02.744 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.recycle_log_file_num: 0 2024-09-12T01:20:02.744 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.allow_fallocate: 1 2024-09-12T01:20:02.744 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.allow_mmap_reads: 0 2024-09-12T01:20:02.744 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.allow_mmap_writes: 0 2024-09-12T01:20:02.744 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.use_direct_reads: 0 2024-09-12T01:20:02.744 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-12T01:20:02.744 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.create_missing_column_families: 0 2024-09-12T01:20:02.744 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.db_log_dir: 2024-09-12T01:20:02.745 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.wal_dir: 2024-09-12T01:20:02.745 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-12T01:20:02.745 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-12T01:20:02.745 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-12T01:20:02.745 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-12T01:20:02.745 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-12T01:20:02.745 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-12T01:20:02.745 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.advise_random_on_open: 1 2024-09-12T01:20:02.745 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.db_write_buffer_size: 0 2024-09-12T01:20:02.745 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.write_buffer_manager: 0x555ebc612dc0 2024-09-12T01:20:02.745 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-12T01:20:02.746 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-12T01:20:02.746 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-12T01:20:02.746 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.rate_limiter: (nil) 2024-09-12T01:20:02.746 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-12T01:20:02.746 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.wal_recovery_mode: 2 2024-09-12T01:20:02.746 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.enable_thread_tracking: 0 2024-09-12T01:20:02.746 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.enable_pipelined_write: 0 2024-09-12T01:20:02.746 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.unordered_write: 0 2024-09-12T01:20:02.746 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-12T01:20:02.746 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-12T01:20:02.746 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-12T01:20:02.746 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-12T01:20:02.746 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.row_cache: None 2024-09-12T01:20:02.747 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.wal_filter: None 2024-09-12T01:20:02.747 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-12T01:20:02.747 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.allow_ingest_behind: 0 2024-09-12T01:20:02.747 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.two_write_queues: 0 2024-09-12T01:20:02.747 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.manual_wal_flush: 0 2024-09-12T01:20:02.747 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.wal_compression: 0 2024-09-12T01:20:02.747 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.atomic_flush: 0 2024-09-12T01:20:02.747 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-12T01:20:02.747 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-12T01:20:02.747 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-12T01:20:02.747 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.log_readahead_size: 0 2024-09-12T01:20:02.747 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-12T01:20:02.747 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.best_efforts_recovery: 0 2024-09-12T01:20:02.747 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-12T01:20:02.747 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-12T01:20:02.748 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.allow_data_in_errors: 0 2024-09-12T01:20:02.748 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.db_host_id: __hostname__ 2024-09-12T01:20:02.748 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-12T01:20:02.748 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.max_background_jobs: 2 2024-09-12T01:20:02.748 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.max_background_compactions: -1 2024-09-12T01:20:02.748 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.max_subcompactions: 1 2024-09-12T01:20:02.748 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-12T01:20:02.748 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-12T01:20:02.748 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-12T01:20:02.748 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.max_total_wal_size: 0 2024-09-12T01:20:02.749 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-12T01:20:02.749 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-12T01:20:02.749 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-12T01:20:02.749 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-12T01:20:02.749 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.max_open_files: -1 2024-09-12T01:20:02.749 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.bytes_per_sync: 0 2024-09-12T01:20:02.749 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-12T01:20:02.749 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-12T01:20:02.749 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.compaction_readahead_size: 0 2024-09-12T01:20:02.749 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.max_background_flushes: -1 2024-09-12T01:20:02.749 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Compression algorithms supported: 2024-09-12T01:20:02.749 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: kZSTD supported: 0 2024-09-12T01:20:02.749 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: kXpressCompression supported: 0 2024-09-12T01:20:02.749 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: kBZip2Compression supported: 0 2024-09-12T01:20:02.750 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-12T01:20:02.750 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: kLZ4Compression supported: 1 2024-09-12T01:20:02.750 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: kZlibCompression supported: 1 2024-09-12T01:20:02.750 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: kLZ4HCCompression supported: 1 2024-09-12T01:20:02.750 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: kSnappyCompression supported: 1 2024-09-12T01:20:02.750 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-12T01:20:02.750 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-12T01:20:02.750 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-12T01:20:02.750 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-12T01:20:02.750 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-12T01:20:02.750 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.merge_operator: 2024-09-12T01:20:02.750 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.compaction_filter: None 2024-09-12T01:20:02.750 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.compaction_filter_factory: None 2024-09-12T01:20:02.750 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.sst_partitioner_factory: None 2024-09-12T01:20:02.751 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-12T01:20:02.751 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-12T01:20:02.751 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x555ebc603320) 2024-09-12T01:20:02.751 INFO:journalctl@ceph.mon.b.smithi053.stdout: cache_index_and_filter_blocks: 1 2024-09-12T01:20:02.751 INFO:journalctl@ceph.mon.b.smithi053.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-12T01:20:02.751 INFO:journalctl@ceph.mon.b.smithi053.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-12T01:20:02.751 INFO:journalctl@ceph.mon.b.smithi053.stdout: pin_top_level_index_and_filter: 1 2024-09-12T01:20:02.751 INFO:journalctl@ceph.mon.b.smithi053.stdout: index_type: 0 2024-09-12T01:20:02.751 INFO:journalctl@ceph.mon.b.smithi053.stdout: data_block_index_type: 0 2024-09-12T01:20:02.751 INFO:journalctl@ceph.mon.b.smithi053.stdout: index_shortening: 1 2024-09-12T01:20:02.751 INFO:journalctl@ceph.mon.b.smithi053.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-12T01:20:02.751 INFO:journalctl@ceph.mon.b.smithi053.stdout: checksum: 4 2024-09-12T01:20:02.751 INFO:journalctl@ceph.mon.b.smithi053.stdout: no_block_cache: 0 2024-09-12T01:20:02.751 INFO:journalctl@ceph.mon.b.smithi053.stdout: block_cache: 0x555ebc5f71f0 2024-09-12T01:20:02.752 INFO:journalctl@ceph.mon.b.smithi053.stdout: block_cache_name: BinnedLRUCache 2024-09-12T01:20:02.752 INFO:journalctl@ceph.mon.b.smithi053.stdout: block_cache_options: 2024-09-12T01:20:02.752 INFO:journalctl@ceph.mon.b.smithi053.stdout: capacity : 536870912 2024-09-12T01:20:02.752 INFO:journalctl@ceph.mon.b.smithi053.stdout: num_shard_bits : 4 2024-09-12T01:20:02.752 INFO:journalctl@ceph.mon.b.smithi053.stdout: strict_capacity_limit : 0 2024-09-12T01:20:02.752 INFO:journalctl@ceph.mon.b.smithi053.stdout: high_pri_pool_ratio: 0.000 2024-09-12T01:20:02.752 INFO:journalctl@ceph.mon.b.smithi053.stdout: block_cache_compressed: (nil) 2024-09-12T01:20:02.752 INFO:journalctl@ceph.mon.b.smithi053.stdout: persistent_cache: (nil) 2024-09-12T01:20:02.752 INFO:journalctl@ceph.mon.b.smithi053.stdout: block_size: 4096 2024-09-12T01:20:02.752 INFO:journalctl@ceph.mon.b.smithi053.stdout: block_size_deviation: 10 2024-09-12T01:20:02.752 INFO:journalctl@ceph.mon.b.smithi053.stdout: block_restart_interval: 16 2024-09-12T01:20:02.752 INFO:journalctl@ceph.mon.b.smithi053.stdout: index_block_restart_interval: 1 2024-09-12T01:20:02.752 INFO:journalctl@ceph.mon.b.smithi053.stdout: metadata_block_size: 4096 2024-09-12T01:20:02.752 INFO:journalctl@ceph.mon.b.smithi053.stdout: partition_filters: 0 2024-09-12T01:20:02.753 INFO:journalctl@ceph.mon.b.smithi053.stdout: use_delta_encoding: 1 2024-09-12T01:20:02.753 INFO:journalctl@ceph.mon.b.smithi053.stdout: filter_policy: bloomfilter 2024-09-12T01:20:02.753 INFO:journalctl@ceph.mon.b.smithi053.stdout: whole_key_filtering: 1 2024-09-12T01:20:02.753 INFO:journalctl@ceph.mon.b.smithi053.stdout: verify_compression: 0 2024-09-12T01:20:02.753 INFO:journalctl@ceph.mon.b.smithi053.stdout: read_amp_bytes_per_bit: 0 2024-09-12T01:20:02.753 INFO:journalctl@ceph.mon.b.smithi053.stdout: format_version: 5 2024-09-12T01:20:02.753 INFO:journalctl@ceph.mon.b.smithi053.stdout: enable_index_compression: 1 2024-09-12T01:20:02.753 INFO:journalctl@ceph.mon.b.smithi053.stdout: block_align: 0 2024-09-12T01:20:02.753 INFO:journalctl@ceph.mon.b.smithi053.stdout: max_auto_readahead_size: 262144 2024-09-12T01:20:02.753 INFO:journalctl@ceph.mon.b.smithi053.stdout: prepopulate_block_cache: 0 2024-09-12T01:20:02.753 INFO:journalctl@ceph.mon.b.smithi053.stdout: initial_auto_readahead_size: 8192 2024-09-12T01:20:02.753 INFO:journalctl@ceph.mon.b.smithi053.stdout: num_file_reads_for_auto_readahead: 2 2024-09-12T01:20:02.753 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.write_buffer_size: 33554432 2024-09-12T01:20:02.753 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.max_write_buffer_number: 2 2024-09-12T01:20:02.753 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.compression: NoCompression 2024-09-12T01:20:02.754 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.bottommost_compression: Disabled 2024-09-12T01:20:02.754 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.prefix_extractor: nullptr 2024-09-12T01:20:02.754 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-12T01:20:02.754 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.num_levels: 7 2024-09-12T01:20:02.754 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-12T01:20:02.754 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-12T01:20:02.754 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-12T01:20:02.754 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-12T01:20:02.754 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-12T01:20:02.754 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-12T01:20:02.754 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-12T01:20:02.754 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-12T01:20:02.754 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-12T01:20:02.754 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-12T01:20:02.755 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-12T01:20:02.755 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-12T01:20:02.755 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-12T01:20:02.755 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.compression_opts.level: 32767 2024-09-12T01:20:02.755 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.compression_opts.strategy: 0 2024-09-12T01:20:02.755 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-12T01:20:02.755 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-12T01:20:02.755 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-12T01:20:02.755 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-12T01:20:02.755 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.compression_opts.enabled: false 2024-09-12T01:20:02.755 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-12T01:20:02.755 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-12T01:20:02.755 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-12T01:20:02.755 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-12T01:20:02.755 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.target_file_size_base: 67108864 2024-09-12T01:20:02.756 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-12T01:20:02.756 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-12T01:20:02.756 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-12T01:20:02.756 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-12T01:20:02.756 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-12T01:20:02.756 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-12T01:20:02.757 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-12T01:20:02.757 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-12T01:20:02.757 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-12T01:20:02.757 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-12T01:20:02.757 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-12T01:20:02.757 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-12T01:20:02.757 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-12T01:20:02.757 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-12T01:20:02.757 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.arena_block_size: 1048576 2024-09-12T01:20:02.757 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-12T01:20:02.757 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-12T01:20:02.757 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.disable_auto_compactions: 0 2024-09-12T01:20:02.757 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-12T01:20:02.757 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-12T01:20:02.758 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-12T01:20:02.758 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-12T01:20:02.758 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-12T01:20:02.758 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-12T01:20:02.758 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-12T01:20:02.758 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-12T01:20:02.758 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-12T01:20:02.758 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-12T01:20:02.758 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-12T01:20:02.758 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.inplace_update_support: 0 2024-09-12T01:20:02.758 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-12T01:20:02.758 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-12T01:20:02.758 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-12T01:20:02.758 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-12T01:20:02.759 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.bloom_locality: 0 2024-09-12T01:20:02.759 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.max_successive_merges: 0 2024-09-12T01:20:02.759 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-12T01:20:02.759 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.paranoid_file_checks: 0 2024-09-12T01:20:02.759 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.force_consistency_checks: 1 2024-09-12T01:20:02.759 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.report_bg_io_stats: 0 2024-09-12T01:20:02.759 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.ttl: 2592000 2024-09-12T01:20:02.759 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-12T01:20:02.759 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-12T01:20:02.759 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-12T01:20:02.759 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.enable_blob_files: false 2024-09-12T01:20:02.759 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.min_blob_size: 0 2024-09-12T01:20:02.759 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.blob_file_size: 268435456 2024-09-12T01:20:02.759 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-12T01:20:02.759 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-12T01:20:02.760 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-12T01:20:02.760 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-12T01:20:02.760 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-12T01:20:02.760 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.blob_file_starting_level: 0 2024-09-12T01:20:02.760 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-12T01:20:02.760 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-12T01:20:02.760 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-12T01:20:02.760 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 7259b386-7380-4cd9-b84d-d0cc4bf0b98e 2024-09-12T01:20:02.760 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726104002395107, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-12T01:20:02.760 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-12T01:20:02.760 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726104002395502, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726104002, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "7259b386-7380-4cd9-b84d-d0cc4bf0b98e", "db_session_id": "8ZVL14HYXKZJO4B9ZU26", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-12T01:20:02.760 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726104002395595, "job": 1, "event": "recovery_finished"} 2024-09-12T01:20:02.760 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-12T01:20:02.760 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-12T01:20:02.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x555ebc624e00 2024-09-12T01:20:02.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: DB pointer 0x555ebc720000 2024-09-12T01:20:02.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-12T01:20:02.761 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-12T01:20:02.761 INFO:journalctl@ceph.mon.b.smithi053.stdout: ** DB Stats ** 2024-09-12T01:20:02.761 INFO:journalctl@ceph.mon.b.smithi053.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T01:20:02.761 INFO:journalctl@ceph.mon.b.smithi053.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-12T01:20:02.761 INFO:journalctl@ceph.mon.b.smithi053.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T01:20:02.761 INFO:journalctl@ceph.mon.b.smithi053.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T01:20:02.761 INFO:journalctl@ceph.mon.b.smithi053.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-12T01:20:02.761 INFO:journalctl@ceph.mon.b.smithi053.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T01:20:02.761 INFO:journalctl@ceph.mon.b.smithi053.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T01:20:02.761 INFO:journalctl@ceph.mon.b.smithi053.stdout: 2024-09-12T01:20:02.762 INFO:journalctl@ceph.mon.b.smithi053.stdout: ** Compaction Stats [default] ** 2024-09-12T01:20:02.762 INFO:journalctl@ceph.mon.b.smithi053.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T01:20:02.762 INFO:journalctl@ceph.mon.b.smithi053.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-12T01:20:02.762 INFO:journalctl@ceph.mon.b.smithi053.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-12T01:20:02.762 INFO:journalctl@ceph.mon.b.smithi053.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-12T01:20:02.762 INFO:journalctl@ceph.mon.b.smithi053.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-12T01:20:02.762 INFO:journalctl@ceph.mon.b.smithi053.stdout: 2024-09-12T01:20:02.762 INFO:journalctl@ceph.mon.b.smithi053.stdout: ** Compaction Stats [default] ** 2024-09-12T01:20:02.762 INFO:journalctl@ceph.mon.b.smithi053.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T01:20:02.762 INFO:journalctl@ceph.mon.b.smithi053.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-12T01:20:02.762 INFO:journalctl@ceph.mon.b.smithi053.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-12T01:20:02.762 INFO:journalctl@ceph.mon.b.smithi053.stdout: 2024-09-12T01:20:02.762 INFO:journalctl@ceph.mon.b.smithi053.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-12T01:20:02.762 INFO:journalctl@ceph.mon.b.smithi053.stdout: 2024-09-12T01:20:02.762 INFO:journalctl@ceph.mon.b.smithi053.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T01:20:02.763 INFO:journalctl@ceph.mon.b.smithi053.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-12T01:20:02.763 INFO:journalctl@ceph.mon.b.smithi053.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-12T01:20:02.763 INFO:journalctl@ceph.mon.b.smithi053.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-12T01:20:02.763 INFO:journalctl@ceph.mon.b.smithi053.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-12T01:20:02.763 INFO:journalctl@ceph.mon.b.smithi053.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-12T01:20:02.763 INFO:journalctl@ceph.mon.b.smithi053.stdout: Cumulative compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T01:20:02.763 INFO:journalctl@ceph.mon.b.smithi053.stdout: Interval compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T01:20:02.763 INFO:journalctl@ceph.mon.b.smithi053.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-12T01:20:02.763 INFO:journalctl@ceph.mon.b.smithi053.stdout: Block cache BinnedLRUCache@0x555ebc5f71f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-09-12T01:20:02.763 INFO:journalctl@ceph.mon.b.smithi053.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-12T01:20:02.763 INFO:journalctl@ceph.mon.b.smithi053.stdout: 2024-09-12T01:20:02.763 INFO:journalctl@ceph.mon.b.smithi053.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-12T01:20:02.763 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-12T01:20:02.763 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: using public_addr v2:172.21.15.53:0/0 -> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] 2024-09-12T01:20:02.764 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: starting mon.b rank -1 at public addrs [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] at bind addrs [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid c0e4f180-70a4-11ef-bcea-c7b262605968 2024-09-12T01:20:02.764 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: mon.b@-1(???) e0 preinit fsid c0e4f180-70a4-11ef-bcea-c7b262605968 2024-09-12T01:20:02.764 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: mon.b@-1(synchronizing).mds e1 new map 2024-09-12T01:20:02.764 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-12T01:20:02.764 INFO:journalctl@ceph.mon.b.smithi053.stdout: e1 2024-09-12T01:20:02.764 INFO:journalctl@ceph.mon.b.smithi053.stdout: btime 2024-09-12T01:18:19:337176+0000 2024-09-12T01:20:02.764 INFO:journalctl@ceph.mon.b.smithi053.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T01:20:02.764 INFO:journalctl@ceph.mon.b.smithi053.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T01:20:02.764 INFO:journalctl@ceph.mon.b.smithi053.stdout: legacy client fscid: -1 2024-09-12T01:20:02.764 INFO:journalctl@ceph.mon.b.smithi053.stdout: 2024-09-12T01:20:02.764 INFO:journalctl@ceph.mon.b.smithi053.stdout: No filesystems configured 2024-09-12T01:20:02.764 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-12T01:20:02.764 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-12T01:20:02.764 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-12T01:20:02.764 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-12T01:20:02.765 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-12T01:20:02.765 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-12T01:20:02.765 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-12T01:20:02.765 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T01:20:02.765 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T01:20:02.765 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T01:20:02.765 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:02.765 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi104", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:20:02.765 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:02.765 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:02.765 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:02.765 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:02.765 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi053", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:20:02.766 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:02.766 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:20:02.766 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: Updating smithi053:/etc/ceph/ceph.conf 2024-09-12T01:20:02.766 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: Updating smithi053:/var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:20:02.766 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2024-09-12T01:20:02.766 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: Updating smithi053:/var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T01:20:02.766 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: Deploying cephadm binary to smithi104 2024-09-12T01:20:02.766 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:02.766 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:02.766 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:02.766 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:02.766 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:02.766 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:02.767 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: Added host smithi104 2024-09-12T01:20:02.767 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:20:02.767 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:02.767 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:02.767 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:02.767 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:02.767 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:20:02.767 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/2207965967' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-12T01:20:02.767 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:02.767 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:02.767 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:02.767 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:02.767 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi104", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:20:02.767 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:02.767 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:20:02.768 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:02.768 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: Updating smithi104:/etc/ceph/ceph.conf 2024-09-12T01:20:02.768 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: Updating smithi104:/var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:20:02.768 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/2207965967' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-12T01:20:02.768 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T01:20:02.768 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:02.768 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:02.768 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:02.768 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-09-12T01:20:02.768 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: Updating smithi104:/var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T01:20:02.768 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:02.768 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:20:02.768 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:02.768 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:20:02.769 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:02.769 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T01:20:02.769 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:02.769 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:02.769 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi001:172.21.15.1=a;smithi053:172.21.15.53=b;smithi104:172.21.15.104=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:20:02.769 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: Saving service mon spec with placement smithi001:172.21.15.1=a;smithi053:172.21.15.53=b;smithi104:172.21.15.104=c;count:3 2024-09-12T01:20:02.769 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: Deploying daemon mon.c on smithi104 2024-09-12T01:20:02.769 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:02.769 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='client.? 172.21.15.104:0/2146903470' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T01:20:02.769 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:02.769 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:20:02.769 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:20:02.769 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:20:02.769 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: mon.a calling monitor election 2024-09-12T01:20:02.770 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:02.770 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:20:02.770 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:20:02.770 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: mon.c calling monitor election 2024-09-12T01:20:02.770 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:02.770 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:20:02.770 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:20:02.770 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:20:02.770 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T01:20:02.770 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: monmap epoch 2 2024-09-12T01:20:02.770 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: fsid c0e4f180-70a4-11ef-bcea-c7b262605968 2024-09-12T01:20:02.770 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: last_changed 2024-09-12T01:19:53.648242+0000 2024-09-12T01:20:02.770 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: created 2024-09-12T01:18:15.215734+0000 2024-09-12T01:20:02.770 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: min_mon_release 19 (squid) 2024-09-12T01:20:02.770 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: election_strategy: 1 2024-09-12T01:20:02.771 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-09-12T01:20:02.771 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: 1: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.c 2024-09-12T01:20:02.771 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: fsmap 2024-09-12T01:20:02.771 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T01:20:02.771 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: mgrmap e13: x(active, since 44s) 2024-09-12T01:20:02.771 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: overall HEALTH_OK 2024-09-12T01:20:02.771 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:02.771 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:02.771 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:02.772 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T01:20:02.772 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:02.772 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:02.773 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: Deploying daemon mon.b on smithi053 2024-09-12T01:20:02.773 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='client.? 172.21.15.104:0/3620326590' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T01:20:02.773 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:20:02.773 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: overall HEALTH_OK 2024-09-12T01:20:02.773 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:02.773 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 ceph-mon[35158]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-12T01:20:02.773 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 bash[35139]: a344fc8ebfb66e31006082877ac60329fe62e38c30afe02d745d460e84c957cf 2024-09-12T01:20:02.773 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:02 smithi053 systemd[1]: Started Ceph mon.b for c0e4f180-70a4-11ef-bcea-c7b262605968. 2024-09-12T01:20:03.850 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T01:20:03.850 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T01:20:04.112 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:20:07.476 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.476+0000 7efe6d0dd640 1 -- 172.21.15.104:0/668750081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe54003530 msgr2=0x7efe540059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:20:07.477 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.476+0000 7efe6d0dd640 1 --2- 172.21.15.104:0/668750081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe54003530 0x7efe540059c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7efe681016c0 tx=0x7efe5802bce0 comp rx=0 tx=0).stop 2024-09-12T01:20:07.477 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.476+0000 7efe6d0dd640 1 -- 172.21.15.104:0/668750081 shutdown_connections 2024-09-12T01:20:07.477 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.476+0000 7efe6d0dd640 1 --2- 172.21.15.104:0/668750081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe54003530 0x7efe540059c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:20:07.477 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.476+0000 7efe6d0dd640 1 -- 172.21.15.104:0/668750081 >> 172.21.15.104:0/668750081 conn(0x7efe680fc280 msgr2=0x7efe680fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:20:07.477 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.476+0000 7efe6d0dd640 1 -- 172.21.15.104:0/668750081 shutdown_connections 2024-09-12T01:20:07.477 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.476+0000 7efe6d0dd640 1 -- 172.21.15.104:0/668750081 wait complete. 2024-09-12T01:20:07.478 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.477+0000 7efe6d0dd640 1 Processor -- start 2024-09-12T01:20:07.482 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.478+0000 7efe6d0dd640 1 -- start start 2024-09-12T01:20:07.482 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.478+0000 7efe6d0dd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe54003530 0x7efe68102210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:20:07.482 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.478+0000 7efe6d0dd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe68102750 0x7efe68102bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:20:07.482 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.478+0000 7efe6d0dd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe681a3590 0x7efe681a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:20:07.483 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.478+0000 7efe6d0dd640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efe68108680 con 0x7efe681a3590 2024-09-12T01:20:07.483 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.478+0000 7efe6d0dd640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7efe68108500 con 0x7efe68102750 2024-09-12T01:20:07.483 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.478+0000 7efe6d0dd640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efe68108800 con 0x7efe54003530 2024-09-12T01:20:07.483 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.478+0000 7efe66d9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe54003530 0x7efe68102210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:20:07.483 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.478+0000 7efe66d9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe54003530 0x7efe68102210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38532/0 (socket says 172.21.15.104:38532) 2024-09-12T01:20:07.483 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.478+0000 7efe66d9d640 1 -- 172.21.15.104:0/1955818431 learned_addr learned my addr 172.21.15.104:0/1955818431 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-09-12T01:20:07.484 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.478+0000 7efe6759e640 1 --2- 172.21.15.104:0/1955818431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe681a3590 0x7efe681a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:20:07.484 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.478+0000 7efe66d9d640 1 -- 172.21.15.104:0/1955818431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe54003530 msgr2=0x7efe68102210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-12T01:20:07.484 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.478+0000 7efe66d9d640 1 -- 172.21.15.104:0/1955818431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe54003530 msgr2=0x7efe68102210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T01:20:07.484 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.478+0000 7efe66d9d640 1 --2- 172.21.15.104:0/1955818431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe54003530 0x7efe68102210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T01:20:07.485 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.478+0000 7efe66d9d640 1 --2- 172.21.15.104:0/1955818431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe54003530 0x7efe68102210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T01:20:07.485 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.478+0000 7efe6659c640 1 --2- 172.21.15.104:0/1955818431 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe68102750 0x7efe68102bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:20:07.485 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.479+0000 7efe6659c640 1 -- 172.21.15.104:0/1955818431 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe68102750 msgr2=0x7efe68102bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-12T01:20:07.485 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.479+0000 7efe6659c640 1 -- 172.21.15.104:0/1955818431 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe68102750 msgr2=0x7efe68102bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T01:20:07.485 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.479+0000 7efe6659c640 1 --2- 172.21.15.104:0/1955818431 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe68102750 0x7efe68102bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T01:20:07.485 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.479+0000 7efe6659c640 1 --2- 172.21.15.104:0/1955818431 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe68102750 0x7efe68102bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T01:20:07.486 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.479+0000 7efe6759e640 1 -- 172.21.15.104:0/1955818431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe54003530 msgr2=0x7efe68102210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T01:20:07.486 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.479+0000 7efe6759e640 1 --2- 172.21.15.104:0/1955818431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe54003530 0x7efe68102210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:20:07.486 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.479+0000 7efe6759e640 1 -- 172.21.15.104:0/1955818431 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe68102750 msgr2=0x7efe68102bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T01:20:07.486 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.479+0000 7efe6759e640 1 --2- 172.21.15.104:0/1955818431 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe68102750 0x7efe68102bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:20:07.486 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.479+0000 7efe6759e640 1 -- 172.21.15.104:0/1955818431 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe681a6050 con 0x7efe681a3590 2024-09-12T01:20:07.486 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.479+0000 7efe6759e640 1 --2- 172.21.15.104:0/1955818431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe681a3590 0x7efe681a5980 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efe6000bd50 tx=0x7efe6000be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:20:07.487 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.481+0000 7efe5ffff640 1 -- 172.21.15.104:0/1955818431 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe60019070 con 0x7efe681a3590 2024-09-12T01:20:07.487 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.481+0000 7efe5ffff640 1 -- 172.21.15.104:0/1955818431 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe60014920 con 0x7efe681a3590 2024-09-12T01:20:07.487 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.481+0000 7efe5ffff640 1 -- 172.21.15.104:0/1955818431 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe60012990 con 0x7efe681a3590 2024-09-12T01:20:07.487 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.481+0000 7efe6d0dd640 1 -- 172.21.15.104:0/1955818431 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe681a6340 con 0x7efe681a3590 2024-09-12T01:20:07.487 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.481+0000 7efe6d0dd640 1 -- 172.21.15.104:0/1955818431 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe681a6830 con 0x7efe681a3590 2024-09-12T01:20:07.487 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.482+0000 7efe5ffff640 1 -- 172.21.15.104:0/1955818431 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7efe6001c070 con 0x7efe681a3590 2024-09-12T01:20:07.488 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.482+0000 7efe6d0dd640 1 -- 172.21.15.104:0/1955818431 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe681097f0 con 0x7efe681a3590 2024-09-12T01:20:07.488 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.482+0000 7efe5ffff640 1 --2- 172.21.15.104:0/1955818431 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7efe480394f0 0x7efe4803b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:20:07.488 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.483+0000 7efe5ffff640 1 -- 172.21.15.104:0/1955818431 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7efe6004bf30 con 0x7efe681a3590 2024-09-12T01:20:07.488 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.483+0000 7efe66d9d640 1 --2- 172.21.15.104:0/1955818431 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7efe480394f0 0x7efe4803b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:20:07.489 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.483+0000 7efe66d9d640 1 --2- 172.21.15.104:0/1955818431 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7efe480394f0 0x7efe4803b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efe58005940 tx=0x7efe580058b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:20:07.491 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.492+0000 7efe5ffff640 1 -- 172.21.15.104:0/1955818431 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efe60013c30 con 0x7efe681a3590 2024-09-12T01:20:07.731 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.732+0000 7efe6d0dd640 1 -- 172.21.15.104:0/1955818431 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7efe681a6bc0 con 0x7efe681a3590 2024-09-12T01:20:07.732 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:07 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:20:07.732 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:07 smithi104 ceph-mon[35219]: mon.a calling monitor election 2024-09-12T01:20:07.732 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:07 smithi104 ceph-mon[35219]: mon.c calling monitor election 2024-09-12T01:20:07.732 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:07 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:20:07.732 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:07 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:20:07.732 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:07 smithi104 ceph-mon[35219]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:07.732 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:07 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:20:07.732 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:07 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:20:07.733 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:07 smithi104 ceph-mon[35219]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:07.733 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:07 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:20:07.733 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:07 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:20:07.733 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:07 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:20:07.734 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:07 smithi104 ceph-mon[35219]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T01:20:07.734 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:07 smithi104 ceph-mon[35219]: monmap epoch 3 2024-09-12T01:20:07.734 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:07 smithi104 ceph-mon[35219]: fsid c0e4f180-70a4-11ef-bcea-c7b262605968 2024-09-12T01:20:07.734 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:07 smithi104 ceph-mon[35219]: last_changed 2024-09-12T01:20:02.419593+0000 2024-09-12T01:20:07.734 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:07 smithi104 ceph-mon[35219]: created 2024-09-12T01:18:15.215734+0000 2024-09-12T01:20:07.734 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:07 smithi104 ceph-mon[35219]: min_mon_release 19 (squid) 2024-09-12T01:20:07.734 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:07 smithi104 ceph-mon[35219]: election_strategy: 1 2024-09-12T01:20:07.734 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:07 smithi104 ceph-mon[35219]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-09-12T01:20:07.734 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:07 smithi104 ceph-mon[35219]: 1: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.c 2024-09-12T01:20:07.734 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:07 smithi104 ceph-mon[35219]: 2: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.b 2024-09-12T01:20:07.735 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:07 smithi104 ceph-mon[35219]: fsmap 2024-09-12T01:20:07.735 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:07 smithi104 ceph-mon[35219]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T01:20:07.735 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:07 smithi104 ceph-mon[35219]: mgrmap e13: x(active, since 52s) 2024-09-12T01:20:07.735 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:07 smithi104 ceph-mon[35219]: overall HEALTH_OK 2024-09-12T01:20:07.735 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:07 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:07.735 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:07 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:07.735 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:07 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:07.735 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:07 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:07.735 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.733+0000 7efe5ffff640 1 -- 172.21.15.104:0/1955818431 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1275 (secure 0 0 0) 0x7efe60014b40 con 0x7efe681a3590 2024-09-12T01:20:07.736 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-12T01:20:07.736 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":3,"fsid":"c0e4f180-70a4-11ef-bcea-c7b262605968","modified":"2024-09-12T01:20:02.419593Z","created":"2024-09-12T01:18:15.215734Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:3300","nonce":0},{"type":"v1","addr":"172.21.15.104:6789","nonce":0}]},"addr":"172.21.15.104:6789/0","public_addr":"172.21.15.104:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:3300","nonce":0},{"type":"v1","addr":"172.21.15.53:6789","nonce":0}]},"addr":"172.21.15.53:6789/0","public_addr":"172.21.15.53:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-12T01:20:07.737 INFO:teuthology.orchestra.run.smithi104.stderr:dumped monmap epoch 3 2024-09-12T01:20:07.737 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.735+0000 7efe6d0dd640 1 -- 172.21.15.104:0/1955818431 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7efe480394f0 msgr2=0x7efe4803b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:20:07.737 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.735+0000 7efe6d0dd640 1 --2- 172.21.15.104:0/1955818431 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7efe480394f0 0x7efe4803b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efe58005940 tx=0x7efe580058b0 comp rx=0 tx=0).stop 2024-09-12T01:20:07.737 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.735+0000 7efe6d0dd640 1 -- 172.21.15.104:0/1955818431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe681a3590 msgr2=0x7efe681a5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:20:07.737 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.735+0000 7efe6d0dd640 1 --2- 172.21.15.104:0/1955818431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe681a3590 0x7efe681a5980 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efe6000bd50 tx=0x7efe6000be40 comp rx=0 tx=0).stop 2024-09-12T01:20:07.738 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.735+0000 7efe6d0dd640 1 -- 172.21.15.104:0/1955818431 shutdown_connections 2024-09-12T01:20:07.738 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.735+0000 7efe6d0dd640 1 --2- 172.21.15.104:0/1955818431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe681a3590 0x7efe681a5980 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:20:07.738 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.735+0000 7efe6d0dd640 1 --2- 172.21.15.104:0/1955818431 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe68102750 0x7efe68102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:20:07.738 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.735+0000 7efe6d0dd640 1 --2- 172.21.15.104:0/1955818431 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7efe480394f0 0x7efe4803b9b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:20:07.738 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.735+0000 7efe6d0dd640 1 --2- 172.21.15.104:0/1955818431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe54003530 0x7efe68102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:20:07.738 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.735+0000 7efe6d0dd640 1 -- 172.21.15.104:0/1955818431 >> 172.21.15.104:0/1955818431 conn(0x7efe680fc280 msgr2=0x7efe68105940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:20:07.738 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.735+0000 7efe6d0dd640 1 -- 172.21.15.104:0/1955818431 shutdown_connections 2024-09-12T01:20:07.738 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:07.735+0000 7efe6d0dd640 1 -- 172.21.15.104:0/1955818431 wait complete. 2024-09-12T01:20:07.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:07 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:20:07.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:07 smithi001 ceph-mon[30387]: mon.a calling monitor election 2024-09-12T01:20:07.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:07 smithi001 ceph-mon[30387]: mon.c calling monitor election 2024-09-12T01:20:07.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:07 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:20:07.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:07 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:20:07.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:07 smithi001 ceph-mon[30387]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:07.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:07 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:20:07.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:07 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:20:07.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:07 smithi001 ceph-mon[30387]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:07.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:07 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:20:07.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:07 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:20:07.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:07 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:20:07.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:07 smithi001 ceph-mon[30387]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T01:20:07.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:07 smithi001 ceph-mon[30387]: monmap epoch 3 2024-09-12T01:20:07.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:07 smithi001 ceph-mon[30387]: fsid c0e4f180-70a4-11ef-bcea-c7b262605968 2024-09-12T01:20:07.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:07 smithi001 ceph-mon[30387]: last_changed 2024-09-12T01:20:02.419593+0000 2024-09-12T01:20:07.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:07 smithi001 ceph-mon[30387]: created 2024-09-12T01:18:15.215734+0000 2024-09-12T01:20:07.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:07 smithi001 ceph-mon[30387]: min_mon_release 19 (squid) 2024-09-12T01:20:07.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:07 smithi001 ceph-mon[30387]: election_strategy: 1 2024-09-12T01:20:07.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:07 smithi001 ceph-mon[30387]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-09-12T01:20:07.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:07 smithi001 ceph-mon[30387]: 1: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.c 2024-09-12T01:20:07.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:07 smithi001 ceph-mon[30387]: 2: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.b 2024-09-12T01:20:07.930 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:07 smithi001 ceph-mon[30387]: fsmap 2024-09-12T01:20:07.930 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:07 smithi001 ceph-mon[30387]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T01:20:07.930 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:07 smithi001 ceph-mon[30387]: mgrmap e13: x(active, since 52s) 2024-09-12T01:20:07.930 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:07 smithi001 ceph-mon[30387]: overall HEALTH_OK 2024-09-12T01:20:07.930 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:07 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:07.930 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:07 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:07.930 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:07 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:07.930 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:07 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:08.249 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-12T01:20:08.249 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-12T01:20:08.631 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:20:08.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:08 smithi001 ceph-mon[30387]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:08.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:08 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:20:08.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:08 smithi001 ceph-mon[30387]: from='client.? 172.21.15.104:0/1955818431' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T01:20:08.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:08 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:20:08.784 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:08 smithi104 ceph-mon[35219]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:08.784 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:08 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:20:08.784 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:08 smithi104 ceph-mon[35219]: from='client.? 172.21.15.104:0/1955818431' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T01:20:08.784 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:08 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:20:09.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.166+0000 7fed07e00640 1 -- 172.21.15.1:0/3811645927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed08101610 msgr2=0x7fed081019f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:20:09.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.166+0000 7fed07e00640 1 --2- 172.21.15.1:0/3811645927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed08101610 0x7fed081019f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fecf8009880 tx=0x7fecf802a5d0 comp rx=0 tx=0).stop 2024-09-12T01:20:09.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.166+0000 7fed07e00640 1 -- 172.21.15.1:0/3811645927 shutdown_connections 2024-09-12T01:20:09.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.166+0000 7fed07e00640 1 --2- 172.21.15.1:0/3811645927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed08101610 0x7fed081019f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:20:09.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.166+0000 7fed07e00640 1 -- 172.21.15.1:0/3811645927 >> 172.21.15.1:0/3811645927 conn(0x7fed080fd0b0 msgr2=0x7fed080ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:20:09.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.166+0000 7fed07e00640 1 -- 172.21.15.1:0/3811645927 shutdown_connections 2024-09-12T01:20:09.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.166+0000 7fed07e00640 1 -- 172.21.15.1:0/3811645927 wait complete. 2024-09-12T01:20:09.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.167+0000 7fed07e00640 1 Processor -- start 2024-09-12T01:20:09.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.167+0000 7fed07e00640 1 -- start start 2024-09-12T01:20:09.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.168+0000 7fed07e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed08101610 0x7fed08199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:20:09.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.168+0000 7fed07e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed08199980 0x7fed0819dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:20:09.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.168+0000 7fed07e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed0819e250 0x7fed0819e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:20:09.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.168+0000 7fed07e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fed0810ba10 con 0x7fed08101610 2024-09-12T01:20:09.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.168+0000 7fed07e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fed0810b890 con 0x7fed0819e250 2024-09-12T01:20:09.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.168+0000 7fed07e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fed0810bb90 con 0x7fed08199980 2024-09-12T01:20:09.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.168+0000 7fed06000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed08199980 0x7fed0819dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:20:09.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.168+0000 7fed06a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed08101610 0x7fed08199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:20:09.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.168+0000 7fed06000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed08199980 0x7fed0819dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:60280/0 (socket says 172.21.15.1:60280) 2024-09-12T01:20:09.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.168+0000 7fed06a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed08101610 0x7fed08199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55784/0 (socket says 172.21.15.1:55784) 2024-09-12T01:20:09.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.168+0000 7fed06000640 1 -- 172.21.15.1:0/2703679621 learned_addr learned my addr 172.21.15.1:0/2703679621 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:20:09.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.169+0000 7fed07400640 1 --2- 172.21.15.1:0/2703679621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed0819e250 0x7fed0819e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:20:09.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.169+0000 7fed07400640 1 -- 172.21.15.1:0/2703679621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed0819e250 msgr2=0x7fed0819e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-12T01:20:09.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.169+0000 7fed07400640 1 -- 172.21.15.1:0/2703679621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed0819e250 msgr2=0x7fed0819e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T01:20:09.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.169+0000 7fed06000640 1 -- 172.21.15.1:0/2703679621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed0819e250 msgr2=0x7fed0819e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:20:09.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.169+0000 7fed07400640 1 --2- 172.21.15.1:0/2703679621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed0819e250 0x7fed0819e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T01:20:09.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.169+0000 7fed07400640 1 --2- 172.21.15.1:0/2703679621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed0819e250 0x7fed0819e700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T01:20:09.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.169+0000 7fed06000640 1 --2- 172.21.15.1:0/2703679621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed0819e250 0x7fed0819e700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:20:09.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.169+0000 7fed06000640 1 -- 172.21.15.1:0/2703679621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed08101610 msgr2=0x7fed08199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:20:09.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.169+0000 7fed06000640 1 --2- 172.21.15.1:0/2703679621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed08101610 0x7fed08199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:20:09.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.169+0000 7fed06000640 1 -- 172.21.15.1:0/2703679621 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed081a5fc0 con 0x7fed08199980 2024-09-12T01:20:09.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.169+0000 7fed06a00640 1 --2- 172.21.15.1:0/2703679621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed08101610 0x7fed08199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:20:09.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.169+0000 7fed06000640 1 --2- 172.21.15.1:0/2703679621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed08199980 0x7fed0819dd10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fecf400b520 tx=0x7fecf400b9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:20:09.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.169+0000 7fecff400640 1 -- 172.21.15.1:0/2703679621 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf4004230 con 0x7fed08199980 2024-09-12T01:20:09.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.170+0000 7fecff400640 1 -- 172.21.15.1:0/2703679621 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecf40043d0 con 0x7fed08199980 2024-09-12T01:20:09.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.170+0000 7fecff400640 1 -- 172.21.15.1:0/2703679621 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf4003d60 con 0x7fed08199980 2024-09-12T01:20:09.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.170+0000 7fed07e00640 1 -- 172.21.15.1:0/2703679621 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed081a61f0 con 0x7fed08199980 2024-09-12T01:20:09.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.170+0000 7fed07e00640 1 -- 172.21.15.1:0/2703679621 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed081a66b0 con 0x7fed08199980 2024-09-12T01:20:09.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.171+0000 7fecff400640 1 -- 172.21.15.1:0/2703679621 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fecf400e070 con 0x7fed08199980 2024-09-12T01:20:09.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.171+0000 7fed07e00640 1 -- 172.21.15.1:0/2703679621 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed08101a70 con 0x7fed08199980 2024-09-12T01:20:09.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.172+0000 7fecff400640 1 --2- 172.21.15.1:0/2703679621 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fecec0394f0 0x7fecec03b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:20:09.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.172+0000 7fecff400640 1 -- 172.21.15.1:0/2703679621 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fecf404baf0 con 0x7fed08199980 2024-09-12T01:20:09.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.172+0000 7fed06a00640 1 --2- 172.21.15.1:0/2703679621 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fecec0394f0 0x7fecec03b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:20:09.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.172+0000 7fed06a00640 1 --2- 172.21.15.1:0/2703679621 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fecec0394f0 0x7fecec03b9b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fecf8009230 tx=0x7fecf8005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:20:09.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.176+0000 7fecff400640 1 -- 172.21.15.1:0/2703679621 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fecf4014020 con 0x7fed08199980 2024-09-12T01:20:09.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.424+0000 7fed07e00640 1 -- 172.21.15.1:0/2703679621 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fed081a6a10 con 0x7fed08199980 2024-09-12T01:20:09.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.424+0000 7fecff400640 1 -- 172.21.15.1:0/2703679621 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+271 (secure 0 0 0) 0x7fecf401fde0 con 0x7fed08199980 2024-09-12T01:20:09.424 INFO:teuthology.orchestra.run.smithi001.stdout:# minimal ceph.conf for c0e4f180-70a4-11ef-bcea-c7b262605968 2024-09-12T01:20:09.424 INFO:teuthology.orchestra.run.smithi001.stdout:[global] 2024-09-12T01:20:09.424 INFO:teuthology.orchestra.run.smithi001.stdout: fsid = c0e4f180-70a4-11ef-bcea-c7b262605968 2024-09-12T01:20:09.424 INFO:teuthology.orchestra.run.smithi001.stdout: mon_host = [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] 2024-09-12T01:20:09.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.426+0000 7fed07e00640 1 -- 172.21.15.1:0/2703679621 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fecec0394f0 msgr2=0x7fecec03b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:20:09.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.426+0000 7fed07e00640 1 --2- 172.21.15.1:0/2703679621 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fecec0394f0 0x7fecec03b9b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fecf8009230 tx=0x7fecf8005a40 comp rx=0 tx=0).stop 2024-09-12T01:20:09.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.426+0000 7fed07e00640 1 -- 172.21.15.1:0/2703679621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed08199980 msgr2=0x7fed0819dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:20:09.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.426+0000 7fed07e00640 1 --2- 172.21.15.1:0/2703679621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed08199980 0x7fed0819dd10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fecf400b520 tx=0x7fecf400b9e0 comp rx=0 tx=0).stop 2024-09-12T01:20:09.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.427+0000 7fed07e00640 1 -- 172.21.15.1:0/2703679621 shutdown_connections 2024-09-12T01:20:09.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.427+0000 7fed07e00640 1 --2- 172.21.15.1:0/2703679621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed0819e250 0x7fed0819e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:20:09.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.427+0000 7fed07e00640 1 --2- 172.21.15.1:0/2703679621 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fecec0394f0 0x7fecec03b9b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:20:09.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.427+0000 7fed07e00640 1 --2- 172.21.15.1:0/2703679621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed08199980 0x7fed0819dd10 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:20:09.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.427+0000 7fed07e00640 1 --2- 172.21.15.1:0/2703679621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed08101610 0x7fed08199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:20:09.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.427+0000 7fed07e00640 1 -- 172.21.15.1:0/2703679621 >> 172.21.15.1:0/2703679621 conn(0x7fed080fd0b0 msgr2=0x7fed08108f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:20:09.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.427+0000 7fed07e00640 1 -- 172.21.15.1:0/2703679621 shutdown_connections 2024-09-12T01:20:09.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:09.427+0000 7fed07e00640 1 -- 172.21.15.1:0/2703679621 wait complete. 2024-09-12T01:20:09.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:09 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:20:09.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:09 smithi053 ceph-mon[35158]: mon.a calling monitor election 2024-09-12T01:20:09.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:09 smithi053 ceph-mon[35158]: mon.c calling monitor election 2024-09-12T01:20:09.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:09 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:20:09.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:09 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:20:09.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:09 smithi053 ceph-mon[35158]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:09.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:09 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:20:09.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:09 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:20:09.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:09 smithi053 ceph-mon[35158]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:09.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:09 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:20:09.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:09 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:20:09.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:09 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:20:09.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:09 smithi053 ceph-mon[35158]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T01:20:09.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:09 smithi053 ceph-mon[35158]: monmap epoch 3 2024-09-12T01:20:09.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:09 smithi053 ceph-mon[35158]: fsid c0e4f180-70a4-11ef-bcea-c7b262605968 2024-09-12T01:20:09.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:09 smithi053 ceph-mon[35158]: last_changed 2024-09-12T01:20:02.419593+0000 2024-09-12T01:20:09.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:09 smithi053 ceph-mon[35158]: created 2024-09-12T01:18:15.215734+0000 2024-09-12T01:20:09.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:09 smithi053 ceph-mon[35158]: min_mon_release 19 (squid) 2024-09-12T01:20:09.742 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:09 smithi053 ceph-mon[35158]: election_strategy: 1 2024-09-12T01:20:09.742 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:09 smithi053 ceph-mon[35158]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-09-12T01:20:09.742 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:09 smithi053 ceph-mon[35158]: 1: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.c 2024-09-12T01:20:09.742 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:09 smithi053 ceph-mon[35158]: 2: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.b 2024-09-12T01:20:09.742 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:09 smithi053 ceph-mon[35158]: fsmap 2024-09-12T01:20:09.742 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:09 smithi053 ceph-mon[35158]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T01:20:09.742 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:09 smithi053 ceph-mon[35158]: mgrmap e13: x(active, since 52s) 2024-09-12T01:20:09.742 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:09 smithi053 ceph-mon[35158]: overall HEALTH_OK 2024-09-12T01:20:09.742 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:09 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:09.742 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:09 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:09.742 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:09 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:09.743 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:09 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:09.743 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:09 smithi053 ceph-mon[35158]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:09.743 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:09 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:20:09.743 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:09 smithi053 ceph-mon[35158]: from='client.? 172.21.15.104:0/1955818431' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T01:20:09.743 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:09 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:20:10.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:10 smithi001 ceph-mon[30387]: mon.b calling monitor election 2024-09-12T01:20:10.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:10 smithi001 ceph-mon[30387]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:10.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:10 smithi001 ceph-mon[30387]: Updating smithi001:/etc/ceph/ceph.conf 2024-09-12T01:20:10.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:10 smithi001 ceph-mon[30387]: Updating smithi053:/etc/ceph/ceph.conf 2024-09-12T01:20:10.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:10 smithi001 ceph-mon[30387]: Updating smithi104:/etc/ceph/ceph.conf 2024-09-12T01:20:10.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:10 smithi001 ceph-mon[30387]: Updating smithi104:/var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:20:10.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:10 smithi001 ceph-mon[30387]: Updating smithi053:/var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:20:10.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:10 smithi001 ceph-mon[30387]: mon.b calling monitor election 2024-09-12T01:20:10.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:10 smithi001 ceph-mon[30387]: mon.c calling monitor election 2024-09-12T01:20:10.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:10 smithi001 ceph-mon[30387]: mon.a calling monitor election 2024-09-12T01:20:10.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:10 smithi001 ceph-mon[30387]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-12T01:20:10.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:10 smithi001 ceph-mon[30387]: monmap epoch 3 2024-09-12T01:20:10.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:10 smithi001 ceph-mon[30387]: fsid c0e4f180-70a4-11ef-bcea-c7b262605968 2024-09-12T01:20:10.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:10 smithi001 ceph-mon[30387]: last_changed 2024-09-12T01:20:02.419593+0000 2024-09-12T01:20:10.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:10 smithi001 ceph-mon[30387]: created 2024-09-12T01:18:15.215734+0000 2024-09-12T01:20:10.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:10 smithi001 ceph-mon[30387]: min_mon_release 19 (squid) 2024-09-12T01:20:10.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:10 smithi001 ceph-mon[30387]: election_strategy: 1 2024-09-12T01:20:10.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:10 smithi001 ceph-mon[30387]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-09-12T01:20:10.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:10 smithi001 ceph-mon[30387]: 1: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.c 2024-09-12T01:20:10.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:10 smithi001 ceph-mon[30387]: 2: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.b 2024-09-12T01:20:10.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:10 smithi001 ceph-mon[30387]: fsmap 2024-09-12T01:20:10.180 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:10 smithi001 ceph-mon[30387]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T01:20:10.180 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:10 smithi001 ceph-mon[30387]: mgrmap e13: x(active, since 54s) 2024-09-12T01:20:10.180 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:10 smithi001 ceph-mon[30387]: overall HEALTH_OK 2024-09-12T01:20:10.180 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:10 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:10.180 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:10 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:10.180 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:10 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:10.180 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:10 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:10.180 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:10 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:10.254 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-12T01:20:10.254 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-12T01:20:10.254 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/ceph/ceph.conf 2024-09-12T01:20:10.318 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-12T01:20:10.318 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T01:20:10.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:10 smithi104 ceph-mon[35219]: mon.b calling monitor election 2024-09-12T01:20:10.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:10 smithi104 ceph-mon[35219]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:10.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:10 smithi104 ceph-mon[35219]: Updating smithi001:/etc/ceph/ceph.conf 2024-09-12T01:20:10.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:10 smithi104 ceph-mon[35219]: Updating smithi053:/etc/ceph/ceph.conf 2024-09-12T01:20:10.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:10 smithi104 ceph-mon[35219]: Updating smithi104:/etc/ceph/ceph.conf 2024-09-12T01:20:10.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:10 smithi104 ceph-mon[35219]: Updating smithi104:/var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:20:10.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:10 smithi104 ceph-mon[35219]: Updating smithi053:/var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:20:10.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:10 smithi104 ceph-mon[35219]: mon.b calling monitor election 2024-09-12T01:20:10.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:10 smithi104 ceph-mon[35219]: mon.c calling monitor election 2024-09-12T01:20:10.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:10 smithi104 ceph-mon[35219]: mon.a calling monitor election 2024-09-12T01:20:10.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:10 smithi104 ceph-mon[35219]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-12T01:20:10.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:10 smithi104 ceph-mon[35219]: monmap epoch 3 2024-09-12T01:20:10.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:10 smithi104 ceph-mon[35219]: fsid c0e4f180-70a4-11ef-bcea-c7b262605968 2024-09-12T01:20:10.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:10 smithi104 ceph-mon[35219]: last_changed 2024-09-12T01:20:02.419593+0000 2024-09-12T01:20:10.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:10 smithi104 ceph-mon[35219]: created 2024-09-12T01:18:15.215734+0000 2024-09-12T01:20:10.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:10 smithi104 ceph-mon[35219]: min_mon_release 19 (squid) 2024-09-12T01:20:10.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:10 smithi104 ceph-mon[35219]: election_strategy: 1 2024-09-12T01:20:10.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:10 smithi104 ceph-mon[35219]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-09-12T01:20:10.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:10 smithi104 ceph-mon[35219]: 1: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.c 2024-09-12T01:20:10.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:10 smithi104 ceph-mon[35219]: 2: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.b 2024-09-12T01:20:10.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:10 smithi104 ceph-mon[35219]: fsmap 2024-09-12T01:20:10.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:10 smithi104 ceph-mon[35219]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T01:20:10.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:10 smithi104 ceph-mon[35219]: mgrmap e13: x(active, since 54s) 2024-09-12T01:20:10.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:10 smithi104 ceph-mon[35219]: overall HEALTH_OK 2024-09-12T01:20:10.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:10 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:10.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:10 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:10.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:10 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:10.327 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:10 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:10.327 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:10 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:10.418 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-12T01:20:10.418 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/ceph/ceph.conf 2024-09-12T01:20:10.447 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:10 smithi053 ceph-mon[35158]: mon.b calling monitor election 2024-09-12T01:20:10.447 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:10 smithi053 ceph-mon[35158]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:10.447 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:10 smithi053 ceph-mon[35158]: Updating smithi001:/etc/ceph/ceph.conf 2024-09-12T01:20:10.447 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:10 smithi053 ceph-mon[35158]: Updating smithi053:/etc/ceph/ceph.conf 2024-09-12T01:20:10.448 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:10 smithi053 ceph-mon[35158]: Updating smithi104:/etc/ceph/ceph.conf 2024-09-12T01:20:10.448 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:10 smithi053 ceph-mon[35158]: Updating smithi104:/var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:20:10.448 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:10 smithi053 ceph-mon[35158]: Updating smithi053:/var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:20:10.448 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:10 smithi053 ceph-mon[35158]: mon.b calling monitor election 2024-09-12T01:20:10.448 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:10 smithi053 ceph-mon[35158]: mon.c calling monitor election 2024-09-12T01:20:10.448 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:10 smithi053 ceph-mon[35158]: mon.a calling monitor election 2024-09-12T01:20:10.448 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:10 smithi053 ceph-mon[35158]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-12T01:20:10.448 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:10 smithi053 ceph-mon[35158]: monmap epoch 3 2024-09-12T01:20:10.448 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:10 smithi053 ceph-mon[35158]: fsid c0e4f180-70a4-11ef-bcea-c7b262605968 2024-09-12T01:20:10.448 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:10 smithi053 ceph-mon[35158]: last_changed 2024-09-12T01:20:02.419593+0000 2024-09-12T01:20:10.448 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:10 smithi053 ceph-mon[35158]: created 2024-09-12T01:18:15.215734+0000 2024-09-12T01:20:10.448 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:10 smithi053 ceph-mon[35158]: min_mon_release 19 (squid) 2024-09-12T01:20:10.448 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:10 smithi053 ceph-mon[35158]: election_strategy: 1 2024-09-12T01:20:10.448 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:10 smithi053 ceph-mon[35158]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-09-12T01:20:10.449 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:10 smithi053 ceph-mon[35158]: 1: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.c 2024-09-12T01:20:10.449 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:10 smithi053 ceph-mon[35158]: 2: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.b 2024-09-12T01:20:10.449 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:10 smithi053 ceph-mon[35158]: fsmap 2024-09-12T01:20:10.449 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:10 smithi053 ceph-mon[35158]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T01:20:10.449 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:10 smithi053 ceph-mon[35158]: mgrmap e13: x(active, since 54s) 2024-09-12T01:20:10.449 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:10 smithi053 ceph-mon[35158]: overall HEALTH_OK 2024-09-12T01:20:10.449 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:10 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:10.449 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:10 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:10.449 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:10 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:10.449 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:10 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:10.449 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:10 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:10.452 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-12T01:20:10.452 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T01:20:10.523 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-12T01:20:10.523 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/ceph/ceph.conf 2024-09-12T01:20:10.556 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-12T01:20:10.556 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T01:20:10.625 INFO:tasks.cephadm:Adding mgr.x on smithi001 2024-09-12T01:20:10.626 INFO:tasks.cephadm:Adding mgr.y on smithi053 2024-09-12T01:20:10.626 INFO:tasks.cephadm:Adding mgr.z on smithi104 2024-09-12T01:20:10.626 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph orch apply mgr '3;smithi001=x;smithi053=y;smithi104=z' 2024-09-12T01:20:10.909 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:20:11.280 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:11 smithi104 ceph-mon[35219]: Updating smithi001:/var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:20:11.280 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:11 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:11.280 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:11 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:11.280 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:11 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:11.280 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:11 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:11.280 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:11 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:11.280 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:11 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:11.281 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:11 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:11.281 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:11 smithi104 ceph-mon[35219]: Reconfiguring mon.a (unknown last config time)... 2024-09-12T01:20:11.281 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:11 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T01:20:11.281 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:11 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T01:20:11.281 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:11 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:11.281 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:11 smithi104 ceph-mon[35219]: Reconfiguring daemon mon.a on smithi001 2024-09-12T01:20:11.281 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:11 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:20:11.364 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.362+0000 7fc1b2dc7640 1 -- 172.21.15.104:0/3754480947 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1ac0ff9a0 msgr2=0x7fc1ac0ffda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:20:11.364 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.362+0000 7fc1b2dc7640 1 --2- 172.21.15.104:0/3754480947 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1ac0ff9a0 0x7fc1ac0ffda0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0009880 tx=0x7fc1a002a5d0 comp rx=0 tx=0).stop 2024-09-12T01:20:11.364 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.362+0000 7fc1b2dc7640 1 -- 172.21.15.104:0/3754480947 shutdown_connections 2024-09-12T01:20:11.364 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.362+0000 7fc1b2dc7640 1 --2- 172.21.15.104:0/3754480947 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1ac0ff9a0 0x7fc1ac0ffda0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:20:11.364 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.362+0000 7fc1b2dc7640 1 -- 172.21.15.104:0/3754480947 >> 172.21.15.104:0/3754480947 conn(0x7fc1ac0fb510 msgr2=0x7fc1ac0fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:20:11.364 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.362+0000 7fc1b2dc7640 1 -- 172.21.15.104:0/3754480947 shutdown_connections 2024-09-12T01:20:11.364 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.362+0000 7fc1b2dc7640 1 -- 172.21.15.104:0/3754480947 wait complete. 2024-09-12T01:20:11.365 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.363+0000 7fc1b2dc7640 1 Processor -- start 2024-09-12T01:20:11.365 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.363+0000 7fc1b2dc7640 1 -- start start 2024-09-12T01:20:11.365 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.363+0000 7fc1b2dc7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1ac0ff9a0 0x7fc1ac196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:20:11.365 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.363+0000 7fc1b2dc7640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1ac197510 0x7fc1ac19c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:20:11.365 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.363+0000 7fc1b2dc7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1ac197990 0x7fc1ac19eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:20:11.365 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.363+0000 7fc1b2dc7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ac1077c0 con 0x7fc1ac197990 2024-09-12T01:20:11.365 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.363+0000 7fc1b2dc7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ac107640 con 0x7fc1ac0ff9a0 2024-09-12T01:20:11.365 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.363+0000 7fc1b2dc7640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ac107940 con 0x7fc1ac197510 2024-09-12T01:20:11.365 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.363+0000 7fc1abfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1ac197510 0x7fc1ac19c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:20:11.365 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.363+0000 7fc1abfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1ac197510 0x7fc1ac19c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38538/0 (socket says 172.21.15.104:38538) 2024-09-12T01:20:11.365 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.363+0000 7fc1abfff640 1 -- 172.21.15.104:0/738328975 learned_addr learned my addr 172.21.15.104:0/738328975 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-09-12T01:20:11.365 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.363+0000 7fc1b1364640 1 --2- 172.21.15.104:0/738328975 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1ac197990 0x7fc1ac19eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:20:11.365 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.363+0000 7fc1b0b63640 1 --2- 172.21.15.104:0/738328975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1ac0ff9a0 0x7fc1ac196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:20:11.366 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.363+0000 7fc1abfff640 1 -- 172.21.15.104:0/738328975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1ac0ff9a0 msgr2=0x7fc1ac196fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:20:11.366 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.363+0000 7fc1abfff640 1 --2- 172.21.15.104:0/738328975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1ac0ff9a0 0x7fc1ac196fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:20:11.366 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.363+0000 7fc1abfff640 1 -- 172.21.15.104:0/738328975 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1ac197990 msgr2=0x7fc1ac19eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:20:11.366 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.363+0000 7fc1abfff640 1 --2- 172.21.15.104:0/738328975 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1ac197990 0x7fc1ac19eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:20:11.366 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.363+0000 7fc1abfff640 1 -- 172.21.15.104:0/738328975 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1ac19f3d0 con 0x7fc1ac197510 2024-09-12T01:20:11.366 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.363+0000 7fc1b0b63640 1 --2- 172.21.15.104:0/738328975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1ac0ff9a0 0x7fc1ac196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:20:11.366 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.363+0000 7fc1abfff640 1 --2- 172.21.15.104:0/738328975 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1ac197510 0x7fc1ac19c580 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc19800b520 tx=0x7fc19800b9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:20:11.366 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.364+0000 7fc1a9ffb640 1 -- 172.21.15.104:0/738328975 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc198004230 con 0x7fc1ac197510 2024-09-12T01:20:11.366 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.364+0000 7fc1a9ffb640 1 -- 172.21.15.104:0/738328975 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1980043d0 con 0x7fc1ac197510 2024-09-12T01:20:11.366 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.364+0000 7fc1b2dc7640 1 -- 172.21.15.104:0/738328975 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1ac19f6c0 con 0x7fc1ac197510 2024-09-12T01:20:11.366 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.364+0000 7fc1a9ffb640 1 -- 172.21.15.104:0/738328975 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc198003d60 con 0x7fc1ac197510 2024-09-12T01:20:11.366 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.364+0000 7fc1b2dc7640 1 -- 172.21.15.104:0/738328975 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1ac19fc80 con 0x7fc1ac197510 2024-09-12T01:20:11.367 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.364+0000 7fc1a9ffb640 1 -- 172.21.15.104:0/738328975 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fc19800e070 con 0x7fc1ac197510 2024-09-12T01:20:11.367 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.365+0000 7fc1a9ffb640 1 --2- 172.21.15.104:0/738328975 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fc19c0394f0 0x7fc19c03b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:20:11.367 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.365+0000 7fc1a9ffb640 1 -- 172.21.15.104:0/738328975 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc19804ba70 con 0x7fc1ac197510 2024-09-12T01:20:11.367 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.365+0000 7fc1b2dc7640 1 -- 172.21.15.104:0/738328975 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1ac190580 con 0x7fc1ac197510 2024-09-12T01:20:11.367 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.365+0000 7fc1b0b63640 1 --2- 172.21.15.104:0/738328975 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fc19c0394f0 0x7fc19c03b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:20:11.367 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.366+0000 7fc1b0b63640 1 --2- 172.21.15.104:0/738328975 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fc19c0394f0 0x7fc19c03b9b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0000c00 tx=0x7fc1a00315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:20:11.370 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.370+0000 7fc1a9ffb640 1 -- 172.21.15.104:0/738328975 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc198014020 con 0x7fc1ac197510 2024-09-12T01:20:11.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:11 smithi001 ceph-mon[30387]: Updating smithi001:/var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:20:11.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:11 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:11.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:11 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:11.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:11 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:11.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:11 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:11.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:11 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:11.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:11 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:11.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:11 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:11.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:11 smithi001 ceph-mon[30387]: Reconfiguring mon.a (unknown last config time)... 2024-09-12T01:20:11.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:11 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T01:20:11.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:11 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T01:20:11.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:11 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:11.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:11 smithi001 ceph-mon[30387]: Reconfiguring daemon mon.a on smithi001 2024-09-12T01:20:11.429 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:11 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:20:11.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:11 smithi053 ceph-mon[35158]: Updating smithi001:/var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:20:11.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:11 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:11.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:11 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:11.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:11 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:11.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:11 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:11.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:11 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:11.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:11 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:11.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:11 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:11.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:11 smithi053 ceph-mon[35158]: Reconfiguring mon.a (unknown last config time)... 2024-09-12T01:20:11.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:11 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T01:20:11.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:11 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T01:20:11.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:11 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:11.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:11 smithi053 ceph-mon[35158]: Reconfiguring daemon mon.a on smithi001 2024-09-12T01:20:11.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:11 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:20:11.546 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.546+0000 7fc1b2dc7640 1 -- 172.21.15.104:0/738328975 --> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi001=x;smithi053=y;smithi104=z", "target": ["mon-mgr", ""]}) -- 0x7fc1ac062780 con 0x7fc19c0394f0 2024-09-12T01:20:11.552 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.552+0000 7fc1a9ffb640 1 -- 172.21.15.104:0/738328975 <== mgr.14150 v2:172.21.15.1:6800/1928662456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fc1ac062780 con 0x7fc19c0394f0 2024-09-12T01:20:11.552 INFO:teuthology.orchestra.run.smithi104.stdout:Scheduled mgr update... 2024-09-12T01:20:11.554 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.554+0000 7fc1b2dc7640 1 -- 172.21.15.104:0/738328975 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fc19c0394f0 msgr2=0x7fc19c03b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:20:11.554 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.554+0000 7fc1b2dc7640 1 --2- 172.21.15.104:0/738328975 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fc19c0394f0 0x7fc19c03b9b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0000c00 tx=0x7fc1a00315b0 comp rx=0 tx=0).stop 2024-09-12T01:20:11.554 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.554+0000 7fc1b2dc7640 1 -- 172.21.15.104:0/738328975 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1ac197510 msgr2=0x7fc1ac19c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:20:11.555 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.554+0000 7fc1b2dc7640 1 --2- 172.21.15.104:0/738328975 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1ac197510 0x7fc1ac19c580 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc19800b520 tx=0x7fc19800b9e0 comp rx=0 tx=0).stop 2024-09-12T01:20:11.555 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.554+0000 7fc1b2dc7640 1 -- 172.21.15.104:0/738328975 shutdown_connections 2024-09-12T01:20:11.555 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.554+0000 7fc1b2dc7640 1 --2- 172.21.15.104:0/738328975 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1ac197990 0x7fc1ac19eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:20:11.555 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.554+0000 7fc1b2dc7640 1 --2- 172.21.15.104:0/738328975 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fc19c0394f0 0x7fc19c03b9b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:20:11.555 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.554+0000 7fc1b2dc7640 1 --2- 172.21.15.104:0/738328975 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1ac197510 0x7fc1ac19c580 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:20:11.555 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.554+0000 7fc1b2dc7640 1 --2- 172.21.15.104:0/738328975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1ac0ff9a0 0x7fc1ac196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:20:11.555 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.554+0000 7fc1b2dc7640 1 -- 172.21.15.104:0/738328975 >> 172.21.15.104:0/738328975 conn(0x7fc1ac0fb510 msgr2=0x7fc1ac0fc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:20:11.555 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.554+0000 7fc1b2dc7640 1 -- 172.21.15.104:0/738328975 shutdown_connections 2024-09-12T01:20:11.555 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:20:11.554+0000 7fc1b2dc7640 1 -- 172.21.15.104:0/738328975 wait complete. 2024-09-12T01:20:12.087 DEBUG:teuthology.orchestra.run.smithi053:mgr.y> sudo journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mgr.y.service 2024-09-12T01:20:12.091 DEBUG:teuthology.orchestra.run.smithi104:mgr.z> sudo journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mgr.z.service 2024-09-12T01:20:12.093 INFO:tasks.cephadm:Deploying OSDs... 2024-09-12T01:20:12.093 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-12T01:20:12.093 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T01:20:12.135 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T01:20:12.135 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_1 2024-09-12T01:20:12.214 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T01:20:12.215 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T01:20:12.215 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-09-12T01:20:12.215 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:20:12.215 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T01:20:12.215 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-09-12 01:19:28.542384325 +0000 2024-09-12T01:20:12.215 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-09-12 01:14:30.931477913 +0000 2024-09-12T01:20:12.215 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-09-12 01:14:30.931477913 +0000 2024-09-12T01:20:12.215 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: 2024-09-12 01:14:30.931477913 +0000 2024-09-12T01:20:12.216 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T01:20:12.311 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2024-09-12T01:20:12.311 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2024-09-12T01:20:12.311 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 0.000174683 s, 2.9 MB/s 2024-09-12T01:20:12.314 DEBUG:teuthology.orchestra.run.smithi001:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T01:20:12.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:12 smithi104 ceph-mon[35219]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:12.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:12 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:12.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:12 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:12.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:12 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:12.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:12 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T01:20:12.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:12 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T01:20:12.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:12 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:12.352 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_2 2024-09-12T01:20:12.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:12 smithi001 ceph-mon[30387]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:12.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:12 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:12.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:12 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:12.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:12 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:12.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:12 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T01:20:12.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:12 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T01:20:12.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:12 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:12.434 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T01:20:12.434 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T01:20:12.434 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-09-12T01:20:12.434 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:20:12.435 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T01:20:12.435 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-09-12 01:19:28.586383763 +0000 2024-09-12T01:20:12.435 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-09-12 01:14:31.364472530 +0000 2024-09-12T01:20:12.435 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-09-12 01:14:31.364472530 +0000 2024-09-12T01:20:12.436 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: 2024-09-12 01:14:31.364472530 +0000 2024-09-12T01:20:12.436 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T01:20:12.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:12 smithi053 ceph-mon[35158]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:12.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:12 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:12.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:12 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:12.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:12 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:12.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:12 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T01:20:12.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:12 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T01:20:12.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:12 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:12.543 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2024-09-12T01:20:12.543 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2024-09-12T01:20:12.543 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 0.000185188 s, 2.8 MB/s 2024-09-12T01:20:12.546 DEBUG:teuthology.orchestra.run.smithi001:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T01:20:12.585 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_3 2024-09-12T01:20:12.661 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T01:20:12.661 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T01:20:12.661 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 6h/6d Inode: 672 Links: 1 2024-09-12T01:20:12.661 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:20:12.661 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T01:20:12.662 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-09-12 01:19:28.649382959 +0000 2024-09-12T01:20:12.662 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-09-12 01:14:31.783467321 +0000 2024-09-12T01:20:12.662 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-09-12 01:14:31.783467321 +0000 2024-09-12T01:20:12.662 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: 2024-09-12 01:14:31.783467321 +0000 2024-09-12T01:20:12.662 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T01:20:12.758 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2024-09-12T01:20:12.759 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2024-09-12T01:20:12.759 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 0.000182403 s, 2.8 MB/s 2024-09-12T01:20:12.761 DEBUG:teuthology.orchestra.run.smithi001:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T01:20:12.799 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_4 2024-09-12T01:20:12.878 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T01:20:12.878 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T01:20:12.878 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-09-12T01:20:12.878 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:20:12.878 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T01:20:12.878 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-09-12 01:19:28.692382410 +0000 2024-09-12T01:20:12.878 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-09-12 01:14:32.284461092 +0000 2024-09-12T01:20:12.878 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-09-12 01:14:32.284461092 +0000 2024-09-12T01:20:12.879 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: 2024-09-12 01:14:32.284461092 +0000 2024-09-12T01:20:12.879 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T01:20:12.971 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2024-09-12T01:20:12.971 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2024-09-12T01:20:12.971 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 0.000166982 s, 3.1 MB/s 2024-09-12T01:20:12.974 DEBUG:teuthology.orchestra.run.smithi001:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T01:20:13.011 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-12T01:20:13.011 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T01:20:13.030 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T01:20:13.030 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_1 2024-09-12T01:20:13.088 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T01:20:13.088 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T01:20:13.088 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-12T01:20:13.088 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:20:13.088 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T01:20:13.088 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-09-12 01:20:01.840627674 +0000 2024-09-12T01:20:13.089 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-09-12 01:13:55.110936511 +0000 2024-09-12T01:20:13.089 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-09-12 01:13:55.110936511 +0000 2024-09-12T01:20:13.089 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-09-12 01:13:55.110936511 +0000 2024-09-12T01:20:13.093 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T01:20:13.155 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-09-12T01:20:13.155 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-09-12T01:20:13.155 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000127686 s, 4.0 MB/s 2024-09-12T01:20:13.156 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T01:20:13.214 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_2 2024-09-12T01:20:13.270 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T01:20:13.271 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T01:20:13.271 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-12T01:20:13.271 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:20:13.271 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T01:20:13.271 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-09-12 01:20:01.841627655 +0000 2024-09-12T01:20:13.271 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-09-12 01:13:55.428930108 +0000 2024-09-12T01:20:13.271 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-09-12 01:13:55.428930108 +0000 2024-09-12T01:20:13.271 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-09-12 01:13:55.428930108 +0000 2024-09-12T01:20:13.271 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T01:20:13.336 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-09-12T01:20:13.336 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-09-12T01:20:13.336 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000115817 s, 4.4 MB/s 2024-09-12T01:20:13.337 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T01:20:13.396 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_3 2024-09-12T01:20:13.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:13 smithi001 ceph-mon[30387]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi001=x;smithi053=y;smithi104=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:20:13.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:13 smithi001 ceph-mon[30387]: Saving service mgr spec with placement smithi001=x;smithi053=y;smithi104=z;count:3 2024-09-12T01:20:13.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:13 smithi001 ceph-mon[30387]: Reconfiguring mgr.x (unknown last config time)... 2024-09-12T01:20:13.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:13 smithi001 ceph-mon[30387]: Reconfiguring daemon mgr.x on smithi001 2024-09-12T01:20:13.452 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T01:20:13.453 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T01:20:13.453 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-12T01:20:13.453 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:20:13.453 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T01:20:13.453 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-09-12 01:20:01.841627655 +0000 2024-09-12T01:20:13.453 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-09-12 01:13:55.743923765 +0000 2024-09-12T01:20:13.453 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-09-12 01:13:55.743923765 +0000 2024-09-12T01:20:13.453 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-09-12 01:13:55.743923765 +0000 2024-09-12T01:20:13.453 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T01:20:13.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:13 smithi053 ceph-mon[35158]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi001=x;smithi053=y;smithi104=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:20:13.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:13 smithi053 ceph-mon[35158]: Saving service mgr spec with placement smithi001=x;smithi053=y;smithi104=z;count:3 2024-09-12T01:20:13.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:13 smithi053 ceph-mon[35158]: Reconfiguring mgr.x (unknown last config time)... 2024-09-12T01:20:13.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:13 smithi053 ceph-mon[35158]: Reconfiguring daemon mgr.x on smithi001 2024-09-12T01:20:13.514 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-09-12T01:20:13.514 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-09-12T01:20:13.514 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000109551 s, 4.7 MB/s 2024-09-12T01:20:13.516 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T01:20:13.573 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_4 2024-09-12T01:20:13.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:13 smithi104 ceph-mon[35219]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi001=x;smithi053=y;smithi104=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:20:13.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:13 smithi104 ceph-mon[35219]: Saving service mgr spec with placement smithi001=x;smithi053=y;smithi104=z;count:3 2024-09-12T01:20:13.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:13 smithi104 ceph-mon[35219]: Reconfiguring mgr.x (unknown last config time)... 2024-09-12T01:20:13.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:13 smithi104 ceph-mon[35219]: Reconfiguring daemon mgr.x on smithi001 2024-09-12T01:20:13.630 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T01:20:13.630 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T01:20:13.631 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-12T01:20:13.631 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:20:13.631 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T01:20:13.631 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-09-12 01:20:01.841627655 +0000 2024-09-12T01:20:13.631 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-09-12 01:13:56.058917422 +0000 2024-09-12T01:20:13.631 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-09-12 01:13:56.058917422 +0000 2024-09-12T01:20:13.631 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-09-12 01:13:56.058917422 +0000 2024-09-12T01:20:13.631 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T01:20:13.695 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-09-12T01:20:13.695 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-09-12T01:20:13.695 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 9.5804e-05 s, 5.3 MB/s 2024-09-12T01:20:13.696 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T01:20:13.752 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-12T01:20:13.752 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T01:20:13.771 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T01:20:13.771 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_1 2024-09-12T01:20:13.830 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T01:20:13.830 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T01:20:13.831 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-12T01:20:13.831 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:20:13.831 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T01:20:13.831 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-09-12 01:19:52.955687561 +0000 2024-09-12T01:20:13.831 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-09-12 01:13:52.159341279 +0000 2024-09-12T01:20:13.831 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-09-12 01:13:52.159341279 +0000 2024-09-12T01:20:13.831 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-09-12 01:13:52.159341279 +0000 2024-09-12T01:20:13.832 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T01:20:13.899 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-09-12T01:20:13.900 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-09-12T01:20:13.900 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000360737 s, 1.4 MB/s 2024-09-12T01:20:13.901 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T01:20:13.961 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_2 2024-09-12T01:20:14.020 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T01:20:14.020 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T01:20:14.020 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-12T01:20:14.021 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:20:14.021 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T01:20:14.021 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-09-12 01:19:52.956687567 +0000 2024-09-12T01:20:14.021 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-09-12 01:13:52.482343376 +0000 2024-09-12T01:20:14.021 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-09-12 01:13:52.482343376 +0000 2024-09-12T01:20:14.021 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-09-12 01:13:52.482343376 +0000 2024-09-12T01:20:14.021 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T01:20:14.086 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-09-12T01:20:14.086 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-09-12T01:20:14.086 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000200475 s, 2.6 MB/s 2024-09-12T01:20:14.088 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T01:20:14.145 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_3 2024-09-12T01:20:14.202 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T01:20:14.203 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T01:20:14.203 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-12T01:20:14.203 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:20:14.203 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T01:20:14.203 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-09-12 01:19:52.956687567 +0000 2024-09-12T01:20:14.203 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-09-12 01:13:52.806345479 +0000 2024-09-12T01:20:14.203 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-09-12 01:13:52.806345479 +0000 2024-09-12T01:20:14.203 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-09-12 01:13:52.806345479 +0000 2024-09-12T01:20:14.203 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T01:20:14.243 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:14 smithi053 ceph-mon[35158]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:14.243 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:14 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:14.243 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:14 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:14.243 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:14 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T01:20:14.244 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:14 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T01:20:14.244 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:14 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:14.268 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-09-12T01:20:14.268 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-09-12T01:20:14.268 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000241075 s, 2.1 MB/s 2024-09-12T01:20:14.269 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T01:20:14.327 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_4 2024-09-12T01:20:14.383 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T01:20:14.383 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T01:20:14.383 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-12T01:20:14.383 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:20:14.383 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T01:20:14.383 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-09-12 01:19:52.956687567 +0000 2024-09-12T01:20:14.383 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-09-12 01:13:53.135347616 +0000 2024-09-12T01:20:14.383 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-09-12 01:13:53.135347616 +0000 2024-09-12T01:20:14.384 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-09-12 01:13:53.135347616 +0000 2024-09-12T01:20:14.384 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T01:20:14.447 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-09-12T01:20:14.447 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-09-12T01:20:14.447 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000206963 s, 2.5 MB/s 2024-09-12T01:20:14.448 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T01:20:14.506 INFO:tasks.cephadm:Deploying osd.0 on smithi001 with /dev/vg_nvme/lv_4... 2024-09-12T01:20:14.506 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-12T01:20:14.560 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:14 smithi001 ceph-mon[30387]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:14.560 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:14 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:14.560 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:14 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:14.560 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:14 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T01:20:14.560 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:14 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T01:20:14.560 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:14 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:14.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:14 smithi104 ceph-mon[35219]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:14.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:14 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:14.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:14 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:14.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:14 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T01:20:14.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:14 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T01:20:14.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:14 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:14.884 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:20:15.305 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:15 smithi001 ceph-mon[30387]: Reconfiguring mon.b (monmap changed)... 2024-09-12T01:20:15.305 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:15 smithi001 ceph-mon[30387]: Reconfiguring daemon mon.b on smithi053 2024-09-12T01:20:15.305 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:15 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:15.305 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:15 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:15.305 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:15 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T01:20:15.305 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:15 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T01:20:15.305 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:15 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:15.461 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:15 smithi104 ceph-mon[35219]: Reconfiguring mon.b (monmap changed)... 2024-09-12T01:20:15.461 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:15 smithi104 ceph-mon[35219]: Reconfiguring daemon mon.b on smithi053 2024-09-12T01:20:15.461 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:15 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:15.461 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:15 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:15.461 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:15 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T01:20:15.461 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:15 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T01:20:15.461 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:15 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:15.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:15 smithi053 ceph-mon[35158]: Reconfiguring mon.b (monmap changed)... 2024-09-12T01:20:15.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:15 smithi053 ceph-mon[35158]: Reconfiguring daemon mon.b on smithi053 2024-09-12T01:20:15.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:15 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:15.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:15 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:15.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:15 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T01:20:15.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:15 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T01:20:15.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:15 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:16.322 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:16 smithi001 ceph-mon[30387]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:16.323 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:16 smithi001 ceph-mon[30387]: Reconfiguring mon.c (monmap changed)... 2024-09-12T01:20:16.323 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:16 smithi001 ceph-mon[30387]: Reconfiguring daemon mon.c on smithi104 2024-09-12T01:20:16.323 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:16 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:16.323 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:16 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:16.323 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:16 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:20:16.323 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:16 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:16.323 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:16 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:20:16.323 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:16 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:16.323 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:16 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T01:20:16.323 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:16 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T01:20:16.323 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:16 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T01:20:16.323 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:16 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:16.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:16 smithi104 ceph-mon[35219]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:16.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:16 smithi104 ceph-mon[35219]: Reconfiguring mon.c (monmap changed)... 2024-09-12T01:20:16.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:16 smithi104 ceph-mon[35219]: Reconfiguring daemon mon.c on smithi104 2024-09-12T01:20:16.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:16 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:16.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:16 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:16.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:16 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:20:16.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:16 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:16.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:16 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:20:16.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:16 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:16.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:16 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T01:20:16.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:16 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T01:20:16.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:16 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T01:20:16.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:16 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:16.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:16 smithi053 ceph-mon[35158]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:16.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:16 smithi053 ceph-mon[35158]: Reconfiguring mon.c (monmap changed)... 2024-09-12T01:20:16.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:16 smithi053 ceph-mon[35158]: Reconfiguring daemon mon.c on smithi104 2024-09-12T01:20:16.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:16 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:16.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:16 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:16.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:16 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:20:16.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:16 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:16.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:16 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:20:16.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:16 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:16.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:16 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T01:20:16.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:16 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T01:20:16.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:16 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T01:20:16.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:16 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:17.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:17 smithi001 ceph-mon[30387]: Deploying daemon mgr.z on smithi104 2024-09-12T01:20:17.485 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:17 smithi104 ceph-mon[35219]: Deploying daemon mgr.z on smithi104 2024-09-12T01:20:17.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:17 smithi053 ceph-mon[35158]: Deploying daemon mgr.z on smithi104 2024-09-12T01:20:17.865 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:20:17.898 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi001:vg_nvme/lv_4 2024-09-12T01:20:18.266 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:20:18.452 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:18 smithi104 ceph-mon[35219]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:18.453 INFO:journalctl@ceph.mgr.z.smithi104.stdout:Sep 12 01:20:18 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-z[37028]: 2024-09-12T01:20:18.386+0000 7fd3a6481040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T01:20:18.453 INFO:journalctl@ceph.mgr.z.smithi104.stdout:Sep 12 01:20:18 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-z[37028]: 2024-09-12T01:20:18.452+0000 7fd3a6481040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T01:20:18.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:18 smithi053 ceph-mon[35158]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:18.560 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:18 smithi001 ceph-mon[30387]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:18.810 INFO:journalctl@ceph.mgr.z.smithi104.stdout:Sep 12 01:20:18 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-z[37028]: 2024-09-12T01:20:18.598+0000 7fd3a6481040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T01:20:18.810 INFO:journalctl@ceph.mgr.z.smithi104.stdout:Sep 12 01:20:18 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-z[37028]: 2024-09-12T01:20:18.663+0000 7fd3a6481040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T01:20:18.810 INFO:journalctl@ceph.mgr.z.smithi104.stdout:Sep 12 01:20:18 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-z[37028]: 2024-09-12T01:20:18.810+0000 7fd3a6481040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T01:20:18.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:18.826+0000 7fecf5400640 1 -- 172.21.15.1:0/4189907933 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecf0103600 msgr2=0x7fecf0109e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:20:18.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:18.826+0000 7fecf5400640 1 --2- 172.21.15.1:0/4189907933 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecf0103600 0x7fecf0109e90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fece8009ca0 tx=0x7fece802a8c0 comp rx=0 tx=0).stop 2024-09-12T01:20:18.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:18.826+0000 7fecf5400640 1 -- 172.21.15.1:0/4189907933 shutdown_connections 2024-09-12T01:20:18.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:18.826+0000 7fecf5400640 1 --2- 172.21.15.1:0/4189907933 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecf0103600 0x7fecf0109e90 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:20:18.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:18.826+0000 7fecf5400640 1 --2- 172.21.15.1:0/4189907933 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fecf0102c40 0x7fecf01030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:20:18.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:18.826+0000 7fecf5400640 1 --2- 172.21.15.1:0/4189907933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecf0101a40 0x7fecf0101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:20:18.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:18.826+0000 7fecf5400640 1 -- 172.21.15.1:0/4189907933 >> 172.21.15.1:0/4189907933 conn(0x7fecf00fd1f0 msgr2=0x7fecf00ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:20:18.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:18.826+0000 7fecf5400640 1 -- 172.21.15.1:0/4189907933 shutdown_connections 2024-09-12T01:20:18.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:18.827+0000 7fecf5400640 1 -- 172.21.15.1:0/4189907933 wait complete. 2024-09-12T01:20:18.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:18.828+0000 7fecf5400640 1 Processor -- start 2024-09-12T01:20:18.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:18.828+0000 7fecf5400640 1 -- start start 2024-09-12T01:20:18.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:18.828+0000 7fecf5400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecf0101a40 0x7fecf0075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:20:18.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:18.828+0000 7fecf5400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecf0102c40 0x7fecf00745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:20:18.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:18.828+0000 7fecf5400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fecf0103600 0x7fecf0074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:20:18.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:18.828+0000 7fecf5400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fecf010d6c0 con 0x7fecf0101a40 2024-09-12T01:20:18.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:18.828+0000 7fecf5400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fecf010d540 con 0x7fecf0102c40 2024-09-12T01:20:18.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:18.828+0000 7fecf5400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fecf010d840 con 0x7fecf0103600 2024-09-12T01:20:18.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:18.829+0000 7fecefe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecf0101a40 0x7fecf0075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:20:18.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:18.829+0000 7fece7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecf0102c40 0x7fecf00745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:20:18.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:18.829+0000 7fecf4a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fecf0103600 0x7fecf0074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:20:18.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:18.829+0000 7fecefe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecf0101a40 0x7fecf0075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55806/0 (socket says 172.21.15.1:55806) 2024-09-12T01:20:18.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:18.829+0000 7fece7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecf0102c40 0x7fecf00745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:34996/0 (socket says 172.21.15.1:34996) 2024-09-12T01:20:18.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:18.829+0000 7fecefe00640 1 -- 172.21.15.1:0/4031772770 learned_addr learned my addr 172.21.15.1:0/4031772770 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:20:18.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:18.829+0000 7fece7e00640 1 -- 172.21.15.1:0/4031772770 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fecf0103600 msgr2=0x7fecf0074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:20:18.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:18.829+0000 7fece7e00640 1 --2- 172.21.15.1:0/4031772770 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fecf0103600 0x7fecf0074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:20:18.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:18.829+0000 7fece7e00640 1 -- 172.21.15.1:0/4031772770 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecf0101a40 msgr2=0x7fecf0075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:20:18.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:18.829+0000 7fece7e00640 1 --2- 172.21.15.1:0/4031772770 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecf0101a40 0x7fecf0075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:20:18.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:18.829+0000 7fece7e00640 1 -- 172.21.15.1:0/4031772770 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecf00753e0 con 0x7fecf0102c40 2024-09-12T01:20:18.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:18.829+0000 7fece7e00640 1 --2- 172.21.15.1:0/4031772770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecf0102c40 0x7fecf00745e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fecd800e9f0 tx=0x7fecd800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:20:18.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:18.829+0000 7feced600640 1 -- 172.21.15.1:0/4031772770 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd800ccc0 con 0x7fecf0102c40 2024-09-12T01:20:18.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:18.830+0000 7feced600640 1 -- 172.21.15.1:0/4031772770 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecd8005020 con 0x7fecf0102c40 2024-09-12T01:20:18.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:18.830+0000 7fecf5400640 1 -- 172.21.15.1:0/4031772770 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecf01a1e50 con 0x7fecf0102c40 2024-09-12T01:20:18.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:18.830+0000 7feced600640 1 -- 172.21.15.1:0/4031772770 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd80049e0 con 0x7fecf0102c40 2024-09-12T01:20:18.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:18.830+0000 7fecf5400640 1 -- 172.21.15.1:0/4031772770 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecf01a22b0 con 0x7fecf0102c40 2024-09-12T01:20:18.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:18.831+0000 7fecf5400640 1 -- 172.21.15.1:0/4031772770 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecf0108e40 con 0x7fecf0102c40 2024-09-12T01:20:18.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:18.831+0000 7feced600640 1 -- 172.21.15.1:0/4031772770 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fecd8004cc0 con 0x7fecf0102c40 2024-09-12T01:20:18.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:18.831+0000 7feced600640 1 --2- 172.21.15.1:0/4031772770 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fecd4039290 0x7fecd403b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:20:18.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:18.831+0000 7feced600640 1 -- 172.21.15.1:0/4031772770 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fecd804c510 con 0x7fecf0102c40 2024-09-12T01:20:18.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:18.832+0000 7fecefe00640 1 --2- 172.21.15.1:0/4031772770 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fecd4039290 0x7fecd403b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:20:18.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:18.832+0000 7fecefe00640 1 --2- 172.21.15.1:0/4031772770 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fecd4039290 0x7fecd403b750 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fece00095f0 tx=0x7fece0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:20:18.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:18.836+0000 7feced600640 1 -- 172.21.15.1:0/4031772770 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fecd8010cd0 con 0x7fecf0102c40 2024-09-12T01:20:19.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:19.029+0000 7fecf5400640 1 -- 172.21.15.1:0/4031772770 --> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fecf0062630 con 0x7fecd4039290 2024-09-12T01:20:19.074 INFO:journalctl@ceph.mgr.z.smithi104.stdout:Sep 12 01:20:18 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-z[37028]: 2024-09-12T01:20:18.947+0000 7fd3a6481040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T01:20:19.574 INFO:journalctl@ceph.mgr.z.smithi104.stdout:Sep 12 01:20:19 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-z[37028]: 2024-09-12T01:20:19.256+0000 7fd3a6481040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T01:20:19.574 INFO:journalctl@ceph.mgr.z.smithi104.stdout:Sep 12 01:20:19 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-z[37028]: 2024-09-12T01:20:19.316+0000 7fd3a6481040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T01:20:19.575 INFO:journalctl@ceph.mgr.z.smithi104.stdout:Sep 12 01:20:19 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-z[37028]: 2024-09-12T01:20:19.504+0000 7fd3a6481040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T01:20:19.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:19 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:19.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:19 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:19.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:19 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:19.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:19 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T01:20:19.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:19 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T01:20:19.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:19 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T01:20:19.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:19 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:19.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:19 smithi104 ceph-mon[35219]: Deploying daemon mgr.y on smithi053 2024-09-12T01:20:19.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:19 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:20:19.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:19 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:20:19.576 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:19 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:19.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:19 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:19.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:19 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:19.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:19 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:19.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:19 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T01:20:19.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:19 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T01:20:19.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:19 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T01:20:19.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:19 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:19.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:19 smithi001 ceph-mon[30387]: Deploying daemon mgr.y on smithi053 2024-09-12T01:20:19.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:19 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:20:19.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:19 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:20:19.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:19 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:19.736 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:19 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:19.736 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:19 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:19.736 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:19 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:19.736 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:19 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T01:20:19.736 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:19 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T01:20:19.736 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:19 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T01:20:19.736 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:19 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:19.736 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:19 smithi053 ceph-mon[35158]: Deploying daemon mgr.y on smithi053 2024-09-12T01:20:19.736 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:19 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:20:19.736 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:19 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:20:19.736 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:19 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:20.433 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 12 01:20:20 smithi053 podman[36511]: 2024-09-12 01:20:20.219024449 +0000 UTC m=+0.027932372 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:20:20.669 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:20 smithi104 ceph-mon[35219]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:20.669 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:20 smithi104 ceph-mon[35219]: from='client.24107 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:20:20.670 INFO:journalctl@ceph.mgr.z.smithi104.stdout:Sep 12 01:20:20 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-z[37028]: 2024-09-12T01:20:20.408+0000 7fd3a6481040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T01:20:20.670 INFO:journalctl@ceph.mgr.z.smithi104.stdout:Sep 12 01:20:20 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-z[37028]: 2024-09-12T01:20:20.478+0000 7fd3a6481040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T01:20:20.695 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:20 smithi053 ceph-mon[35158]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:20.695 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:20 smithi053 ceph-mon[35158]: from='client.24107 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:20:20.695 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 12 01:20:20 smithi053 podman[36511]: 2024-09-12 01:20:20.56671301 +0000 UTC m=+0.375620931 container init d40466187460d25b017b58541ba5974f7c2626c107cbc66d861b3c0a8cd985f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T01:20:20.695 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 12 01:20:20 smithi053 podman[36511]: 2024-09-12 01:20:20.56941618 +0000 UTC m=+0.378324101 container start d40466187460d25b017b58541ba5974f7c2626c107cbc66d861b3c0a8cd985f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD) 2024-09-12T01:20:20.695 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 12 01:20:20 smithi053 ceph-mgr[36531]: -- 172.21.15.53:0/750823466 <== mon.2 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56264cda50e0 con 0x56264cd83000 2024-09-12T01:20:20.695 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 12 01:20:20 smithi053 bash[36511]: d40466187460d25b017b58541ba5974f7c2626c107cbc66d861b3c0a8cd985f1 2024-09-12T01:20:20.695 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 12 01:20:20 smithi053 systemd[1]: Started Ceph mgr.y for c0e4f180-70a4-11ef-bcea-c7b262605968. 2024-09-12T01:20:20.695 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 12 01:20:20 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-y[36527]: 2024-09-12T01:20:20.695+0000 7fdb17581040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T01:20:20.775 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:20 smithi001 ceph-mon[30387]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:20.775 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:20 smithi001 ceph-mon[30387]: from='client.24107 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:20:20.988 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 12 01:20:20 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-y[36527]: 2024-09-12T01:20:20.760+0000 7fdb17581040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T01:20:20.989 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 12 01:20:20 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-y[36527]: 2024-09-12T01:20:20.916+0000 7fdb17581040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T01:20:20.989 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 12 01:20:20 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-y[36527]: 2024-09-12T01:20:20.989+0000 7fdb17581040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T01:20:21.239 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 12 01:20:21 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-y[36527]: 2024-09-12T01:20:21.146+0000 7fdb17581040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T01:20:21.498 INFO:journalctl@ceph.mgr.z.smithi104.stdout:Sep 12 01:20:21 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-z[37028]: 2024-09-12T01:20:21.232+0000 7fd3a6481040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T01:20:21.499 INFO:journalctl@ceph.mgr.z.smithi104.stdout:Sep 12 01:20:21 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-z[37028]: 2024-09-12T01:20:21.355+0000 7fd3a6481040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T01:20:21.499 INFO:journalctl@ceph.mgr.z.smithi104.stdout:Sep 12 01:20:21 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-z[37028]: 2024-09-12T01:20:21.498+0000 7fd3a6481040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T01:20:21.648 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 12 01:20:21 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-y[36527]: 2024-09-12T01:20:21.316+0000 7fdb17581040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T01:20:21.789 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:21 smithi104 ceph-mon[35219]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:21.789 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:21 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:21.789 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:21 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:21.789 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:21 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:21.789 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:21 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:21.789 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:21 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:20:21.790 INFO:journalctl@ceph.mgr.z.smithi104.stdout:Sep 12 01:20:21 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-z[37028]: 2024-09-12T01:20:21.563+0000 7fd3a6481040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T01:20:21.790 INFO:journalctl@ceph.mgr.z.smithi104.stdout:Sep 12 01:20:21 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-z[37028]: 2024-09-12T01:20:21.717+0000 7fd3a6481040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T01:20:21.790 INFO:journalctl@ceph.mgr.z.smithi104.stdout:Sep 12 01:20:21 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-z[37028]: 2024-09-12T01:20:21.789+0000 7fd3a6481040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T01:20:21.908 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:21 smithi053 ceph-mon[35158]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:21.909 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:21 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:21.909 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:21 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:21.909 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:21 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:21.909 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:21 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:21.909 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:21 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:20:21.909 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 12 01:20:21 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-y[36527]: 2024-09-12T01:20:21.652+0000 7fdb17581040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T01:20:21.909 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 12 01:20:21 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-y[36527]: 2024-09-12T01:20:21.715+0000 7fdb17581040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T01:20:21.909 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 12 01:20:21 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-y[36527]: 2024-09-12T01:20:21.909+0000 7fdb17581040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T01:20:22.074 INFO:journalctl@ceph.mgr.z.smithi104.stdout:Sep 12 01:20:21 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-z[37028]: 2024-09-12T01:20:21.849+0000 7fd3a6481040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T01:20:22.074 INFO:journalctl@ceph.mgr.z.smithi104.stdout:Sep 12 01:20:22 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-z[37028]: 2024-09-12T01:20:22.015+0000 7fd3a6481040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T01:20:22.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:21 smithi001 ceph-mon[30387]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:22.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:21 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:22.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:21 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:22.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:21 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:22.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:21 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:22.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:21 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:20:22.355 INFO:journalctl@ceph.mgr.z.smithi104.stdout:Sep 12 01:20:22 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-z[37028]: 2024-09-12T01:20:22.081+0000 7fd3a6481040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T01:20:22.355 INFO:journalctl@ceph.mgr.z.smithi104.stdout:Sep 12 01:20:22 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-z[37028]: 2024-09-12T01:20:22.146+0000 7fd3a6481040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T01:20:22.355 INFO:journalctl@ceph.mgr.z.smithi104.stdout:Sep 12 01:20:22 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-z[37028]: 2024-09-12T01:20:22.233+0000 7fd3a6481040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T01:20:22.355 INFO:journalctl@ceph.mgr.z.smithi104.stdout:Sep 12 01:20:22 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-z[37028]: 2024-09-12T01:20:22.292+0000 7fd3a6481040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T01:20:22.355 INFO:journalctl@ceph.mgr.z.smithi104.stdout:Sep 12 01:20:22 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-z[37028]: 2024-09-12T01:20:22.354+0000 7fd3a6481040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T01:20:23.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:22 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:23.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:22 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:23.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:22 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:23.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:22 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:23.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:22 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:23.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:22 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:20:23.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:22 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:23.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:22 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/1385900249' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b9a396ae-bf97-462f-9ef9-b10ef6b9d337"}]: dispatch 2024-09-12T01:20:23.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:22 smithi001 ceph-mon[30387]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b9a396ae-bf97-462f-9ef9-b10ef6b9d337"}]: dispatch 2024-09-12T01:20:23.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:22 smithi001 ceph-mon[30387]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b9a396ae-bf97-462f-9ef9-b10ef6b9d337"}]': finished 2024-09-12T01:20:23.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:22 smithi001 ceph-mon[30387]: osdmap e5: 1 total, 0 up, 1 in 2024-09-12T01:20:23.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:22 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:20:23.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:22 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:20:23.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:22 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:23.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:22 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:20:23.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:22 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:23.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:22 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/3358738870' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:20:23.239 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 12 01:20:22 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-y[36527]: 2024-09-12T01:20:22.842+0000 7fdb17581040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T01:20:23.240 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 12 01:20:22 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-y[36527]: 2024-09-12T01:20:22.912+0000 7fdb17581040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T01:20:23.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:22 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:23.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:22 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:23.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:22 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:23.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:22 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:23.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:22 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:23.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:22 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:20:23.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:22 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:23.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:22 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/1385900249' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b9a396ae-bf97-462f-9ef9-b10ef6b9d337"}]: dispatch 2024-09-12T01:20:23.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:22 smithi053 ceph-mon[35158]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b9a396ae-bf97-462f-9ef9-b10ef6b9d337"}]: dispatch 2024-09-12T01:20:23.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:22 smithi053 ceph-mon[35158]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b9a396ae-bf97-462f-9ef9-b10ef6b9d337"}]': finished 2024-09-12T01:20:23.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:22 smithi053 ceph-mon[35158]: osdmap e5: 1 total, 0 up, 1 in 2024-09-12T01:20:23.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:22 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:20:23.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:22 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:20:23.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:22 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:23.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:22 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:20:23.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:22 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:23.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:22 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/3358738870' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:20:23.324 INFO:journalctl@ceph.mgr.z.smithi104.stdout:Sep 12 01:20:23 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-z[37028]: 2024-09-12T01:20:23.139+0000 7fd3a6481040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T01:20:23.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:22 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:23.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:22 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:23.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:22 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:23.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:22 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:23.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:22 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:23.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:22 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:20:23.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:22 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:23.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:22 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/1385900249' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b9a396ae-bf97-462f-9ef9-b10ef6b9d337"}]: dispatch 2024-09-12T01:20:23.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:22 smithi104 ceph-mon[35219]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b9a396ae-bf97-462f-9ef9-b10ef6b9d337"}]: dispatch 2024-09-12T01:20:23.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:22 smithi104 ceph-mon[35219]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b9a396ae-bf97-462f-9ef9-b10ef6b9d337"}]': finished 2024-09-12T01:20:23.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:22 smithi104 ceph-mon[35219]: osdmap e5: 1 total, 0 up, 1 in 2024-09-12T01:20:23.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:22 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:20:23.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:22 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:20:23.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:22 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:23.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:22 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:20:23.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:22 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:23.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:22 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/3358738870' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:20:23.925 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 12 01:20:23 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-y[36527]: 2024-09-12T01:20:23.658+0000 7fdb17581040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T01:20:23.925 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 12 01:20:23 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-y[36527]: 2024-09-12T01:20:23.778+0000 7fdb17581040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T01:20:23.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:23.997+0000 7feced600640 1 -- 172.21.15.1:0/4031772770 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 14) ==== 92059+0+0 (secure 0 0 0) 0x7fecd800f3c0 con 0x7fecf0102c40 2024-09-12T01:20:24.214 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:23 smithi053 ceph-mon[35158]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:24.214 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:23 smithi053 ceph-mon[35158]: Standby manager daemon z started 2024-09-12T01:20:24.214 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:23 smithi053 ceph-mon[35158]: from='mgr.? 172.21.15.104:0/2126018391' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T01:20:24.214 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:23 smithi053 ceph-mon[35158]: from='mgr.? 172.21.15.104:0/2126018391' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T01:20:24.214 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:23 smithi053 ceph-mon[35158]: from='mgr.? 172.21.15.104:0/2126018391' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T01:20:24.214 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:23 smithi053 ceph-mon[35158]: from='mgr.? 172.21.15.104:0/2126018391' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T01:20:24.215 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 12 01:20:23 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-y[36527]: 2024-09-12T01:20:23.925+0000 7fdb17581040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T01:20:24.215 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 12 01:20:23 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-y[36527]: 2024-09-12T01:20:23.989+0000 7fdb17581040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T01:20:24.215 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 12 01:20:24 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-y[36527]: 2024-09-12T01:20:24.144+0000 7fdb17581040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T01:20:24.215 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 12 01:20:24 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-y[36527]: 2024-09-12T01:20:24.214+0000 7fdb17581040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T01:20:24.247 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:23 smithi104 ceph-mon[35219]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:24.247 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:23 smithi104 ceph-mon[35219]: Standby manager daemon z started 2024-09-12T01:20:24.247 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:23 smithi104 ceph-mon[35219]: from='mgr.? 172.21.15.104:0/2126018391' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T01:20:24.247 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:23 smithi104 ceph-mon[35219]: from='mgr.? 172.21.15.104:0/2126018391' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T01:20:24.248 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:23 smithi104 ceph-mon[35219]: from='mgr.? 172.21.15.104:0/2126018391' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T01:20:24.248 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:23 smithi104 ceph-mon[35219]: from='mgr.? 172.21.15.104:0/2126018391' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T01:20:24.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:23 smithi001 ceph-mon[30387]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:24.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:23 smithi001 ceph-mon[30387]: Standby manager daemon z started 2024-09-12T01:20:24.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:23 smithi001 ceph-mon[30387]: from='mgr.? 172.21.15.104:0/2126018391' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T01:20:24.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:23 smithi001 ceph-mon[30387]: from='mgr.? 172.21.15.104:0/2126018391' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T01:20:24.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:23 smithi001 ceph-mon[30387]: from='mgr.? 172.21.15.104:0/2126018391' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T01:20:24.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:23 smithi001 ceph-mon[30387]: from='mgr.? 172.21.15.104:0/2126018391' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T01:20:24.489 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 12 01:20:24 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-y[36527]: 2024-09-12T01:20:24.276+0000 7fdb17581040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T01:20:24.489 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 12 01:20:24 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-y[36527]: 2024-09-12T01:20:24.440+0000 7fdb17581040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T01:20:24.783 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 12 01:20:24 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-y[36527]: 2024-09-12T01:20:24.507+0000 7fdb17581040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T01:20:24.783 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 12 01:20:24 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-y[36527]: 2024-09-12T01:20:24.576+0000 7fdb17581040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T01:20:24.784 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 12 01:20:24 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-y[36527]: 2024-09-12T01:20:24.662+0000 7fdb17581040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T01:20:24.784 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 12 01:20:24 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-y[36527]: 2024-09-12T01:20:24.723+0000 7fdb17581040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T01:20:24.784 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 12 01:20:24 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-y[36527]: 2024-09-12T01:20:24.784+0000 7fdb17581040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T01:20:25.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:24 smithi053 ceph-mon[35158]: mgrmap e14: x(active, since 69s), standbys: z 2024-09-12T01:20:25.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:24 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T01:20:25.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:24 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:25.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:24 smithi104 ceph-mon[35219]: mgrmap e14: x(active, since 69s), standbys: z 2024-09-12T01:20:25.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:24 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T01:20:25.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:24 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:25.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:24 smithi001 ceph-mon[30387]: mgrmap e14: x(active, since 69s), standbys: z 2024-09-12T01:20:25.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:24 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T01:20:25.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:24 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:25.989 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 12 01:20:25 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-y[36527]: 2024-09-12T01:20:25.605+0000 7fdb17581040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T01:20:26.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:26.007+0000 7feced600640 1 -- 172.21.15.1:0/4031772770 <== mon.2 v2:172.21.15.53:3300/0 8 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7fecd805ee90 con 0x7fecf0102c40 2024-09-12T01:20:26.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:26 smithi104 ceph-mon[35219]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:26.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:26 smithi104 ceph-mon[35219]: Standby manager daemon y started 2024-09-12T01:20:26.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:26 smithi104 ceph-mon[35219]: from='mgr.? 172.21.15.53:0/1521945383' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T01:20:26.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:26 smithi104 ceph-mon[35219]: from='mgr.? 172.21.15.53:0/1521945383' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T01:20:26.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:26 smithi104 ceph-mon[35219]: from='mgr.? 172.21.15.53:0/1521945383' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T01:20:26.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:26 smithi104 ceph-mon[35219]: from='mgr.? 172.21.15.53:0/1521945383' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T01:20:26.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:26 smithi001 ceph-mon[30387]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:26.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:26 smithi001 ceph-mon[30387]: Standby manager daemon y started 2024-09-12T01:20:26.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:26 smithi001 ceph-mon[30387]: from='mgr.? 172.21.15.53:0/1521945383' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T01:20:26.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:26 smithi001 ceph-mon[30387]: from='mgr.? 172.21.15.53:0/1521945383' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T01:20:26.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:26 smithi001 ceph-mon[30387]: from='mgr.? 172.21.15.53:0/1521945383' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T01:20:26.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:26 smithi001 ceph-mon[30387]: from='mgr.? 172.21.15.53:0/1521945383' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T01:20:26.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:26 smithi053 ceph-mon[35158]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:26.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:26 smithi053 ceph-mon[35158]: Standby manager daemon y started 2024-09-12T01:20:26.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:26 smithi053 ceph-mon[35158]: from='mgr.? 172.21.15.53:0/1521945383' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T01:20:26.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:26 smithi053 ceph-mon[35158]: from='mgr.? 172.21.15.53:0/1521945383' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T01:20:26.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:26 smithi053 ceph-mon[35158]: from='mgr.? 172.21.15.53:0/1521945383' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T01:20:26.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:26 smithi053 ceph-mon[35158]: from='mgr.? 172.21.15.53:0/1521945383' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T01:20:27.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:27 smithi104 ceph-mon[35219]: mgrmap e15: x(active, since 71s), standbys: z, y 2024-09-12T01:20:27.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:27 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T01:20:27.326 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:27 smithi001 ceph-mon[30387]: mgrmap e15: x(active, since 71s), standbys: z, y 2024-09-12T01:20:27.326 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:27 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T01:20:27.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:27 smithi053 ceph-mon[35158]: mgrmap e15: x(active, since 71s), standbys: z, y 2024-09-12T01:20:27.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:27 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T01:20:28.125 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:28 smithi001 ceph-mon[30387]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:28.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:28 smithi104 ceph-mon[35219]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:28.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:28 smithi053 ceph-mon[35158]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:30.269 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:30 smithi001 ceph-mon[30387]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:30.291 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:30 smithi053 ceph-mon[35158]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:30.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:30 smithi104 ceph-mon[35219]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:32.312 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:32 smithi001 ceph-mon[30387]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:32.312 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:32 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-12T01:20:32.312 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:32 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:32.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:32 smithi104 ceph-mon[35219]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:32.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:32 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-12T01:20:32.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:32 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:32.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:32 smithi053 ceph-mon[35158]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:32.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:32 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-12T01:20:32.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:32 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:33.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:33 smithi104 ceph-mon[35219]: Deploying daemon osd.0 on smithi001 2024-09-12T01:20:33.354 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:33 smithi001 ceph-mon[30387]: Deploying daemon osd.0 on smithi001 2024-09-12T01:20:33.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:33 smithi053 ceph-mon[35158]: Deploying daemon osd.0 on smithi001 2024-09-12T01:20:34.158 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:34 smithi001 ceph-mon[30387]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:34.316 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:34 smithi104 ceph-mon[35219]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:34.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:34 smithi053 ceph-mon[35158]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:36.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:36 smithi104 ceph-mon[35219]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:36.341 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:36 smithi001 ceph-mon[30387]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:36.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:36 smithi053 ceph-mon[35158]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:38.297 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:38 smithi001 ceph-mon[30387]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:38.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:38 smithi104 ceph-mon[35219]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:38.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:38 smithi053 ceph-mon[35158]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:40.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:40 smithi104 ceph-mon[35219]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:40.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:40 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:20:40.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:40 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:40.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:40 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:40.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:40 smithi001 ceph-mon[30387]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:40.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:40 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:20:40.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:40 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:40.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:40 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:40.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:40 smithi053 ceph-mon[35158]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:40.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:40 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:20:40.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:40 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:40.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:40 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:42.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:42 smithi104 ceph-mon[35219]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:42.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:42 smithi001 ceph-mon[30387]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:42.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:42 smithi053 ceph-mon[35158]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:44.150 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:43 smithi001 ceph-mon[30387]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:44.150 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:43 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:44.151 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:43 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:44.151 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:43 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:44.151 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:43 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:20:44.151 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:43 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:44.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:43 smithi053 ceph-mon[35158]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:44.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:43 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:44.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:43 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:44.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:43 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:44.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:43 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:20:44.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:43 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:44.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:43 smithi104 ceph-mon[35219]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:44.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:43 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:44.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:43 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:44.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:43 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:44.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:43 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:20:44.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:43 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:44.902 INFO:teuthology.orchestra.run.smithi001.stdout:Created osd(s) 0 on host 'smithi001' 2024-09-12T01:20:44.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:44.901+0000 7feced600640 1 -- 172.21.15.1:0/4031772770 <== mgr.14150 v2:172.21.15.1:6800/1928662456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fecf0062630 con 0x7fecd4039290 2024-09-12T01:20:44.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:44.904+0000 7fecf5400640 1 -- 172.21.15.1:0/4031772770 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fecd4039290 msgr2=0x7fecd403b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:20:44.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:44.904+0000 7fecf5400640 1 --2- 172.21.15.1:0/4031772770 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fecd4039290 0x7fecd403b750 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fece00095f0 tx=0x7fece0005f90 comp rx=0 tx=0).stop 2024-09-12T01:20:44.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:44.904+0000 7fecf5400640 1 -- 172.21.15.1:0/4031772770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecf0102c40 msgr2=0x7fecf00745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:20:44.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:44.904+0000 7fecf5400640 1 --2- 172.21.15.1:0/4031772770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecf0102c40 0x7fecf00745e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fecd800e9f0 tx=0x7fecd800eeb0 comp rx=0 tx=0).stop 2024-09-12T01:20:44.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:44.904+0000 7fecf5400640 1 -- 172.21.15.1:0/4031772770 shutdown_connections 2024-09-12T01:20:44.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:44.904+0000 7fecf5400640 1 --2- 172.21.15.1:0/4031772770 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fecd4039290 0x7fecd403b750 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:20:44.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:44.904+0000 7fecf5400640 1 --2- 172.21.15.1:0/4031772770 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fecf0103600 0x7fecf0074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:20:44.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:44.904+0000 7fecf5400640 1 --2- 172.21.15.1:0/4031772770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecf0102c40 0x7fecf00745e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:20:44.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:44.904+0000 7fecf5400640 1 --2- 172.21.15.1:0/4031772770 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecf0101a40 0x7fecf0075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:20:44.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:44.904+0000 7fecf5400640 1 -- 172.21.15.1:0/4031772770 >> 172.21.15.1:0/4031772770 conn(0x7fecf00fd1f0 msgr2=0x7fecf0106010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:20:44.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:44.904+0000 7fecf5400640 1 -- 172.21.15.1:0/4031772770 shutdown_connections 2024-09-12T01:20:44.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:20:44.904+0000 7fecf5400640 1 -- 172.21.15.1:0/4031772770 wait complete. 2024-09-12T01:20:45.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:44 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:20:45.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:44 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:45.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:44 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:45.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:44 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:20:45.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:44 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:45.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:44 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:45.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:44 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:20:45.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:44 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:45.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:44 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:45.655 DEBUG:teuthology.orchestra.run.smithi001:osd.0> sudo journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@osd.0.service 2024-09-12T01:20:45.661 INFO:tasks.cephadm:Deploying osd.1 on smithi053 with /dev/vg_nvme/lv_4... 2024-09-12T01:20:45.661 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-12T01:20:45.927 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T01:20:45.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:45 smithi053 ceph-mon[35158]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:46.171 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:45 smithi001 ceph-mon[30387]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:46.172 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 12 01:20:45 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-0[44328]: 2024-09-12T01:20:45.917+0000 7f0ab0273640 -1 osd.0 0 log_to_monitors true 2024-09-12T01:20:46.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:45 smithi104 ceph-mon[35219]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:46.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:46 smithi001 ceph-mon[30387]: from='osd.0 [v2:172.21.15.1:6802/1125335136,v1:172.21.15.1:6803/1125335136]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T01:20:46.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:46 smithi053 ceph-mon[35158]: from='osd.0 [v2:172.21.15.1:6802/1125335136,v1:172.21.15.1:6803/1125335136]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T01:20:47.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:46 smithi104 ceph-mon[35219]: from='osd.0 [v2:172.21.15.1:6802/1125335136,v1:172.21.15.1:6803/1125335136]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T01:20:47.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:47 smithi053 ceph-mon[35158]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:47.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:47 smithi053 ceph-mon[35158]: from='osd.0 [v2:172.21.15.1:6802/1125335136,v1:172.21.15.1:6803/1125335136]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-12T01:20:47.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:47 smithi053 ceph-mon[35158]: osdmap e6: 1 total, 0 up, 1 in 2024-09-12T01:20:47.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:47 smithi053 ceph-mon[35158]: from='osd.0 [v2:172.21.15.1:6802/1125335136,v1:172.21.15.1:6803/1125335136]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-12T01:20:47.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:47 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:20:48.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:47 smithi001 ceph-mon[30387]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:48.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:47 smithi001 ceph-mon[30387]: from='osd.0 [v2:172.21.15.1:6802/1125335136,v1:172.21.15.1:6803/1125335136]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-12T01:20:48.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:47 smithi001 ceph-mon[30387]: osdmap e6: 1 total, 0 up, 1 in 2024-09-12T01:20:48.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:47 smithi001 ceph-mon[30387]: from='osd.0 [v2:172.21.15.1:6802/1125335136,v1:172.21.15.1:6803/1125335136]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-12T01:20:48.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:47 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:20:48.309 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-12T01:20:48.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:47 smithi104 ceph-mon[35219]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:48.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:47 smithi104 ceph-mon[35219]: from='osd.0 [v2:172.21.15.1:6802/1125335136,v1:172.21.15.1:6803/1125335136]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-12T01:20:48.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:47 smithi104 ceph-mon[35219]: osdmap e6: 1 total, 0 up, 1 in 2024-09-12T01:20:48.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:47 smithi104 ceph-mon[35219]: from='osd.0 [v2:172.21.15.1:6802/1125335136,v1:172.21.15.1:6803/1125335136]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-12T01:20:48.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:47 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:20:48.336 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi053:vg_nvme/lv_4 2024-09-12T01:20:48.590 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T01:20:48.923 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 12 01:20:48 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-0[44328]: 2024-09-12T01:20:48.524+0000 7f0aaae00640 -1 osd.0 0 waiting for initial osdmap 2024-09-12T01:20:48.923 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 12 01:20:48 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-0[44328]: 2024-09-12T01:20:48.538+0000 7f0aa0e00640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T01:20:49.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:20:49.021+0000 7f5550063640 1 -- 172.21.15.53:0/4277652559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5548102690 msgr2=0x7f5548108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:20:49.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:20:49.021+0000 7f5550063640 1 --2- 172.21.15.53:0/4277652559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5548102690 0x7f5548108f20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5540009ca0 tx=0x7f554002a8c0 comp rx=0 tx=0).stop 2024-09-12T01:20:49.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:20:49.021+0000 7f5550063640 1 -- 172.21.15.53:0/4277652559 shutdown_connections 2024-09-12T01:20:49.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:20:49.021+0000 7f5550063640 1 --2- 172.21.15.53:0/4277652559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5548102690 0x7f5548108f20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:20:49.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:20:49.021+0000 7f5550063640 1 --2- 172.21.15.53:0/4277652559 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5548101cd0 0x7f5548102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:20:49.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:20:49.021+0000 7f5550063640 1 --2- 172.21.15.53:0/4277652559 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5548100ad0 0x7f5548100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:20:49.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:20:49.021+0000 7f5550063640 1 -- 172.21.15.53:0/4277652559 >> 172.21.15.53:0/4277652559 conn(0x7f55480fc280 msgr2=0x7f55480fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:20:49.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:20:49.021+0000 7f5550063640 1 -- 172.21.15.53:0/4277652559 shutdown_connections 2024-09-12T01:20:49.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:20:49.021+0000 7f5550063640 1 -- 172.21.15.53:0/4277652559 wait complete. 2024-09-12T01:20:49.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:20:49.022+0000 7f5550063640 1 Processor -- start 2024-09-12T01:20:49.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:20:49.022+0000 7f5550063640 1 -- start start 2024-09-12T01:20:49.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:20:49.022+0000 7f5550063640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5548100ad0 0x7f5548197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:20:49.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:20:49.022+0000 7f5550063640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5548101cd0 0x7f55481983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:20:49.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:20:49.022+0000 7f5550063640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5548102690 0x7f554819f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:20:49.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:20:49.022+0000 7f5550063640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f554810c780 con 0x7f5548100ad0 2024-09-12T01:20:49.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:20:49.022+0000 7f5550063640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f554810c600 con 0x7f5548102690 2024-09-12T01:20:49.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:20:49.022+0000 7f5550063640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f554810c900 con 0x7f5548101cd0 2024-09-12T01:20:49.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:20:49.022+0000 7f554e600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5548102690 0x7f554819f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:20:49.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:20:49.022+0000 7f554e600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5548102690 0x7f554819f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60178/0 (socket says 172.21.15.53:60178) 2024-09-12T01:20:49.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:20:49.022+0000 7f554e600640 1 -- 172.21.15.53:0/78943234 learned_addr learned my addr 172.21.15.53:0/78943234 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-12T01:20:49.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:20:49.022+0000 7f554d5fe640 1 --2- 172.21.15.53:0/78943234 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5548101cd0 0x7f55481983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:20:49.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:20:49.023+0000 7f554e600640 1 -- 172.21.15.53:0/78943234 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5548101cd0 msgr2=0x7f55481983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:20:49.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:20:49.023+0000 7f554e600640 1 --2- 172.21.15.53:0/78943234 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5548101cd0 0x7f55481983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:20:49.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:20:49.023+0000 7f554e600640 1 -- 172.21.15.53:0/78943234 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5548100ad0 msgr2=0x7f5548197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:20:49.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:20:49.023+0000 7f554e600640 1 --2- 172.21.15.53:0/78943234 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5548100ad0 0x7f5548197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:20:49.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:20:49.023+0000 7f554e600640 1 -- 172.21.15.53:0/78943234 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f554819fac0 con 0x7f5548102690 2024-09-12T01:20:49.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:20:49.023+0000 7f554d5fe640 1 --2- 172.21.15.53:0/78943234 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5548101cd0 0x7f55481983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:20:49.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:20:49.023+0000 7f554e600640 1 --2- 172.21.15.53:0/78943234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5548102690 0x7f554819f450 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f55400319f0 tx=0x7f554002f450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:20:49.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:20:49.023+0000 7f553affd640 1 -- 172.21.15.53:0/78943234 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5540003c70 con 0x7f5548102690 2024-09-12T01:20:49.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:20:49.023+0000 7f553affd640 1 -- 172.21.15.53:0/78943234 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5540003e10 con 0x7f5548102690 2024-09-12T01:20:49.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:20:49.023+0000 7f5550063640 1 -- 172.21.15.53:0/78943234 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f554819fd50 con 0x7f5548102690 2024-09-12T01:20:49.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:20:49.023+0000 7f553affd640 1 -- 172.21.15.53:0/78943234 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554002fd30 con 0x7f5548102690 2024-09-12T01:20:49.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:20:49.023+0000 7f5550063640 1 -- 172.21.15.53:0/78943234 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55481a0250 con 0x7f5548102690 2024-09-12T01:20:49.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:20:49.024+0000 7f5550063640 1 -- 172.21.15.53:0/78943234 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55481a04b0 con 0x7f5548102690 2024-09-12T01:20:49.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:20:49.025+0000 7f553affd640 1 -- 172.21.15.53:0/78943234 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7f5540004040 con 0x7f5548102690 2024-09-12T01:20:49.032 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:20:49.026+0000 7f553affd640 1 --2- 172.21.15.53:0/78943234 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f55340a2ba0 0x7f55340a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:20:49.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:20:49.026+0000 7f553affd640 1 -- 172.21.15.53:0/78943234 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7f55400eea10 con 0x7f5548102690 2024-09-12T01:20:49.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:20:49.026+0000 7f554ddff640 1 --2- 172.21.15.53:0/78943234 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f55340a2ba0 0x7f55340a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:20:49.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:20:49.027+0000 7f554ddff640 1 --2- 172.21.15.53:0/78943234 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f55340a2ba0 0x7f55340a5060 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5544005fd0 tx=0x7f554400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:20:49.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:20:49.030+0000 7f553affd640 1 -- 172.21.15.53:0/78943234 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f55400bd500 con 0x7f5548102690 2024-09-12T01:20:49.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:48 smithi001 ceph-mon[30387]: purged_snaps scrub starts 2024-09-12T01:20:49.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:48 smithi001 ceph-mon[30387]: purged_snaps scrub ok 2024-09-12T01:20:49.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:48 smithi001 ceph-mon[30387]: from='osd.0 [v2:172.21.15.1:6802/1125335136,v1:172.21.15.1:6803/1125335136]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-09-12T01:20:49.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:48 smithi001 ceph-mon[30387]: osdmap e7: 1 total, 0 up, 1 in 2024-09-12T01:20:49.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:48 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:20:49.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:48 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:20:49.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:48 smithi001 ceph-mon[30387]: from='osd.0 [v2:172.21.15.1:6802/1125335136,v1:172.21.15.1:6803/1125335136]' entity='osd.0' 2024-09-12T01:20:49.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:20:49.215+0000 7f5550063640 1 -- 172.21.15.53:0/78943234 --> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f55481a0680 con 0x7f55340a2ba0 2024-09-12T01:20:49.215 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:48 smithi053 ceph-mon[35158]: purged_snaps scrub starts 2024-09-12T01:20:49.215 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:48 smithi053 ceph-mon[35158]: purged_snaps scrub ok 2024-09-12T01:20:49.215 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:48 smithi053 ceph-mon[35158]: from='osd.0 [v2:172.21.15.1:6802/1125335136,v1:172.21.15.1:6803/1125335136]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-09-12T01:20:49.215 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:48 smithi053 ceph-mon[35158]: osdmap e7: 1 total, 0 up, 1 in 2024-09-12T01:20:49.215 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:48 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:20:49.216 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:48 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:20:49.216 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:48 smithi053 ceph-mon[35158]: from='osd.0 [v2:172.21.15.1:6802/1125335136,v1:172.21.15.1:6803/1125335136]' entity='osd.0' 2024-09-12T01:20:49.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:48 smithi104 ceph-mon[35219]: purged_snaps scrub starts 2024-09-12T01:20:49.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:48 smithi104 ceph-mon[35219]: purged_snaps scrub ok 2024-09-12T01:20:49.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:48 smithi104 ceph-mon[35219]: from='osd.0 [v2:172.21.15.1:6802/1125335136,v1:172.21.15.1:6803/1125335136]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-09-12T01:20:49.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:48 smithi104 ceph-mon[35219]: osdmap e7: 1 total, 0 up, 1 in 2024-09-12T01:20:49.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:48 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:20:49.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:48 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:20:49.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:48 smithi104 ceph-mon[35219]: from='osd.0 [v2:172.21.15.1:6802/1125335136,v1:172.21.15.1:6803/1125335136]' entity='osd.0' 2024-09-12T01:20:50.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:49 smithi001 ceph-mon[30387]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:50.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:49 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:20:50.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:49 smithi001 ceph-mon[30387]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:20:50.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:49 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:20:50.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:49 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:20:50.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:49 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:50.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:49 smithi001 ceph-mon[30387]: osd.0 [v2:172.21.15.1:6802/1125335136,v1:172.21.15.1:6803/1125335136] boot 2024-09-12T01:20:50.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:49 smithi001 ceph-mon[30387]: osdmap e8: 1 total, 1 up, 1 in 2024-09-12T01:20:50.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:49 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:20:50.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:49 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:50.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:49 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:50.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:49 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:20:50.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:49 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:50.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:49 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:50.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:49 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:20:50.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:49 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:50.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:49 smithi053 ceph-mon[35158]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:50.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:49 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:20:50.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:49 smithi053 ceph-mon[35158]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:20:50.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:49 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:20:50.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:49 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:20:50.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:49 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:50.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:49 smithi053 ceph-mon[35158]: osd.0 [v2:172.21.15.1:6802/1125335136,v1:172.21.15.1:6803/1125335136] boot 2024-09-12T01:20:50.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:49 smithi053 ceph-mon[35158]: osdmap e8: 1 total, 1 up, 1 in 2024-09-12T01:20:50.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:49 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:20:50.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:49 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:50.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:49 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:50.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:49 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:20:50.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:49 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:50.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:49 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:50.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:49 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:20:50.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:49 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:50.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:49 smithi104 ceph-mon[35219]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:20:50.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:49 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:20:50.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:49 smithi104 ceph-mon[35219]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:20:50.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:49 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:20:50.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:49 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:20:50.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:49 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:50.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:49 smithi104 ceph-mon[35219]: osd.0 [v2:172.21.15.1:6802/1125335136,v1:172.21.15.1:6803/1125335136] boot 2024-09-12T01:20:50.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:49 smithi104 ceph-mon[35219]: osdmap e8: 1 total, 1 up, 1 in 2024-09-12T01:20:50.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:49 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:20:50.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:49 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:50.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:49 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:50.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:49 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:20:50.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:49 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:50.327 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:49 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:20:50.327 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:49 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:20:50.327 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:49 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:20:51.159 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:50 smithi053 ceph-mon[35158]: Detected new or changed devices on smithi001 2024-09-12T01:20:51.160 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:50 smithi053 ceph-mon[35158]: Adjusting osd_memory_target on smithi001 to 17094M 2024-09-12T01:20:51.160 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:50 smithi053 ceph-mon[35158]: osdmap e9: 1 total, 1 up, 1 in 2024-09-12T01:20:51.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:50 smithi104 ceph-mon[35219]: Detected new or changed devices on smithi001 2024-09-12T01:20:51.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:50 smithi104 ceph-mon[35219]: Adjusting osd_memory_target on smithi001 to 17094M 2024-09-12T01:20:51.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:50 smithi104 ceph-mon[35219]: osdmap e9: 1 total, 1 up, 1 in 2024-09-12T01:20:51.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:50 smithi001 ceph-mon[30387]: Detected new or changed devices on smithi001 2024-09-12T01:20:51.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:50 smithi001 ceph-mon[30387]: Adjusting osd_memory_target on smithi001 to 17094M 2024-09-12T01:20:51.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:50 smithi001 ceph-mon[30387]: osdmap e9: 1 total, 1 up, 1 in 2024-09-12T01:20:51.992 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:51 smithi053 ceph-mon[35158]: pgmap v46: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:20:51.992 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:51 smithi053 ceph-mon[35158]: from='client.? 172.21.15.53:0/1938087285' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dfe7a8a3-3700-4789-ac64-88660de9e0d9"}]: dispatch 2024-09-12T01:20:51.993 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:51 smithi053 ceph-mon[35158]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dfe7a8a3-3700-4789-ac64-88660de9e0d9"}]: dispatch 2024-09-12T01:20:51.993 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:51 smithi053 ceph-mon[35158]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dfe7a8a3-3700-4789-ac64-88660de9e0d9"}]': finished 2024-09-12T01:20:51.993 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:51 smithi053 ceph-mon[35158]: osdmap e10: 2 total, 1 up, 2 in 2024-09-12T01:20:51.993 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:51 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:20:52.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:51 smithi104 ceph-mon[35219]: pgmap v46: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:20:52.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:51 smithi104 ceph-mon[35219]: from='client.? 172.21.15.53:0/1938087285' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dfe7a8a3-3700-4789-ac64-88660de9e0d9"}]: dispatch 2024-09-12T01:20:52.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:51 smithi104 ceph-mon[35219]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dfe7a8a3-3700-4789-ac64-88660de9e0d9"}]: dispatch 2024-09-12T01:20:52.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:51 smithi104 ceph-mon[35219]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dfe7a8a3-3700-4789-ac64-88660de9e0d9"}]': finished 2024-09-12T01:20:52.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:51 smithi104 ceph-mon[35219]: osdmap e10: 2 total, 1 up, 2 in 2024-09-12T01:20:52.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:51 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:20:52.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:51 smithi001 ceph-mon[30387]: pgmap v46: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:20:52.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:51 smithi001 ceph-mon[30387]: from='client.? 172.21.15.53:0/1938087285' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dfe7a8a3-3700-4789-ac64-88660de9e0d9"}]: dispatch 2024-09-12T01:20:52.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:51 smithi001 ceph-mon[30387]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dfe7a8a3-3700-4789-ac64-88660de9e0d9"}]: dispatch 2024-09-12T01:20:52.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:51 smithi001 ceph-mon[30387]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dfe7a8a3-3700-4789-ac64-88660de9e0d9"}]': finished 2024-09-12T01:20:52.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:51 smithi001 ceph-mon[30387]: osdmap e10: 2 total, 1 up, 2 in 2024-09-12T01:20:52.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:51 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:20:53.197 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:52 smithi001 ceph-mon[30387]: from='client.? 172.21.15.53:0/24479959' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:20:53.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:52 smithi053 ceph-mon[35158]: from='client.? 172.21.15.53:0/24479959' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:20:53.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:52 smithi104 ceph-mon[35219]: from='client.? 172.21.15.53:0/24479959' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:20:54.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:53 smithi001 ceph-mon[30387]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:20:54.225 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:53 smithi104 ceph-mon[35219]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:20:54.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:53 smithi053 ceph-mon[35158]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:20:56.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:55 smithi053 ceph-mon[35158]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:20:56.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:55 smithi104 ceph-mon[35219]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:20:56.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:55 smithi001 ceph-mon[30387]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:20:58.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:57 smithi053 ceph-mon[35158]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:20:58.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:57 smithi104 ceph-mon[35219]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:20:58.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:57 smithi001 ceph-mon[30387]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:21:00.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:59 smithi053 ceph-mon[35158]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:21:00.243 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:59 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-12T01:21:00.243 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:20:59 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:21:00.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:59 smithi104 ceph-mon[35219]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:21:00.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:59 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-12T01:21:00.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:20:59 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:21:00.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:59 smithi001 ceph-mon[30387]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:21:00.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:59 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-12T01:21:00.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:20:59 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:21:01.177 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:00 smithi053 ceph-mon[35158]: Deploying daemon osd.1 on smithi053 2024-09-12T01:21:01.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:00 smithi104 ceph-mon[35219]: Deploying daemon osd.1 on smithi053 2024-09-12T01:21:01.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:00 smithi001 ceph-mon[30387]: Deploying daemon osd.1 on smithi053 2024-09-12T01:21:01.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:01 smithi053 ceph-mon[35158]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:21:02.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:01 smithi104 ceph-mon[35219]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:21:02.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:01 smithi001 ceph-mon[30387]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:21:04.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:03 smithi053 ceph-mon[35158]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:21:04.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:03 smithi104 ceph-mon[35219]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:21:04.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:03 smithi001 ceph-mon[30387]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:21:06.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:05 smithi053 ceph-mon[35158]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:21:06.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:05 smithi104 ceph-mon[35219]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:21:06.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:05 smithi001 ceph-mon[30387]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:21:07.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:06 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:21:07.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:06 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:07.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:06 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:07.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:06 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:21:07.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:06 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:07.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:06 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:07.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:06 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:21:07.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:06 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:07.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:06 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:08.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:08 smithi001 ceph-mon[30387]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:21:08.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:08 smithi053 ceph-mon[35158]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:21:08.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:08 smithi104 ceph-mon[35219]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:21:09.911 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:09 smithi053 ceph-mon[35158]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:21:09.912 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:09 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:09.912 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:09 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:09.912 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:09 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:21:09.912 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:09 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:21:09.912 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:09 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:09.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:21:09.953+0000 7f553affd640 1 -- 172.21.15.53:0/78943234 <== mgr.14150 v2:172.21.15.1:6800/1928662456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f55481a0680 con 0x7f55340a2ba0 2024-09-12T01:21:09.952 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 1 on host 'smithi053' 2024-09-12T01:21:09.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:21:09.955+0000 7f5550063640 1 -- 172.21.15.53:0/78943234 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f55340a2ba0 msgr2=0x7f55340a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:21:09.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:21:09.955+0000 7f5550063640 1 --2- 172.21.15.53:0/78943234 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f55340a2ba0 0x7f55340a5060 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5544005fd0 tx=0x7f554400b410 comp rx=0 tx=0).stop 2024-09-12T01:21:09.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:21:09.955+0000 7f5550063640 1 -- 172.21.15.53:0/78943234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5548102690 msgr2=0x7f554819f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:21:09.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:21:09.955+0000 7f5550063640 1 --2- 172.21.15.53:0/78943234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5548102690 0x7f554819f450 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f55400319f0 tx=0x7f554002f450 comp rx=0 tx=0).stop 2024-09-12T01:21:09.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:21:09.955+0000 7f5550063640 1 -- 172.21.15.53:0/78943234 shutdown_connections 2024-09-12T01:21:09.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:21:09.955+0000 7f5550063640 1 --2- 172.21.15.53:0/78943234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5548102690 0x7f554819f450 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:21:09.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:21:09.955+0000 7f5550063640 1 --2- 172.21.15.53:0/78943234 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f55340a2ba0 0x7f55340a5060 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:21:09.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:21:09.955+0000 7f5550063640 1 --2- 172.21.15.53:0/78943234 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5548101cd0 0x7f55481983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:21:09.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:21:09.955+0000 7f5550063640 1 --2- 172.21.15.53:0/78943234 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5548100ad0 0x7f5548197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:21:09.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:21:09.955+0000 7f5550063640 1 -- 172.21.15.53:0/78943234 >> 172.21.15.53:0/78943234 conn(0x7f55480fc280 msgr2=0x7f55480fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:21:09.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:21:09.955+0000 7f5550063640 1 -- 172.21.15.53:0/78943234 shutdown_connections 2024-09-12T01:21:09.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:21:09.955+0000 7f5550063640 1 -- 172.21.15.53:0/78943234 wait complete. 2024-09-12T01:21:10.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:09 smithi104 ceph-mon[35219]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:21:10.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:09 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:10.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:09 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:10.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:09 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:21:10.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:09 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:21:10.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:09 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:10.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:09 smithi001 ceph-mon[30387]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:21:10.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:09 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:10.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:09 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:10.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:09 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:21:10.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:09 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:21:10.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:09 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:10.628 DEBUG:teuthology.orchestra.run.smithi053:osd.1> sudo journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@osd.1.service 2024-09-12T01:21:10.630 INFO:tasks.cephadm:Deploying osd.2 on smithi104 with /dev/vg_nvme/lv_4... 2024-09-12T01:21:10.630 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-12T01:21:10.865 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:10 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:21:10.865 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:10 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:10.866 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:10 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:10.885 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:21:11.005 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:10 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:21:11.006 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:10 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:11.006 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:10 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:11.080 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:10 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:21:11.080 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:10 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:11.080 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:10 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:11.951 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 12 01:21:11 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-1[39630]: 2024-09-12T01:21:11.689+0000 7fdf79f3f640 -1 osd.1 0 log_to_monitors true 2024-09-12T01:21:12.169 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:11 smithi104 ceph-mon[35219]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:21:12.170 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:11 smithi104 ceph-mon[35219]: from='osd.1 [v2:172.21.15.53:6800/3131049165,v1:172.21.15.53:6801/3131049165]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T01:21:12.170 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:11 smithi104 ceph-mon[35219]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T01:21:12.233 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:11 smithi053 ceph-mon[35158]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:21:12.233 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:11 smithi053 ceph-mon[35158]: from='osd.1 [v2:172.21.15.53:6800/3131049165,v1:172.21.15.53:6801/3131049165]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T01:21:12.233 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:11 smithi053 ceph-mon[35158]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T01:21:12.425 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:11 smithi001 ceph-mon[30387]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:21:12.425 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:11 smithi001 ceph-mon[30387]: from='osd.1 [v2:172.21.15.53:6800/3131049165,v1:172.21.15.53:6801/3131049165]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T01:21:12.425 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:11 smithi001 ceph-mon[30387]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T01:21:13.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:12 smithi001 ceph-mon[30387]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-12T01:21:13.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:12 smithi001 ceph-mon[30387]: from='osd.1 [v2:172.21.15.53:6800/3131049165,v1:172.21.15.53:6801/3131049165]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-12T01:21:13.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:12 smithi001 ceph-mon[30387]: osdmap e11: 2 total, 1 up, 2 in 2024-09-12T01:21:13.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:12 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:21:13.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:12 smithi001 ceph-mon[30387]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-12T01:21:13.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:12 smithi053 ceph-mon[35158]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-12T01:21:13.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:12 smithi053 ceph-mon[35158]: from='osd.1 [v2:172.21.15.53:6800/3131049165,v1:172.21.15.53:6801/3131049165]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-12T01:21:13.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:12 smithi053 ceph-mon[35158]: osdmap e11: 2 total, 1 up, 2 in 2024-09-12T01:21:13.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:12 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:21:13.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:12 smithi053 ceph-mon[35158]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-12T01:21:13.259 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:12 smithi104 ceph-mon[35219]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-12T01:21:13.259 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:12 smithi104 ceph-mon[35219]: from='osd.1 [v2:172.21.15.53:6800/3131049165,v1:172.21.15.53:6801/3131049165]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-12T01:21:13.259 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:12 smithi104 ceph-mon[35219]: osdmap e11: 2 total, 1 up, 2 in 2024-09-12T01:21:13.259 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:12 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:21:13.259 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:12 smithi104 ceph-mon[35219]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-12T01:21:13.261 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-12T01:21:13.287 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi104:vg_nvme/lv_4 2024-09-12T01:21:13.489 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 12 01:21:13 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-1[39630]: 2024-09-12T01:21:13.445+0000 7fdf75ec2640 -1 osd.1 0 waiting for initial osdmap 2024-09-12T01:21:13.489 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 12 01:21:13 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-1[39630]: 2024-09-12T01:21:13.449+0000 7fdf6dce2640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T01:21:13.532 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:21:14.012 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:14.006+0000 7f0734023640 1 -- 172.21.15.104:0/3948068714 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f072c102660 msgr2=0x7f072c104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:21:14.012 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:14.006+0000 7f0734023640 1 --2- 172.21.15.104:0/3948068714 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f072c102660 0x7f072c104a50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0728009900 tx=0x7f072802a410 comp rx=0 tx=0).stop 2024-09-12T01:21:14.012 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:14.006+0000 7f0734023640 1 -- 172.21.15.104:0/3948068714 shutdown_connections 2024-09-12T01:21:14.012 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:14.006+0000 7f0734023640 1 --2- 172.21.15.104:0/3948068714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f072c1078c0 0x7f072c109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:21:14.012 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:14.006+0000 7f0734023640 1 --2- 172.21.15.104:0/3948068714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f072c104f90 0x7f072c107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:21:14.012 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:14.006+0000 7f0734023640 1 --2- 172.21.15.104:0/3948068714 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f072c102660 0x7f072c104a50 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:21:14.012 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:14.006+0000 7f0734023640 1 -- 172.21.15.104:0/3948068714 >> 172.21.15.104:0/3948068714 conn(0x7f072c0fc280 msgr2=0x7f072c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:21:14.012 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:14.006+0000 7f0734023640 1 -- 172.21.15.104:0/3948068714 shutdown_connections 2024-09-12T01:21:14.013 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:14.006+0000 7f0734023640 1 -- 172.21.15.104:0/3948068714 wait complete. 2024-09-12T01:21:14.013 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:14.007+0000 7f0734023640 1 Processor -- start 2024-09-12T01:21:14.013 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:14.007+0000 7f0734023640 1 -- start start 2024-09-12T01:21:14.013 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:14.007+0000 7f0734023640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f072c102660 0x7f072c101fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:21:14.013 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:14.007+0000 7f0734023640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f072c104f90 0x7f072c100610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:21:14.013 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:14.007+0000 7f0734023640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f072c1078c0 0x7f072c100b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:21:14.013 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:14.007+0000 7f0734023640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f072c10d900 con 0x7f072c104f90 2024-09-12T01:21:14.013 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:14.007+0000 7f0734023640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f072c10d780 con 0x7f072c102660 2024-09-12T01:21:14.013 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:14.007+0000 7f0734023640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f072c10da80 con 0x7f072c1078c0 2024-09-12T01:21:14.013 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:14.007+0000 7f07325c0640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f072c1078c0 0x7f072c100b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:21:14.013 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:14.007+0000 7f07325c0640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f072c1078c0 0x7f072c100b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:49406/0 (socket says 172.21.15.104:49406) 2024-09-12T01:21:14.013 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:14.007+0000 7f0731dbf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f072c102660 0x7f072c101fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:21:14.014 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:14.007+0000 7f07325c0640 1 -- 172.21.15.104:0/3533164470 learned_addr learned my addr 172.21.15.104:0/3533164470 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-09-12T01:21:14.014 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:14.007+0000 7f07325c0640 1 -- 172.21.15.104:0/3533164470 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f072c102660 msgr2=0x7f072c101fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:21:14.014 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:14.007+0000 7f07325c0640 1 --2- 172.21.15.104:0/3533164470 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f072c102660 0x7f072c101fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:21:14.014 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:14.007+0000 7f07325c0640 1 -- 172.21.15.104:0/3533164470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f072c104f90 msgr2=0x7f072c100610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:21:14.014 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:14.007+0000 7f07325c0640 1 --2- 172.21.15.104:0/3533164470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f072c104f90 0x7f072c100610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:21:14.014 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:14.008+0000 7f07325c0640 1 -- 172.21.15.104:0/3533164470 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f072c101410 con 0x7f072c1078c0 2024-09-12T01:21:14.014 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:14.008+0000 7f07325c0640 1 --2- 172.21.15.104:0/3533164470 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f072c1078c0 0x7f072c100b50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f072400eb10 tx=0x7f072400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:21:14.014 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:14.008+0000 7f0722ffd640 1 -- 172.21.15.104:0/3533164470 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0724004b20 con 0x7f072c1078c0 2024-09-12T01:21:14.014 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:14.008+0000 7f0722ffd640 1 -- 172.21.15.104:0/3533164470 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0724004cc0 con 0x7f072c1078c0 2024-09-12T01:21:14.014 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:14.008+0000 7f0734023640 1 -- 172.21.15.104:0/3533164470 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f072c1a2110 con 0x7f072c1078c0 2024-09-12T01:21:14.014 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:14.008+0000 7f0722ffd640 1 -- 172.21.15.104:0/3533164470 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0724010760 con 0x7f072c1078c0 2024-09-12T01:21:14.014 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:14.008+0000 7f0734023640 1 -- 172.21.15.104:0/3533164470 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f072c1a26a0 con 0x7f072c1078c0 2024-09-12T01:21:14.015 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:14.009+0000 7f0734023640 1 -- 172.21.15.104:0/3533164470 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f072c0fdd70 con 0x7f072c1078c0 2024-09-12T01:21:14.015 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:14.010+0000 7f0722ffd640 1 -- 172.21.15.104:0/3533164470 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7f0724005020 con 0x7f072c1078c0 2024-09-12T01:21:14.015 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:14.011+0000 7f0722ffd640 1 --2- 172.21.15.104:0/3533164470 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f07180a2af0 0x7f07180a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:21:14.015 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:14.011+0000 7f0722ffd640 1 -- 172.21.15.104:0/3533164470 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7f07240cddb0 con 0x7f072c1078c0 2024-09-12T01:21:14.015 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:14.011+0000 7f0731dbf640 1 --2- 172.21.15.104:0/3533164470 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f07180a2af0 0x7f07180a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:21:14.015 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:14.012+0000 7f0731dbf640 1 --2- 172.21.15.104:0/3533164470 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f07180a2af0 0x7f07180a4fb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f07280314e0 tx=0x7f0728000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:21:14.015 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:14.015+0000 7f0722ffd640 1 -- 172.21.15.104:0/3533164470 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f072409c680 con 0x7f072c1078c0 2024-09-12T01:21:14.194 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:14.195+0000 7f0734023640 1 -- 172.21.15.104:0/3533164470 --> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f072c062630 con 0x7f07180a2af0 2024-09-12T01:21:14.195 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:13 smithi104 ceph-mon[35219]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:21:14.195 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:13 smithi104 ceph-mon[35219]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-09-12T01:21:14.195 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:13 smithi104 ceph-mon[35219]: osdmap e12: 2 total, 1 up, 2 in 2024-09-12T01:21:14.195 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:13 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:21:14.195 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:13 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:21:14.195 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:13 smithi104 ceph-mon[35219]: from='osd.1 ' entity='osd.1' 2024-09-12T01:21:14.195 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:13 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:14.195 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:13 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:14.195 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:13 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:21:14.195 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:13 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:14.196 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:13 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:21:14.196 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:13 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:21:14.196 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:13 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:14.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:13 smithi053 ceph-mon[35158]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:21:14.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:13 smithi053 ceph-mon[35158]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-09-12T01:21:14.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:13 smithi053 ceph-mon[35158]: osdmap e12: 2 total, 1 up, 2 in 2024-09-12T01:21:14.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:13 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:21:14.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:13 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:21:14.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:13 smithi053 ceph-mon[35158]: from='osd.1 ' entity='osd.1' 2024-09-12T01:21:14.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:13 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:14.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:13 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:14.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:13 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:21:14.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:13 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:14.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:13 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:21:14.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:13 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:21:14.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:13 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:14.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:13 smithi001 ceph-mon[30387]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:21:14.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:13 smithi001 ceph-mon[30387]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-09-12T01:21:14.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:13 smithi001 ceph-mon[30387]: osdmap e12: 2 total, 1 up, 2 in 2024-09-12T01:21:14.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:13 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:21:14.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:13 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:21:14.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:13 smithi001 ceph-mon[30387]: from='osd.1 ' entity='osd.1' 2024-09-12T01:21:14.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:13 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:14.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:13 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:14.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:13 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:21:14.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:13 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:14.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:13 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:21:14.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:13 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:21:14.429 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:13 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:15.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:14 smithi104 ceph-mon[35219]: purged_snaps scrub starts 2024-09-12T01:21:15.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:14 smithi104 ceph-mon[35219]: purged_snaps scrub ok 2024-09-12T01:21:15.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:14 smithi104 ceph-mon[35219]: Detected new or changed devices on smithi053 2024-09-12T01:21:15.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:14 smithi104 ceph-mon[35219]: Adjusting osd_memory_target on smithi053 to 17092M 2024-09-12T01:21:15.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:14 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:21:15.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:14 smithi104 ceph-mon[35219]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:21:15.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:14 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:21:15.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:14 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:21:15.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:14 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:21:15.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:14 smithi104 ceph-mon[35219]: osd.1 [v2:172.21.15.53:6800/3131049165,v1:172.21.15.53:6801/3131049165] boot 2024-09-12T01:21:15.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:14 smithi104 ceph-mon[35219]: osdmap e13: 2 total, 2 up, 2 in 2024-09-12T01:21:15.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:14 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:21:15.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:14 smithi053 ceph-mon[35158]: purged_snaps scrub starts 2024-09-12T01:21:15.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:14 smithi053 ceph-mon[35158]: purged_snaps scrub ok 2024-09-12T01:21:15.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:14 smithi053 ceph-mon[35158]: Detected new or changed devices on smithi053 2024-09-12T01:21:15.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:14 smithi053 ceph-mon[35158]: Adjusting osd_memory_target on smithi053 to 17092M 2024-09-12T01:21:15.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:14 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:21:15.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:14 smithi053 ceph-mon[35158]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:21:15.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:14 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:21:15.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:14 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:21:15.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:14 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:21:15.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:14 smithi053 ceph-mon[35158]: osd.1 [v2:172.21.15.53:6800/3131049165,v1:172.21.15.53:6801/3131049165] boot 2024-09-12T01:21:15.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:14 smithi053 ceph-mon[35158]: osdmap e13: 2 total, 2 up, 2 in 2024-09-12T01:21:15.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:14 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:21:15.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:14 smithi001 ceph-mon[30387]: purged_snaps scrub starts 2024-09-12T01:21:15.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:14 smithi001 ceph-mon[30387]: purged_snaps scrub ok 2024-09-12T01:21:15.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:14 smithi001 ceph-mon[30387]: Detected new or changed devices on smithi053 2024-09-12T01:21:15.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:14 smithi001 ceph-mon[30387]: Adjusting osd_memory_target on smithi053 to 17092M 2024-09-12T01:21:15.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:14 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:21:15.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:14 smithi001 ceph-mon[30387]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:21:15.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:14 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:21:15.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:14 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:21:15.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:14 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:21:15.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:14 smithi001 ceph-mon[30387]: osd.1 [v2:172.21.15.53:6800/3131049165,v1:172.21.15.53:6801/3131049165] boot 2024-09-12T01:21:15.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:14 smithi001 ceph-mon[30387]: osdmap e13: 2 total, 2 up, 2 in 2024-09-12T01:21:15.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:14 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:21:16.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:16 smithi053 ceph-mon[35158]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:21:16.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:16 smithi053 ceph-mon[35158]: osdmap e14: 2 total, 2 up, 2 in 2024-09-12T01:21:16.768 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:16 smithi104 ceph-mon[35219]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:21:16.768 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:16 smithi104 ceph-mon[35219]: osdmap e14: 2 total, 2 up, 2 in 2024-09-12T01:21:16.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:16 smithi001 ceph-mon[30387]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:21:16.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:16 smithi001 ceph-mon[30387]: osdmap e14: 2 total, 2 up, 2 in 2024-09-12T01:21:17.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:17 smithi053 ceph-mon[35158]: from='client.? 172.21.15.104:0/4046635824' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d9d3603-ba7c-4115-9267-12abcb20d55b"}]: dispatch 2024-09-12T01:21:17.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:17 smithi053 ceph-mon[35158]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d9d3603-ba7c-4115-9267-12abcb20d55b"}]: dispatch 2024-09-12T01:21:17.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:17 smithi053 ceph-mon[35158]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d9d3603-ba7c-4115-9267-12abcb20d55b"}]': finished 2024-09-12T01:21:17.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:17 smithi053 ceph-mon[35158]: osdmap e15: 3 total, 2 up, 3 in 2024-09-12T01:21:17.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:17 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:21:17.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:17 smithi053 ceph-mon[35158]: from='client.? 172.21.15.104:0/2327107587' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:21:17.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:17 smithi104 ceph-mon[35219]: from='client.? 172.21.15.104:0/4046635824' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d9d3603-ba7c-4115-9267-12abcb20d55b"}]: dispatch 2024-09-12T01:21:17.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:17 smithi104 ceph-mon[35219]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d9d3603-ba7c-4115-9267-12abcb20d55b"}]: dispatch 2024-09-12T01:21:17.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:17 smithi104 ceph-mon[35219]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d9d3603-ba7c-4115-9267-12abcb20d55b"}]': finished 2024-09-12T01:21:17.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:17 smithi104 ceph-mon[35219]: osdmap e15: 3 total, 2 up, 3 in 2024-09-12T01:21:17.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:17 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:21:17.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:17 smithi104 ceph-mon[35219]: from='client.? 172.21.15.104:0/2327107587' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:21:17.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:17 smithi001 ceph-mon[30387]: from='client.? 172.21.15.104:0/4046635824' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d9d3603-ba7c-4115-9267-12abcb20d55b"}]: dispatch 2024-09-12T01:21:17.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:17 smithi001 ceph-mon[30387]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d9d3603-ba7c-4115-9267-12abcb20d55b"}]: dispatch 2024-09-12T01:21:17.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:17 smithi001 ceph-mon[30387]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d9d3603-ba7c-4115-9267-12abcb20d55b"}]': finished 2024-09-12T01:21:17.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:17 smithi001 ceph-mon[30387]: osdmap e15: 3 total, 2 up, 3 in 2024-09-12T01:21:17.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:17 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:21:17.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:17 smithi001 ceph-mon[30387]: from='client.? 172.21.15.104:0/2327107587' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:21:18.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:18 smithi053 ceph-mon[35158]: pgmap v66: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:21:18.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:18 smithi104 ceph-mon[35219]: pgmap v66: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:21:18.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:18 smithi001 ceph-mon[30387]: pgmap v66: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:21:20.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:20 smithi053 ceph-mon[35158]: pgmap v67: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:21:20.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:20 smithi104 ceph-mon[35219]: pgmap v67: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:21:20.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:20 smithi001 ceph-mon[30387]: pgmap v67: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:21:22.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:22 smithi053 ceph-mon[35158]: pgmap v68: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:21:22.803 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:22 smithi104 ceph-mon[35219]: pgmap v68: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:21:22.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:22 smithi001 ceph-mon[30387]: pgmap v68: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:21:24.704 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:24 smithi104 ceph-mon[35219]: pgmap v69: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:21:24.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:24 smithi053 ceph-mon[35158]: pgmap v69: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:21:24.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:24 smithi001 ceph-mon[30387]: pgmap v69: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:21:25.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:25 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-12T01:21:25.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:25 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:21:25.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:25 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-12T01:21:25.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:25 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:21:25.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:25 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-12T01:21:25.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:25 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:21:26.553 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:26 smithi104 ceph-mon[35219]: pgmap v70: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:21:26.553 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:26 smithi104 ceph-mon[35219]: Deploying daemon osd.2 on smithi104 2024-09-12T01:21:26.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:26 smithi053 ceph-mon[35158]: pgmap v70: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:21:26.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:26 smithi053 ceph-mon[35158]: Deploying daemon osd.2 on smithi104 2024-09-12T01:21:26.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:26 smithi001 ceph-mon[30387]: pgmap v70: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:21:26.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:26 smithi001 ceph-mon[30387]: Deploying daemon osd.2 on smithi104 2024-09-12T01:21:28.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:28 smithi104 ceph-mon[35219]: pgmap v71: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:21:28.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:28 smithi001 ceph-mon[30387]: pgmap v71: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:21:28.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:28 smithi053 ceph-mon[35158]: pgmap v71: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:21:30.777 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:30 smithi104 ceph-mon[35219]: pgmap v72: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:21:30.923 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:30 smithi053 ceph-mon[35158]: pgmap v72: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:21:30.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:30 smithi001 ceph-mon[30387]: pgmap v72: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:21:31.807 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:31 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:21:31.807 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:31 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:31.807 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:31 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:31.899 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:31 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:21:31.899 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:31 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:31.899 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:31 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:31.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:31 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:21:31.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:31 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:31.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:31 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:32.700 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:32 smithi104 ceph-mon[35219]: pgmap v73: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:21:32.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:32 smithi001 ceph-mon[30387]: pgmap v73: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:21:32.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:32 smithi053 ceph-mon[35158]: pgmap v73: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:21:34.560 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:34 smithi104 ceph-mon[35219]: pgmap v74: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:21:34.561 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:34 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:34.561 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:34 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:34.561 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:34 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:21:34.561 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:34 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:21:34.561 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:34 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:34.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:34 smithi001 ceph-mon[30387]: pgmap v74: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:21:34.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:34 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:34.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:34 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:34.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:34 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:21:34.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:34 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:21:34.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:34 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:34.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:34 smithi053 ceph-mon[35158]: pgmap v74: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:21:34.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:34 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:34.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:34 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:34.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:34 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:21:34.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:34 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:21:34.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:34 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:35.200 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:35.201+0000 7f0722ffd640 1 -- 172.21.15.104:0/3533164470 <== mgr.14150 v2:172.21.15.1:6800/1928662456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f072c062630 con 0x7f07180a2af0 2024-09-12T01:21:35.201 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 2 on host 'smithi104' 2024-09-12T01:21:35.203 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:35.203+0000 7f0734023640 1 -- 172.21.15.104:0/3533164470 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f07180a2af0 msgr2=0x7f07180a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:21:35.203 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:35.203+0000 7f0734023640 1 --2- 172.21.15.104:0/3533164470 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f07180a2af0 0x7f07180a4fb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f07280314e0 tx=0x7f0728000bc0 comp rx=0 tx=0).stop 2024-09-12T01:21:35.203 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:35.203+0000 7f0734023640 1 -- 172.21.15.104:0/3533164470 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f072c1078c0 msgr2=0x7f072c100b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:21:35.203 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:35.203+0000 7f0734023640 1 --2- 172.21.15.104:0/3533164470 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f072c1078c0 0x7f072c100b50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f072400eb10 tx=0x7f072400c2d0 comp rx=0 tx=0).stop 2024-09-12T01:21:35.203 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:35.203+0000 7f0734023640 1 -- 172.21.15.104:0/3533164470 shutdown_connections 2024-09-12T01:21:35.203 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:35.203+0000 7f0734023640 1 --2- 172.21.15.104:0/3533164470 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f07180a2af0 0x7f07180a4fb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:21:35.203 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:35.203+0000 7f0734023640 1 --2- 172.21.15.104:0/3533164470 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f072c1078c0 0x7f072c100b50 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:21:35.204 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:35.203+0000 7f0734023640 1 --2- 172.21.15.104:0/3533164470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f072c104f90 0x7f072c100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:21:35.204 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:35.203+0000 7f0734023640 1 --2- 172.21.15.104:0/3533164470 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f072c102660 0x7f072c101fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:21:35.204 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:35.203+0000 7f0734023640 1 -- 172.21.15.104:0/3533164470 >> 172.21.15.104:0/3533164470 conn(0x7f072c0fc280 msgr2=0x7f072c1030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:21:35.204 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:35.203+0000 7f0734023640 1 -- 172.21.15.104:0/3533164470 shutdown_connections 2024-09-12T01:21:35.204 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:21:35.204+0000 7f0734023640 1 -- 172.21.15.104:0/3533164470 wait complete. 2024-09-12T01:21:35.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:35 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:21:35.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:35 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:35.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:35 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:35.897 DEBUG:teuthology.orchestra.run.smithi104:osd.2> sudo journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@osd.2.service 2024-09-12T01:21:35.900 INFO:tasks.cephadm:Deploying osd.3 on smithi001 with /dev/vg_nvme/lv_3... 2024-09-12T01:21:35.900 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-12T01:21:35.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:35 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:21:35.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:35 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:35.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:35 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:35.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:35 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:21:35.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:35 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:35.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:35 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:36.277 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:21:36.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:36 smithi001 ceph-mon[30387]: pgmap v75: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:21:36.819 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:36 smithi104 ceph-mon[35219]: pgmap v75: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:21:36.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:36 smithi053 ceph-mon[35158]: pgmap v75: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:21:37.075 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 12 01:21:36 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-2[40211]: 2024-09-12T01:21:36.839+0000 7f05669c5640 -1 osd.2 0 log_to_monitors true 2024-09-12T01:21:37.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:37 smithi104 ceph-mon[35219]: from='osd.2 [v2:172.21.15.104:6800/1133285636,v1:172.21.15.104:6801/1133285636]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T01:21:37.773 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:37 smithi104 ceph-mon[35219]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T01:21:37.788 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:37 smithi001 ceph-mon[30387]: from='osd.2 [v2:172.21.15.104:6800/1133285636,v1:172.21.15.104:6801/1133285636]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T01:21:37.788 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:37 smithi001 ceph-mon[30387]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T01:21:37.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:37 smithi053 ceph-mon[35158]: from='osd.2 [v2:172.21.15.104:6800/1133285636,v1:172.21.15.104:6801/1133285636]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T01:21:37.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:37 smithi053 ceph-mon[35158]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T01:21:38.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:38 smithi104 ceph-mon[35219]: pgmap v76: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:21:38.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:38 smithi104 ceph-mon[35219]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-12T01:21:38.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:38 smithi104 ceph-mon[35219]: osdmap e16: 3 total, 2 up, 3 in 2024-09-12T01:21:38.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:38 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:21:38.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:38 smithi104 ceph-mon[35219]: from='osd.2 [v2:172.21.15.104:6800/1133285636,v1:172.21.15.104:6801/1133285636]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-12T01:21:38.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:38 smithi104 ceph-mon[35219]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-12T01:21:38.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:38 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:38.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:38 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:38.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:38 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:21:38.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:38 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:38.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:38 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:21:38.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:38 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:21:38.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:38 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:38.908 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:38 smithi001 ceph-mon[30387]: pgmap v76: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:21:38.908 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:38 smithi001 ceph-mon[30387]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-12T01:21:38.908 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:38 smithi001 ceph-mon[30387]: osdmap e16: 3 total, 2 up, 3 in 2024-09-12T01:21:38.908 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:38 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:21:38.908 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:38 smithi001 ceph-mon[30387]: from='osd.2 [v2:172.21.15.104:6800/1133285636,v1:172.21.15.104:6801/1133285636]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-12T01:21:38.908 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:38 smithi001 ceph-mon[30387]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-12T01:21:38.908 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:38 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:38.909 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:38 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:38.909 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:38 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:21:38.909 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:38 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:38.909 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:38 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:21:38.909 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:38 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:21:38.909 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:38 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:38.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:38 smithi053 ceph-mon[35158]: pgmap v76: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:21:38.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:38 smithi053 ceph-mon[35158]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-12T01:21:38.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:38 smithi053 ceph-mon[35158]: osdmap e16: 3 total, 2 up, 3 in 2024-09-12T01:21:38.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:38 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:21:38.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:38 smithi053 ceph-mon[35158]: from='osd.2 [v2:172.21.15.104:6800/1133285636,v1:172.21.15.104:6801/1133285636]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-12T01:21:38.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:38 smithi053 ceph-mon[35158]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-12T01:21:38.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:38 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:38.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:38 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:38.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:38 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:21:38.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:38 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:38.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:38 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:21:38.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:38 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:21:38.991 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:38 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:39.531 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 12 01:21:39 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-2[40211]: 2024-09-12T01:21:39.167+0000 7f056315c640 -1 osd.2 0 waiting for initial osdmap 2024-09-12T01:21:39.531 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 12 01:21:39 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-2[40211]: 2024-09-12T01:21:39.172+0000 7f055a768640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T01:21:39.794 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:39 smithi001 ceph-mon[30387]: Detected new or changed devices on smithi104 2024-09-12T01:21:39.794 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:39 smithi001 ceph-mon[30387]: Adjusting osd_memory_target on smithi104 to 17088M 2024-09-12T01:21:39.794 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:39 smithi001 ceph-mon[30387]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-12T01:21:39.794 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:39 smithi001 ceph-mon[30387]: osdmap e17: 3 total, 2 up, 3 in 2024-09-12T01:21:39.794 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:39 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:21:39.794 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:39 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:21:39.795 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:39 smithi001 ceph-mon[30387]: from='osd.2 ' entity='osd.2' 2024-09-12T01:21:39.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:39 smithi104 ceph-mon[35219]: Detected new or changed devices on smithi104 2024-09-12T01:21:39.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:39 smithi104 ceph-mon[35219]: Adjusting osd_memory_target on smithi104 to 17088M 2024-09-12T01:21:39.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:39 smithi104 ceph-mon[35219]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-12T01:21:39.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:39 smithi104 ceph-mon[35219]: osdmap e17: 3 total, 2 up, 3 in 2024-09-12T01:21:39.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:39 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:21:39.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:39 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:21:39.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:39 smithi104 ceph-mon[35219]: from='osd.2 ' entity='osd.2' 2024-09-12T01:21:39.921 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:21:39.953 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi001:vg_nvme/lv_3 2024-09-12T01:21:39.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:39 smithi053 ceph-mon[35158]: Detected new or changed devices on smithi104 2024-09-12T01:21:39.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:39 smithi053 ceph-mon[35158]: Adjusting osd_memory_target on smithi104 to 17088M 2024-09-12T01:21:39.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:39 smithi053 ceph-mon[35158]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-12T01:21:39.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:39 smithi053 ceph-mon[35158]: osdmap e17: 3 total, 2 up, 3 in 2024-09-12T01:21:39.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:39 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:21:39.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:39 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:21:39.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:39 smithi053 ceph-mon[35158]: from='osd.2 ' entity='osd.2' 2024-09-12T01:21:40.363 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:21:40.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:40 smithi104 ceph-mon[35219]: purged_snaps scrub starts 2024-09-12T01:21:40.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:40 smithi104 ceph-mon[35219]: purged_snaps scrub ok 2024-09-12T01:21:40.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:40 smithi104 ceph-mon[35219]: pgmap v79: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:21:40.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:40 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:21:40.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:40 smithi104 ceph-mon[35219]: osd.2 [v2:172.21.15.104:6800/1133285636,v1:172.21.15.104:6801/1133285636] boot 2024-09-12T01:21:40.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:40 smithi104 ceph-mon[35219]: osdmap e18: 3 total, 3 up, 3 in 2024-09-12T01:21:40.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:40 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:21:40.883 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:40 smithi001 ceph-mon[30387]: purged_snaps scrub starts 2024-09-12T01:21:40.884 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:40 smithi001 ceph-mon[30387]: purged_snaps scrub ok 2024-09-12T01:21:40.884 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:40 smithi001 ceph-mon[30387]: pgmap v79: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:21:40.884 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:40 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:21:40.884 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:40 smithi001 ceph-mon[30387]: osd.2 [v2:172.21.15.104:6800/1133285636,v1:172.21.15.104:6801/1133285636] boot 2024-09-12T01:21:40.884 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:40 smithi001 ceph-mon[30387]: osdmap e18: 3 total, 3 up, 3 in 2024-09-12T01:21:40.884 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:40 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:21:40.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:40 smithi053 ceph-mon[35158]: purged_snaps scrub starts 2024-09-12T01:21:40.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:40 smithi053 ceph-mon[35158]: purged_snaps scrub ok 2024-09-12T01:21:40.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:40 smithi053 ceph-mon[35158]: pgmap v79: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:21:40.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:40 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:21:40.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:40 smithi053 ceph-mon[35158]: osd.2 [v2:172.21.15.104:6800/1133285636,v1:172.21.15.104:6801/1133285636] boot 2024-09-12T01:21:40.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:40 smithi053 ceph-mon[35158]: osdmap e18: 3 total, 3 up, 3 in 2024-09-12T01:21:40.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:40 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:21:41.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:21:41.001+0000 7fcb41200640 1 -- 172.21.15.1:0/3882529249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb3c078790 msgr2=0x7fcb3c079030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:21:41.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:21:41.001+0000 7fcb41200640 1 --2- 172.21.15.1:0/3882529249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb3c078790 0x7fcb3c079030 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcb34009ca0 tx=0x7fcb3402a8c0 comp rx=0 tx=0).stop 2024-09-12T01:21:41.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:21:41.001+0000 7fcb41200640 1 -- 172.21.15.1:0/3882529249 shutdown_connections 2024-09-12T01:21:41.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:21:41.001+0000 7fcb41200640 1 --2- 172.21.15.1:0/3882529249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb3c078790 0x7fcb3c079030 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:21:41.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:21:41.001+0000 7fcb41200640 1 --2- 172.21.15.1:0/3882529249 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb3c077df0 0x7fcb3c078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:21:41.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:21:41.001+0000 7fcb41200640 1 --2- 172.21.15.1:0/3882529249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb3c076ba0 0x7fcb3c076fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:21:41.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:21:41.001+0000 7fcb41200640 1 -- 172.21.15.1:0/3882529249 >> 172.21.15.1:0/3882529249 conn(0x7fcb3c0ff5b0 msgr2=0x7fcb3c1019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:21:41.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:21:41.001+0000 7fcb41200640 1 -- 172.21.15.1:0/3882529249 shutdown_connections 2024-09-12T01:21:41.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:21:41.002+0000 7fcb41200640 1 -- 172.21.15.1:0/3882529249 wait complete. 2024-09-12T01:21:41.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:21:41.002+0000 7fcb41200640 1 Processor -- start 2024-09-12T01:21:41.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:21:41.002+0000 7fcb41200640 1 -- start start 2024-09-12T01:21:41.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:21:41.003+0000 7fcb41200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb3c076ba0 0x7fcb3c19e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:21:41.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:21:41.003+0000 7fcb41200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb3c077df0 0x7fcb3c19e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:21:41.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:21:41.003+0000 7fcb41200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb3c078790 0x7fcb3c1a5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:21:41.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:21:41.003+0000 7fcb41200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3c113f10 con 0x7fcb3c078790 2024-09-12T01:21:41.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:21:41.003+0000 7fcb41200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3c113d90 con 0x7fcb3c077df0 2024-09-12T01:21:41.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:21:41.003+0000 7fcb41200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3c114090 con 0x7fcb3c076ba0 2024-09-12T01:21:41.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:21:41.003+0000 7fcb39e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb3c076ba0 0x7fcb3c19e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:21:41.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:21:41.003+0000 7fcb39e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb3c076ba0 0x7fcb3c19e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:40862/0 (socket says 172.21.15.1:40862) 2024-09-12T01:21:41.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:21:41.003+0000 7fcb39e00640 1 -- 172.21.15.1:0/2732766532 learned_addr learned my addr 172.21.15.1:0/2732766532 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:21:41.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:21:41.003+0000 7fcb39400640 1 --2- 172.21.15.1:0/2732766532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb3c077df0 0x7fcb3c19e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:21:41.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:21:41.003+0000 7fcb3a800640 1 --2- 172.21.15.1:0/2732766532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb3c078790 0x7fcb3c1a5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:21:41.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:21:41.004+0000 7fcb39e00640 1 -- 172.21.15.1:0/2732766532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb3c077df0 msgr2=0x7fcb3c19e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:21:41.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:21:41.004+0000 7fcb39e00640 1 --2- 172.21.15.1:0/2732766532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb3c077df0 0x7fcb3c19e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:21:41.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:21:41.004+0000 7fcb39e00640 1 -- 172.21.15.1:0/2732766532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb3c078790 msgr2=0x7fcb3c1a5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:21:41.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:21:41.004+0000 7fcb39e00640 1 --2- 172.21.15.1:0/2732766532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb3c078790 0x7fcb3c1a5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:21:41.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:21:41.004+0000 7fcb39e00640 1 -- 172.21.15.1:0/2732766532 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb3c1a60a0 con 0x7fcb3c076ba0 2024-09-12T01:21:41.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:21:41.004+0000 7fcb39e00640 1 --2- 172.21.15.1:0/2732766532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb3c076ba0 0x7fcb3c19e470 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c00c890 tx=0x7fcb2c00cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:21:41.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:21:41.004+0000 7fcb32a00640 1 -- 172.21.15.1:0/2732766532 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c004220 con 0x7fcb3c076ba0 2024-09-12T01:21:41.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:21:41.004+0000 7fcb32a00640 1 -- 172.21.15.1:0/2732766532 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb2c0043c0 con 0x7fcb3c076ba0 2024-09-12T01:21:41.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:21:41.004+0000 7fcb32a00640 1 -- 172.21.15.1:0/2732766532 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c0049e0 con 0x7fcb3c076ba0 2024-09-12T01:21:41.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:21:41.004+0000 7fcb41200640 1 -- 172.21.15.1:0/2732766532 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb3c1a6390 con 0x7fcb3c076ba0 2024-09-12T01:21:41.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:21:41.004+0000 7fcb41200640 1 -- 172.21.15.1:0/2732766532 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb3c1a6880 con 0x7fcb3c076ba0 2024-09-12T01:21:41.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:21:41.005+0000 7fcb41200640 1 -- 172.21.15.1:0/2732766532 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb3c110780 con 0x7fcb3c076ba0 2024-09-12T01:21:41.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:21:41.007+0000 7fcb32a00640 1 -- 172.21.15.1:0/2732766532 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7fcb2c0079b0 con 0x7fcb3c076ba0 2024-09-12T01:21:41.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:21:41.009+0000 7fcb32a00640 1 --2- 172.21.15.1:0/2732766532 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fcb200a2ba0 0x7fcb200a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:21:41.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:21:41.009+0000 7fcb32a00640 1 -- 172.21.15.1:0/2732766532 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2851+0+0 (secure 0 0 0) 0x7fcb2c0cdb90 con 0x7fcb3c076ba0 2024-09-12T01:21:41.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:21:41.009+0000 7fcb39400640 1 --2- 172.21.15.1:0/2732766532 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fcb200a2ba0 0x7fcb200a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:21:41.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:21:41.010+0000 7fcb39400640 1 --2- 172.21.15.1:0/2732766532 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fcb200a2ba0 0x7fcb200a5060 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c19f990 tx=0x7fcb28005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:21:41.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:21:41.011+0000 7fcb32a00640 1 -- 172.21.15.1:0/2732766532 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcb2c014030 con 0x7fcb3c076ba0 2024-09-12T01:21:41.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:21:41.207+0000 7fcb41200640 1 -- 172.21.15.1:0/2732766532 --> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fcb3c062630 con 0x7fcb200a2ba0 2024-09-12T01:21:41.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:41 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-12T01:21:41.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:41 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:21:41.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:41 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:21:41.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:41 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:21:41.804 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:41 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-12T01:21:41.804 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:41 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:21:41.804 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:41 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:21:41.804 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:41 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:21:41.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:41 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-12T01:21:41.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:41 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:21:41.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:41 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:21:41.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:41 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:21:42.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:42 smithi104 ceph-mon[35219]: pgmap v81: 0 pgs: ; 0 B data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:21:42.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:42 smithi104 ceph-mon[35219]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:21:42.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:42 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-12T01:21:42.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:42 smithi104 ceph-mon[35219]: osdmap e19: 3 total, 3 up, 3 in 2024-09-12T01:21:42.826 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:42 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T01:21:42.865 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:42 smithi001 ceph-mon[30387]: pgmap v81: 0 pgs: ; 0 B data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:21:42.865 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:42 smithi001 ceph-mon[30387]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:21:42.865 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:42 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-12T01:21:42.865 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:42 smithi001 ceph-mon[30387]: osdmap e19: 3 total, 3 up, 3 in 2024-09-12T01:21:42.866 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:42 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T01:21:42.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:42 smithi053 ceph-mon[35158]: pgmap v81: 0 pgs: ; 0 B data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:21:42.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:42 smithi053 ceph-mon[35158]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:21:42.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:42 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-12T01:21:42.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:42 smithi053 ceph-mon[35158]: osdmap e19: 3 total, 3 up, 3 in 2024-09-12T01:21:42.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:42 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T01:21:43.473 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 12 01:21:43 smithi001 sudo[48691]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T01:21:43.474 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 12 01:21:43 smithi001 sudo[48691]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T01:21:43.474 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 12 01:21:43 smithi001 sudo[48691]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T01:21:43.474 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 12 01:21:43 smithi001 sudo[48691]: pam_unix(sudo:session): session closed for user root 2024-09-12T01:21:43.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:21:43.555+0000 7fcb32a00640 1 -- 172.21.15.1:0/2732766532 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fcb2c098070 con 0x7fcb3c076ba0 2024-09-12T01:21:43.787 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:43 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-12T01:21:43.788 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:43 smithi001 ceph-mon[30387]: osdmap e20: 3 total, 3 up, 3 in 2024-09-12T01:21:43.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:43 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-12T01:21:43.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:43 smithi104 ceph-mon[35219]: osdmap e20: 3 total, 3 up, 3 in 2024-09-12T01:21:43.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:43 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-12T01:21:43.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:43 smithi053 ceph-mon[35158]: osdmap e20: 3 total, 3 up, 3 in 2024-09-12T01:21:44.089 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 12 01:21:43 smithi001 sudo[48701]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T01:21:44.089 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 12 01:21:43 smithi001 sudo[48701]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T01:21:44.089 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 12 01:21:43 smithi001 sudo[48701]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T01:21:44.089 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 12 01:21:43 smithi001 sudo[48701]: pam_unix(sudo:session): session closed for user root 2024-09-12T01:21:44.354 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 12 01:21:44 smithi053 sudo[43620]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T01:21:44.354 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 12 01:21:44 smithi053 sudo[43620]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T01:21:44.354 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 12 01:21:44 smithi053 sudo[43620]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T01:21:44.354 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 12 01:21:44 smithi053 sudo[43620]: pam_unix(sudo:session): session closed for user root 2024-09-12T01:21:44.739 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 12 01:21:44 smithi053 sudo[43623]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T01:21:44.739 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 12 01:21:44 smithi053 sudo[43623]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T01:21:44.740 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 12 01:21:44 smithi053 sudo[43623]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T01:21:44.740 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 12 01:21:44 smithi053 sudo[43623]: pam_unix(sudo:session): session closed for user root 2024-09-12T01:21:44.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:44 smithi053 ceph-mon[35158]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:21:44.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:44 smithi053 ceph-mon[35158]: osdmap e21: 3 total, 3 up, 3 in 2024-09-12T01:21:44.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:44 smithi053 ceph-mon[35158]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-12T01:21:44.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:44 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:44.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:44 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:44.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:44 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/3762457491' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e063c66-41b4-4db1-a964-28e122ce628e"}]: dispatch 2024-09-12T01:21:44.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:44 smithi053 ceph-mon[35158]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e063c66-41b4-4db1-a964-28e122ce628e"}]: dispatch 2024-09-12T01:21:44.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:44 smithi053 ceph-mon[35158]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e063c66-41b4-4db1-a964-28e122ce628e"}]': finished 2024-09-12T01:21:44.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:44 smithi053 ceph-mon[35158]: osdmap e22: 4 total, 3 up, 4 in 2024-09-12T01:21:44.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:44 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:21:44.824 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 12 01:21:44 smithi104 sudo[43833]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T01:21:44.824 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 12 01:21:44 smithi104 sudo[43833]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T01:21:44.825 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 12 01:21:44 smithi104 sudo[43833]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T01:21:44.825 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 12 01:21:44 smithi104 sudo[43833]: pam_unix(sudo:session): session closed for user root 2024-09-12T01:21:44.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:44 smithi104 ceph-mon[35219]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:21:44.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:44 smithi104 ceph-mon[35219]: osdmap e21: 3 total, 3 up, 3 in 2024-09-12T01:21:44.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:44 smithi104 ceph-mon[35219]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-12T01:21:44.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:44 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:44.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:44 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:44.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:44 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/3762457491' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e063c66-41b4-4db1-a964-28e122ce628e"}]: dispatch 2024-09-12T01:21:44.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:44 smithi104 ceph-mon[35219]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e063c66-41b4-4db1-a964-28e122ce628e"}]: dispatch 2024-09-12T01:21:44.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:44 smithi104 ceph-mon[35219]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e063c66-41b4-4db1-a964-28e122ce628e"}]': finished 2024-09-12T01:21:44.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:44 smithi104 ceph-mon[35219]: osdmap e22: 4 total, 3 up, 4 in 2024-09-12T01:21:44.826 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:44 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:21:44.848 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:44 smithi001 ceph-mon[30387]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:21:44.849 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:44 smithi001 ceph-mon[30387]: osdmap e21: 3 total, 3 up, 3 in 2024-09-12T01:21:44.849 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:44 smithi001 ceph-mon[30387]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-12T01:21:44.849 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:44 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:44.849 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:44 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:44.849 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:44 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/3762457491' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e063c66-41b4-4db1-a964-28e122ce628e"}]: dispatch 2024-09-12T01:21:44.849 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:44 smithi001 ceph-mon[30387]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e063c66-41b4-4db1-a964-28e122ce628e"}]: dispatch 2024-09-12T01:21:44.849 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:44 smithi001 ceph-mon[30387]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e063c66-41b4-4db1-a964-28e122ce628e"}]': finished 2024-09-12T01:21:44.849 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:44 smithi001 ceph-mon[30387]: osdmap e22: 4 total, 3 up, 4 in 2024-09-12T01:21:44.849 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:44 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:21:45.324 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 12 01:21:44 smithi104 sudo[43836]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T01:21:45.325 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 12 01:21:44 smithi104 sudo[43836]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T01:21:45.325 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 12 01:21:44 smithi104 sudo[43836]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T01:21:45.325 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 12 01:21:44 smithi104 sudo[43836]: pam_unix(sudo:session): session closed for user root 2024-09-12T01:21:45.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:45 smithi001 sudo[48798]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T01:21:45.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:45 smithi001 sudo[48798]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T01:21:45.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:45 smithi001 sudo[48798]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T01:21:45.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:45 smithi001 sudo[48798]: pam_unix(sudo:session): session closed for user root 2024-09-12T01:21:46.296 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:46 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:46.296 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:46 smithi104 ceph-mon[35219]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T01:21:46.296 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:46 smithi104 sudo[43839]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T01:21:46.296 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:46 smithi104 sudo[43839]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T01:21:46.296 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:46 smithi104 sudo[43839]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T01:21:46.297 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:46 smithi104 sudo[43839]: pam_unix(sudo:session): session closed for user root 2024-09-12T01:21:46.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:46 smithi001 sudo[48829]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T01:21:46.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:46 smithi001 sudo[48829]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T01:21:46.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:46 smithi001 sudo[48829]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T01:21:46.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:46 smithi001 sudo[48829]: pam_unix(sudo:session): session closed for user root 2024-09-12T01:21:46.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:46 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:46.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:46 smithi001 ceph-mon[30387]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T01:21:46.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:46 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:21:46.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:46 smithi053 ceph-mon[35158]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T01:21:46.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:46 smithi104 sudo[43842]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T01:21:46.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:46 smithi104 sudo[43842]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T01:21:46.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:46 smithi104 sudo[43842]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T01:21:46.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:46 smithi104 sudo[43842]: pam_unix(sudo:session): session closed for user root 2024-09-12T01:21:47.058 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:46 smithi053 sudo[43626]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T01:21:47.058 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:46 smithi053 sudo[43626]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T01:21:47.058 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:46 smithi053 sudo[43626]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T01:21:47.058 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:46 smithi053 sudo[43626]: pam_unix(sudo:session): session closed for user root 2024-09-12T01:21:47.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:47 smithi104 ceph-mon[35219]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:21:47.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:47 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/930732823' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:21:47.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:47 smithi104 ceph-mon[35219]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T01:21:47.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:47 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:21:47.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:47 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:21:47.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:47 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:21:47.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:47 smithi104 ceph-mon[35219]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T01:21:47.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:47 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:21:47.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:47 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:21:47.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:47 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:21:47.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:47 smithi104 ceph-mon[35219]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T01:21:47.327 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:47 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:21:47.327 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:47 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:21:47.327 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:47 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:21:47.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:47 smithi001 ceph-mon[30387]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:21:47.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:47 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/930732823' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:21:47.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:47 smithi001 ceph-mon[30387]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T01:21:47.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:47 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:21:47.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:47 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:21:47.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:47 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:21:47.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:47 smithi001 ceph-mon[30387]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T01:21:47.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:47 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:21:47.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:47 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:21:47.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:47 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:21:47.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:47 smithi001 ceph-mon[30387]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T01:21:47.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:47 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:21:47.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:47 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:21:47.429 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:47 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:21:47.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:47 smithi053 sudo[43629]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T01:21:47.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:47 smithi053 sudo[43629]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T01:21:47.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:47 smithi053 sudo[43629]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T01:21:47.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:47 smithi053 sudo[43629]: pam_unix(sudo:session): session closed for user root 2024-09-12T01:21:47.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:47 smithi053 ceph-mon[35158]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:21:47.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:47 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/930732823' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:21:47.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:47 smithi053 ceph-mon[35158]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T01:21:47.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:47 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:21:47.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:47 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:21:47.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:47 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:21:47.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:47 smithi053 ceph-mon[35158]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T01:21:47.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:47 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:21:47.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:47 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:21:47.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:47 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:21:47.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:47 smithi053 ceph-mon[35158]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T01:21:47.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:47 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:21:47.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:47 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:21:47.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:47 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:21:48.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:48 smithi104 ceph-mon[35219]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T01:21:48.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:48 smithi104 ceph-mon[35219]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:21:48.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:48 smithi104 ceph-mon[35219]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T01:21:48.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:48 smithi001 ceph-mon[30387]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T01:21:48.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:48 smithi001 ceph-mon[30387]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:21:48.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:48 smithi001 ceph-mon[30387]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T01:21:48.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:48 smithi053 ceph-mon[35158]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T01:21:48.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:48 smithi053 ceph-mon[35158]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:21:48.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:48 smithi053 ceph-mon[35158]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T01:21:50.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:50 smithi104 ceph-mon[35219]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:21:50.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:50 smithi001 ceph-mon[30387]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:21:50.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:50 smithi053 ceph-mon[35158]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:21:52.415 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:52 smithi001 ceph-mon[30387]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-09-12T01:21:52.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:52 smithi053 ceph-mon[35158]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-09-12T01:21:52.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:52 smithi104 ceph-mon[35219]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-09-12T01:21:54.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:54 smithi001 ceph-mon[30387]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-12T01:21:54.377 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:54 smithi104 ceph-mon[35219]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-12T01:21:54.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:54 smithi053 ceph-mon[35158]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-12T01:21:55.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:55 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-12T01:21:55.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:55 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:21:55.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:55 smithi001 ceph-mon[30387]: Deploying daemon osd.3 on smithi001 2024-09-12T01:21:55.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:55 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-12T01:21:55.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:55 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:21:55.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:55 smithi053 ceph-mon[35158]: Deploying daemon osd.3 on smithi001 2024-09-12T01:21:55.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:55 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-12T01:21:55.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:55 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:21:55.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:55 smithi104 ceph-mon[35219]: Deploying daemon osd.3 on smithi001 2024-09-12T01:21:56.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:56 smithi001 ceph-mon[30387]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-12T01:21:56.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:56 smithi053 ceph-mon[35158]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-12T01:21:56.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:56 smithi104 ceph-mon[35219]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-12T01:21:58.307 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:21:58 smithi001 ceph-mon[30387]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-12T01:21:58.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:21:58 smithi053 ceph-mon[35158]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-12T01:21:58.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:21:58 smithi104 ceph-mon[35219]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-12T01:22:00.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:00 smithi001 ceph-mon[30387]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-12T01:22:00.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:00 smithi053 ceph-mon[35158]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-12T01:22:00.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:00 smithi104 ceph-mon[35219]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-12T01:22:02.216 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:02 smithi001 ceph-mon[30387]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-12T01:22:02.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:02 smithi053 ceph-mon[35158]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-12T01:22:02.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:02 smithi104 ceph-mon[35219]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-12T01:22:03.345 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:03 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:22:03.345 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:03 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:03.345 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:03 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:03.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:03 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:22:03.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:03 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:03.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:03 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:03.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:03 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:22:03.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:03 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:03.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:03 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:04.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:04 smithi053 ceph-mon[35158]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:22:04.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:04 smithi104 ceph-mon[35219]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:22:04.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:04 smithi001 ceph-mon[30387]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:22:06.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:06 smithi001 ceph-mon[30387]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:22:06.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:06 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:06.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:06 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:06.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:06 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:22:06.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:06 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:22:06.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:06 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:06.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:06 smithi053 ceph-mon[35158]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:22:06.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:06 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:06.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:06 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:06.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:06 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:22:06.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:06 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:22:06.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:06 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:06.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:06 smithi104 ceph-mon[35219]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:22:06.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:06 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:06.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:06 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:06.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:06 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:22:06.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:06 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:22:06.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:06 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:07.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:22:07.349+0000 7fcb32a00640 1 -- 172.21.15.1:0/2732766532 <== mgr.14150 v2:172.21.15.1:6800/1928662456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fcb3c062630 con 0x7fcb200a2ba0 2024-09-12T01:22:07.350 INFO:teuthology.orchestra.run.smithi001.stdout:Created osd(s) 3 on host 'smithi001' 2024-09-12T01:22:07.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:22:07.352+0000 7fcb41200640 1 -- 172.21.15.1:0/2732766532 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fcb200a2ba0 msgr2=0x7fcb200a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:22:07.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:22:07.352+0000 7fcb41200640 1 --2- 172.21.15.1:0/2732766532 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fcb200a2ba0 0x7fcb200a5060 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c19f990 tx=0x7fcb28005f90 comp rx=0 tx=0).stop 2024-09-12T01:22:07.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:22:07.352+0000 7fcb41200640 1 -- 172.21.15.1:0/2732766532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb3c076ba0 msgr2=0x7fcb3c19e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:22:07.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:22:07.352+0000 7fcb41200640 1 --2- 172.21.15.1:0/2732766532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb3c076ba0 0x7fcb3c19e470 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c00c890 tx=0x7fcb2c00cd50 comp rx=0 tx=0).stop 2024-09-12T01:22:07.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:22:07.352+0000 7fcb41200640 1 -- 172.21.15.1:0/2732766532 shutdown_connections 2024-09-12T01:22:07.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:22:07.352+0000 7fcb41200640 1 --2- 172.21.15.1:0/2732766532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb3c078790 0x7fcb3c1a5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:22:07.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:22:07.352+0000 7fcb41200640 1 --2- 172.21.15.1:0/2732766532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb3c077df0 0x7fcb3c19e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:22:07.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:22:07.352+0000 7fcb41200640 1 --2- 172.21.15.1:0/2732766532 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fcb200a2ba0 0x7fcb200a5060 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:22:07.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:22:07.352+0000 7fcb41200640 1 --2- 172.21.15.1:0/2732766532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb3c076ba0 0x7fcb3c19e470 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:22:07.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:22:07.352+0000 7fcb41200640 1 -- 172.21.15.1:0/2732766532 >> 172.21.15.1:0/2732766532 conn(0x7fcb3c0ff5b0 msgr2=0x7fcb3c101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:22:07.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:22:07.352+0000 7fcb41200640 1 -- 172.21.15.1:0/2732766532 shutdown_connections 2024-09-12T01:22:07.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:22:07.352+0000 7fcb41200640 1 -- 172.21.15.1:0/2732766532 wait complete. 2024-09-12T01:22:08.090 DEBUG:teuthology.orchestra.run.smithi001:osd.3> sudo journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@osd.3.service 2024-09-12T01:22:08.094 INFO:tasks.cephadm:Deploying osd.4 on smithi053 with /dev/vg_nvme/lv_3... 2024-09-12T01:22:08.094 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-12T01:22:08.325 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:08 smithi001 ceph-mon[30387]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:22:08.325 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:08 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:22:08.325 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:08 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:08.325 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:08 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:08.325 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:08 smithi001 ceph-mon[30387]: from='osd.3 [v2:172.21.15.1:6810/1247468803,v1:172.21.15.1:6811/1247468803]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T01:22:08.326 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:08 smithi001 ceph-mon[30387]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T01:22:08.352 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T01:22:08.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:08 smithi053 ceph-mon[35158]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:22:08.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:08 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:22:08.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:08 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:08.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:08 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:08.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:08 smithi053 ceph-mon[35158]: from='osd.3 [v2:172.21.15.1:6810/1247468803,v1:172.21.15.1:6811/1247468803]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T01:22:08.473 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:08 smithi053 ceph-mon[35158]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T01:22:08.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:08 smithi104 ceph-mon[35219]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:22:08.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:08 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:22:08.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:08 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:08.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:08 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:08.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:08 smithi104 ceph-mon[35219]: from='osd.3 [v2:172.21.15.1:6810/1247468803,v1:172.21.15.1:6811/1247468803]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T01:22:08.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:08 smithi104 ceph-mon[35219]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T01:22:09.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:09 smithi001 ceph-mon[30387]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-12T01:22:09.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:09 smithi001 ceph-mon[30387]: osdmap e23: 4 total, 3 up, 4 in 2024-09-12T01:22:09.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:09 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:22:09.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:09 smithi001 ceph-mon[30387]: from='osd.3 [v2:172.21.15.1:6810/1247468803,v1:172.21.15.1:6811/1247468803]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-12T01:22:09.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:09 smithi001 ceph-mon[30387]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-12T01:22:09.624 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:09 smithi053 ceph-mon[35158]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-12T01:22:09.624 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:09 smithi053 ceph-mon[35158]: osdmap e23: 4 total, 3 up, 4 in 2024-09-12T01:22:09.624 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:09 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:22:09.624 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:09 smithi053 ceph-mon[35158]: from='osd.3 [v2:172.21.15.1:6810/1247468803,v1:172.21.15.1:6811/1247468803]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-12T01:22:09.625 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:09 smithi053 ceph-mon[35158]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-12T01:22:09.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:09 smithi104 ceph-mon[35219]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-12T01:22:09.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:09 smithi104 ceph-mon[35219]: osdmap e23: 4 total, 3 up, 4 in 2024-09-12T01:22:09.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:09 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:22:09.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:09 smithi104 ceph-mon[35219]: from='osd.3 [v2:172.21.15.1:6810/1247468803,v1:172.21.15.1:6811/1247468803]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-12T01:22:09.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:09 smithi104 ceph-mon[35219]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-12T01:22:10.181 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 12 01:22:09 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-3[50823]: 2024-09-12T01:22:09.997+0000 7f6b5b000640 -1 osd.3 0 waiting for initial osdmap 2024-09-12T01:22:10.181 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 12 01:22:10 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-3[50823]: 2024-09-12T01:22:10.015+0000 7f6b51000640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T01:22:10.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:10 smithi001 ceph-mon[30387]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:22:10.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:10 smithi001 ceph-mon[30387]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-09-12T01:22:10.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:10 smithi001 ceph-mon[30387]: osdmap e24: 4 total, 3 up, 4 in 2024-09-12T01:22:10.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:10 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:22:10.679 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:10 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:22:10.679 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:10 smithi001 ceph-mon[30387]: from='osd.3 ' entity='osd.3' 2024-09-12T01:22:10.679 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:10 smithi001 ceph-mon[30387]: osd.3 [v2:172.21.15.1:6810/1247468803,v1:172.21.15.1:6811/1247468803] boot 2024-09-12T01:22:10.679 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:10 smithi001 ceph-mon[30387]: osdmap e25: 4 total, 4 up, 4 in 2024-09-12T01:22:10.679 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:10 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:22:10.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:10 smithi053 ceph-mon[35158]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:22:10.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:10 smithi053 ceph-mon[35158]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-09-12T01:22:10.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:10 smithi053 ceph-mon[35158]: osdmap e24: 4 total, 3 up, 4 in 2024-09-12T01:22:10.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:10 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:22:10.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:10 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:22:10.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:10 smithi053 ceph-mon[35158]: from='osd.3 ' entity='osd.3' 2024-09-12T01:22:10.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:10 smithi053 ceph-mon[35158]: osd.3 [v2:172.21.15.1:6810/1247468803,v1:172.21.15.1:6811/1247468803] boot 2024-09-12T01:22:10.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:10 smithi053 ceph-mon[35158]: osdmap e25: 4 total, 4 up, 4 in 2024-09-12T01:22:10.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:10 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:22:10.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:10 smithi104 ceph-mon[35219]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:22:10.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:10 smithi104 ceph-mon[35219]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-09-12T01:22:10.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:10 smithi104 ceph-mon[35219]: osdmap e24: 4 total, 3 up, 4 in 2024-09-12T01:22:10.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:10 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:22:10.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:10 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:22:10.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:10 smithi104 ceph-mon[35219]: from='osd.3 ' entity='osd.3' 2024-09-12T01:22:10.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:10 smithi104 ceph-mon[35219]: osd.3 [v2:172.21.15.1:6810/1247468803,v1:172.21.15.1:6811/1247468803] boot 2024-09-12T01:22:10.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:10 smithi104 ceph-mon[35219]: osdmap e25: 4 total, 4 up, 4 in 2024-09-12T01:22:10.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:10 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:22:11.154 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-12T01:22:11.183 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi053:vg_nvme/lv_3 2024-09-12T01:22:11.437 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T01:22:11.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:11 smithi053 ceph-mon[35158]: purged_snaps scrub starts 2024-09-12T01:22:11.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:11 smithi053 ceph-mon[35158]: purged_snaps scrub ok 2024-09-12T01:22:11.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:11 smithi053 ceph-mon[35158]: osdmap e26: 4 total, 4 up, 4 in 2024-09-12T01:22:11.580 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:11 smithi001 ceph-mon[30387]: purged_snaps scrub starts 2024-09-12T01:22:11.580 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:11 smithi001 ceph-mon[30387]: purged_snaps scrub ok 2024-09-12T01:22:11.580 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:11 smithi001 ceph-mon[30387]: osdmap e26: 4 total, 4 up, 4 in 2024-09-12T01:22:11.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:11 smithi104 ceph-mon[35219]: purged_snaps scrub starts 2024-09-12T01:22:11.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:11 smithi104 ceph-mon[35219]: purged_snaps scrub ok 2024-09-12T01:22:11.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:11 smithi104 ceph-mon[35219]: osdmap e26: 4 total, 4 up, 4 in 2024-09-12T01:22:12.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:12.065+0000 7f6f79b01640 1 -- 172.21.15.53:0/1881570747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f74105070 msgr2=0x7f6f74107460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:22:12.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:12.065+0000 7f6f79b01640 1 --2- 172.21.15.53:0/1881570747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f74105070 0x7f6f74107460 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c009900 tx=0x7f6f5c02a410 comp rx=0 tx=0).stop 2024-09-12T01:22:12.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:12.065+0000 7f6f79b01640 1 -- 172.21.15.53:0/1881570747 shutdown_connections 2024-09-12T01:22:12.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:12.065+0000 7f6f79b01640 1 --2- 172.21.15.53:0/1881570747 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f741079a0 0x7f6f74109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:22:12.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:12.065+0000 7f6f79b01640 1 --2- 172.21.15.53:0/1881570747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f74105070 0x7f6f74107460 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:22:12.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:12.065+0000 7f6f79b01640 1 --2- 172.21.15.53:0/1881570747 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f74102740 0x7f6f74104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:22:12.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:12.065+0000 7f6f79b01640 1 -- 172.21.15.53:0/1881570747 >> 172.21.15.53:0/1881570747 conn(0x7f6f740fc2a0 msgr2=0x7f6f740fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:22:12.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:12.065+0000 7f6f79b01640 1 -- 172.21.15.53:0/1881570747 shutdown_connections 2024-09-12T01:22:12.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:12.065+0000 7f6f79b01640 1 -- 172.21.15.53:0/1881570747 wait complete. 2024-09-12T01:22:12.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:12.065+0000 7f6f79b01640 1 Processor -- start 2024-09-12T01:22:12.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:12.065+0000 7f6f79b01640 1 -- start start 2024-09-12T01:22:12.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:12.066+0000 7f6f79b01640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f74102740 0x7f6f741020a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:22:12.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:12.066+0000 7f6f79b01640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f74105070 0x7f6f741006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:22:12.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:12.066+0000 7f6f79b01640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f741079a0 0x7f6f74100c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:22:12.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:12.066+0000 7f6f79b01640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6f7410d8f0 con 0x7f6f741079a0 2024-09-12T01:22:12.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:12.066+0000 7f6f79b01640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6f7410d770 con 0x7f6f74105070 2024-09-12T01:22:12.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:12.066+0000 7f6f79b01640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6f7410da70 con 0x7f6f74102740 2024-09-12T01:22:12.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:12.066+0000 7f6f6affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f74105070 0x7f6f741006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:22:12.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:12.066+0000 7f6f6affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f74105070 0x7f6f741006f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47362/0 (socket says 172.21.15.53:47362) 2024-09-12T01:22:12.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:12.066+0000 7f6f6affd640 1 -- 172.21.15.53:0/2021232742 learned_addr learned my addr 172.21.15.53:0/2021232742 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-12T01:22:12.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:12.066+0000 7f6f73fff640 1 --2- 172.21.15.53:0/2021232742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f741079a0 0x7f6f74100c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:22:12.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:12.066+0000 7f6f737fe640 1 --2- 172.21.15.53:0/2021232742 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f74102740 0x7f6f741020a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:22:12.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:12.066+0000 7f6f6affd640 1 -- 172.21.15.53:0/2021232742 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f74102740 msgr2=0x7f6f741020a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:22:12.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:12.066+0000 7f6f6affd640 1 --2- 172.21.15.53:0/2021232742 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f74102740 0x7f6f741020a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:22:12.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:12.066+0000 7f6f6affd640 1 -- 172.21.15.53:0/2021232742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f741079a0 msgr2=0x7f6f74100c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:22:12.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:12.066+0000 7f6f6affd640 1 --2- 172.21.15.53:0/2021232742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f741079a0 0x7f6f74100c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:22:12.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:12.066+0000 7f6f6affd640 1 -- 172.21.15.53:0/2021232742 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f741014f0 con 0x7f6f74105070 2024-09-12T01:22:12.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:12.066+0000 7f6f73fff640 1 --2- 172.21.15.53:0/2021232742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f741079a0 0x7f6f74100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:22:12.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:12.066+0000 7f6f6affd640 1 --2- 172.21.15.53:0/2021232742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f74105070 0x7f6f741006f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c031610 tx=0x7f6f5c0043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:22:12.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:12.067+0000 7f6f717fa640 1 -- 172.21.15.53:0/2021232742 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c03f030 con 0x7f6f74105070 2024-09-12T01:22:12.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:12.067+0000 7f6f717fa640 1 -- 172.21.15.53:0/2021232742 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f5c003ce0 con 0x7f6f74105070 2024-09-12T01:22:12.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:12.067+0000 7f6f79b01640 1 -- 172.21.15.53:0/2021232742 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f741a20d0 con 0x7f6f74105070 2024-09-12T01:22:12.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:12.067+0000 7f6f717fa640 1 -- 172.21.15.53:0/2021232742 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c035070 con 0x7f6f74105070 2024-09-12T01:22:12.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:12.067+0000 7f6f79b01640 1 -- 172.21.15.53:0/2021232742 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f741a2600 con 0x7f6f74105070 2024-09-12T01:22:12.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:12.068+0000 7f6f79b01640 1 -- 172.21.15.53:0/2021232742 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f740fdd90 con 0x7f6f74105070 2024-09-12T01:22:12.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:12.069+0000 7f6f717fa640 1 -- 172.21.15.53:0/2021232742 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f6f5c042030 con 0x7f6f74105070 2024-09-12T01:22:12.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:12.070+0000 7f6f717fa640 1 --2- 172.21.15.53:0/2021232742 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f6f600a2b80 0x7f6f600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:22:12.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:12.070+0000 7f6f717fa640 1 -- 172.21.15.53:0/2021232742 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 3698+0+0 (secure 0 0 0) 0x7f6f5c0ee720 con 0x7f6f74105070 2024-09-12T01:22:12.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:12.071+0000 7f6f737fe640 1 --2- 172.21.15.53:0/2021232742 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f6f600a2b80 0x7f6f600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:22:12.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:12.071+0000 7f6f737fe640 1 --2- 172.21.15.53:0/2021232742 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f6f600a2b80 0x7f6f600a5040 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6f64009890 tx=0x7f6f64008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:22:12.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:12.074+0000 7f6f717fa640 1 -- 172.21.15.53:0/2021232742 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6f5c0bc880 con 0x7f6f74105070 2024-09-12T01:22:12.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:12.261+0000 7f6f79b01640 1 -- 172.21.15.53:0/2021232742 --> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f6f74062630 con 0x7f6f600a2b80 2024-09-12T01:22:12.668 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:12 smithi053 ceph-mon[35158]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 136 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:22:12.669 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:12 smithi053 ceph-mon[35158]: osdmap e27: 4 total, 4 up, 4 in 2024-09-12T01:22:12.669 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:12 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:12.669 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:12 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:12.669 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:12 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:22:12.669 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:12 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:22:12.669 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:12 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:12.669 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:12 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:22:12.669 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:12 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:22:12.670 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:12 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:12.670 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:12 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:22:12.670 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:12 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:22:12.670 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:12 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:22:12.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:12 smithi001 ceph-mon[30387]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 136 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:22:12.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:12 smithi001 ceph-mon[30387]: osdmap e27: 4 total, 4 up, 4 in 2024-09-12T01:22:12.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:12 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:12.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:12 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:12.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:12 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:22:12.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:12 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:22:12.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:12 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:12.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:12 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:22:12.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:12 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:22:12.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:12 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:12.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:12 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:22:12.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:12 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:22:12.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:12 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:22:12.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:12 smithi104 ceph-mon[35219]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 136 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:22:12.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:12 smithi104 ceph-mon[35219]: osdmap e27: 4 total, 4 up, 4 in 2024-09-12T01:22:12.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:12 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:12.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:12 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:12.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:12 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:22:12.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:12 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:22:12.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:12 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:12.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:12 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:22:12.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:12 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:22:12.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:12 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:12.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:12 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:22:12.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:12 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:22:12.826 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:12 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:22:13.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:13 smithi001 ceph-mon[30387]: Detected new or changed devices on smithi001 2024-09-12T01:22:13.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:13 smithi001 ceph-mon[30387]: Adjusting osd_memory_target on smithi001 to 8547M 2024-09-12T01:22:13.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:13 smithi001 ceph-mon[30387]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:22:13.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:13 smithi053 ceph-mon[35158]: Detected new or changed devices on smithi001 2024-09-12T01:22:13.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:13 smithi053 ceph-mon[35158]: Adjusting osd_memory_target on smithi001 to 8547M 2024-09-12T01:22:13.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:13 smithi053 ceph-mon[35158]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:22:13.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:13 smithi104 ceph-mon[35219]: Detected new or changed devices on smithi001 2024-09-12T01:22:13.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:13 smithi104 ceph-mon[35219]: Adjusting osd_memory_target on smithi001 to 8547M 2024-09-12T01:22:13.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:13 smithi104 ceph-mon[35219]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:22:14.679 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:14 smithi001 ceph-mon[30387]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 136 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:22:14.679 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:14 smithi001 ceph-mon[30387]: osdmap e28: 4 total, 4 up, 4 in 2024-09-12T01:22:14.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:14 smithi053 ceph-mon[35158]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 136 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:22:14.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:14 smithi053 ceph-mon[35158]: osdmap e28: 4 total, 4 up, 4 in 2024-09-12T01:22:14.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:14 smithi104 ceph-mon[35219]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 136 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:22:14.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:14 smithi104 ceph-mon[35219]: osdmap e28: 4 total, 4 up, 4 in 2024-09-12T01:22:15.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:15 smithi001 ceph-mon[30387]: osdmap e29: 4 total, 4 up, 4 in 2024-09-12T01:22:15.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:15 smithi001 ceph-mon[30387]: from='client.? 172.21.15.53:0/2451360898' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe89ece0-2171-45d5-a06d-e400f77233e5"}]: dispatch 2024-09-12T01:22:15.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:15 smithi001 ceph-mon[30387]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe89ece0-2171-45d5-a06d-e400f77233e5"}]: dispatch 2024-09-12T01:22:15.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:15 smithi001 ceph-mon[30387]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fe89ece0-2171-45d5-a06d-e400f77233e5"}]': finished 2024-09-12T01:22:15.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:15 smithi001 ceph-mon[30387]: osdmap e30: 5 total, 4 up, 5 in 2024-09-12T01:22:15.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:15 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:22:15.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:15 smithi053 ceph-mon[35158]: osdmap e29: 4 total, 4 up, 4 in 2024-09-12T01:22:15.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:15 smithi053 ceph-mon[35158]: from='client.? 172.21.15.53:0/2451360898' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe89ece0-2171-45d5-a06d-e400f77233e5"}]: dispatch 2024-09-12T01:22:15.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:15 smithi053 ceph-mon[35158]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe89ece0-2171-45d5-a06d-e400f77233e5"}]: dispatch 2024-09-12T01:22:15.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:15 smithi053 ceph-mon[35158]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fe89ece0-2171-45d5-a06d-e400f77233e5"}]': finished 2024-09-12T01:22:15.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:15 smithi053 ceph-mon[35158]: osdmap e30: 5 total, 4 up, 5 in 2024-09-12T01:22:15.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:15 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:22:15.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:15 smithi104 ceph-mon[35219]: osdmap e29: 4 total, 4 up, 4 in 2024-09-12T01:22:15.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:15 smithi104 ceph-mon[35219]: from='client.? 172.21.15.53:0/2451360898' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe89ece0-2171-45d5-a06d-e400f77233e5"}]: dispatch 2024-09-12T01:22:15.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:15 smithi104 ceph-mon[35219]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe89ece0-2171-45d5-a06d-e400f77233e5"}]: dispatch 2024-09-12T01:22:15.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:15 smithi104 ceph-mon[35219]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fe89ece0-2171-45d5-a06d-e400f77233e5"}]': finished 2024-09-12T01:22:15.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:15 smithi104 ceph-mon[35219]: osdmap e30: 5 total, 4 up, 5 in 2024-09-12T01:22:15.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:15 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:22:16.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:16 smithi001 ceph-mon[30387]: pgmap v109: 1 pgs: 1 peering; 577 KiB data, 136 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:22:16.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:16 smithi001 ceph-mon[30387]: from='client.? 172.21.15.53:0/3988035717' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:22:16.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:16 smithi053 ceph-mon[35158]: pgmap v109: 1 pgs: 1 peering; 577 KiB data, 136 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:22:16.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:16 smithi053 ceph-mon[35158]: from='client.? 172.21.15.53:0/3988035717' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:22:16.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:16 smithi104 ceph-mon[35219]: pgmap v109: 1 pgs: 1 peering; 577 KiB data, 136 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:22:16.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:16 smithi104 ceph-mon[35219]: from='client.? 172.21.15.53:0/3988035717' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:22:18.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:18 smithi001 ceph-mon[30387]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-09-12T01:22:18.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:18 smithi053 ceph-mon[35158]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-09-12T01:22:18.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:18 smithi104 ceph-mon[35219]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-09-12T01:22:20.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:20 smithi001 ceph-mon[30387]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T01:22:20.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:20 smithi053 ceph-mon[35158]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T01:22:20.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:20 smithi104 ceph-mon[35219]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T01:22:22.455 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:22 smithi053 ceph-mon[35158]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-12T01:22:22.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:22 smithi001 ceph-mon[30387]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-12T01:22:22.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:22 smithi104 ceph-mon[35219]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-12T01:22:23.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:23 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-12T01:22:23.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:23 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:22:23.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:23 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-12T01:22:23.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:23 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:22:23.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:23 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-12T01:22:23.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:23 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:22:24.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:24 smithi001 ceph-mon[30387]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-12T01:22:24.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:24 smithi001 ceph-mon[30387]: Deploying daemon osd.4 on smithi053 2024-09-12T01:22:24.726 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:24 smithi104 ceph-mon[35219]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-12T01:22:24.726 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:24 smithi104 ceph-mon[35219]: Deploying daemon osd.4 on smithi053 2024-09-12T01:22:24.731 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:24 smithi053 ceph-mon[35158]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-12T01:22:24.732 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:24 smithi053 ceph-mon[35158]: Deploying daemon osd.4 on smithi053 2024-09-12T01:22:26.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:26 smithi001 ceph-mon[30387]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T01:22:26.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:26 smithi053 ceph-mon[35158]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T01:22:26.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:26 smithi104 ceph-mon[35219]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T01:22:28.532 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:28 smithi053 ceph-mon[35158]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-12T01:22:28.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:28 smithi001 ceph-mon[30387]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-12T01:22:28.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:28 smithi104 ceph-mon[35219]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-12T01:22:30.621 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:30 smithi053 ceph-mon[35158]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:22:30.621 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:30 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:22:30.622 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:30 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:30.622 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:30 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:30.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:30 smithi001 ceph-mon[30387]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:22:30.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:30 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:22:30.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:30 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:30.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:30 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:30.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:30 smithi104 ceph-mon[35219]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:22:30.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:30 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:22:30.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:30 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:30.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:30 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:32.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:32 smithi001 ceph-mon[30387]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:22:32.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:32 smithi053 ceph-mon[35158]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:22:32.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:32 smithi104 ceph-mon[35219]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:22:34.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:33 smithi104 ceph-mon[35219]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:22:34.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:33 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:34.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:33 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:34.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:33 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:22:34.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:33 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:22:34.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:33 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:34.076 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:33 smithi053 ceph-mon[35158]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:22:34.076 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:33 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:34.076 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:33 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:34.076 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:33 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:22:34.076 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:33 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:22:34.076 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:33 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:34.122 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 4 on host 'smithi053' 2024-09-12T01:22:34.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:34.123+0000 7f6f717fa640 1 -- 172.21.15.53:0/2021232742 <== mgr.14150 v2:172.21.15.1:6800/1928662456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6f74062630 con 0x7f6f600a2b80 2024-09-12T01:22:34.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:34.125+0000 7f6f79b01640 1 -- 172.21.15.53:0/2021232742 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f6f600a2b80 msgr2=0x7f6f600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:22:34.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:34.125+0000 7f6f79b01640 1 --2- 172.21.15.53:0/2021232742 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f6f600a2b80 0x7f6f600a5040 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6f64009890 tx=0x7f6f64008040 comp rx=0 tx=0).stop 2024-09-12T01:22:34.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:34.125+0000 7f6f79b01640 1 -- 172.21.15.53:0/2021232742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f74105070 msgr2=0x7f6f741006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:22:34.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:34.125+0000 7f6f79b01640 1 --2- 172.21.15.53:0/2021232742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f74105070 0x7f6f741006f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c031610 tx=0x7f6f5c0043b0 comp rx=0 tx=0).stop 2024-09-12T01:22:34.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:34.125+0000 7f6f79b01640 1 -- 172.21.15.53:0/2021232742 shutdown_connections 2024-09-12T01:22:34.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:34.125+0000 7f6f79b01640 1 --2- 172.21.15.53:0/2021232742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f741079a0 0x7f6f74100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:22:34.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:34.125+0000 7f6f79b01640 1 --2- 172.21.15.53:0/2021232742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f74105070 0x7f6f741006f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:22:34.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:34.125+0000 7f6f79b01640 1 --2- 172.21.15.53:0/2021232742 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f6f600a2b80 0x7f6f600a5040 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:22:34.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:34.125+0000 7f6f79b01640 1 --2- 172.21.15.53:0/2021232742 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f74102740 0x7f6f741020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:22:34.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:34.125+0000 7f6f79b01640 1 -- 172.21.15.53:0/2021232742 >> 172.21.15.53:0/2021232742 conn(0x7f6f740fc2a0 msgr2=0x7f6f74103120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:22:34.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:34.126+0000 7f6f79b01640 1 -- 172.21.15.53:0/2021232742 shutdown_connections 2024-09-12T01:22:34.126 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:22:34.126+0000 7f6f79b01640 1 -- 172.21.15.53:0/2021232742 wait complete. 2024-09-12T01:22:34.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:33 smithi001 ceph-mon[30387]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:22:34.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:33 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:34.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:33 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:34.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:33 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:22:34.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:33 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:22:34.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:33 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:34.816 DEBUG:teuthology.orchestra.run.smithi053:osd.4> sudo journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@osd.4.service 2024-09-12T01:22:34.818 INFO:tasks.cephadm:Deploying osd.5 on smithi104 with /dev/vg_nvme/lv_3... 2024-09-12T01:22:34.818 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-12T01:22:35.071 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:22:35.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:34 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:22:35.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:34 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:35.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:34 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:35.106 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:34 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:22:35.106 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:34 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:35.106 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:34 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:35.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:34 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:22:35.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:34 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:35.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:34 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:35.740 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 12 01:22:35 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-4[46372]: 2024-09-12T01:22:35.447+0000 7f3590b3f640 -1 osd.4 0 log_to_monitors true 2024-09-12T01:22:36.183 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:36 smithi104 ceph-mon[35219]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:22:36.183 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:36 smithi104 ceph-mon[35219]: from='osd.4 [v2:172.21.15.53:6808/2673413639,v1:172.21.15.53:6809/2673413639]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T01:22:36.184 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:36 smithi104 ceph-mon[35219]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T01:22:36.334 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:36 smithi053 ceph-mon[35158]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:22:36.334 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:36 smithi053 ceph-mon[35158]: from='osd.4 [v2:172.21.15.53:6808/2673413639,v1:172.21.15.53:6809/2673413639]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T01:22:36.334 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:36 smithi053 ceph-mon[35158]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T01:22:36.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:36 smithi001 ceph-mon[30387]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:22:36.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:36 smithi001 ceph-mon[30387]: from='osd.4 [v2:172.21.15.53:6808/2673413639,v1:172.21.15.53:6809/2673413639]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T01:22:36.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:36 smithi001 ceph-mon[30387]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T01:22:37.402 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 12 01:22:37 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-4[46372]: 2024-09-12T01:22:37.134+0000 7f358cac2640 -1 osd.4 0 waiting for initial osdmap 2024-09-12T01:22:37.403 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 12 01:22:37 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-4[46372]: 2024-09-12T01:22:37.142+0000 7f35888ea640 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T01:22:37.403 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:37 smithi053 ceph-mon[35158]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-12T01:22:37.403 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:37 smithi053 ceph-mon[35158]: from='osd.4 [v2:172.21.15.53:6808/2673413639,v1:172.21.15.53:6809/2673413639]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-12T01:22:37.403 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:37 smithi053 ceph-mon[35158]: osdmap e31: 5 total, 4 up, 5 in 2024-09-12T01:22:37.403 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:37 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:22:37.403 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:37 smithi053 ceph-mon[35158]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-12T01:22:37.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:37 smithi001 ceph-mon[30387]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-12T01:22:37.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:37 smithi001 ceph-mon[30387]: from='osd.4 [v2:172.21.15.53:6808/2673413639,v1:172.21.15.53:6809/2673413639]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-12T01:22:37.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:37 smithi001 ceph-mon[30387]: osdmap e31: 5 total, 4 up, 5 in 2024-09-12T01:22:37.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:37 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:22:37.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:37 smithi001 ceph-mon[30387]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-12T01:22:37.576 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:37 smithi104 ceph-mon[35219]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-12T01:22:37.576 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:37 smithi104 ceph-mon[35219]: from='osd.4 [v2:172.21.15.53:6808/2673413639,v1:172.21.15.53:6809/2673413639]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-12T01:22:37.577 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:37 smithi104 ceph-mon[35219]: osdmap e31: 5 total, 4 up, 5 in 2024-09-12T01:22:37.577 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:37 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:22:37.577 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:37 smithi104 ceph-mon[35219]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-12T01:22:37.586 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-12T01:22:37.611 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi104:vg_nvme/lv_3 2024-09-12T01:22:37.865 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:22:38.361 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:38.355+0000 7fee6d408640 1 -- 172.21.15.104:0/2920515532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee681037b0 msgr2=0x7fee6810a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:22:38.362 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:38.355+0000 7fee6d408640 1 --2- 172.21.15.104:0/2920515532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee681037b0 0x7fee6810a040 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fee60009ca0 tx=0x7fee6002a8c0 comp rx=0 tx=0).stop 2024-09-12T01:22:38.362 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:38.355+0000 7fee6d408640 1 -- 172.21.15.104:0/2920515532 shutdown_connections 2024-09-12T01:22:38.362 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:38.355+0000 7fee6d408640 1 --2- 172.21.15.104:0/2920515532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee681037b0 0x7fee6810a040 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:22:38.362 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:38.355+0000 7fee6d408640 1 --2- 172.21.15.104:0/2920515532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee68102df0 0x7fee68103270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:22:38.362 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:38.355+0000 7fee6d408640 1 --2- 172.21.15.104:0/2920515532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee68101bf0 0x7fee68101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:22:38.362 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:38.355+0000 7fee6d408640 1 -- 172.21.15.104:0/2920515532 >> 172.21.15.104:0/2920515532 conn(0x7fee680fd3c0 msgr2=0x7fee680ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:22:38.362 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:38.355+0000 7fee6d408640 1 -- 172.21.15.104:0/2920515532 shutdown_connections 2024-09-12T01:22:38.362 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:38.355+0000 7fee6d408640 1 -- 172.21.15.104:0/2920515532 wait complete. 2024-09-12T01:22:38.362 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:38.356+0000 7fee6d408640 1 Processor -- start 2024-09-12T01:22:38.362 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:38.356+0000 7fee6d408640 1 -- start start 2024-09-12T01:22:38.362 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:38.356+0000 7fee6d408640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee68101bf0 0x7fee68197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:22:38.363 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:38.356+0000 7fee6d408640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee68102df0 0x7fee68198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:22:38.363 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:38.356+0000 7fee6d408640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee681037b0 0x7fee6819f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:22:38.363 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:38.356+0000 7fee6d408640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fee6810d850 con 0x7fee681037b0 2024-09-12T01:22:38.363 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:38.356+0000 7fee6d408640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fee6810d6d0 con 0x7fee68101bf0 2024-09-12T01:22:38.363 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:38.356+0000 7fee6d408640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fee6810d9d0 con 0x7fee68102df0 2024-09-12T01:22:38.363 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:38.356+0000 7fee667fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee68102df0 0x7fee68198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:22:38.363 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:38.356+0000 7fee667fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee68102df0 0x7fee68198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:34302/0 (socket says 172.21.15.104:34302) 2024-09-12T01:22:38.363 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:38.356+0000 7fee667fc640 1 -- 172.21.15.104:0/522314461 learned_addr learned my addr 172.21.15.104:0/522314461 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-09-12T01:22:38.363 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:38.356+0000 7fee66ffd640 1 --2- 172.21.15.104:0/522314461 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee68101bf0 0x7fee68197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:22:38.363 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:38.356+0000 7fee677fe640 1 --2- 172.21.15.104:0/522314461 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee681037b0 0x7fee6819f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:22:38.363 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:38.356+0000 7fee667fc640 1 -- 172.21.15.104:0/522314461 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee68101bf0 msgr2=0x7fee68197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:22:38.363 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:38.356+0000 7fee667fc640 1 --2- 172.21.15.104:0/522314461 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee68101bf0 0x7fee68197dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:22:38.364 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:38.356+0000 7fee667fc640 1 -- 172.21.15.104:0/522314461 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee681037b0 msgr2=0x7fee6819f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:22:38.364 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:38.356+0000 7fee667fc640 1 --2- 172.21.15.104:0/522314461 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee681037b0 0x7fee6819f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:22:38.364 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:38.356+0000 7fee667fc640 1 -- 172.21.15.104:0/522314461 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee6819f9f0 con 0x7fee68102df0 2024-09-12T01:22:38.364 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:38.356+0000 7fee667fc640 1 --2- 172.21.15.104:0/522314461 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee68102df0 0x7fee68198300 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fee5800d950 tx=0x7fee5800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:22:38.364 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:38.357+0000 7fee4ffff640 1 -- 172.21.15.104:0/522314461 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee58017030 con 0x7fee68102df0 2024-09-12T01:22:38.364 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:38.357+0000 7fee4ffff640 1 -- 172.21.15.104:0/522314461 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee58004020 con 0x7fee68102df0 2024-09-12T01:22:38.364 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:38.357+0000 7fee6d408640 1 -- 172.21.15.104:0/522314461 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee6819fce0 con 0x7fee68102df0 2024-09-12T01:22:38.364 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:38.357+0000 7fee4ffff640 1 -- 172.21.15.104:0/522314461 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee580047a0 con 0x7fee68102df0 2024-09-12T01:22:38.364 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:38.357+0000 7fee6d408640 1 -- 172.21.15.104:0/522314461 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee681a01c0 con 0x7fee68102df0 2024-09-12T01:22:38.364 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:38.358+0000 7fee6d408640 1 -- 172.21.15.104:0/522314461 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee68108f20 con 0x7fee68102df0 2024-09-12T01:22:38.364 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:38.359+0000 7fee4ffff640 1 -- 172.21.15.104:0/522314461 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fee580041d0 con 0x7fee68102df0 2024-09-12T01:22:38.364 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:38.360+0000 7fee4ffff640 1 --2- 172.21.15.104:0/522314461 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fee500a2bb0 0x7fee500a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:22:38.364 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:38.360+0000 7fee4ffff640 1 -- 172.21.15.104:0/522314461 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4101+0+0 (secure 0 0 0) 0x7fee580ce070 con 0x7fee68102df0 2024-09-12T01:22:38.364 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:38.360+0000 7fee66ffd640 1 --2- 172.21.15.104:0/522314461 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fee500a2bb0 0x7fee500a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:22:38.365 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:38.361+0000 7fee66ffd640 1 --2- 172.21.15.104:0/522314461 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fee500a2bb0 0x7fee500a5070 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fee5c005fd0 tx=0x7fee5c005ce0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:22:38.365 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:38.364+0000 7fee4ffff640 1 -- 172.21.15.104:0/522314461 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fee5809c040 con 0x7fee68102df0 2024-09-12T01:22:38.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:38 smithi001 ceph-mon[30387]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:22:38.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:38 smithi001 ceph-mon[30387]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-09-12T01:22:38.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:38 smithi001 ceph-mon[30387]: osdmap e32: 5 total, 4 up, 5 in 2024-09-12T01:22:38.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:38 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:22:38.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:38 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:22:38.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:38 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:38.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:38 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:38.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:38 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:22:38.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:38 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:22:38.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:38 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:38.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:38 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:22:38.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:38 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:22:38.429 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:38 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:38.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:38 smithi053 ceph-mon[35158]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:22:38.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:38 smithi053 ceph-mon[35158]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-09-12T01:22:38.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:38 smithi053 ceph-mon[35158]: osdmap e32: 5 total, 4 up, 5 in 2024-09-12T01:22:38.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:38 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:22:38.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:38 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:22:38.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:38 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:38.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:38 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:38.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:38 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:22:38.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:38 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:22:38.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:38 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:38.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:38 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:22:38.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:38 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:22:38.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:38 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:38.542 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:38.543+0000 7fee6d408640 1 -- 172.21.15.104:0/522314461 --> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fee68062630 con 0x7fee500a2bb0 2024-09-12T01:22:38.542 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:38 smithi104 ceph-mon[35219]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:22:38.542 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:38 smithi104 ceph-mon[35219]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-09-12T01:22:38.543 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:38 smithi104 ceph-mon[35219]: osdmap e32: 5 total, 4 up, 5 in 2024-09-12T01:22:38.543 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:38 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:22:38.543 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:38 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:22:38.543 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:38 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:38.543 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:38 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:38.543 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:38 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:22:38.543 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:38 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:22:38.543 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:38 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:38.543 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:38 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:22:38.543 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:38 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:22:38.543 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:38 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:39.171 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:39 smithi104 ceph-mon[35219]: purged_snaps scrub starts 2024-09-12T01:22:39.172 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:39 smithi104 ceph-mon[35219]: purged_snaps scrub ok 2024-09-12T01:22:39.172 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:39 smithi104 ceph-mon[35219]: Detected new or changed devices on smithi053 2024-09-12T01:22:39.172 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:39 smithi104 ceph-mon[35219]: Adjusting osd_memory_target on smithi053 to 8546M 2024-09-12T01:22:39.172 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:39 smithi104 ceph-mon[35219]: osd.4 [v2:172.21.15.53:6808/2673413639,v1:172.21.15.53:6809/2673413639] boot 2024-09-12T01:22:39.172 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:39 smithi104 ceph-mon[35219]: osdmap e33: 5 total, 5 up, 5 in 2024-09-12T01:22:39.172 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:39 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:22:39.172 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:39 smithi104 ceph-mon[35219]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:22:39.172 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:39 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:22:39.172 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:39 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:22:39.172 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:39 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:22:39.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:39 smithi001 ceph-mon[30387]: purged_snaps scrub starts 2024-09-12T01:22:39.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:39 smithi001 ceph-mon[30387]: purged_snaps scrub ok 2024-09-12T01:22:39.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:39 smithi001 ceph-mon[30387]: Detected new or changed devices on smithi053 2024-09-12T01:22:39.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:39 smithi001 ceph-mon[30387]: Adjusting osd_memory_target on smithi053 to 8546M 2024-09-12T01:22:39.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:39 smithi001 ceph-mon[30387]: osd.4 [v2:172.21.15.53:6808/2673413639,v1:172.21.15.53:6809/2673413639] boot 2024-09-12T01:22:39.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:39 smithi001 ceph-mon[30387]: osdmap e33: 5 total, 5 up, 5 in 2024-09-12T01:22:39.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:39 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:22:39.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:39 smithi001 ceph-mon[30387]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:22:39.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:39 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:22:39.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:39 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:22:39.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:39 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:22:39.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:39 smithi053 ceph-mon[35158]: purged_snaps scrub starts 2024-09-12T01:22:39.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:39 smithi053 ceph-mon[35158]: purged_snaps scrub ok 2024-09-12T01:22:39.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:39 smithi053 ceph-mon[35158]: Detected new or changed devices on smithi053 2024-09-12T01:22:39.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:39 smithi053 ceph-mon[35158]: Adjusting osd_memory_target on smithi053 to 8546M 2024-09-12T01:22:39.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:39 smithi053 ceph-mon[35158]: osd.4 [v2:172.21.15.53:6808/2673413639,v1:172.21.15.53:6809/2673413639] boot 2024-09-12T01:22:39.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:39 smithi053 ceph-mon[35158]: osdmap e33: 5 total, 5 up, 5 in 2024-09-12T01:22:39.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:39 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:22:39.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:39 smithi053 ceph-mon[35158]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:22:39.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:39 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:22:39.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:39 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:22:39.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:39 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:22:40.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:40 smithi001 ceph-mon[30387]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:22:40.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:40 smithi001 ceph-mon[30387]: osdmap e34: 5 total, 5 up, 5 in 2024-09-12T01:22:40.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:40 smithi053 ceph-mon[35158]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:22:40.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:40 smithi053 ceph-mon[35158]: osdmap e34: 5 total, 5 up, 5 in 2024-09-12T01:22:40.527 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:40 smithi104 ceph-mon[35219]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:22:40.527 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:40 smithi104 ceph-mon[35219]: osdmap e34: 5 total, 5 up, 5 in 2024-09-12T01:22:41.418 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:41 smithi104 ceph-mon[35219]: from='client.? 172.21.15.104:0/3600399384' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "daab795c-9942-4d16-aa34-8bba30a4b99b"}]: dispatch 2024-09-12T01:22:41.418 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:41 smithi104 ceph-mon[35219]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "daab795c-9942-4d16-aa34-8bba30a4b99b"}]: dispatch 2024-09-12T01:22:41.418 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:41 smithi104 ceph-mon[35219]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "daab795c-9942-4d16-aa34-8bba30a4b99b"}]': finished 2024-09-12T01:22:41.419 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:41 smithi104 ceph-mon[35219]: osdmap e35: 6 total, 5 up, 6 in 2024-09-12T01:22:41.419 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:41 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:22:41.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:41 smithi001 ceph-mon[30387]: from='client.? 172.21.15.104:0/3600399384' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "daab795c-9942-4d16-aa34-8bba30a4b99b"}]: dispatch 2024-09-12T01:22:41.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:41 smithi001 ceph-mon[30387]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "daab795c-9942-4d16-aa34-8bba30a4b99b"}]: dispatch 2024-09-12T01:22:41.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:41 smithi001 ceph-mon[30387]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "daab795c-9942-4d16-aa34-8bba30a4b99b"}]': finished 2024-09-12T01:22:41.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:41 smithi001 ceph-mon[30387]: osdmap e35: 6 total, 5 up, 6 in 2024-09-12T01:22:41.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:41 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:22:41.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:41 smithi053 ceph-mon[35158]: from='client.? 172.21.15.104:0/3600399384' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "daab795c-9942-4d16-aa34-8bba30a4b99b"}]: dispatch 2024-09-12T01:22:41.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:41 smithi053 ceph-mon[35158]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "daab795c-9942-4d16-aa34-8bba30a4b99b"}]: dispatch 2024-09-12T01:22:41.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:41 smithi053 ceph-mon[35158]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "daab795c-9942-4d16-aa34-8bba30a4b99b"}]': finished 2024-09-12T01:22:41.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:41 smithi053 ceph-mon[35158]: osdmap e35: 6 total, 5 up, 6 in 2024-09-12T01:22:41.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:41 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:22:42.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:42 smithi053 ceph-mon[35158]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:22:42.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:42 smithi053 ceph-mon[35158]: from='client.? 172.21.15.104:0/1535078975' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:22:42.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:42 smithi001 ceph-mon[30387]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:22:42.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:42 smithi001 ceph-mon[30387]: from='client.? 172.21.15.104:0/1535078975' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:22:42.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:42 smithi104 ceph-mon[35219]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:22:42.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:42 smithi104 ceph-mon[35219]: from='client.? 172.21.15.104:0/1535078975' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:22:44.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:44 smithi001 ceph-mon[30387]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:22:44.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:44 smithi053 ceph-mon[35158]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:22:44.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:44 smithi104 ceph-mon[35219]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:22:46.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:46 smithi104 ceph-mon[35219]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:22:46.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:46 smithi001 ceph-mon[30387]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:22:46.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:46 smithi053 ceph-mon[35158]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:22:48.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:48 smithi001 ceph-mon[30387]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:22:48.458 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:48 smithi104 ceph-mon[35219]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:22:48.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:48 smithi053 ceph-mon[35158]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:22:50.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:50 smithi001 ceph-mon[30387]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:22:50.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:50 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-12T01:22:50.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:50 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:22:50.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:50 smithi001 ceph-mon[30387]: Deploying daemon osd.5 on smithi104 2024-09-12T01:22:50.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:50 smithi053 ceph-mon[35158]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:22:50.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:50 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-12T01:22:50.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:50 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:22:50.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:50 smithi053 ceph-mon[35158]: Deploying daemon osd.5 on smithi104 2024-09-12T01:22:50.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:50 smithi104 ceph-mon[35219]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:22:50.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:50 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-12T01:22:50.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:50 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:22:50.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:50 smithi104 ceph-mon[35219]: Deploying daemon osd.5 on smithi104 2024-09-12T01:22:52.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:52 smithi001 ceph-mon[30387]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:22:52.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:52 smithi053 ceph-mon[35158]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:22:52.512 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:52 smithi104 ceph-mon[35219]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:22:54.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:54 smithi001 ceph-mon[30387]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:22:54.477 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:54 smithi104 ceph-mon[35219]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:22:54.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:54 smithi053 ceph-mon[35158]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:22:56.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:56 smithi001 ceph-mon[30387]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:22:56.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:56 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:22:56.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:56 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:56.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:56 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:56.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:56 smithi053 ceph-mon[35158]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:22:56.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:56 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:22:56.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:56 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:56.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:56 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:56.505 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:56 smithi104 ceph-mon[35219]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:22:56.505 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:56 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:22:56.505 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:56 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:56.505 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:56 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:58.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:58 smithi104 ceph-mon[35219]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:22:58.503 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:58 smithi053 ceph-mon[35158]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:22:58.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:58 smithi001 ceph-mon[30387]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:22:59.937 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:59 smithi104 ceph-mon[35219]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:22:59.937 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:59 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:59.937 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:59 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:59.937 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:59 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:22:59.938 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:59 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:22:59.938 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:22:59 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:59.976 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:59.976+0000 7fee4ffff640 1 -- 172.21.15.104:0/522314461 <== mgr.14150 v2:172.21.15.1:6800/1928662456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fee68062630 con 0x7fee500a2bb0 2024-09-12T01:22:59.977 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 5 on host 'smithi104' 2024-09-12T01:22:59.978 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:59.978+0000 7fee6d408640 1 -- 172.21.15.104:0/522314461 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fee500a2bb0 msgr2=0x7fee500a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:22:59.979 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:59.978+0000 7fee6d408640 1 --2- 172.21.15.104:0/522314461 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fee500a2bb0 0x7fee500a5070 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fee5c005fd0 tx=0x7fee5c005ce0 comp rx=0 tx=0).stop 2024-09-12T01:22:59.979 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:59.978+0000 7fee6d408640 1 -- 172.21.15.104:0/522314461 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee68102df0 msgr2=0x7fee68198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:22:59.979 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:59.978+0000 7fee6d408640 1 --2- 172.21.15.104:0/522314461 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee68102df0 0x7fee68198300 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fee5800d950 tx=0x7fee5800de10 comp rx=0 tx=0).stop 2024-09-12T01:22:59.979 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:59.978+0000 7fee6d408640 1 -- 172.21.15.104:0/522314461 shutdown_connections 2024-09-12T01:22:59.979 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:59.978+0000 7fee6d408640 1 --2- 172.21.15.104:0/522314461 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee681037b0 0x7fee6819f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:22:59.979 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:59.978+0000 7fee6d408640 1 --2- 172.21.15.104:0/522314461 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fee500a2bb0 0x7fee500a5070 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:22:59.979 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:59.978+0000 7fee6d408640 1 --2- 172.21.15.104:0/522314461 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee68102df0 0x7fee68198300 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:22:59.979 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:59.978+0000 7fee6d408640 1 --2- 172.21.15.104:0/522314461 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee68101bf0 0x7fee68197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:22:59.979 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:59.978+0000 7fee6d408640 1 -- 172.21.15.104:0/522314461 >> 172.21.15.104:0/522314461 conn(0x7fee680fd3c0 msgr2=0x7fee681060f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:22:59.979 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:59.979+0000 7fee6d408640 1 -- 172.21.15.104:0/522314461 shutdown_connections 2024-09-12T01:22:59.980 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:22:59.979+0000 7fee6d408640 1 -- 172.21.15.104:0/522314461 wait complete. 2024-09-12T01:22:59.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:59 smithi053 ceph-mon[35158]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:22:59.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:59 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:59.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:59 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:22:59.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:59 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:22:59.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:59 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:22:59.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:22:59 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:00.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:59 smithi001 ceph-mon[30387]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:23:00.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:59 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:00.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:59 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:00.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:59 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:23:00.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:59 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:23:00.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:22:59 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:00.768 DEBUG:teuthology.orchestra.run.smithi104:osd.5> sudo journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@osd.5.service 2024-09-12T01:23:00.770 INFO:tasks.cephadm:Deploying osd.6 on smithi001 with /dev/vg_nvme/lv_2... 2024-09-12T01:23:00.771 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-12T01:23:00.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:00 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:23:00.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:00 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:00.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:00 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:01.010 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:00 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:23:01.010 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:00 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:01.011 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:00 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:01.151 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:23:01.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:00 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:23:01.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:00 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:01.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:00 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:01.575 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:23:01 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5[46675]: 2024-09-12T01:23:01.266+0000 7f5871a95640 -1 osd.5 0 log_to_monitors true 2024-09-12T01:23:02.224 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:01 smithi104 ceph-mon[35219]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:23:02.225 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:01 smithi104 ceph-mon[35219]: from='osd.5 [v2:172.21.15.104:6808/3131498790,v1:172.21.15.104:6809/3131498790]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T01:23:02.225 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:01 smithi104 ceph-mon[35219]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T01:23:02.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:01 smithi053 ceph-mon[35158]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:23:02.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:01 smithi053 ceph-mon[35158]: from='osd.5 [v2:172.21.15.104:6808/3131498790,v1:172.21.15.104:6809/3131498790]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T01:23:02.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:01 smithi053 ceph-mon[35158]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T01:23:02.316 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:01 smithi001 ceph-mon[30387]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:23:02.316 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:01 smithi001 ceph-mon[30387]: from='osd.5 [v2:172.21.15.104:6808/3131498790,v1:172.21.15.104:6809/3131498790]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T01:23:02.317 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:01 smithi001 ceph-mon[30387]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T01:23:03.034 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:02 smithi001 ceph-mon[30387]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-12T01:23:03.035 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:02 smithi001 ceph-mon[30387]: osdmap e36: 6 total, 5 up, 6 in 2024-09-12T01:23:03.035 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:02 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:23:03.035 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:02 smithi001 ceph-mon[30387]: from='osd.5 [v2:172.21.15.104:6808/3131498790,v1:172.21.15.104:6809/3131498790]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-12T01:23:03.035 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:02 smithi001 ceph-mon[30387]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-12T01:23:03.262 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:02 smithi104 ceph-mon[35219]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-12T01:23:03.263 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:02 smithi104 ceph-mon[35219]: osdmap e36: 6 total, 5 up, 6 in 2024-09-12T01:23:03.263 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:02 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:23:03.263 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:02 smithi104 ceph-mon[35219]: from='osd.5 [v2:172.21.15.104:6808/3131498790,v1:172.21.15.104:6809/3131498790]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-12T01:23:03.263 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:02 smithi104 ceph-mon[35219]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-12T01:23:03.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:02 smithi053 ceph-mon[35158]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-12T01:23:03.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:02 smithi053 ceph-mon[35158]: osdmap e36: 6 total, 5 up, 6 in 2024-09-12T01:23:03.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:23:03.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:02 smithi053 ceph-mon[35158]: from='osd.5 [v2:172.21.15.104:6808/3131498790,v1:172.21.15.104:6809/3131498790]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-12T01:23:03.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:02 smithi053 ceph-mon[35158]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-12T01:23:03.992 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:23:03 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5[46675]: 2024-09-12T01:23:03.710+0000 7f586da18640 -1 osd.5 0 waiting for initial osdmap 2024-09-12T01:23:03.992 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:23:03 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5[46675]: 2024-09-12T01:23:03.717+0000 7f5865838640 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T01:23:04.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:03 smithi001 ceph-mon[30387]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:23:04.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:03 smithi001 ceph-mon[30387]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-12T01:23:04.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:03 smithi001 ceph-mon[30387]: osdmap e37: 6 total, 5 up, 6 in 2024-09-12T01:23:04.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:03 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:23:04.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:03 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:23:04.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:03 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:04.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:03 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:04.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:03 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:23:04.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:03 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:23:04.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:03 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:04.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:03 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:23:04.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:03 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:23:04.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:03 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:04.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:03 smithi001 ceph-mon[30387]: from='osd.5 ' entity='osd.5' 2024-09-12T01:23:04.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:03 smithi104 ceph-mon[35219]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:23:04.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:03 smithi104 ceph-mon[35219]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-12T01:23:04.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:03 smithi104 ceph-mon[35219]: osdmap e37: 6 total, 5 up, 6 in 2024-09-12T01:23:04.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:03 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:23:04.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:03 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:23:04.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:03 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:04.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:03 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:04.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:03 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:23:04.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:03 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:23:04.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:03 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:04.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:03 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:23:04.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:03 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:23:04.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:03 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:04.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:03 smithi104 ceph-mon[35219]: from='osd.5 ' entity='osd.5' 2024-09-12T01:23:04.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:03 smithi053 ceph-mon[35158]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:23:04.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:03 smithi053 ceph-mon[35158]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-12T01:23:04.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:03 smithi053 ceph-mon[35158]: osdmap e37: 6 total, 5 up, 6 in 2024-09-12T01:23:04.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:03 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:23:04.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:03 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:23:04.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:03 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:04.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:03 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:04.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:03 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:23:04.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:03 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:23:04.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:03 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:04.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:03 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:23:04.492 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:03 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:23:04.492 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:03 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:04.492 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:03 smithi053 ceph-mon[35158]: from='osd.5 ' entity='osd.5' 2024-09-12T01:23:04.751 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:23:04.784 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi001:vg_nvme/lv_2 2024-09-12T01:23:05.161 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:23:05.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:04 smithi001 ceph-mon[30387]: purged_snaps scrub starts 2024-09-12T01:23:05.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:04 smithi001 ceph-mon[30387]: purged_snaps scrub ok 2024-09-12T01:23:05.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:04 smithi001 ceph-mon[30387]: Detected new or changed devices on smithi104 2024-09-12T01:23:05.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:04 smithi001 ceph-mon[30387]: Adjusting osd_memory_target on smithi104 to 8544M 2024-09-12T01:23:05.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:04 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:23:05.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:04 smithi001 ceph-mon[30387]: osd.5 [v2:172.21.15.104:6808/3131498790,v1:172.21.15.104:6809/3131498790] boot 2024-09-12T01:23:05.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:04 smithi001 ceph-mon[30387]: osdmap e38: 6 total, 6 up, 6 in 2024-09-12T01:23:05.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:04 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:23:05.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:04 smithi104 ceph-mon[35219]: purged_snaps scrub starts 2024-09-12T01:23:05.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:04 smithi104 ceph-mon[35219]: purged_snaps scrub ok 2024-09-12T01:23:05.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:04 smithi104 ceph-mon[35219]: Detected new or changed devices on smithi104 2024-09-12T01:23:05.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:04 smithi104 ceph-mon[35219]: Adjusting osd_memory_target on smithi104 to 8544M 2024-09-12T01:23:05.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:04 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:23:05.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:04 smithi104 ceph-mon[35219]: osd.5 [v2:172.21.15.104:6808/3131498790,v1:172.21.15.104:6809/3131498790] boot 2024-09-12T01:23:05.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:04 smithi104 ceph-mon[35219]: osdmap e38: 6 total, 6 up, 6 in 2024-09-12T01:23:05.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:04 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:23:05.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:04 smithi053 ceph-mon[35158]: purged_snaps scrub starts 2024-09-12T01:23:05.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:04 smithi053 ceph-mon[35158]: purged_snaps scrub ok 2024-09-12T01:23:05.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:04 smithi053 ceph-mon[35158]: Detected new or changed devices on smithi104 2024-09-12T01:23:05.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:04 smithi053 ceph-mon[35158]: Adjusting osd_memory_target on smithi104 to 8544M 2024-09-12T01:23:05.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:04 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:23:05.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:04 smithi053 ceph-mon[35158]: osd.5 [v2:172.21.15.104:6808/3131498790,v1:172.21.15.104:6809/3131498790] boot 2024-09-12T01:23:05.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:04 smithi053 ceph-mon[35158]: osdmap e38: 6 total, 6 up, 6 in 2024-09-12T01:23:05.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:04 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:23:05.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:05.772+0000 7ff169c00640 1 -- 172.21.15.1:0/3713223840 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1641037c0 msgr2=0x7ff16410a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:23:05.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:05.772+0000 7ff169c00640 1 --2- 172.21.15.1:0/3713223840 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1641037c0 0x7ff16410a050 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff15c009ca0 tx=0x7ff15c02a8c0 comp rx=0 tx=0).stop 2024-09-12T01:23:05.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:05.772+0000 7ff169c00640 1 -- 172.21.15.1:0/3713223840 shutdown_connections 2024-09-12T01:23:05.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:05.772+0000 7ff169c00640 1 --2- 172.21.15.1:0/3713223840 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1641037c0 0x7ff16410a050 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:23:05.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:05.772+0000 7ff169c00640 1 --2- 172.21.15.1:0/3713223840 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff164102e00 0x7ff164103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:23:05.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:05.772+0000 7ff169c00640 1 --2- 172.21.15.1:0/3713223840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff164101c00 0x7ff164102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:23:05.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:05.772+0000 7ff169c00640 1 -- 172.21.15.1:0/3713223840 >> 172.21.15.1:0/3713223840 conn(0x7ff1640fd390 msgr2=0x7ff1640ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:23:05.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:05.772+0000 7ff169c00640 1 -- 172.21.15.1:0/3713223840 shutdown_connections 2024-09-12T01:23:05.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:05.772+0000 7ff169c00640 1 -- 172.21.15.1:0/3713223840 wait complete. 2024-09-12T01:23:05.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:05.773+0000 7ff169c00640 1 Processor -- start 2024-09-12T01:23:05.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:05.773+0000 7ff169c00640 1 -- start start 2024-09-12T01:23:05.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:05.773+0000 7ff169c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff164101c00 0x7ff164197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:23:05.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:05.773+0000 7ff169c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff164102e00 0x7ff164198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:23:05.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:05.773+0000 7ff169c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1641037c0 0x7ff16419f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:23:05.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:05.773+0000 7ff169c00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff16410d860 con 0x7ff164101c00 2024-09-12T01:23:05.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:05.773+0000 7ff169c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff16410d6e0 con 0x7ff164102e00 2024-09-12T01:23:05.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:05.773+0000 7ff169c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff16410d9e0 con 0x7ff1641037c0 2024-09-12T01:23:05.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:05.774+0000 7ff162a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff164101c00 0x7ff164197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:23:05.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:05.774+0000 7ff162a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff164101c00 0x7ff164197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51940/0 (socket says 172.21.15.1:51940) 2024-09-12T01:23:05.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:05.774+0000 7ff162a00640 1 -- 172.21.15.1:0/2048247695 learned_addr learned my addr 172.21.15.1:0/2048247695 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:23:05.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:05.774+0000 7ff162000640 1 --2- 172.21.15.1:0/2048247695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff164102e00 0x7ff164198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:23:05.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:05.774+0000 7ff163400640 1 --2- 172.21.15.1:0/2048247695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1641037c0 0x7ff16419f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:23:05.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:05.774+0000 7ff162a00640 1 -- 172.21.15.1:0/2048247695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1641037c0 msgr2=0x7ff16419f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:23:05.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:05.774+0000 7ff162a00640 1 --2- 172.21.15.1:0/2048247695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1641037c0 0x7ff16419f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:23:05.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:05.774+0000 7ff162a00640 1 -- 172.21.15.1:0/2048247695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff164102e00 msgr2=0x7ff164198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:23:05.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:05.774+0000 7ff162a00640 1 --2- 172.21.15.1:0/2048247695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff164102e00 0x7ff164198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:23:05.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:05.774+0000 7ff162a00640 1 -- 172.21.15.1:0/2048247695 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff16419fa80 con 0x7ff164101c00 2024-09-12T01:23:05.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:05.774+0000 7ff162000640 1 --2- 172.21.15.1:0/2048247695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff164102e00 0x7ff164198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:23:05.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:05.774+0000 7ff162a00640 1 --2- 172.21.15.1:0/2048247695 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff164101c00 0x7ff164197e50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff15400e9c0 tx=0x7ff15400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:23:05.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:05.775+0000 7ff15b400640 1 -- 172.21.15.1:0/2048247695 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15400cde0 con 0x7ff164101c00 2024-09-12T01:23:05.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:05.775+0000 7ff15b400640 1 -- 172.21.15.1:0/2048247695 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff154010910 con 0x7ff164101c00 2024-09-12T01:23:05.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:05.775+0000 7ff169c00640 1 -- 172.21.15.1:0/2048247695 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff16419fd70 con 0x7ff164101c00 2024-09-12T01:23:05.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:05.775+0000 7ff15b400640 1 -- 172.21.15.1:0/2048247695 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff154018a00 con 0x7ff164101c00 2024-09-12T01:23:05.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:05.775+0000 7ff169c00640 1 -- 172.21.15.1:0/2048247695 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1641a01f0 con 0x7ff164101c00 2024-09-12T01:23:05.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:05.776+0000 7ff169c00640 1 -- 172.21.15.1:0/2048247695 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff164108f30 con 0x7ff164101c00 2024-09-12T01:23:05.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:05.778+0000 7ff15b400640 1 -- 172.21.15.1:0/2048247695 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7ff15401f020 con 0x7ff164101c00 2024-09-12T01:23:05.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:05.779+0000 7ff15b400640 1 --2- 172.21.15.1:0/2048247695 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff1480a2bb0 0x7ff1480a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:23:05.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:05.779+0000 7ff15b400640 1 -- 172.21.15.1:0/2048247695 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4562+0+0 (secure 0 0 0) 0x7ff1540ce9b0 con 0x7ff164101c00 2024-09-12T01:23:05.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:05.780+0000 7ff162000640 1 --2- 172.21.15.1:0/2048247695 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff1480a2bb0 0x7ff1480a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:23:05.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:05.780+0000 7ff162000640 1 --2- 172.21.15.1:0/2048247695 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff1480a2bb0 0x7ff1480a5070 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff164199370 tx=0x7ff150005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:23:05.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:05.782+0000 7ff15b400640 1 -- 172.21.15.1:0/2048247695 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff15409c710 con 0x7ff164101c00 2024-09-12T01:23:05.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:05.978+0000 7ff169c00640 1 -- 172.21.15.1:0/2048247695 --> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7ff164062630 con 0x7ff1480a2bb0 2024-09-12T01:23:06.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:06 smithi104 ceph-mon[35219]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:23:06.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:06 smithi104 ceph-mon[35219]: osdmap e39: 6 total, 6 up, 6 in 2024-09-12T01:23:06.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:06 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:23:06.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:06 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:23:06.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:06 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:23:06.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:06 smithi001 ceph-mon[30387]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:23:06.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:06 smithi001 ceph-mon[30387]: osdmap e39: 6 total, 6 up, 6 in 2024-09-12T01:23:06.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:06 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:23:06.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:06 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:23:06.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:06 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:23:06.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:06 smithi053 ceph-mon[35158]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:23:06.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:06 smithi053 ceph-mon[35158]: osdmap e39: 6 total, 6 up, 6 in 2024-09-12T01:23:06.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:06 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:23:06.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:06 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:23:06.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:06 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:23:07.125 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:07 smithi001 ceph-mon[30387]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:23:07.125 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:07 smithi001 ceph-mon[30387]: osdmap e40: 6 total, 6 up, 6 in 2024-09-12T01:23:07.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:07 smithi104 ceph-mon[35219]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:23:07.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:07 smithi104 ceph-mon[35219]: osdmap e40: 6 total, 6 up, 6 in 2024-09-12T01:23:07.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:07 smithi053 ceph-mon[35158]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:23:07.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:07 smithi053 ceph-mon[35158]: osdmap e40: 6 total, 6 up, 6 in 2024-09-12T01:23:08.171 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:08 smithi001 ceph-mon[30387]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:23:08.171 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:08 smithi001 ceph-mon[30387]: osdmap e41: 6 total, 6 up, 6 in 2024-09-12T01:23:08.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:08 smithi104 ceph-mon[35219]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:23:08.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:08 smithi104 ceph-mon[35219]: osdmap e41: 6 total, 6 up, 6 in 2024-09-12T01:23:08.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:08 smithi053 ceph-mon[35158]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:23:08.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:08 smithi053 ceph-mon[35158]: osdmap e41: 6 total, 6 up, 6 in 2024-09-12T01:23:09.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:09 smithi001 ceph-mon[30387]: osdmap e42: 6 total, 6 up, 6 in 2024-09-12T01:23:09.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:09 smithi104 ceph-mon[35219]: osdmap e42: 6 total, 6 up, 6 in 2024-09-12T01:23:09.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:09 smithi053 ceph-mon[35158]: osdmap e42: 6 total, 6 up, 6 in 2024-09-12T01:23:10.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:10 smithi104 ceph-mon[35219]: pgmap v149: 1 pgs: 1 remapped+peering; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:23:10.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:10 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/2896734477' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f31d8633-e435-499b-9d46-0aa11a2481cd"}]: dispatch 2024-09-12T01:23:10.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:10 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/2896734477' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f31d8633-e435-499b-9d46-0aa11a2481cd"}]': finished 2024-09-12T01:23:10.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:10 smithi104 ceph-mon[35219]: osdmap e43: 7 total, 6 up, 7 in 2024-09-12T01:23:10.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:10 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:23:10.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:10 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/2442601184' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:23:10.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:10 smithi001 ceph-mon[30387]: pgmap v149: 1 pgs: 1 remapped+peering; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:23:10.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:10 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/2896734477' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f31d8633-e435-499b-9d46-0aa11a2481cd"}]: dispatch 2024-09-12T01:23:10.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:10 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/2896734477' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f31d8633-e435-499b-9d46-0aa11a2481cd"}]': finished 2024-09-12T01:23:10.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:10 smithi001 ceph-mon[30387]: osdmap e43: 7 total, 6 up, 7 in 2024-09-12T01:23:10.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:10 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:23:10.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:10 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/2442601184' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:23:10.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:10 smithi053 ceph-mon[35158]: pgmap v149: 1 pgs: 1 remapped+peering; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:23:10.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:10 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/2896734477' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f31d8633-e435-499b-9d46-0aa11a2481cd"}]: dispatch 2024-09-12T01:23:10.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:10 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/2896734477' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f31d8633-e435-499b-9d46-0aa11a2481cd"}]': finished 2024-09-12T01:23:10.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:10 smithi053 ceph-mon[35158]: osdmap e43: 7 total, 6 up, 7 in 2024-09-12T01:23:10.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:10 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:23:10.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:10 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/2442601184' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:23:12.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:12 smithi104 ceph-mon[35219]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-09-12T01:23:12.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:12 smithi001 ceph-mon[30387]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-09-12T01:23:12.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:12 smithi053 ceph-mon[35158]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-09-12T01:23:14.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:14 smithi001 ceph-mon[30387]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T01:23:14.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:14 smithi104 ceph-mon[35219]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T01:23:14.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:14 smithi053 ceph-mon[35158]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T01:23:16.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:16 smithi104 ceph-mon[35219]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-09-12T01:23:16.409 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:16 smithi001 ceph-mon[30387]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-09-12T01:23:16.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:16 smithi053 ceph-mon[35158]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-09-12T01:23:18.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:18 smithi104 ceph-mon[35219]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-12T01:23:18.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:18 smithi001 ceph-mon[30387]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-12T01:23:18.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:18 smithi053 ceph-mon[35158]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-12T01:23:20.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:20 smithi104 ceph-mon[35219]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T01:23:20.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:20 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-12T01:23:20.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:20 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:23:20.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:20 smithi104 ceph-mon[35219]: Deploying daemon osd.6 on smithi001 2024-09-12T01:23:20.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:20 smithi001 ceph-mon[30387]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T01:23:20.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:20 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-12T01:23:20.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:20 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:23:20.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:20 smithi001 ceph-mon[30387]: Deploying daemon osd.6 on smithi001 2024-09-12T01:23:20.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:20 smithi053 ceph-mon[35158]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T01:23:20.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:20 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-12T01:23:20.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:20 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:23:20.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:20 smithi053 ceph-mon[35158]: Deploying daemon osd.6 on smithi001 2024-09-12T01:23:22.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:22 smithi104 ceph-mon[35219]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-12T01:23:22.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:22 smithi001 ceph-mon[30387]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-12T01:23:22.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:22 smithi053 ceph-mon[35158]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-12T01:23:24.305 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:24 smithi001 ceph-mon[30387]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:23:24.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:24 smithi104 ceph-mon[35219]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:23:24.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:24 smithi053 ceph-mon[35158]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:23:26.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:26 smithi104 ceph-mon[35219]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:23:26.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:26 smithi001 ceph-mon[30387]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:23:26.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:26 smithi053 ceph-mon[35158]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:23:28.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:28 smithi001 ceph-mon[30387]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:23:28.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:28 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:23:28.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:28 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:28.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:28 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:28.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:28 smithi104 ceph-mon[35219]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:23:28.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:28 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:23:28.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:28 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:28.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:28 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:28.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:28 smithi053 ceph-mon[35158]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:23:28.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:28 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:23:28.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:28 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:28.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:28 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:30.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:30 smithi104 ceph-mon[35219]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:23:30.384 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:30 smithi001 ceph-mon[30387]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:23:30.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:30 smithi053 ceph-mon[35158]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:23:31.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:31 smithi001 ceph-mon[30387]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:23:31.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:31 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:31.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:31 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:31.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:31 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:23:31.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:31 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:23:31.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:31 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:32.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:31 smithi104 ceph-mon[35219]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:23:32.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:31 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:32.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:31 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:32.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:31 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:23:32.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:31 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:23:32.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:31 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:32.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:32.078+0000 7ff15b400640 1 -- 172.21.15.1:0/2048247695 <== mgr.14150 v2:172.21.15.1:6800/1928662456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff164062630 con 0x7ff1480a2bb0 2024-09-12T01:23:32.078 INFO:teuthology.orchestra.run.smithi001.stdout:Created osd(s) 6 on host 'smithi001' 2024-09-12T01:23:32.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:32.081+0000 7ff169c00640 1 -- 172.21.15.1:0/2048247695 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff1480a2bb0 msgr2=0x7ff1480a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:23:32.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:32.081+0000 7ff169c00640 1 --2- 172.21.15.1:0/2048247695 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff1480a2bb0 0x7ff1480a5070 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff164199370 tx=0x7ff150005f90 comp rx=0 tx=0).stop 2024-09-12T01:23:32.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:32.081+0000 7ff169c00640 1 -- 172.21.15.1:0/2048247695 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff164101c00 msgr2=0x7ff164197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:23:32.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:32.081+0000 7ff169c00640 1 --2- 172.21.15.1:0/2048247695 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff164101c00 0x7ff164197e50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff15400e9c0 tx=0x7ff15400ee80 comp rx=0 tx=0).stop 2024-09-12T01:23:32.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:32.081+0000 7ff169c00640 1 -- 172.21.15.1:0/2048247695 shutdown_connections 2024-09-12T01:23:32.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:32.081+0000 7ff169c00640 1 --2- 172.21.15.1:0/2048247695 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff1480a2bb0 0x7ff1480a5070 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:23:32.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:32.081+0000 7ff169c00640 1 --2- 172.21.15.1:0/2048247695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1641037c0 0x7ff16419f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:23:32.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:32.081+0000 7ff169c00640 1 --2- 172.21.15.1:0/2048247695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff164102e00 0x7ff164198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:23:32.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:32.081+0000 7ff169c00640 1 --2- 172.21.15.1:0/2048247695 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff164101c00 0x7ff164197e50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:23:32.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:32.081+0000 7ff169c00640 1 -- 172.21.15.1:0/2048247695 >> 172.21.15.1:0/2048247695 conn(0x7ff1640fd390 msgr2=0x7ff164106100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:23:32.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:32.081+0000 7ff169c00640 1 -- 172.21.15.1:0/2048247695 shutdown_connections 2024-09-12T01:23:32.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:23:32.081+0000 7ff169c00640 1 -- 172.21.15.1:0/2048247695 wait complete. 2024-09-12T01:23:32.125 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:31 smithi053 ceph-mon[35158]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:23:32.125 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:31 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:32.125 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:31 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:32.125 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:31 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:23:32.126 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:31 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:23:32.126 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:31 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:33.027 DEBUG:teuthology.orchestra.run.smithi001:osd.6> sudo journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@osd.6.service 2024-09-12T01:23:33.033 INFO:tasks.cephadm:Deploying osd.7 on smithi053 with /dev/vg_nvme/lv_2... 2024-09-12T01:23:33.033 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-12T01:23:33.061 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:32 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:23:33.061 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:32 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:33.064 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:32 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:33.064 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:32 smithi053 ceph-mon[35158]: from='osd.6 [v2:172.21.15.1:6818/3413067918,v1:172.21.15.1:6819/3413067918]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T01:23:33.064 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:32 smithi053 ceph-mon[35158]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T01:23:33.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:32 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:23:33.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:32 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:33.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:32 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:33.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:32 smithi104 ceph-mon[35219]: from='osd.6 [v2:172.21.15.1:6818/3413067918,v1:172.21.15.1:6819/3413067918]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T01:23:33.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:32 smithi104 ceph-mon[35219]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T01:23:33.081 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:32 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:23:33.081 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:32 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:33.081 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:32 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:33.081 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:32 smithi001 ceph-mon[30387]: from='osd.6 [v2:172.21.15.1:6818/3413067918,v1:172.21.15.1:6819/3413067918]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T01:23:33.081 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:32 smithi001 ceph-mon[30387]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T01:23:33.295 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T01:23:34.377 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:34 smithi001 ceph-mon[30387]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:23:34.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:34 smithi001 ceph-mon[30387]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-12T01:23:34.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:34 smithi001 ceph-mon[30387]: osdmap e44: 7 total, 6 up, 7 in 2024-09-12T01:23:34.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:34 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:23:34.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:34 smithi001 ceph-mon[30387]: from='osd.6 [v2:172.21.15.1:6818/3413067918,v1:172.21.15.1:6819/3413067918]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-12T01:23:34.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:34 smithi001 ceph-mon[30387]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-12T01:23:34.378 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 12 01:23:34 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-6[57769]: 2024-09-12T01:23:34.100+0000 7fabff000640 -1 osd.6 0 waiting for initial osdmap 2024-09-12T01:23:34.378 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 12 01:23:34 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-6[57769]: 2024-09-12T01:23:34.116+0000 7fabfa000640 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T01:23:34.446 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:34 smithi053 ceph-mon[35158]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:23:34.446 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:34 smithi053 ceph-mon[35158]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-12T01:23:34.446 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:34 smithi053 ceph-mon[35158]: osdmap e44: 7 total, 6 up, 7 in 2024-09-12T01:23:34.446 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:34 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:23:34.446 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:34 smithi053 ceph-mon[35158]: from='osd.6 [v2:172.21.15.1:6818/3413067918,v1:172.21.15.1:6819/3413067918]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-12T01:23:34.446 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:34 smithi053 ceph-mon[35158]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-12T01:23:34.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:34 smithi104 ceph-mon[35219]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:23:34.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:34 smithi104 ceph-mon[35219]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-12T01:23:34.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:34 smithi104 ceph-mon[35219]: osdmap e44: 7 total, 6 up, 7 in 2024-09-12T01:23:34.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:34 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:23:34.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:34 smithi104 ceph-mon[35219]: from='osd.6 [v2:172.21.15.1:6818/3413067918,v1:172.21.15.1:6819/3413067918]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-12T01:23:34.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:34 smithi104 ceph-mon[35219]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-12T01:23:35.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:35 smithi001 ceph-mon[30387]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-09-12T01:23:35.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:35 smithi001 ceph-mon[30387]: osdmap e45: 7 total, 6 up, 7 in 2024-09-12T01:23:35.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:35 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:23:35.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:35 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:23:35.282 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:35 smithi053 ceph-mon[35158]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-09-12T01:23:35.282 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:35 smithi053 ceph-mon[35158]: osdmap e45: 7 total, 6 up, 7 in 2024-09-12T01:23:35.282 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:35 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:23:35.282 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:35 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:23:35.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:35 smithi104 ceph-mon[35219]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-09-12T01:23:35.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:35 smithi104 ceph-mon[35219]: osdmap e45: 7 total, 6 up, 7 in 2024-09-12T01:23:35.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:35 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:23:35.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:35 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:23:36.011 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-12T01:23:36.039 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi053:vg_nvme/lv_2 2024-09-12T01:23:36.334 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T01:23:36.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:36 smithi001 ceph-mon[30387]: purged_snaps scrub starts 2024-09-12T01:23:36.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:36 smithi001 ceph-mon[30387]: purged_snaps scrub ok 2024-09-12T01:23:36.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:36 smithi001 ceph-mon[30387]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:23:36.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:36 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:23:36.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:36 smithi001 ceph-mon[30387]: osd.6 [v2:172.21.15.1:6818/3413067918,v1:172.21.15.1:6819/3413067918] boot 2024-09-12T01:23:36.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:36 smithi001 ceph-mon[30387]: osdmap e46: 7 total, 7 up, 7 in 2024-09-12T01:23:36.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:36 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:23:36.454 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:36 smithi053 ceph-mon[35158]: purged_snaps scrub starts 2024-09-12T01:23:36.454 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:36 smithi053 ceph-mon[35158]: purged_snaps scrub ok 2024-09-12T01:23:36.454 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:36 smithi053 ceph-mon[35158]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:23:36.455 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:36 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:23:36.455 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:36 smithi053 ceph-mon[35158]: osd.6 [v2:172.21.15.1:6818/3413067918,v1:172.21.15.1:6819/3413067918] boot 2024-09-12T01:23:36.455 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:36 smithi053 ceph-mon[35158]: osdmap e46: 7 total, 7 up, 7 in 2024-09-12T01:23:36.455 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:36 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:23:36.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:36 smithi104 ceph-mon[35219]: purged_snaps scrub starts 2024-09-12T01:23:36.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:36 smithi104 ceph-mon[35219]: purged_snaps scrub ok 2024-09-12T01:23:36.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:36 smithi104 ceph-mon[35219]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:23:36.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:36 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:23:36.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:36 smithi104 ceph-mon[35219]: osd.6 [v2:172.21.15.1:6818/3413067918,v1:172.21.15.1:6819/3413067918] boot 2024-09-12T01:23:36.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:36 smithi104 ceph-mon[35219]: osdmap e46: 7 total, 7 up, 7 in 2024-09-12T01:23:36.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:36 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:23:36.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:36.804+0000 7fdc1a4eb640 1 -- 172.21.15.53:0/2394252209 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc14101d10 msgr2=0x7fdc14102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:23:36.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:36.804+0000 7fdc1a4eb640 1 --2- 172.21.15.53:0/2394252209 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc14101d10 0x7fdc14102190 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdc08009900 tx=0x7fdc0802a410 comp rx=0 tx=0).stop 2024-09-12T01:23:36.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:36.804+0000 7fdc1a4eb640 1 -- 172.21.15.53:0/2394252209 shutdown_connections 2024-09-12T01:23:36.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:36.804+0000 7fdc1a4eb640 1 --2- 172.21.15.53:0/2394252209 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc141026d0 0x7fdc14108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:23:36.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:36.804+0000 7fdc1a4eb640 1 --2- 172.21.15.53:0/2394252209 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc14101d10 0x7fdc14102190 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:23:36.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:36.804+0000 7fdc1a4eb640 1 --2- 172.21.15.53:0/2394252209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc14100b10 0x7fdc14100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:23:36.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:36.804+0000 7fdc1a4eb640 1 -- 172.21.15.53:0/2394252209 >> 172.21.15.53:0/2394252209 conn(0x7fdc140fc2a0 msgr2=0x7fdc140fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:23:36.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:36.804+0000 7fdc1a4eb640 1 -- 172.21.15.53:0/2394252209 shutdown_connections 2024-09-12T01:23:36.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:36.804+0000 7fdc1a4eb640 1 -- 172.21.15.53:0/2394252209 wait complete. 2024-09-12T01:23:36.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:36.805+0000 7fdc1a4eb640 1 Processor -- start 2024-09-12T01:23:36.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:36.805+0000 7fdc1a4eb640 1 -- start start 2024-09-12T01:23:36.829 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:36.805+0000 7fdc1a4eb640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc14100b10 0x7fdc14197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:23:36.829 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:36.805+0000 7fdc1a4eb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc14101d10 0x7fdc14198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:23:36.829 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:36.805+0000 7fdc1a4eb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc141026d0 0x7fdc1419f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:23:36.829 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:36.805+0000 7fdc1a4eb640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1410c790 con 0x7fdc14101d10 2024-09-12T01:23:36.829 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:36.805+0000 7fdc1a4eb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1410c610 con 0x7fdc141026d0 2024-09-12T01:23:36.829 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:36.805+0000 7fdc1a4eb640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1410c910 con 0x7fdc14100b10 2024-09-12T01:23:36.829 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:36.805+0000 7fdc18a88640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc141026d0 0x7fdc1419f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:23:36.829 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:36.805+0000 7fdc18a88640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc141026d0 0x7fdc1419f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48868/0 (socket says 172.21.15.53:48868) 2024-09-12T01:23:36.829 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:36.805+0000 7fdc18a88640 1 -- 172.21.15.53:0/615048746 learned_addr learned my addr 172.21.15.53:0/615048746 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-12T01:23:36.829 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:36.805+0000 7fdc137fe640 1 --2- 172.21.15.53:0/615048746 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc14101d10 0x7fdc14198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:23:36.829 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:36.805+0000 7fdc13fff640 1 --2- 172.21.15.53:0/615048746 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc14100b10 0x7fdc14197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:23:36.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:36.806+0000 7fdc18a88640 1 -- 172.21.15.53:0/615048746 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc14100b10 msgr2=0x7fdc14197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:23:36.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:36.806+0000 7fdc18a88640 1 --2- 172.21.15.53:0/615048746 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc14100b10 0x7fdc14197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:23:36.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:36.806+0000 7fdc18a88640 1 -- 172.21.15.53:0/615048746 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc14101d10 msgr2=0x7fdc14198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:23:36.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:36.806+0000 7fdc18a88640 1 --2- 172.21.15.53:0/615048746 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc14101d10 0x7fdc14198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:23:36.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:36.806+0000 7fdc18a88640 1 -- 172.21.15.53:0/615048746 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc1419fb60 con 0x7fdc141026d0 2024-09-12T01:23:36.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:36.806+0000 7fdc13fff640 1 --2- 172.21.15.53:0/615048746 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc14100b10 0x7fdc14197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:23:36.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:36.806+0000 7fdc18a88640 1 --2- 172.21.15.53:0/615048746 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc141026d0 0x7fdc1419f4f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00eaa0 tx=0x7fdc0c00ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:23:36.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:36.806+0000 7fdc117fa640 1 -- 172.21.15.53:0/615048746 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c00cdb0 con 0x7fdc141026d0 2024-09-12T01:23:36.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:36.806+0000 7fdc117fa640 1 -- 172.21.15.53:0/615048746 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc0c005020 con 0x7fdc141026d0 2024-09-12T01:23:36.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:36.806+0000 7fdc1a4eb640 1 -- 172.21.15.53:0/615048746 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc1419fe50 con 0x7fdc141026d0 2024-09-12T01:23:36.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:36.806+0000 7fdc117fa640 1 -- 172.21.15.53:0/615048746 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c0116b0 con 0x7fdc141026d0 2024-09-12T01:23:36.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:36.806+0000 7fdc1a4eb640 1 -- 172.21.15.53:0/615048746 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc141a0300 con 0x7fdc141026d0 2024-09-12T01:23:36.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:36.807+0000 7fdc1a4eb640 1 -- 172.21.15.53:0/615048746 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc141a0590 con 0x7fdc141026d0 2024-09-12T01:23:36.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:36.808+0000 7fdc117fa640 1 -- 172.21.15.53:0/615048746 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fdc0c005560 con 0x7fdc141026d0 2024-09-12T01:23:36.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:36.809+0000 7fdc117fa640 1 --2- 172.21.15.53:0/615048746 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fdbfc0a2c30 0x7fdbfc0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:23:36.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:36.809+0000 7fdc117fa640 1 -- 172.21.15.53:0/615048746 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7fdc0c0ce4e0 con 0x7fdc141026d0 2024-09-12T01:23:36.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:36.809+0000 7fdc13fff640 1 --2- 172.21.15.53:0/615048746 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fdbfc0a2c30 0x7fdbfc0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:23:36.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:36.810+0000 7fdc13fff640 1 --2- 172.21.15.53:0/615048746 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fdbfc0a2c30 0x7fdbfc0a50f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdc040058c0 tx=0x7fdc04005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:23:36.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:36.814+0000 7fdc117fa640 1 -- 172.21.15.53:0/615048746 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdc0c016030 con 0x7fdc141026d0 2024-09-12T01:23:37.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:37.001+0000 7fdc1a4eb640 1 -- 172.21.15.53:0/615048746 --> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fdc141a0760 con 0x7fdbfc0a2c30 2024-09-12T01:23:37.411 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:37 smithi053 ceph-mon[35158]: osdmap e47: 7 total, 7 up, 7 in 2024-09-12T01:23:37.411 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:37 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:23:37.411 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:37 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:23:37.411 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:37 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:23:37.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:37 smithi001 ceph-mon[30387]: osdmap e47: 7 total, 7 up, 7 in 2024-09-12T01:23:37.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:37 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:23:37.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:37 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:23:37.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:37 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:23:37.576 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:37 smithi104 ceph-mon[35219]: osdmap e47: 7 total, 7 up, 7 in 2024-09-12T01:23:37.577 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:37 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:23:37.577 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:37 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:23:37.577 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:37 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:23:38.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:38 smithi001 ceph-mon[30387]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:23:38.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:38 smithi001 ceph-mon[30387]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:23:38.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:38 smithi001 ceph-mon[30387]: Detected new or changed devices on smithi001 2024-09-12T01:23:38.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:38 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:38.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:38 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:38.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:38 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:23:38.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:38 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:23:38.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:38 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:23:38.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:38 smithi001 ceph-mon[30387]: Adjusting osd_memory_target on smithi001 to 5698M 2024-09-12T01:23:38.429 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:38 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:38.429 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:38 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:23:38.429 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:38 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:23:38.429 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:38 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:38.451 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:38 smithi053 ceph-mon[35158]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:23:38.452 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:38 smithi053 ceph-mon[35158]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:23:38.452 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:38 smithi053 ceph-mon[35158]: Detected new or changed devices on smithi001 2024-09-12T01:23:38.452 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:38 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:38.452 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:38 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:38.452 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:38 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:23:38.452 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:38 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:23:38.452 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:38 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:23:38.452 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:38 smithi053 ceph-mon[35158]: Adjusting osd_memory_target on smithi001 to 5698M 2024-09-12T01:23:38.452 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:38 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:38.452 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:38 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:23:38.452 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:38 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:23:38.453 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:38 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:38.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:38 smithi104 ceph-mon[35219]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:23:38.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:38 smithi104 ceph-mon[35219]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:23:38.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:38 smithi104 ceph-mon[35219]: Detected new or changed devices on smithi001 2024-09-12T01:23:38.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:38 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:38.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:38 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:38.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:38 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:23:38.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:38 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:23:38.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:38 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:23:38.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:38 smithi104 ceph-mon[35219]: Adjusting osd_memory_target on smithi001 to 5698M 2024-09-12T01:23:38.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:38 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:38.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:38 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:23:38.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:38 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:23:38.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:38 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:40.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:40 smithi001 ceph-mon[30387]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:23:40.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:40 smithi001 ceph-mon[30387]: from='client.? 172.21.15.53:0/2713489698' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e64da49e-d31e-4801-b6cc-634a233b4761"}]: dispatch 2024-09-12T01:23:40.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:40 smithi001 ceph-mon[30387]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e64da49e-d31e-4801-b6cc-634a233b4761"}]: dispatch 2024-09-12T01:23:40.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:40 smithi001 ceph-mon[30387]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e64da49e-d31e-4801-b6cc-634a233b4761"}]': finished 2024-09-12T01:23:40.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:40 smithi001 ceph-mon[30387]: osdmap e48: 8 total, 7 up, 8 in 2024-09-12T01:23:40.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:40 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:23:40.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:40 smithi001 ceph-mon[30387]: from='client.? 172.21.15.53:0/2621456897' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:23:40.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:40 smithi053 ceph-mon[35158]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:23:40.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:40 smithi053 ceph-mon[35158]: from='client.? 172.21.15.53:0/2713489698' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e64da49e-d31e-4801-b6cc-634a233b4761"}]: dispatch 2024-09-12T01:23:40.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:40 smithi053 ceph-mon[35158]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e64da49e-d31e-4801-b6cc-634a233b4761"}]: dispatch 2024-09-12T01:23:40.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:40 smithi053 ceph-mon[35158]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e64da49e-d31e-4801-b6cc-634a233b4761"}]': finished 2024-09-12T01:23:40.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:40 smithi053 ceph-mon[35158]: osdmap e48: 8 total, 7 up, 8 in 2024-09-12T01:23:40.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:40 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:23:40.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:40 smithi053 ceph-mon[35158]: from='client.? 172.21.15.53:0/2621456897' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:23:40.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:40 smithi104 ceph-mon[35219]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:23:40.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:40 smithi104 ceph-mon[35219]: from='client.? 172.21.15.53:0/2713489698' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e64da49e-d31e-4801-b6cc-634a233b4761"}]: dispatch 2024-09-12T01:23:40.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:40 smithi104 ceph-mon[35219]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e64da49e-d31e-4801-b6cc-634a233b4761"}]: dispatch 2024-09-12T01:23:40.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:40 smithi104 ceph-mon[35219]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e64da49e-d31e-4801-b6cc-634a233b4761"}]': finished 2024-09-12T01:23:40.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:40 smithi104 ceph-mon[35219]: osdmap e48: 8 total, 7 up, 8 in 2024-09-12T01:23:40.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:40 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:23:40.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:40 smithi104 ceph-mon[35219]: from='client.? 172.21.15.53:0/2621456897' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:23:42.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:42 smithi001 ceph-mon[30387]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:23:42.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:42 smithi053 ceph-mon[35158]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:23:42.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:42 smithi104 ceph-mon[35219]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:23:44.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:44 smithi001 ceph-mon[30387]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:23:44.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:44 smithi053 ceph-mon[35158]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:23:44.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:44 smithi104 ceph-mon[35219]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:23:46.300 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:46 smithi053 ceph-mon[35158]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:23:46.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:46 smithi001 ceph-mon[30387]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:23:46.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:46 smithi104 ceph-mon[35219]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:23:48.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:48 smithi001 ceph-mon[30387]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:23:48.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:48 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-12T01:23:48.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:48 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:23:48.453 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:48 smithi053 ceph-mon[35158]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:23:48.453 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:48 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-12T01:23:48.453 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:48 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:23:48.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:48 smithi104 ceph-mon[35219]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:23:48.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:48 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-12T01:23:48.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:48 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:23:49.215 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:49 smithi053 ceph-mon[35158]: Deploying daemon osd.7 on smithi053 2024-09-12T01:23:49.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:49 smithi001 ceph-mon[30387]: Deploying daemon osd.7 on smithi053 2024-09-12T01:23:49.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:49 smithi104 ceph-mon[35219]: Deploying daemon osd.7 on smithi053 2024-09-12T01:23:50.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:50 smithi001 ceph-mon[30387]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:23:50.444 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:50 smithi053 ceph-mon[35158]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:23:50.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:50 smithi104 ceph-mon[35219]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:23:52.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:52 smithi001 ceph-mon[30387]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:23:52.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:52 smithi053 ceph-mon[35158]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:23:52.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:52 smithi104 ceph-mon[35219]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:23:54.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:54 smithi001 ceph-mon[30387]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:23:54.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:54 smithi053 ceph-mon[35158]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:23:54.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:54 smithi104 ceph-mon[35219]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:23:55.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:55 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:23:55.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:55 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:55.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:55 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:55.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:55 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:23:55.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:55 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:55.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:55 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:55.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:55 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:23:55.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:55 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:55.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:55 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:56.187 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:56 smithi053 ceph-mon[35158]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:23:56.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:56 smithi104 ceph-mon[35219]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:23:56.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:56 smithi001 ceph-mon[30387]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:23:57.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:57.969+0000 7fdc117fa640 1 -- 172.21.15.53:0/615048746 <== mgr.14150 v2:172.21.15.1:6800/1928662456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdc141a0760 con 0x7fdbfc0a2c30 2024-09-12T01:23:57.970 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 7 on host 'smithi053' 2024-09-12T01:23:57.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:57.972+0000 7fdc1a4eb640 1 -- 172.21.15.53:0/615048746 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fdbfc0a2c30 msgr2=0x7fdbfc0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:23:57.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:57.972+0000 7fdc1a4eb640 1 --2- 172.21.15.53:0/615048746 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fdbfc0a2c30 0x7fdbfc0a50f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdc040058c0 tx=0x7fdc04005850 comp rx=0 tx=0).stop 2024-09-12T01:23:57.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:57.972+0000 7fdc1a4eb640 1 -- 172.21.15.53:0/615048746 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc141026d0 msgr2=0x7fdc1419f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:23:57.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:57.972+0000 7fdc1a4eb640 1 --2- 172.21.15.53:0/615048746 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc141026d0 0x7fdc1419f4f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00eaa0 tx=0x7fdc0c00ef60 comp rx=0 tx=0).stop 2024-09-12T01:23:57.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:57.972+0000 7fdc1a4eb640 1 -- 172.21.15.53:0/615048746 shutdown_connections 2024-09-12T01:23:57.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:57.972+0000 7fdc1a4eb640 1 --2- 172.21.15.53:0/615048746 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc141026d0 0x7fdc1419f4f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:23:57.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:57.972+0000 7fdc1a4eb640 1 --2- 172.21.15.53:0/615048746 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc14101d10 0x7fdc14198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:23:57.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:57.972+0000 7fdc1a4eb640 1 --2- 172.21.15.53:0/615048746 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fdbfc0a2c30 0x7fdbfc0a50f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:23:57.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:57.972+0000 7fdc1a4eb640 1 --2- 172.21.15.53:0/615048746 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc14100b10 0x7fdc14197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:23:57.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:57.972+0000 7fdc1a4eb640 1 -- 172.21.15.53:0/615048746 >> 172.21.15.53:0/615048746 conn(0x7fdc140fc2a0 msgr2=0x7fdc1406a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:23:57.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:57.972+0000 7fdc1a4eb640 1 -- 172.21.15.53:0/615048746 shutdown_connections 2024-09-12T01:23:57.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:23:57.972+0000 7fdc1a4eb640 1 -- 172.21.15.53:0/615048746 wait complete. 2024-09-12T01:23:58.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:58 smithi053 ceph-mon[35158]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:23:58.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:58 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:58.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:58 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:58.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:58 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:23:58.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:58 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:23:58.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:58 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:58.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:58 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:23:58.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:58 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:58.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:23:58 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:58.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:58 smithi104 ceph-mon[35219]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:23:58.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:58 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:58.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:58 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:58.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:58 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:23:58.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:58 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:23:58.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:58 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:58.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:58 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:23:58.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:58 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:58.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:23:58 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:58.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:58 smithi001 ceph-mon[30387]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:23:58.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:58 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:58.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:58 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:58.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:58 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:23:58.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:58 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:23:58.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:58 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:58.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:58 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:23:58.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:58 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:58.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:23:58 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:23:58.696 DEBUG:teuthology.orchestra.run.smithi053:osd.7> sudo journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@osd.7.service 2024-09-12T01:23:58.699 INFO:tasks.cephadm:Deploying osd.8 on smithi104 with /dev/vg_nvme/lv_2... 2024-09-12T01:23:58.699 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-12T01:23:58.952 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:24:00.183 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:00 smithi053 ceph-mon[35158]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:24:00.184 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:00 smithi053 ceph-mon[35158]: from='osd.7 [v2:172.21.15.53:6816/2841484136,v1:172.21.15.53:6817/2841484136]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T01:24:00.184 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:00 smithi053 ceph-mon[35158]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T01:24:00.184 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:23:59 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:23:59.966+0000 7fe01c982640 -1 osd.7 0 log_to_monitors true 2024-09-12T01:24:00.299 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:00 smithi104 ceph-mon[35219]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:24:00.299 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:00 smithi104 ceph-mon[35219]: from='osd.7 [v2:172.21.15.53:6816/2841484136,v1:172.21.15.53:6817/2841484136]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T01:24:00.299 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:00 smithi104 ceph-mon[35219]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T01:24:00.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:00 smithi001 ceph-mon[30387]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:24:00.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:00 smithi001 ceph-mon[30387]: from='osd.7 [v2:172.21.15.53:6816/2841484136,v1:172.21.15.53:6817/2841484136]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T01:24:00.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:00 smithi001 ceph-mon[30387]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T01:24:01.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:01 smithi053 ceph-mon[35158]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-12T01:24:01.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:01 smithi053 ceph-mon[35158]: from='osd.7 [v2:172.21.15.53:6816/2841484136,v1:172.21.15.53:6817/2841484136]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-12T01:24:01.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:01 smithi053 ceph-mon[35158]: osdmap e49: 8 total, 7 up, 8 in 2024-09-12T01:24:01.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:01 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:24:01.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:01 smithi053 ceph-mon[35158]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-12T01:24:01.412 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:01 smithi104 ceph-mon[35219]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-12T01:24:01.412 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:01 smithi104 ceph-mon[35219]: from='osd.7 [v2:172.21.15.53:6816/2841484136,v1:172.21.15.53:6817/2841484136]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-12T01:24:01.412 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:01 smithi104 ceph-mon[35219]: osdmap e49: 8 total, 7 up, 8 in 2024-09-12T01:24:01.412 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:01 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:24:01.412 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:01 smithi104 ceph-mon[35219]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-12T01:24:01.415 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-12T01:24:01.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:01 smithi001 ceph-mon[30387]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-12T01:24:01.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:01 smithi001 ceph-mon[30387]: from='osd.7 [v2:172.21.15.53:6816/2841484136,v1:172.21.15.53:6817/2841484136]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-12T01:24:01.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:01 smithi001 ceph-mon[30387]: osdmap e49: 8 total, 7 up, 8 in 2024-09-12T01:24:01.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:01 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:24:01.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:01 smithi001 ceph-mon[30387]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-12T01:24:01.440 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi104:vg_nvme/lv_2 2024-09-12T01:24:01.682 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:24:01.908 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:24:01 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:24:01.607+0000 7fe019119640 -1 osd.7 0 waiting for initial osdmap 2024-09-12T01:24:01.908 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:24:01 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:24:01.616+0000 7fe010725640 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T01:24:02.208 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:02.207+0000 7f566ef4e640 1 -- 172.21.15.104:0/690417265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5668100b10 msgr2=0x7f5668100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:24:02.209 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:02.207+0000 7f566ef4e640 1 --2- 172.21.15.104:0/690417265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5668100b10 0x7f5668100f10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f565c009900 tx=0x7f565c02a410 comp rx=0 tx=0).stop 2024-09-12T01:24:02.209 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:02.207+0000 7f566ef4e640 1 -- 172.21.15.104:0/690417265 shutdown_connections 2024-09-12T01:24:02.209 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:02.207+0000 7f566ef4e640 1 --2- 172.21.15.104:0/690417265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56681026d0 0x7f5668108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:24:02.209 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:02.207+0000 7f566ef4e640 1 --2- 172.21.15.104:0/690417265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5668101d10 0x7f5668102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:24:02.209 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:02.207+0000 7f566ef4e640 1 --2- 172.21.15.104:0/690417265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5668100b10 0x7f5668100f10 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:24:02.209 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:02.207+0000 7f566ef4e640 1 -- 172.21.15.104:0/690417265 >> 172.21.15.104:0/690417265 conn(0x7f56680fc2a0 msgr2=0x7f56680fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:24:02.209 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:02.207+0000 7f566ef4e640 1 -- 172.21.15.104:0/690417265 shutdown_connections 2024-09-12T01:24:02.209 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:02.207+0000 7f566ef4e640 1 -- 172.21.15.104:0/690417265 wait complete. 2024-09-12T01:24:02.209 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:02.208+0000 7f566ef4e640 1 Processor -- start 2024-09-12T01:24:02.209 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:02.208+0000 7f566ef4e640 1 -- start start 2024-09-12T01:24:02.209 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:02.208+0000 7f566ef4e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5668100b10 0x7f566806b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:24:02.210 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:02.208+0000 7f566ef4e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5668101d10 0x7f566819f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:24:02.210 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:02.208+0000 7f566ef4e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56681026d0 0x7f56681a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:24:02.210 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:02.208+0000 7f566ef4e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f566810c870 con 0x7f5668101d10 2024-09-12T01:24:02.210 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:02.208+0000 7f566ef4e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f566810c6f0 con 0x7f56681026d0 2024-09-12T01:24:02.210 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:02.208+0000 7f566ef4e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f566810c9f0 con 0x7f5668100b10 2024-09-12T01:24:02.210 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:02.208+0000 7f566ccea640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5668100b10 0x7f566806b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:24:02.210 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:02.208+0000 7f566ccea640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5668100b10 0x7f566806b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:39994/0 (socket says 172.21.15.104:39994) 2024-09-12T01:24:02.210 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:02.208+0000 7f566ccea640 1 -- 172.21.15.104:0/2558323098 learned_addr learned my addr 172.21.15.104:0/2558323098 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-09-12T01:24:02.210 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:02.208+0000 7f566d4eb640 1 --2- 172.21.15.104:0/2558323098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56681026d0 0x7f56681a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:24:02.210 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:02.208+0000 7f5667fff640 1 --2- 172.21.15.104:0/2558323098 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5668101d10 0x7f566819f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:24:02.210 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:02.209+0000 7f566ccea640 1 -- 172.21.15.104:0/2558323098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56681026d0 msgr2=0x7f56681a17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:24:02.210 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:02.209+0000 7f566ccea640 1 --2- 172.21.15.104:0/2558323098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56681026d0 0x7f56681a17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:24:02.210 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:02.209+0000 7f566ccea640 1 -- 172.21.15.104:0/2558323098 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5668101d10 msgr2=0x7f566819f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:24:02.211 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:02.209+0000 7f566ccea640 1 --2- 172.21.15.104:0/2558323098 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5668101d10 0x7f566819f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:24:02.211 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:02.209+0000 7f566ccea640 1 -- 172.21.15.104:0/2558323098 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56681a1fb0 con 0x7f5668100b10 2024-09-12T01:24:02.211 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:02.209+0000 7f566ccea640 1 --2- 172.21.15.104:0/2558323098 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5668100b10 0x7f566806b2f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f565c005ab0 tx=0x7f565c004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:24:02.211 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:02.209+0000 7f5665ffb640 1 -- 172.21.15.104:0/2558323098 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c03f030 con 0x7f5668100b10 2024-09-12T01:24:02.211 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:02.209+0000 7f566ef4e640 1 -- 172.21.15.104:0/2558323098 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56681a21e0 con 0x7f5668100b10 2024-09-12T01:24:02.211 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:02.209+0000 7f5665ffb640 1 -- 172.21.15.104:0/2558323098 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f565c003ce0 con 0x7f5668100b10 2024-09-12T01:24:02.211 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:02.209+0000 7f5665ffb640 1 -- 172.21.15.104:0/2558323098 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c035070 con 0x7f5668100b10 2024-09-12T01:24:02.211 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:02.209+0000 7f566ef4e640 1 -- 172.21.15.104:0/2558323098 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56681a2790 con 0x7f5668100b10 2024-09-12T01:24:02.212 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:02.210+0000 7f566ef4e640 1 -- 172.21.15.104:0/2558323098 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5668107fa0 con 0x7f5668100b10 2024-09-12T01:24:02.212 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:02.211+0000 7f5665ffb640 1 -- 172.21.15.104:0/2558323098 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f565c042030 con 0x7f5668100b10 2024-09-12T01:24:02.212 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:02.212+0000 7f5665ffb640 1 --2- 172.21.15.104:0/2558323098 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f56580a2bb0 0x7f56580a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:24:02.212 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:02.212+0000 7f5665ffb640 1 -- 172.21.15.104:0/2558323098 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5430+0+0 (secure 0 0 0) 0x7f565c0eeb40 con 0x7f5668100b10 2024-09-12T01:24:02.212 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:02.212+0000 7f5667fff640 1 --2- 172.21.15.104:0/2558323098 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f56580a2bb0 0x7f56580a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:24:02.212 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:02.213+0000 7f5667fff640 1 --2- 172.21.15.104:0/2558323098 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f56580a2bb0 0x7f56580a5070 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f566806b870 tx=0x7f5654008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:24:02.215 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:02.216+0000 7f5665ffb640 1 -- 172.21.15.104:0/2558323098 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f565c0bc5e0 con 0x7f5668100b10 2024-09-12T01:24:02.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:02 smithi053 ceph-mon[35158]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:24:02.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:02 smithi053 ceph-mon[35158]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-09-12T01:24:02.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:02 smithi053 ceph-mon[35158]: osdmap e50: 8 total, 7 up, 8 in 2024-09-12T01:24:02.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:24:02.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:24:02.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:02 smithi053 ceph-mon[35158]: from='osd.7 ' entity='osd.7' 2024-09-12T01:24:02.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:02.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:02.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:24:02.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:24:02.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:24:02.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:02.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:24:02.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:24:02.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:02.389 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:02.390+0000 7f566ef4e640 1 -- 172.21.15.104:0/2558323098 --> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f5668062630 con 0x7f56580a2bb0 2024-09-12T01:24:02.389 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:02 smithi104 ceph-mon[35219]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:24:02.389 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:02 smithi104 ceph-mon[35219]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-09-12T01:24:02.389 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:02 smithi104 ceph-mon[35219]: osdmap e50: 8 total, 7 up, 8 in 2024-09-12T01:24:02.389 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:02 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:24:02.390 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:02 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:24:02.390 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:02 smithi104 ceph-mon[35219]: from='osd.7 ' entity='osd.7' 2024-09-12T01:24:02.390 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:02 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:02.390 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:02 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:02.390 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:02 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:24:02.390 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:02 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:24:02.390 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:02 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:24:02.390 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:02 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:02.390 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:02 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:24:02.390 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:02 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:24:02.390 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:02 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:02.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:02 smithi001 ceph-mon[30387]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:24:02.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:02 smithi001 ceph-mon[30387]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-09-12T01:24:02.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:02 smithi001 ceph-mon[30387]: osdmap e50: 8 total, 7 up, 8 in 2024-09-12T01:24:02.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:02 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:24:02.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:02 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:24:02.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:02 smithi001 ceph-mon[30387]: from='osd.7 ' entity='osd.7' 2024-09-12T01:24:02.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:02 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:02.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:02 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:02.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:02 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:24:02.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:02 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:24:02.429 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:02 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:24:02.429 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:02 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:02.429 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:02 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:24:02.429 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:02 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:24:02.429 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:02 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:03.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:03 smithi053 ceph-mon[35158]: purged_snaps scrub starts 2024-09-12T01:24:03.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:03 smithi053 ceph-mon[35158]: purged_snaps scrub ok 2024-09-12T01:24:03.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:03 smithi053 ceph-mon[35158]: Detected new or changed devices on smithi053 2024-09-12T01:24:03.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:03 smithi053 ceph-mon[35158]: Adjusting osd_memory_target on smithi053 to 5697M 2024-09-12T01:24:03.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:03 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:24:03.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:03 smithi053 ceph-mon[35158]: osd.7 [v2:172.21.15.53:6816/2841484136,v1:172.21.15.53:6817/2841484136] boot 2024-09-12T01:24:03.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:03 smithi053 ceph-mon[35158]: osdmap e51: 8 total, 8 up, 8 in 2024-09-12T01:24:03.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:03 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:24:03.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:03 smithi053 ceph-mon[35158]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:24:03.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:03 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:24:03.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:03 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:24:03.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:03 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:24:03.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:03 smithi104 ceph-mon[35219]: purged_snaps scrub starts 2024-09-12T01:24:03.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:03 smithi104 ceph-mon[35219]: purged_snaps scrub ok 2024-09-12T01:24:03.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:03 smithi104 ceph-mon[35219]: Detected new or changed devices on smithi053 2024-09-12T01:24:03.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:03 smithi104 ceph-mon[35219]: Adjusting osd_memory_target on smithi053 to 5697M 2024-09-12T01:24:03.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:03 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:24:03.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:03 smithi104 ceph-mon[35219]: osd.7 [v2:172.21.15.53:6816/2841484136,v1:172.21.15.53:6817/2841484136] boot 2024-09-12T01:24:03.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:03 smithi104 ceph-mon[35219]: osdmap e51: 8 total, 8 up, 8 in 2024-09-12T01:24:03.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:03 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:24:03.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:03 smithi104 ceph-mon[35219]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:24:03.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:03 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:24:03.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:03 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:24:03.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:03 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:24:03.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:03 smithi001 ceph-mon[30387]: purged_snaps scrub starts 2024-09-12T01:24:03.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:03 smithi001 ceph-mon[30387]: purged_snaps scrub ok 2024-09-12T01:24:03.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:03 smithi001 ceph-mon[30387]: Detected new or changed devices on smithi053 2024-09-12T01:24:03.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:03 smithi001 ceph-mon[30387]: Adjusting osd_memory_target on smithi053 to 5697M 2024-09-12T01:24:03.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:03 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:24:03.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:03 smithi001 ceph-mon[30387]: osd.7 [v2:172.21.15.53:6816/2841484136,v1:172.21.15.53:6817/2841484136] boot 2024-09-12T01:24:03.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:03 smithi001 ceph-mon[30387]: osdmap e51: 8 total, 8 up, 8 in 2024-09-12T01:24:03.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:03 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:24:03.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:03 smithi001 ceph-mon[30387]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:24:03.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:03 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:24:03.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:03 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:24:03.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:03 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:24:04.392 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:04 smithi104 ceph-mon[35219]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:24:04.392 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:04 smithi104 ceph-mon[35219]: osdmap e52: 8 total, 8 up, 8 in 2024-09-12T01:24:04.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:04 smithi001 ceph-mon[30387]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:24:04.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:04 smithi001 ceph-mon[30387]: osdmap e52: 8 total, 8 up, 8 in 2024-09-12T01:24:04.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:04 smithi053 ceph-mon[35158]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:24:04.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:04 smithi053 ceph-mon[35158]: osdmap e52: 8 total, 8 up, 8 in 2024-09-12T01:24:05.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:05 smithi001 ceph-mon[30387]: osdmap e53: 8 total, 8 up, 8 in 2024-09-12T01:24:05.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:05 smithi001 ceph-mon[30387]: from='client.? 172.21.15.104:0/4232679520' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fdfb5a42-dca7-4874-b479-176b4a191e21"}]: dispatch 2024-09-12T01:24:05.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:05 smithi001 ceph-mon[30387]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fdfb5a42-dca7-4874-b479-176b4a191e21"}]: dispatch 2024-09-12T01:24:05.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:05 smithi001 ceph-mon[30387]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fdfb5a42-dca7-4874-b479-176b4a191e21"}]': finished 2024-09-12T01:24:05.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:05 smithi001 ceph-mon[30387]: osdmap e54: 9 total, 8 up, 9 in 2024-09-12T01:24:05.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:05 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:24:05.433 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:05 smithi104 ceph-mon[35219]: osdmap e53: 8 total, 8 up, 8 in 2024-09-12T01:24:05.433 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:05 smithi104 ceph-mon[35219]: from='client.? 172.21.15.104:0/4232679520' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fdfb5a42-dca7-4874-b479-176b4a191e21"}]: dispatch 2024-09-12T01:24:05.433 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:05 smithi104 ceph-mon[35219]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fdfb5a42-dca7-4874-b479-176b4a191e21"}]: dispatch 2024-09-12T01:24:05.433 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:05 smithi104 ceph-mon[35219]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fdfb5a42-dca7-4874-b479-176b4a191e21"}]': finished 2024-09-12T01:24:05.434 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:05 smithi104 ceph-mon[35219]: osdmap e54: 9 total, 8 up, 9 in 2024-09-12T01:24:05.434 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:05 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:24:05.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:05 smithi053 ceph-mon[35158]: osdmap e53: 8 total, 8 up, 8 in 2024-09-12T01:24:05.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:05 smithi053 ceph-mon[35158]: from='client.? 172.21.15.104:0/4232679520' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fdfb5a42-dca7-4874-b479-176b4a191e21"}]: dispatch 2024-09-12T01:24:05.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:05 smithi053 ceph-mon[35158]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fdfb5a42-dca7-4874-b479-176b4a191e21"}]: dispatch 2024-09-12T01:24:05.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:05 smithi053 ceph-mon[35158]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fdfb5a42-dca7-4874-b479-176b4a191e21"}]': finished 2024-09-12T01:24:05.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:05 smithi053 ceph-mon[35158]: osdmap e54: 9 total, 8 up, 9 in 2024-09-12T01:24:05.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:05 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:24:06.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:06 smithi001 ceph-mon[30387]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:24:06.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:06 smithi001 ceph-mon[30387]: from='client.? 172.21.15.104:0/3419816113' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:24:06.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:06 smithi053 ceph-mon[35158]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:24:06.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:06 smithi053 ceph-mon[35158]: from='client.? 172.21.15.104:0/3419816113' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:24:06.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:06 smithi104 ceph-mon[35219]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:24:06.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:06 smithi104 ceph-mon[35219]: from='client.? 172.21.15.104:0/3419816113' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:24:07.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:07 smithi001 ceph-mon[30387]: osdmap e55: 9 total, 8 up, 9 in 2024-09-12T01:24:07.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:07 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:24:07.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:07 smithi053 ceph-mon[35158]: osdmap e55: 9 total, 8 up, 9 in 2024-09-12T01:24:07.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:07 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:24:07.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:07 smithi104 ceph-mon[35219]: osdmap e55: 9 total, 8 up, 9 in 2024-09-12T01:24:07.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:07 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:24:08.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:08 smithi001 ceph-mon[30387]: pgmap v191: 1 pgs: 1 active+recovery_wait+undersized+degraded+remapped; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2024-09-12T01:24:08.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:08 smithi001 ceph-mon[30387]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-09-12T01:24:08.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:08 smithi001 ceph-mon[30387]: osdmap e56: 9 total, 8 up, 9 in 2024-09-12T01:24:08.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:08 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:24:08.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:08 smithi053 ceph-mon[35158]: pgmap v191: 1 pgs: 1 active+recovery_wait+undersized+degraded+remapped; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2024-09-12T01:24:08.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:08 smithi053 ceph-mon[35158]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-09-12T01:24:08.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:08 smithi053 ceph-mon[35158]: osdmap e56: 9 total, 8 up, 9 in 2024-09-12T01:24:08.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:08 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:24:08.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:08 smithi104 ceph-mon[35219]: pgmap v191: 1 pgs: 1 active+recovery_wait+undersized+degraded+remapped; 577 KiB data, 261 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2024-09-12T01:24:08.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:08 smithi104 ceph-mon[35219]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-09-12T01:24:08.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:08 smithi104 ceph-mon[35219]: osdmap e56: 9 total, 8 up, 9 in 2024-09-12T01:24:08.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:08 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:24:09.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:09 smithi001 ceph-mon[30387]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-09-12T01:24:09.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:09 smithi001 ceph-mon[30387]: Cluster is now healthy 2024-09-12T01:24:09.472 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:09 smithi104 ceph-mon[35219]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-09-12T01:24:09.472 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:09 smithi104 ceph-mon[35219]: Cluster is now healthy 2024-09-12T01:24:09.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:09 smithi053 ceph-mon[35158]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-09-12T01:24:09.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:09 smithi053 ceph-mon[35158]: Cluster is now healthy 2024-09-12T01:24:10.380 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:10 smithi104 ceph-mon[35219]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:24:10.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:10 smithi001 ceph-mon[30387]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:24:10.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:10 smithi053 ceph-mon[35158]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:24:12.483 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:12 smithi104 ceph-mon[35219]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:24:12.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:12 smithi053 ceph-mon[35158]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:24:12.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:12 smithi001 ceph-mon[30387]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:24:13.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:13 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-12T01:24:13.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:13 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:24:13.528 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:13 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-12T01:24:13.528 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:13 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:24:13.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:13 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-12T01:24:13.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:13 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:24:14.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:14 smithi053 ceph-mon[35158]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:24:14.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:14 smithi053 ceph-mon[35158]: Deploying daemon osd.8 on smithi104 2024-09-12T01:24:14.516 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:14 smithi104 ceph-mon[35219]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:24:14.516 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:14 smithi104 ceph-mon[35219]: Deploying daemon osd.8 on smithi104 2024-09-12T01:24:14.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:14 smithi001 ceph-mon[30387]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:24:14.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:14 smithi001 ceph-mon[30387]: Deploying daemon osd.8 on smithi104 2024-09-12T01:24:16.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:16 smithi053 ceph-mon[35158]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:24:16.491 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:16 smithi104 ceph-mon[35219]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:24:16.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:16 smithi001 ceph-mon[30387]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:24:18.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:18 smithi053 ceph-mon[35158]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:24:18.549 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:18 smithi104 ceph-mon[35219]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:24:18.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:18 smithi001 ceph-mon[30387]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:24:19.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:19 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:24:19.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:19 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:19.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:19 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:19.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:19 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:24:19.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:19 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:19.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:19 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:19.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:19 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:24:19.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:19 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:19.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:19 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:20.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:20 smithi104 ceph-mon[35219]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:24:20.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:20 smithi053 ceph-mon[35158]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:24:20.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:20 smithi001 ceph-mon[30387]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:24:22.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:22 smithi053 ceph-mon[35158]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:24:22.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:22 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:22.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:22 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:22.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:22 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:24:22.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:22 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:24:22.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:22 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:22.552 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:22 smithi104 ceph-mon[35219]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:24:22.552 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:22 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:22.552 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:22 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:22.552 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:22 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:24:22.553 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:22 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:24:22.553 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:22 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:22.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:22 smithi001 ceph-mon[30387]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:24:22.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:22 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:22.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:22 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:22.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:22 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:24:22.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:22 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:24:22.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:22 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:22.869 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 8 on host 'smithi104' 2024-09-12T01:24:22.870 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:22.870+0000 7f5665ffb640 1 -- 172.21.15.104:0/2558323098 <== mgr.14150 v2:172.21.15.1:6800/1928662456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5668062630 con 0x7f56580a2bb0 2024-09-12T01:24:22.871 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:22.872+0000 7f566ef4e640 1 -- 172.21.15.104:0/2558323098 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f56580a2bb0 msgr2=0x7f56580a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:24:22.871 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:22.872+0000 7f566ef4e640 1 --2- 172.21.15.104:0/2558323098 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f56580a2bb0 0x7f56580a5070 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f566806b870 tx=0x7f5654008040 comp rx=0 tx=0).stop 2024-09-12T01:24:22.872 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:22.872+0000 7f566ef4e640 1 -- 172.21.15.104:0/2558323098 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5668100b10 msgr2=0x7f566806b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:24:22.872 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:22.872+0000 7f566ef4e640 1 --2- 172.21.15.104:0/2558323098 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5668100b10 0x7f566806b2f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f565c005ab0 tx=0x7f565c004360 comp rx=0 tx=0).stop 2024-09-12T01:24:22.872 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:22.872+0000 7f566ef4e640 1 -- 172.21.15.104:0/2558323098 shutdown_connections 2024-09-12T01:24:22.872 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:22.872+0000 7f566ef4e640 1 --2- 172.21.15.104:0/2558323098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56681026d0 0x7f56681a17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:24:22.872 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:22.872+0000 7f566ef4e640 1 --2- 172.21.15.104:0/2558323098 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5668101d10 0x7f566819f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:24:22.872 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:22.872+0000 7f566ef4e640 1 --2- 172.21.15.104:0/2558323098 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f56580a2bb0 0x7f56580a5070 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:24:22.872 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:22.872+0000 7f566ef4e640 1 --2- 172.21.15.104:0/2558323098 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5668100b10 0x7f566806b2f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:24:22.872 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:22.872+0000 7f566ef4e640 1 -- 172.21.15.104:0/2558323098 >> 172.21.15.104:0/2558323098 conn(0x7f56680fc2a0 msgr2=0x7f5668105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:24:22.872 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:22.872+0000 7f566ef4e640 1 -- 172.21.15.104:0/2558323098 shutdown_connections 2024-09-12T01:24:22.872 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:24:22.872+0000 7f566ef4e640 1 -- 172.21.15.104:0/2558323098 wait complete. 2024-09-12T01:24:23.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:23 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:24:23.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:23 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:23.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:23 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:23.548 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:23 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:24:23.548 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:23 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:23.549 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:23 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:23.576 DEBUG:teuthology.orchestra.run.smithi104:osd.8> sudo journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@osd.8.service 2024-09-12T01:24:23.578 INFO:tasks.cephadm:Deploying osd.9 on smithi001 with /dev/vg_nvme/lv_1... 2024-09-12T01:24:23.578 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-12T01:24:23.634 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:23 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:24:23.634 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:23 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:23.634 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:23 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:23.968 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:24:24.423 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:24 smithi001 ceph-mon[30387]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:24:24.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:24 smithi053 ceph-mon[35158]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:24:24.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:24 smithi104 ceph-mon[35219]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:24:25.251 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:25 smithi104 ceph-mon[35219]: from='osd.8 [v2:172.21.15.104:6816/941453547,v1:172.21.15.104:6817/941453547]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T01:24:25.251 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:25 smithi104 ceph-mon[35219]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T01:24:25.252 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:24:25 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:24:24.999+0000 7ff710e95640 -1 osd.8 0 log_to_monitors true 2024-09-12T01:24:25.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:25 smithi001 ceph-mon[30387]: from='osd.8 [v2:172.21.15.104:6816/941453547,v1:172.21.15.104:6817/941453547]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T01:24:25.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:25 smithi001 ceph-mon[30387]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T01:24:25.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:25 smithi053 ceph-mon[35158]: from='osd.8 [v2:172.21.15.104:6816/941453547,v1:172.21.15.104:6817/941453547]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T01:24:25.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:25 smithi053 ceph-mon[35158]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T01:24:26.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:26 smithi001 ceph-mon[30387]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:24:26.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:26 smithi001 ceph-mon[30387]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-12T01:24:26.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:26 smithi001 ceph-mon[30387]: osdmap e57: 9 total, 8 up, 9 in 2024-09-12T01:24:26.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:26 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:24:26.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:26 smithi001 ceph-mon[30387]: from='osd.8 [v2:172.21.15.104:6816/941453547,v1:172.21.15.104:6817/941453547]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-12T01:24:26.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:26 smithi001 ceph-mon[30387]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-12T01:24:26.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:26 smithi053 ceph-mon[35158]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:24:26.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:26 smithi053 ceph-mon[35158]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-12T01:24:26.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:26 smithi053 ceph-mon[35158]: osdmap e57: 9 total, 8 up, 9 in 2024-09-12T01:24:26.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:26 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:24:26.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:26 smithi053 ceph-mon[35158]: from='osd.8 [v2:172.21.15.104:6816/941453547,v1:172.21.15.104:6817/941453547]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-12T01:24:26.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:26 smithi053 ceph-mon[35158]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-12T01:24:26.498 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:26 smithi104 ceph-mon[35219]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:24:26.498 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:26 smithi104 ceph-mon[35219]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-12T01:24:26.499 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:26 smithi104 ceph-mon[35219]: osdmap e57: 9 total, 8 up, 9 in 2024-09-12T01:24:26.499 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:26 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:24:26.499 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:26 smithi104 ceph-mon[35219]: from='osd.8 [v2:172.21.15.104:6816/941453547,v1:172.21.15.104:6817/941453547]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-12T01:24:26.499 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:26 smithi104 ceph-mon[35219]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-12T01:24:26.499 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:24:26 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:24:26.241+0000 7ff70ce18640 -1 osd.8 0 waiting for initial osdmap 2024-09-12T01:24:26.499 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:24:26 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:24:26.252+0000 7ff708c40640 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T01:24:27.544 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:27 smithi001 ceph-mon[30387]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-12T01:24:27.544 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:27 smithi001 ceph-mon[30387]: osdmap e58: 9 total, 8 up, 9 in 2024-09-12T01:24:27.544 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:27 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:24:27.544 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:27 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:24:27.544 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:27 smithi001 ceph-mon[30387]: Detected new or changed devices on smithi104 2024-09-12T01:24:27.545 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:27 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:27.545 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:27 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:27.545 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:27 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:24:27.545 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:27 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:24:27.545 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:27 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:24:27.545 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:27 smithi001 ceph-mon[30387]: Adjusting osd_memory_target on smithi104 to 5696M 2024-09-12T01:24:27.545 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:27 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:27.545 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:27 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:24:27.545 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:27 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:24:27.545 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:27 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:27.553 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:24:27.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:27 smithi104 ceph-mon[35219]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-12T01:24:27.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:27 smithi104 ceph-mon[35219]: osdmap e58: 9 total, 8 up, 9 in 2024-09-12T01:24:27.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:27 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:24:27.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:27 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:24:27.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:27 smithi104 ceph-mon[35219]: Detected new or changed devices on smithi104 2024-09-12T01:24:27.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:27 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:27.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:27 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:27.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:27 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:24:27.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:27 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:24:27.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:27 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:24:27.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:27 smithi104 ceph-mon[35219]: Adjusting osd_memory_target on smithi104 to 5696M 2024-09-12T01:24:27.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:27 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:27.576 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:27 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:24:27.576 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:27 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:24:27.576 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:27 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:27.586 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi001:vg_nvme/lv_1 2024-09-12T01:24:27.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:27 smithi053 ceph-mon[35158]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-12T01:24:27.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:27 smithi053 ceph-mon[35158]: osdmap e58: 9 total, 8 up, 9 in 2024-09-12T01:24:27.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:27 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:24:27.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:27 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:24:27.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:27 smithi053 ceph-mon[35158]: Detected new or changed devices on smithi104 2024-09-12T01:24:27.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:27 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:27.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:27 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:27.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:27 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:24:27.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:27 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:24:27.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:27 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:24:27.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:27 smithi053 ceph-mon[35158]: Adjusting osd_memory_target on smithi104 to 5696M 2024-09-12T01:24:27.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:27 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:27.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:27 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:24:27.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:27 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:24:27.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:27 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:27.962 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:24:28.414 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:28 smithi001 ceph-mon[30387]: purged_snaps scrub starts 2024-09-12T01:24:28.414 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:28 smithi001 ceph-mon[30387]: purged_snaps scrub ok 2024-09-12T01:24:28.414 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:28 smithi001 ceph-mon[30387]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:24:28.414 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:28 smithi001 ceph-mon[30387]: osd.8 [v2:172.21.15.104:6816/941453547,v1:172.21.15.104:6817/941453547] boot 2024-09-12T01:24:28.414 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:28 smithi001 ceph-mon[30387]: osdmap e59: 9 total, 9 up, 9 in 2024-09-12T01:24:28.414 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:28 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:24:28.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:28.532+0000 7f1b81e00640 1 -- 172.21.15.1:0/2236413421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b7c1037c0 msgr2=0x7f1b7c10a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:24:28.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:28.532+0000 7f1b81e00640 1 --2- 172.21.15.1:0/2236413421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b7c1037c0 0x7f1b7c10a050 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1b74009ca0 tx=0x7f1b7402a8c0 comp rx=0 tx=0).stop 2024-09-12T01:24:28.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:28.532+0000 7f1b81e00640 1 -- 172.21.15.1:0/2236413421 shutdown_connections 2024-09-12T01:24:28.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:28.532+0000 7f1b81e00640 1 --2- 172.21.15.1:0/2236413421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b7c1037c0 0x7f1b7c10a050 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:24:28.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:28.532+0000 7f1b81e00640 1 --2- 172.21.15.1:0/2236413421 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b7c102e00 0x7f1b7c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:24:28.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:28.532+0000 7f1b81e00640 1 --2- 172.21.15.1:0/2236413421 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b7c101c00 0x7f1b7c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:24:28.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:28.532+0000 7f1b81e00640 1 -- 172.21.15.1:0/2236413421 >> 172.21.15.1:0/2236413421 conn(0x7f1b7c0fd390 msgr2=0x7f1b7c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:24:28.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:28.532+0000 7f1b81e00640 1 -- 172.21.15.1:0/2236413421 shutdown_connections 2024-09-12T01:24:28.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:28.532+0000 7f1b81e00640 1 -- 172.21.15.1:0/2236413421 wait complete. 2024-09-12T01:24:28.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:28.533+0000 7f1b81e00640 1 Processor -- start 2024-09-12T01:24:28.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:28.533+0000 7f1b81e00640 1 -- start start 2024-09-12T01:24:28.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:28.533+0000 7f1b81e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b7c101c00 0x7f1b7c197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:24:28.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:28.533+0000 7f1b81e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b7c102e00 0x7f1b7c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:24:28.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:28.533+0000 7f1b81e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b7c1037c0 0x7f1b7c19f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:24:28.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:28.533+0000 7f1b81e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c10d880 con 0x7f1b7c1037c0 2024-09-12T01:24:28.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:28.533+0000 7f1b81e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c10d700 con 0x7f1b7c102e00 2024-09-12T01:24:28.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:28.534+0000 7f1b81e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c10da00 con 0x7f1b7c101c00 2024-09-12T01:24:28.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:28.534+0000 7f1b73e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b7c102e00 0x7f1b7c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:24:28.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:28.534+0000 7f1b7b400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b7c1037c0 0x7f1b7c19f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:24:28.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:28.534+0000 7f1b73e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b7c102e00 0x7f1b7c1983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:60950/0 (socket says 172.21.15.1:60950) 2024-09-12T01:24:28.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:28.534+0000 7f1b73e00640 1 -- 172.21.15.1:0/619115022 learned_addr learned my addr 172.21.15.1:0/619115022 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:24:28.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:28.534+0000 7f1b7aa00640 1 --2- 172.21.15.1:0/619115022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b7c101c00 0x7f1b7c197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:24:28.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:28.534+0000 7f1b73e00640 1 -- 172.21.15.1:0/619115022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b7c101c00 msgr2=0x7f1b7c197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:24:28.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:28.534+0000 7f1b73e00640 1 --2- 172.21.15.1:0/619115022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b7c101c00 0x7f1b7c197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:24:28.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:28.534+0000 7f1b73e00640 1 -- 172.21.15.1:0/619115022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b7c1037c0 msgr2=0x7f1b7c19f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:24:28.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:28.534+0000 7f1b73e00640 1 --2- 172.21.15.1:0/619115022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b7c1037c0 0x7f1b7c19f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:24:28.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:28.534+0000 7f1b73e00640 1 -- 172.21.15.1:0/619115022 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b7c19fad0 con 0x7f1b7c102e00 2024-09-12T01:24:28.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:28.534+0000 7f1b7aa00640 1 --2- 172.21.15.1:0/619115022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b7c101c00 0x7f1b7c197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:24:28.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:28.535+0000 7f1b73e00640 1 --2- 172.21.15.1:0/619115022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b7c102e00 0x7f1b7c1983e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1b6400ea60 tx=0x7f1b6400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:24:28.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:28.535+0000 7f1b80a00640 1 -- 172.21.15.1:0/619115022 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6400cde0 con 0x7f1b7c102e00 2024-09-12T01:24:28.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:28.535+0000 7f1b81e00640 1 -- 172.21.15.1:0/619115022 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b7c19fdc0 con 0x7f1b7c102e00 2024-09-12T01:24:28.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:28.535+0000 7f1b80a00640 1 -- 172.21.15.1:0/619115022 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b640035f0 con 0x7f1b7c102e00 2024-09-12T01:24:28.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:28.535+0000 7f1b80a00640 1 -- 172.21.15.1:0/619115022 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b64010670 con 0x7f1b7c102e00 2024-09-12T01:24:28.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:28.535+0000 7f1b81e00640 1 -- 172.21.15.1:0/619115022 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b7c1a0280 con 0x7f1b7c102e00 2024-09-12T01:24:28.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:28.536+0000 7f1b81e00640 1 -- 172.21.15.1:0/619115022 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b7c108fd0 con 0x7f1b7c102e00 2024-09-12T01:24:28.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:28.538+0000 7f1b80a00640 1 -- 172.21.15.1:0/619115022 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f1b640037a0 con 0x7f1b7c102e00 2024-09-12T01:24:28.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:28.539+0000 7f1b80a00640 1 --2- 172.21.15.1:0/619115022 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1b600a2c30 0x7f1b600a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:24:28.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:28.539+0000 7f1b80a00640 1 -- 172.21.15.1:0/619115022 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5858+0+0 (secure 0 0 0) 0x7f1b640cea50 con 0x7f1b7c102e00 2024-09-12T01:24:28.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:28.539+0000 7f1b7aa00640 1 --2- 172.21.15.1:0/619115022 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1b600a2c30 0x7f1b600a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:24:28.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:28.540+0000 7f1b7aa00640 1 --2- 172.21.15.1:0/619115022 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1b600a2c30 0x7f1b600a50f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c102c60 tx=0x7f1b6c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:24:28.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:28.544+0000 7f1b80a00640 1 -- 172.21.15.1:0/619115022 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1b6409c2a0 con 0x7f1b7c102e00 2024-09-12T01:24:28.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:28 smithi104 ceph-mon[35219]: purged_snaps scrub starts 2024-09-12T01:24:28.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:28 smithi104 ceph-mon[35219]: purged_snaps scrub ok 2024-09-12T01:24:28.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:28 smithi104 ceph-mon[35219]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:24:28.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:28 smithi104 ceph-mon[35219]: osd.8 [v2:172.21.15.104:6816/941453547,v1:172.21.15.104:6817/941453547] boot 2024-09-12T01:24:28.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:28 smithi104 ceph-mon[35219]: osdmap e59: 9 total, 9 up, 9 in 2024-09-12T01:24:28.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:28 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:24:28.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:28 smithi053 ceph-mon[35158]: purged_snaps scrub starts 2024-09-12T01:24:28.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:28 smithi053 ceph-mon[35158]: purged_snaps scrub ok 2024-09-12T01:24:28.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:28 smithi053 ceph-mon[35158]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:24:28.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:28 smithi053 ceph-mon[35158]: osd.8 [v2:172.21.15.104:6816/941453547,v1:172.21.15.104:6817/941453547] boot 2024-09-12T01:24:28.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:28 smithi053 ceph-mon[35158]: osdmap e59: 9 total, 9 up, 9 in 2024-09-12T01:24:28.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:28 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:24:28.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:28.743+0000 7f1b81e00640 1 -- 172.21.15.1:0/619115022 --> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f1b7c062630 con 0x7f1b600a2c30 2024-09-12T01:24:29.533 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:29 smithi001 ceph-mon[30387]: osdmap e60: 9 total, 9 up, 9 in 2024-09-12T01:24:29.533 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:29 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:24:29.533 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:29 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:24:29.533 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:29 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:24:29.576 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:29 smithi104 ceph-mon[35219]: osdmap e60: 9 total, 9 up, 9 in 2024-09-12T01:24:29.577 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:29 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:24:29.577 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:29 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:24:29.577 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:29 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:24:29.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:29 smithi053 ceph-mon[35158]: osdmap e60: 9 total, 9 up, 9 in 2024-09-12T01:24:29.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:29 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:24:29.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:29 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:24:29.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:29 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:24:30.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:30 smithi001 ceph-mon[30387]: pgmap v207: 1 pgs: 1 remapped+peering; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:24:30.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:30 smithi001 ceph-mon[30387]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:24:30.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:30 smithi001 ceph-mon[30387]: osdmap e61: 9 total, 9 up, 9 in 2024-09-12T01:24:30.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:30 smithi104 ceph-mon[35219]: pgmap v207: 1 pgs: 1 remapped+peering; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:24:30.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:30 smithi104 ceph-mon[35219]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:24:30.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:30 smithi104 ceph-mon[35219]: osdmap e61: 9 total, 9 up, 9 in 2024-09-12T01:24:30.641 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:30 smithi053 ceph-mon[35158]: pgmap v207: 1 pgs: 1 remapped+peering; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:24:30.642 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:30 smithi053 ceph-mon[35158]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:24:30.642 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:30 smithi053 ceph-mon[35158]: osdmap e61: 9 total, 9 up, 9 in 2024-09-12T01:24:31.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:31 smithi104 ceph-mon[35219]: osdmap e62: 9 total, 9 up, 9 in 2024-09-12T01:24:31.641 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:31 smithi001 ceph-mon[30387]: osdmap e62: 9 total, 9 up, 9 in 2024-09-12T01:24:31.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:31 smithi053 ceph-mon[35158]: osdmap e62: 9 total, 9 up, 9 in 2024-09-12T01:24:32.561 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:32 smithi001 ceph-mon[30387]: pgmap v210: 1 pgs: 1 remapped+peering; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:24:32.562 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:32 smithi001 ceph-mon[30387]: osdmap e63: 9 total, 9 up, 9 in 2024-09-12T01:24:32.562 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:32 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/3950435162' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f81537a-cae5-4bb8-aec2-4eb07deb7958"}]: dispatch 2024-09-12T01:24:32.562 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:32 smithi001 ceph-mon[30387]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f81537a-cae5-4bb8-aec2-4eb07deb7958"}]: dispatch 2024-09-12T01:24:32.562 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:32 smithi001 ceph-mon[30387]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7f81537a-cae5-4bb8-aec2-4eb07deb7958"}]': finished 2024-09-12T01:24:32.562 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:32 smithi001 ceph-mon[30387]: osdmap e64: 10 total, 9 up, 10 in 2024-09-12T01:24:32.562 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:32 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:24:32.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:32 smithi104 ceph-mon[35219]: pgmap v210: 1 pgs: 1 remapped+peering; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:24:32.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:32 smithi104 ceph-mon[35219]: osdmap e63: 9 total, 9 up, 9 in 2024-09-12T01:24:32.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:32 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/3950435162' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f81537a-cae5-4bb8-aec2-4eb07deb7958"}]: dispatch 2024-09-12T01:24:32.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:32 smithi104 ceph-mon[35219]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f81537a-cae5-4bb8-aec2-4eb07deb7958"}]: dispatch 2024-09-12T01:24:32.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:32 smithi104 ceph-mon[35219]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7f81537a-cae5-4bb8-aec2-4eb07deb7958"}]': finished 2024-09-12T01:24:32.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:32 smithi104 ceph-mon[35219]: osdmap e64: 10 total, 9 up, 10 in 2024-09-12T01:24:32.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:32 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:24:32.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:32 smithi053 ceph-mon[35158]: pgmap v210: 1 pgs: 1 remapped+peering; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:24:32.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:32 smithi053 ceph-mon[35158]: osdmap e63: 9 total, 9 up, 9 in 2024-09-12T01:24:32.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:32 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/3950435162' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f81537a-cae5-4bb8-aec2-4eb07deb7958"}]: dispatch 2024-09-12T01:24:32.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:32 smithi053 ceph-mon[35158]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f81537a-cae5-4bb8-aec2-4eb07deb7958"}]: dispatch 2024-09-12T01:24:32.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:32 smithi053 ceph-mon[35158]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7f81537a-cae5-4bb8-aec2-4eb07deb7958"}]': finished 2024-09-12T01:24:32.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:32 smithi053 ceph-mon[35158]: osdmap e64: 10 total, 9 up, 10 in 2024-09-12T01:24:32.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:32 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:24:33.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:33 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/3089006631' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:24:33.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:33 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/3089006631' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:24:33.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:33 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/3089006631' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:24:34.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:34 smithi104 ceph-mon[35219]: pgmap v213: 1 pgs: 1 remapped+peering; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:24:34.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:34 smithi001 ceph-mon[30387]: pgmap v213: 1 pgs: 1 remapped+peering; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:24:34.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:34 smithi053 ceph-mon[35158]: pgmap v213: 1 pgs: 1 remapped+peering; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:24:36.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:36 smithi104 ceph-mon[35219]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-09-12T01:24:36.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:36 smithi001 ceph-mon[30387]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-09-12T01:24:36.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:36 smithi053 ceph-mon[35158]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-09-12T01:24:38.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:38 smithi001 ceph-mon[30387]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-12T01:24:38.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:38 smithi104 ceph-mon[35219]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-12T01:24:38.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:38 smithi053 ceph-mon[35158]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-12T01:24:40.406 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:40 smithi001 ceph-mon[30387]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T01:24:40.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:40 smithi104 ceph-mon[35219]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T01:24:40.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:40 smithi053 ceph-mon[35158]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T01:24:42.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:42 smithi001 ceph-mon[30387]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-12T01:24:42.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:42 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-12T01:24:42.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:42 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:24:42.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:42 smithi104 ceph-mon[35219]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-12T01:24:42.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:42 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-12T01:24:42.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:42 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:24:42.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:42 smithi053 ceph-mon[35158]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-12T01:24:42.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:42 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-12T01:24:42.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:42 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:24:43.547 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:43 smithi001 ceph-mon[30387]: Deploying daemon osd.9 on smithi001 2024-09-12T01:24:43.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:43 smithi104 ceph-mon[35219]: Deploying daemon osd.9 on smithi001 2024-09-12T01:24:43.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:43 smithi053 ceph-mon[35158]: Deploying daemon osd.9 on smithi001 2024-09-12T01:24:44.392 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:44 smithi001 ceph-mon[30387]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-12T01:24:44.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:44 smithi104 ceph-mon[35219]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-12T01:24:44.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:44 smithi053 ceph-mon[35158]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-12T01:24:46.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:46 smithi104 ceph-mon[35219]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T01:24:46.608 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:46 smithi001 ceph-mon[30387]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T01:24:46.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:46 smithi053 ceph-mon[35158]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T01:24:48.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:48 smithi104 ceph-mon[35219]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:24:48.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:48 smithi001 ceph-mon[30387]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:24:48.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:48 smithi053 ceph-mon[35158]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:24:50.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:50 smithi104 ceph-mon[35219]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:24:50.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:50 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:24:50.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:50 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:50.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:50 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:50.673 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:50 smithi001 ceph-mon[30387]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:24:50.674 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:50 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:24:50.674 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:50 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:50.674 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:50 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:50.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:50 smithi053 ceph-mon[35158]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:24:50.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:50 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:24:50.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:50 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:50.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:50 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:52.491 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:52 smithi001 ceph-mon[30387]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:24:52.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:52 smithi104 ceph-mon[35219]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:24:52.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:52 smithi053 ceph-mon[35158]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:24:54.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:54.422+0000 7f1b80a00640 1 -- 172.21.15.1:0/619115022 <== mgr.14150 v2:172.21.15.1:6800/1928662456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1b7c062630 con 0x7f1b600a2c30 2024-09-12T01:24:54.421 INFO:teuthology.orchestra.run.smithi001.stdout:Created osd(s) 9 on host 'smithi001' 2024-09-12T01:24:54.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:54.424+0000 7f1b81e00640 1 -- 172.21.15.1:0/619115022 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1b600a2c30 msgr2=0x7f1b600a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:24:54.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:54.424+0000 7f1b81e00640 1 --2- 172.21.15.1:0/619115022 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1b600a2c30 0x7f1b600a50f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c102c60 tx=0x7f1b6c005f90 comp rx=0 tx=0).stop 2024-09-12T01:24:54.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:54.424+0000 7f1b81e00640 1 -- 172.21.15.1:0/619115022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b7c102e00 msgr2=0x7f1b7c1983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:24:54.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:54.424+0000 7f1b81e00640 1 --2- 172.21.15.1:0/619115022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b7c102e00 0x7f1b7c1983e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1b6400ea60 tx=0x7f1b6400ef20 comp rx=0 tx=0).stop 2024-09-12T01:24:54.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:54.424+0000 7f1b81e00640 1 -- 172.21.15.1:0/619115022 shutdown_connections 2024-09-12T01:24:54.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:54.424+0000 7f1b81e00640 1 --2- 172.21.15.1:0/619115022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b7c1037c0 0x7f1b7c19f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:24:54.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:54.424+0000 7f1b81e00640 1 --2- 172.21.15.1:0/619115022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b7c102e00 0x7f1b7c1983e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:24:54.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:54.424+0000 7f1b81e00640 1 --2- 172.21.15.1:0/619115022 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1b600a2c30 0x7f1b600a50f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:24:54.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:54.424+0000 7f1b81e00640 1 --2- 172.21.15.1:0/619115022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b7c101c00 0x7f1b7c197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:24:54.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:54.424+0000 7f1b81e00640 1 -- 172.21.15.1:0/619115022 >> 172.21.15.1:0/619115022 conn(0x7f1b7c0fd390 msgr2=0x7f1b7c1061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:24:54.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:54.425+0000 7f1b81e00640 1 -- 172.21.15.1:0/619115022 shutdown_connections 2024-09-12T01:24:54.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:24:54.425+0000 7f1b81e00640 1 -- 172.21.15.1:0/619115022 wait complete. 2024-09-12T01:24:54.667 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:54 smithi001 ceph-mon[30387]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:24:54.667 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:54 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:54.667 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:54 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:54.667 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:54 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:24:54.667 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:54 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:24:54.667 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:54 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:54.668 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:54 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:24:54.668 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:54 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:54.668 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:54 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:54.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:54 smithi104 ceph-mon[35219]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:24:54.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:54 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:54.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:54 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:54.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:54 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:24:54.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:54 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:24:54.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:54 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:54.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:54 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:24:54.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:54 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:54.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:54 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:54.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:54 smithi053 ceph-mon[35158]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:24:54.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:54 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:54.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:54 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:54.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:54 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:24:54.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:54 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:24:54.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:54 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:54.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:54 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:24:54.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:54 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:54.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:54 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:55.223 DEBUG:teuthology.orchestra.run.smithi001:osd.9> sudo journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@osd.9.service 2024-09-12T01:24:55.228 INFO:tasks.cephadm:Deploying osd.10 on smithi053 with /dev/vg_nvme/lv_1... 2024-09-12T01:24:55.228 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-12T01:24:55.489 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T01:24:56.178 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:24:55 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:24:55.714+0000 7f7de5e30640 -1 osd.9 0 log_to_monitors true 2024-09-12T01:24:56.653 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:56 smithi001 ceph-mon[30387]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:24:56.653 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:56 smithi001 ceph-mon[30387]: from='osd.9 [v2:172.21.15.1:6826/958287586,v1:172.21.15.1:6827/958287586]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T01:24:56.653 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:56 smithi001 ceph-mon[30387]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T01:24:56.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:56 smithi104 ceph-mon[35219]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:24:56.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:56 smithi104 ceph-mon[35219]: from='osd.9 [v2:172.21.15.1:6826/958287586,v1:172.21.15.1:6827/958287586]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T01:24:56.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:56 smithi104 ceph-mon[35219]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T01:24:56.939 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:56 smithi053 ceph-mon[35158]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:24:56.939 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:56 smithi053 ceph-mon[35158]: from='osd.9 [v2:172.21.15.1:6826/958287586,v1:172.21.15.1:6827/958287586]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T01:24:56.939 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:56 smithi053 ceph-mon[35158]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T01:24:57.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:57 smithi104 ceph-mon[35219]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-12T01:24:57.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:57 smithi104 ceph-mon[35219]: osdmap e65: 10 total, 9 up, 10 in 2024-09-12T01:24:57.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:57 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:24:57.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:57 smithi104 ceph-mon[35219]: from='osd.9 [v2:172.21.15.1:6826/958287586,v1:172.21.15.1:6827/958287586]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-12T01:24:57.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:57 smithi104 ceph-mon[35219]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-12T01:24:57.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:57 smithi001 ceph-mon[30387]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-12T01:24:57.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:57 smithi001 ceph-mon[30387]: osdmap e65: 10 total, 9 up, 10 in 2024-09-12T01:24:57.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:57 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:24:57.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:57 smithi001 ceph-mon[30387]: from='osd.9 [v2:172.21.15.1:6826/958287586,v1:172.21.15.1:6827/958287586]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-12T01:24:57.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:57 smithi001 ceph-mon[30387]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-12T01:24:57.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:24:57 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:24:57.571+0000 7f7de0600640 -1 osd.9 0 waiting for initial osdmap 2024-09-12T01:24:57.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:24:57 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:24:57.586+0000 7f7ddb600640 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T01:24:57.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:57 smithi053 ceph-mon[35158]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-12T01:24:57.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:57 smithi053 ceph-mon[35158]: osdmap e65: 10 total, 9 up, 10 in 2024-09-12T01:24:57.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:57 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:24:57.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:57 smithi053 ceph-mon[35158]: from='osd.9 [v2:172.21.15.1:6826/958287586,v1:172.21.15.1:6827/958287586]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-12T01:24:57.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:57 smithi053 ceph-mon[35158]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-12T01:24:58.442 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-12T01:24:58.469 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi053:vg_nvme/lv_1 2024-09-12T01:24:58.721 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T01:24:58.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:58 smithi053 ceph-mon[35158]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:24:58.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:58 smithi053 ceph-mon[35158]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-09-12T01:24:58.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:58 smithi053 ceph-mon[35158]: osdmap e66: 10 total, 9 up, 10 in 2024-09-12T01:24:58.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:58 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:24:58.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:58 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:24:58.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:58 smithi104 ceph-mon[35219]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:24:58.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:58 smithi104 ceph-mon[35219]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-09-12T01:24:58.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:58 smithi104 ceph-mon[35219]: osdmap e66: 10 total, 9 up, 10 in 2024-09-12T01:24:58.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:58 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:24:58.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:58 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:24:58.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:58 smithi001 ceph-mon[30387]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:24:58.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:58 smithi001 ceph-mon[30387]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-09-12T01:24:58.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:58 smithi001 ceph-mon[30387]: osdmap e66: 10 total, 9 up, 10 in 2024-09-12T01:24:58.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:58 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:24:58.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:58 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:24:59.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:24:59.229+0000 7f2d842d3640 1 -- 172.21.15.53:0/1431080807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d7c1078c0 msgr2=0x7f2d7c109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:24:59.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:24:59.229+0000 7f2d842d3640 1 --2- 172.21.15.53:0/1431080807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d7c1078c0 0x7f2d7c109d10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2d74009ca0 tx=0x7f2d7402a8c0 comp rx=0 tx=0).stop 2024-09-12T01:24:59.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:24:59.229+0000 7f2d842d3640 1 -- 172.21.15.53:0/1431080807 shutdown_connections 2024-09-12T01:24:59.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:24:59.229+0000 7f2d842d3640 1 --2- 172.21.15.53:0/1431080807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d7c1078c0 0x7f2d7c109d10 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:24:59.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:24:59.229+0000 7f2d842d3640 1 --2- 172.21.15.53:0/1431080807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d7c104f90 0x7f2d7c107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:24:59.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:24:59.229+0000 7f2d842d3640 1 --2- 172.21.15.53:0/1431080807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d7c102660 0x7f2d7c104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:24:59.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:24:59.229+0000 7f2d842d3640 1 -- 172.21.15.53:0/1431080807 >> 172.21.15.53:0/1431080807 conn(0x7f2d7c0fc280 msgr2=0x7f2d7c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:24:59.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:24:59.229+0000 7f2d842d3640 1 -- 172.21.15.53:0/1431080807 shutdown_connections 2024-09-12T01:24:59.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:24:59.229+0000 7f2d842d3640 1 -- 172.21.15.53:0/1431080807 wait complete. 2024-09-12T01:24:59.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:24:59.229+0000 7f2d842d3640 1 Processor -- start 2024-09-12T01:24:59.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:24:59.229+0000 7f2d842d3640 1 -- start start 2024-09-12T01:24:59.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:24:59.229+0000 7f2d842d3640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d7c102660 0x7f2d7c197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:24:59.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:24:59.230+0000 7f2d842d3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d7c104f90 0x7f2d7c1982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:24:59.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:24:59.230+0000 7f2d842d3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d7c1078c0 0x7f2d7c19f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:24:59.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:24:59.230+0000 7f2d842d3640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7c10d860 con 0x7f2d7c104f90 2024-09-12T01:24:59.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:24:59.230+0000 7f2d842d3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7c10d6e0 con 0x7f2d7c1078c0 2024-09-12T01:24:59.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:24:59.230+0000 7f2d842d3640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7c10d9e0 con 0x7f2d7c102660 2024-09-12T01:24:59.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:24:59.230+0000 7f2d82870640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d7c1078c0 0x7f2d7c19f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:24:59.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:24:59.230+0000 7f2d82870640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d7c1078c0 0x7f2d7c19f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45878/0 (socket says 172.21.15.53:45878) 2024-09-12T01:24:59.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:24:59.230+0000 7f2d82870640 1 -- 172.21.15.53:0/3659977738 learned_addr learned my addr 172.21.15.53:0/3659977738 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-12T01:24:59.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:24:59.230+0000 7f2d8206f640 1 --2- 172.21.15.53:0/3659977738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d7c102660 0x7f2d7c197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:24:59.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:24:59.230+0000 7f2d8186e640 1 --2- 172.21.15.53:0/3659977738 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d7c104f90 0x7f2d7c1982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:24:59.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:24:59.230+0000 7f2d82870640 1 -- 172.21.15.53:0/3659977738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d7c102660 msgr2=0x7f2d7c197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:24:59.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:24:59.230+0000 7f2d82870640 1 --2- 172.21.15.53:0/3659977738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d7c102660 0x7f2d7c197da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:24:59.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:24:59.230+0000 7f2d82870640 1 -- 172.21.15.53:0/3659977738 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d7c104f90 msgr2=0x7f2d7c1982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:24:59.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:24:59.230+0000 7f2d82870640 1 --2- 172.21.15.53:0/3659977738 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d7c104f90 0x7f2d7c1982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:24:59.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:24:59.230+0000 7f2d82870640 1 -- 172.21.15.53:0/3659977738 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d7c19f9d0 con 0x7f2d7c1078c0 2024-09-12T01:24:59.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:24:59.230+0000 7f2d8186e640 1 --2- 172.21.15.53:0/3659977738 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d7c104f90 0x7f2d7c1982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:24:59.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:24:59.230+0000 7f2d82870640 1 --2- 172.21.15.53:0/3659977738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d7c1078c0 0x7f2d7c19f360 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2d740098d0 tx=0x7f2d7402f840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:24:59.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:24:59.230+0000 7f2d737fe640 1 -- 172.21.15.53:0/3659977738 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d7402fc30 con 0x7f2d7c1078c0 2024-09-12T01:24:59.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:24:59.230+0000 7f2d737fe640 1 -- 172.21.15.53:0/3659977738 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d74003e50 con 0x7f2d7c1078c0 2024-09-12T01:24:59.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:24:59.230+0000 7f2d737fe640 1 -- 172.21.15.53:0/3659977738 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d740305c0 con 0x7f2d7c1078c0 2024-09-12T01:24:59.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:24:59.230+0000 7f2d842d3640 1 -- 172.21.15.53:0/3659977738 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d7c19fc60 con 0x7f2d7c1078c0 2024-09-12T01:24:59.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:24:59.230+0000 7f2d842d3640 1 -- 172.21.15.53:0/3659977738 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d7c1a0150 con 0x7f2d7c1078c0 2024-09-12T01:24:59.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:24:59.231+0000 7f2d842d3640 1 -- 172.21.15.53:0/3659977738 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d7c0fdd70 con 0x7f2d7c1078c0 2024-09-12T01:24:59.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:24:59.232+0000 7f2d737fe640 1 -- 172.21.15.53:0/3659977738 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f2d74003970 con 0x7f2d7c1078c0 2024-09-12T01:24:59.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:24:59.233+0000 7f2d737fe640 1 --2- 172.21.15.53:0/3659977738 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f2d680a2bb0 0x7f2d680a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:24:59.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:24:59.233+0000 7f2d737fe640 1 -- 172.21.15.53:0/3659977738 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6261+0+0 (secure 0 0 0) 0x7f2d740efc50 con 0x7f2d7c1078c0 2024-09-12T01:24:59.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:24:59.233+0000 7f2d8206f640 1 --2- 172.21.15.53:0/3659977738 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f2d680a2bb0 0x7f2d680a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:24:59.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:24:59.234+0000 7f2d8206f640 1 --2- 172.21.15.53:0/3659977738 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f2d680a2bb0 0x7f2d680a5070 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2d78005e10 tx=0x7f2d78005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:24:59.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:24:59.237+0000 7f2d737fe640 1 -- 172.21.15.53:0/3659977738 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2d740f5020 con 0x7f2d7c1078c0 2024-09-12T01:24:59.420 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:24:59.420+0000 7f2d842d3640 1 -- 172.21.15.53:0/3659977738 --> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f2d7c062630 con 0x7f2d680a2bb0 2024-09-12T01:24:59.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:59 smithi053 ceph-mon[35158]: purged_snaps scrub starts 2024-09-12T01:24:59.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:59 smithi053 ceph-mon[35158]: purged_snaps scrub ok 2024-09-12T01:24:59.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:59 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:24:59.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:59 smithi053 ceph-mon[35158]: osd.9 [v2:172.21.15.1:6826/958287586,v1:172.21.15.1:6827/958287586] boot 2024-09-12T01:24:59.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:59 smithi053 ceph-mon[35158]: osdmap e67: 10 total, 10 up, 10 in 2024-09-12T01:24:59.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:59 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:24:59.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:59 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:59.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:59 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:59.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:59 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:24:59.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:59 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:24:59.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:59 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:24:59.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:59 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:24:59.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:59 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:59.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:59 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:24:59.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:59 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:24:59.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:59 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:59.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:59 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:24:59.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:59 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:24:59.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:24:59 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:24:59.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:59 smithi104 ceph-mon[35219]: purged_snaps scrub starts 2024-09-12T01:24:59.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:59 smithi104 ceph-mon[35219]: purged_snaps scrub ok 2024-09-12T01:24:59.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:59 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:24:59.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:59 smithi104 ceph-mon[35219]: osd.9 [v2:172.21.15.1:6826/958287586,v1:172.21.15.1:6827/958287586] boot 2024-09-12T01:24:59.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:59 smithi104 ceph-mon[35219]: osdmap e67: 10 total, 10 up, 10 in 2024-09-12T01:24:59.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:59 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:24:59.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:59 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:59.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:59 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:59.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:59 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:24:59.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:59 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:24:59.826 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:59 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:24:59.826 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:59 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:24:59.826 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:59 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:59.826 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:59 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:24:59.826 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:59 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:24:59.826 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:59 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:59.826 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:59 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:24:59.826 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:59 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:24:59.826 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:24:59 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:24:59.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:59 smithi001 ceph-mon[30387]: purged_snaps scrub starts 2024-09-12T01:24:59.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:59 smithi001 ceph-mon[30387]: purged_snaps scrub ok 2024-09-12T01:24:59.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:59 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:24:59.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:59 smithi001 ceph-mon[30387]: osd.9 [v2:172.21.15.1:6826/958287586,v1:172.21.15.1:6827/958287586] boot 2024-09-12T01:24:59.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:59 smithi001 ceph-mon[30387]: osdmap e67: 10 total, 10 up, 10 in 2024-09-12T01:24:59.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:59 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:24:59.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:59 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:59.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:59 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:59.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:59 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:24:59.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:59 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:24:59.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:59 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:24:59.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:59 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:24:59.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:59 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:59.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:59 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:24:59.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:59 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:24:59.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:59 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:24:59.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:59 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:24:59.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:59 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:24:59.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:24:59 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:25:00.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:00 smithi104 ceph-mon[35219]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:25:00.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:00 smithi104 ceph-mon[35219]: Detected new or changed devices on smithi001 2024-09-12T01:25:00.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:00 smithi104 ceph-mon[35219]: Adjusting osd_memory_target on smithi001 to 4273M 2024-09-12T01:25:00.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:00 smithi104 ceph-mon[35219]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:25:00.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:00 smithi104 ceph-mon[35219]: osdmap e68: 10 total, 10 up, 10 in 2024-09-12T01:25:00.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:00 smithi001 ceph-mon[30387]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:25:00.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:00 smithi001 ceph-mon[30387]: Detected new or changed devices on smithi001 2024-09-12T01:25:00.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:00 smithi001 ceph-mon[30387]: Adjusting osd_memory_target on smithi001 to 4273M 2024-09-12T01:25:00.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:00 smithi001 ceph-mon[30387]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:25:00.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:00 smithi001 ceph-mon[30387]: osdmap e68: 10 total, 10 up, 10 in 2024-09-12T01:25:00.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:00 smithi053 ceph-mon[35158]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:25:00.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:00 smithi053 ceph-mon[35158]: Detected new or changed devices on smithi001 2024-09-12T01:25:00.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:00 smithi053 ceph-mon[35158]: Adjusting osd_memory_target on smithi001 to 4273M 2024-09-12T01:25:00.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:00 smithi053 ceph-mon[35158]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:25:00.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:00 smithi053 ceph-mon[35158]: osdmap e68: 10 total, 10 up, 10 in 2024-09-12T01:25:02.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:02 smithi053 ceph-mon[35158]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:25:02.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:02 smithi053 ceph-mon[35158]: from='client.? 172.21.15.53:0/1533726460' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e3c9c37d-46b9-4585-990d-69c447453bd8"}]: dispatch 2024-09-12T01:25:02.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:02 smithi053 ceph-mon[35158]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e3c9c37d-46b9-4585-990d-69c447453bd8"}]: dispatch 2024-09-12T01:25:02.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:02 smithi053 ceph-mon[35158]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e3c9c37d-46b9-4585-990d-69c447453bd8"}]': finished 2024-09-12T01:25:02.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:02 smithi053 ceph-mon[35158]: osdmap e69: 11 total, 10 up, 11 in 2024-09-12T01:25:02.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:25:02.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:02 smithi104 ceph-mon[35219]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:25:02.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:02 smithi104 ceph-mon[35219]: from='client.? 172.21.15.53:0/1533726460' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e3c9c37d-46b9-4585-990d-69c447453bd8"}]: dispatch 2024-09-12T01:25:02.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:02 smithi104 ceph-mon[35219]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e3c9c37d-46b9-4585-990d-69c447453bd8"}]: dispatch 2024-09-12T01:25:02.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:02 smithi104 ceph-mon[35219]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e3c9c37d-46b9-4585-990d-69c447453bd8"}]': finished 2024-09-12T01:25:02.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:02 smithi104 ceph-mon[35219]: osdmap e69: 11 total, 10 up, 11 in 2024-09-12T01:25:02.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:02 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:25:02.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:02 smithi001 ceph-mon[30387]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:25:02.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:02 smithi001 ceph-mon[30387]: from='client.? 172.21.15.53:0/1533726460' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e3c9c37d-46b9-4585-990d-69c447453bd8"}]: dispatch 2024-09-12T01:25:02.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:02 smithi001 ceph-mon[30387]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e3c9c37d-46b9-4585-990d-69c447453bd8"}]: dispatch 2024-09-12T01:25:02.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:02 smithi001 ceph-mon[30387]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e3c9c37d-46b9-4585-990d-69c447453bd8"}]': finished 2024-09-12T01:25:02.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:02 smithi001 ceph-mon[30387]: osdmap e69: 11 total, 10 up, 11 in 2024-09-12T01:25:02.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:02 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:25:03.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:03 smithi104 ceph-mon[35219]: from='client.? 172.21.15.53:0/1713204067' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:25:03.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:03 smithi001 ceph-mon[30387]: from='client.? 172.21.15.53:0/1713204067' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:25:03.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:03 smithi053 ceph-mon[35158]: from='client.? 172.21.15.53:0/1713204067' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:25:04.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:04 smithi104 ceph-mon[35219]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:25:04.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:04 smithi001 ceph-mon[30387]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:25:04.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:04 smithi053 ceph-mon[35158]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:25:06.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:06 smithi104 ceph-mon[35219]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:25:06.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:06 smithi001 ceph-mon[30387]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:25:06.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:06 smithi053 ceph-mon[35158]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:25:08.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:08 smithi001 ceph-mon[30387]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:25:08.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:08 smithi053 ceph-mon[35158]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:25:09.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:08 smithi104 ceph-mon[35219]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:25:10.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:10 smithi001 ceph-mon[30387]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:25:10.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:10 smithi053 ceph-mon[35158]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:25:11.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:10 smithi104 ceph-mon[35219]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:25:11.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:11 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-12T01:25:11.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:11 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:25:11.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:11 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-12T01:25:11.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:11 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:25:12.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:11 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-12T01:25:12.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:11 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:25:12.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:12 smithi053 ceph-mon[35158]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:25:12.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:12 smithi053 ceph-mon[35158]: Deploying daemon osd.10 on smithi053 2024-09-12T01:25:12.846 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:12 smithi001 ceph-mon[30387]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:25:12.846 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:12 smithi001 ceph-mon[30387]: Deploying daemon osd.10 on smithi053 2024-09-12T01:25:13.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:12 smithi104 ceph-mon[35219]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:25:13.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:12 smithi104 ceph-mon[35219]: Deploying daemon osd.10 on smithi053 2024-09-12T01:25:13.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:13 smithi053 ceph-mon[35158]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:25:13.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:13 smithi001 ceph-mon[30387]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:25:14.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:13 smithi104 ceph-mon[35219]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:25:15.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:15 smithi053 ceph-mon[35158]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:25:16.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:15 smithi104 ceph-mon[35219]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:25:16.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:15 smithi001 ceph-mon[30387]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:25:17.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:17 smithi053 ceph-mon[35158]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:25:17.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:17 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:25:17.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:17 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:17.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:17 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:18.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:17 smithi104 ceph-mon[35219]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:25:18.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:17 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:25:18.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:17 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:18.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:17 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:18.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:17 smithi001 ceph-mon[30387]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:25:18.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:17 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:25:18.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:17 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:18.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:17 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:19.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:19 smithi053 ceph-mon[35158]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:25:19.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:19 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:19.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:19 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:19.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:19 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:25:19.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:19 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:25:20.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:19 smithi104 ceph-mon[35219]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:25:20.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:19 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:20.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:19 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:20.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:19 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:25:20.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:19 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:25:20.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:19 smithi001 ceph-mon[30387]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:25:20.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:19 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:20.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:19 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:20.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:19 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:25:20.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:19 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:25:20.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:25:20.541+0000 7f2d737fe640 1 -- 172.21.15.53:0/3659977738 <== mgr.14150 v2:172.21.15.1:6800/1928662456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f2d7c062630 con 0x7f2d680a2bb0 2024-09-12T01:25:20.541 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 10 on host 'smithi053' 2024-09-12T01:25:20.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:25:20.543+0000 7f2d842d3640 1 -- 172.21.15.53:0/3659977738 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f2d680a2bb0 msgr2=0x7f2d680a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:25:20.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:25:20.543+0000 7f2d842d3640 1 --2- 172.21.15.53:0/3659977738 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f2d680a2bb0 0x7f2d680a5070 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2d78005e10 tx=0x7f2d78005da0 comp rx=0 tx=0).stop 2024-09-12T01:25:20.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:25:20.543+0000 7f2d842d3640 1 -- 172.21.15.53:0/3659977738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d7c1078c0 msgr2=0x7f2d7c19f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:25:20.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:25:20.543+0000 7f2d842d3640 1 --2- 172.21.15.53:0/3659977738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d7c1078c0 0x7f2d7c19f360 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2d740098d0 tx=0x7f2d7402f840 comp rx=0 tx=0).stop 2024-09-12T01:25:20.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:25:20.543+0000 7f2d842d3640 1 -- 172.21.15.53:0/3659977738 shutdown_connections 2024-09-12T01:25:20.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:25:20.544+0000 7f2d842d3640 1 --2- 172.21.15.53:0/3659977738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d7c1078c0 0x7f2d7c19f360 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:20.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:25:20.544+0000 7f2d842d3640 1 --2- 172.21.15.53:0/3659977738 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d7c104f90 0x7f2d7c1982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:20.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:25:20.544+0000 7f2d842d3640 1 --2- 172.21.15.53:0/3659977738 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f2d680a2bb0 0x7f2d680a5070 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:20.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:25:20.544+0000 7f2d842d3640 1 --2- 172.21.15.53:0/3659977738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d7c102660 0x7f2d7c197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:20.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:25:20.544+0000 7f2d842d3640 1 -- 172.21.15.53:0/3659977738 >> 172.21.15.53:0/3659977738 conn(0x7f2d7c0fc280 msgr2=0x7f2d7c102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:25:20.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:25:20.544+0000 7f2d842d3640 1 -- 172.21.15.53:0/3659977738 shutdown_connections 2024-09-12T01:25:20.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-12T01:25:20.544+0000 7f2d842d3640 1 -- 172.21.15.53:0/3659977738 wait complete. 2024-09-12T01:25:20.914 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:20 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:20.915 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:20 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:25:20.915 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:20 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:20.915 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:20 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:21.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:20 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:21.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:20 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:25:21.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:20 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:21.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:20 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:21.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:20 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:21.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:20 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:25:21.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:20 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:21.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:20 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:21.284 DEBUG:teuthology.orchestra.run.smithi053:osd.10> sudo journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@osd.10.service 2024-09-12T01:25:21.286 INFO:tasks.cephadm:Deploying osd.11 on smithi104 with /dev/vg_nvme/lv_1... 2024-09-12T01:25:21.286 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-12T01:25:21.542 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:25:22.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:22 smithi053 ceph-mon[35158]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:25:22.791 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:22 smithi104 ceph-mon[35219]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:25:22.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:22 smithi001 ceph-mon[30387]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:25:23.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:25:22 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:25:22.874+0000 7f2a22c8f640 -1 osd.10 0 log_to_monitors true 2024-09-12T01:25:23.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:23 smithi104 ceph-mon[35219]: from='osd.10 [v2:172.21.15.53:6824/3357909498,v1:172.21.15.53:6825/3357909498]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T01:25:23.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:23 smithi104 ceph-mon[35219]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T01:25:23.826 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:23 smithi001 ceph-mon[30387]: from='osd.10 [v2:172.21.15.53:6824/3357909498,v1:172.21.15.53:6825/3357909498]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T01:25:23.826 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:23 smithi001 ceph-mon[30387]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T01:25:23.888 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:23 smithi053 ceph-mon[35158]: from='osd.10 [v2:172.21.15.53:6824/3357909498,v1:172.21.15.53:6825/3357909498]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T01:25:23.889 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:23 smithi053 ceph-mon[35158]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T01:25:24.219 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-12T01:25:24.245 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi104:vg_nvme/lv_1 2024-09-12T01:25:24.499 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:25:24.804 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:24 smithi104 ceph-mon[35219]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:25:24.804 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:24 smithi104 ceph-mon[35219]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-12T01:25:24.804 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:24 smithi104 ceph-mon[35219]: osdmap e70: 11 total, 10 up, 11 in 2024-09-12T01:25:24.804 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:24 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:25:24.804 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:24 smithi104 ceph-mon[35219]: from='osd.10 [v2:172.21.15.53:6824/3357909498,v1:172.21.15.53:6825/3357909498]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-12T01:25:24.804 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:24 smithi104 ceph-mon[35219]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-12T01:25:24.804 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:24 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:24.805 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:24 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:24.805 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:24 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:25:24.805 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:24 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:25:24.805 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:24 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:25:24.805 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:24 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:25:24.805 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:24 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:24.805 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:24 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:25:24.805 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:24 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:25:24.805 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:24 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:24.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:24 smithi001 ceph-mon[30387]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:25:24.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:24 smithi001 ceph-mon[30387]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-12T01:25:24.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:24 smithi001 ceph-mon[30387]: osdmap e70: 11 total, 10 up, 11 in 2024-09-12T01:25:24.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:24 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:25:24.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:24 smithi001 ceph-mon[30387]: from='osd.10 [v2:172.21.15.53:6824/3357909498,v1:172.21.15.53:6825/3357909498]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-12T01:25:24.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:24 smithi001 ceph-mon[30387]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-12T01:25:24.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:24 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:24.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:24 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:24.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:24 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:25:24.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:24 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:25:24.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:24 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:25:24.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:24 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:25:24.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:24 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:24.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:24 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:25:24.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:24 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:25:24.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:24 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:24.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:24 smithi053 ceph-mon[35158]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:25:24.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:24 smithi053 ceph-mon[35158]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-12T01:25:24.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:24 smithi053 ceph-mon[35158]: osdmap e70: 11 total, 10 up, 11 in 2024-09-12T01:25:24.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:24 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:25:24.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:24 smithi053 ceph-mon[35158]: from='osd.10 [v2:172.21.15.53:6824/3357909498,v1:172.21.15.53:6825/3357909498]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-12T01:25:24.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:24 smithi053 ceph-mon[35158]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-12T01:25:24.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:24 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:24.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:24 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:24.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:24 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:25:24.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:24 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:25:24.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:24 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:25:24.991 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:24 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:25:24.991 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:24 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:24.991 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:24 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:25:24.991 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:24 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:25:24.991 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:24 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:25.020 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:25.003+0000 7fde49823640 1 -- 172.21.15.104:0/1764209557 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde44104fc0 msgr2=0x7fde441073b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:25:25.020 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:25.003+0000 7fde49823640 1 --2- 172.21.15.104:0/1764209557 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde44104fc0 0x7fde441073b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fde38009900 tx=0x7fde3802a410 comp rx=0 tx=0).stop 2024-09-12T01:25:25.021 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:25.003+0000 7fde49823640 1 -- 172.21.15.104:0/1764209557 shutdown_connections 2024-09-12T01:25:25.021 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:25.003+0000 7fde49823640 1 --2- 172.21.15.104:0/1764209557 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde441078f0 0x7fde44109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:25.021 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:25.003+0000 7fde49823640 1 --2- 172.21.15.104:0/1764209557 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde44104fc0 0x7fde441073b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:25.021 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:25.003+0000 7fde49823640 1 --2- 172.21.15.104:0/1764209557 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde44102690 0x7fde44104a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:25.021 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:25.003+0000 7fde49823640 1 -- 172.21.15.104:0/1764209557 >> 172.21.15.104:0/1764209557 conn(0x7fde440fc290 msgr2=0x7fde440fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:25:25.021 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:25.004+0000 7fde49823640 1 -- 172.21.15.104:0/1764209557 shutdown_connections 2024-09-12T01:25:25.021 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:25.004+0000 7fde49823640 1 -- 172.21.15.104:0/1764209557 wait complete. 2024-09-12T01:25:25.021 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:25.004+0000 7fde49823640 1 Processor -- start 2024-09-12T01:25:25.021 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:25.004+0000 7fde49823640 1 -- start start 2024-09-12T01:25:25.021 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:25.005+0000 7fde49823640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde44102690 0x7fde44197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:25:25.021 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:25.005+0000 7fde49823640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde44104fc0 0x7fde44198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:25:25.021 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:25.005+0000 7fde49823640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde441078f0 0x7fde4419f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:25:25.021 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:25.005+0000 7fde49823640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fde4410d910 con 0x7fde441078f0 2024-09-12T01:25:25.022 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:25.005+0000 7fde49823640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fde4410d790 con 0x7fde44104fc0 2024-09-12T01:25:25.022 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:25.005+0000 7fde49823640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fde4410da90 con 0x7fde44102690 2024-09-12T01:25:25.022 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:25.005+0000 7fde42ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde44102690 0x7fde44197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:25:25.022 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:25.005+0000 7fde42ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde44102690 0x7fde44197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:60594/0 (socket says 172.21.15.104:60594) 2024-09-12T01:25:25.022 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:25.005+0000 7fde42ffd640 1 -- 172.21.15.104:0/3108062937 learned_addr learned my addr 172.21.15.104:0/3108062937 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-09-12T01:25:25.022 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:25.005+0000 7fde427fc640 1 --2- 172.21.15.104:0/3108062937 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde44104fc0 0x7fde44198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:25:25.022 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:25.005+0000 7fde437fe640 1 --2- 172.21.15.104:0/3108062937 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde441078f0 0x7fde4419f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:25:25.022 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:25.005+0000 7fde42ffd640 1 -- 172.21.15.104:0/3108062937 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde44104fc0 msgr2=0x7fde44198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:25:25.022 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:25.005+0000 7fde42ffd640 1 --2- 172.21.15.104:0/3108062937 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde44104fc0 0x7fde44198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:25.022 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:25.005+0000 7fde42ffd640 1 -- 172.21.15.104:0/3108062937 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde441078f0 msgr2=0x7fde4419f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:25:25.022 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:25.005+0000 7fde42ffd640 1 --2- 172.21.15.104:0/3108062937 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde441078f0 0x7fde4419f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:25.022 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:25.005+0000 7fde42ffd640 1 -- 172.21.15.104:0/3108062937 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde4419fbf0 con 0x7fde44102690 2024-09-12T01:25:25.023 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:25.005+0000 7fde42ffd640 1 --2- 172.21.15.104:0/3108062937 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde44102690 0x7fde44197fc0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fde3400b4a0 tx=0x7fde3400b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:25:25.023 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:25.005+0000 7fde48821640 1 -- 172.21.15.104:0/3108062937 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde34014070 con 0x7fde44102690 2024-09-12T01:25:25.023 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:25.005+0000 7fde48821640 1 -- 172.21.15.104:0/3108062937 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde34010af0 con 0x7fde44102690 2024-09-12T01:25:25.023 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:25.005+0000 7fde48821640 1 -- 172.21.15.104:0/3108062937 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde34017d10 con 0x7fde44102690 2024-09-12T01:25:25.023 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:25.005+0000 7fde49823640 1 -- 172.21.15.104:0/3108062937 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde4419fee0 con 0x7fde44102690 2024-09-12T01:25:25.023 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:25.005+0000 7fde49823640 1 -- 172.21.15.104:0/3108062937 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde441a0300 con 0x7fde44102690 2024-09-12T01:25:25.023 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:25.007+0000 7fde49823640 1 -- 172.21.15.104:0/3108062937 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde440fdd80 con 0x7fde44102690 2024-09-12T01:25:25.023 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:25.008+0000 7fde48821640 1 -- 172.21.15.104:0/3108062937 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fde34017070 con 0x7fde44102690 2024-09-12T01:25:25.023 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:25.008+0000 7fde48821640 1 --2- 172.21.15.104:0/3108062937 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fde2c0a2c30 0x7fde2c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:25:25.023 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:25.008+0000 7fde48821640 1 -- 172.21.15.104:0/3108062937 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6414+0+0 (secure 0 0 0) 0x7fde340cdba0 con 0x7fde44102690 2024-09-12T01:25:25.023 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:25.008+0000 7fde427fc640 1 --2- 172.21.15.104:0/3108062937 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fde2c0a2c30 0x7fde2c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:25:25.023 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:25.009+0000 7fde427fc640 1 --2- 172.21.15.104:0/3108062937 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fde2c0a2c30 0x7fde2c0a50f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fde38005ab0 tx=0x7fde38000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:25:25.023 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:25.013+0000 7fde48821640 1 -- 172.21.15.104:0/3108062937 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fde3409b270 con 0x7fde44102690 2024-09-12T01:25:25.192 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:25.192+0000 7fde49823640 1 -- 172.21.15.104:0/3108062937 --> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fde44062630 con 0x7fde2c0a2c30 2024-09-12T01:25:25.489 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:25:25 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:25:25.076+0000 7f2a1ec12640 -1 osd.10 0 waiting for initial osdmap 2024-09-12T01:25:25.490 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:25:25 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:25:25.087+0000 7f2a16a32640 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T01:25:25.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:25 smithi104 ceph-mon[35219]: Detected new or changed devices on smithi053 2024-09-12T01:25:25.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:25 smithi104 ceph-mon[35219]: Adjusting osd_memory_target on smithi053 to 4273M 2024-09-12T01:25:25.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:25 smithi104 ceph-mon[35219]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-09-12T01:25:25.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:25 smithi104 ceph-mon[35219]: osdmap e71: 11 total, 10 up, 11 in 2024-09-12T01:25:25.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:25 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:25:25.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:25 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:25:25.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:25 smithi104 ceph-mon[35219]: from='osd.10 ' entity='osd.10' 2024-09-12T01:25:25.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:25 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:25:25.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:25 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:25:25.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:25 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:25:25.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:25 smithi001 ceph-mon[30387]: Detected new or changed devices on smithi053 2024-09-12T01:25:25.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:25 smithi001 ceph-mon[30387]: Adjusting osd_memory_target on smithi053 to 4273M 2024-09-12T01:25:25.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:25 smithi001 ceph-mon[30387]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-09-12T01:25:25.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:25 smithi001 ceph-mon[30387]: osdmap e71: 11 total, 10 up, 11 in 2024-09-12T01:25:25.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:25 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:25:25.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:25 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:25:25.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:25 smithi001 ceph-mon[30387]: from='osd.10 ' entity='osd.10' 2024-09-12T01:25:25.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:25 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:25:25.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:25 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:25:25.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:25 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:25:25.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:25 smithi053 ceph-mon[35158]: Detected new or changed devices on smithi053 2024-09-12T01:25:25.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:25 smithi053 ceph-mon[35158]: Adjusting osd_memory_target on smithi053 to 4273M 2024-09-12T01:25:25.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:25 smithi053 ceph-mon[35158]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-09-12T01:25:25.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:25 smithi053 ceph-mon[35158]: osdmap e71: 11 total, 10 up, 11 in 2024-09-12T01:25:25.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:25 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:25:25.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:25 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:25:25.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:25 smithi053 ceph-mon[35158]: from='osd.10 ' entity='osd.10' 2024-09-12T01:25:25.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:25 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:25:25.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:25 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:25:25.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:25 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:25:26.561 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:26 smithi104 ceph-mon[35219]: purged_snaps scrub starts 2024-09-12T01:25:26.561 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:26 smithi104 ceph-mon[35219]: purged_snaps scrub ok 2024-09-12T01:25:26.561 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:26 smithi104 ceph-mon[35219]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:25:26.562 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:26 smithi104 ceph-mon[35219]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:25:26.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:26 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:25:26.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:26 smithi104 ceph-mon[35219]: osd.10 [v2:172.21.15.53:6824/3357909498,v1:172.21.15.53:6825/3357909498] boot 2024-09-12T01:25:26.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:26 smithi104 ceph-mon[35219]: osdmap e72: 11 total, 11 up, 11 in 2024-09-12T01:25:26.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:26 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:25:26.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:26 smithi104 ceph-mon[35219]: osdmap e73: 11 total, 11 up, 11 in 2024-09-12T01:25:26.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:26 smithi001 ceph-mon[30387]: purged_snaps scrub starts 2024-09-12T01:25:26.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:26 smithi001 ceph-mon[30387]: purged_snaps scrub ok 2024-09-12T01:25:26.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:26 smithi001 ceph-mon[30387]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:25:26.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:26 smithi001 ceph-mon[30387]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:25:26.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:26 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:25:26.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:26 smithi001 ceph-mon[30387]: osd.10 [v2:172.21.15.53:6824/3357909498,v1:172.21.15.53:6825/3357909498] boot 2024-09-12T01:25:26.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:26 smithi001 ceph-mon[30387]: osdmap e72: 11 total, 11 up, 11 in 2024-09-12T01:25:26.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:26 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:25:26.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:26 smithi001 ceph-mon[30387]: osdmap e73: 11 total, 11 up, 11 in 2024-09-12T01:25:26.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:26 smithi053 ceph-mon[35158]: purged_snaps scrub starts 2024-09-12T01:25:26.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:26 smithi053 ceph-mon[35158]: purged_snaps scrub ok 2024-09-12T01:25:26.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:26 smithi053 ceph-mon[35158]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:25:26.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:26 smithi053 ceph-mon[35158]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:25:26.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:26 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:25:26.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:26 smithi053 ceph-mon[35158]: osd.10 [v2:172.21.15.53:6824/3357909498,v1:172.21.15.53:6825/3357909498] boot 2024-09-12T01:25:26.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:26 smithi053 ceph-mon[35158]: osdmap e72: 11 total, 11 up, 11 in 2024-09-12T01:25:26.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:26 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:25:26.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:26 smithi053 ceph-mon[35158]: osdmap e73: 11 total, 11 up, 11 in 2024-09-12T01:25:28.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:28 smithi104 ceph-mon[35219]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:25:28.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:28 smithi104 ceph-mon[35219]: osdmap e74: 11 total, 11 up, 11 in 2024-09-12T01:25:28.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:28 smithi104 ceph-mon[35219]: from='client.? 172.21.15.104:0/169872043' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "474b5345-f2ae-4e0e-a197-53ab11f58c04"}]: dispatch 2024-09-12T01:25:28.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:28 smithi104 ceph-mon[35219]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "474b5345-f2ae-4e0e-a197-53ab11f58c04"}]: dispatch 2024-09-12T01:25:28.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:28 smithi104 ceph-mon[35219]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "474b5345-f2ae-4e0e-a197-53ab11f58c04"}]': finished 2024-09-12T01:25:28.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:28 smithi104 ceph-mon[35219]: osdmap e75: 12 total, 11 up, 12 in 2024-09-12T01:25:28.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:28 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:25:28.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:28 smithi104 ceph-mon[35219]: from='client.? 172.21.15.104:0/910573473' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:25:28.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:28 smithi001 ceph-mon[30387]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:25:28.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:28 smithi001 ceph-mon[30387]: osdmap e74: 11 total, 11 up, 11 in 2024-09-12T01:25:28.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:28 smithi001 ceph-mon[30387]: from='client.? 172.21.15.104:0/169872043' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "474b5345-f2ae-4e0e-a197-53ab11f58c04"}]: dispatch 2024-09-12T01:25:28.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:28 smithi001 ceph-mon[30387]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "474b5345-f2ae-4e0e-a197-53ab11f58c04"}]: dispatch 2024-09-12T01:25:28.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:28 smithi001 ceph-mon[30387]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "474b5345-f2ae-4e0e-a197-53ab11f58c04"}]': finished 2024-09-12T01:25:28.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:28 smithi001 ceph-mon[30387]: osdmap e75: 12 total, 11 up, 12 in 2024-09-12T01:25:28.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:28 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:25:28.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:28 smithi001 ceph-mon[30387]: from='client.? 172.21.15.104:0/910573473' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:25:28.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:28 smithi053 ceph-mon[35158]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:25:28.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:28 smithi053 ceph-mon[35158]: osdmap e74: 11 total, 11 up, 11 in 2024-09-12T01:25:28.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:28 smithi053 ceph-mon[35158]: from='client.? 172.21.15.104:0/169872043' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "474b5345-f2ae-4e0e-a197-53ab11f58c04"}]: dispatch 2024-09-12T01:25:28.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:28 smithi053 ceph-mon[35158]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "474b5345-f2ae-4e0e-a197-53ab11f58c04"}]: dispatch 2024-09-12T01:25:28.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:28 smithi053 ceph-mon[35158]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "474b5345-f2ae-4e0e-a197-53ab11f58c04"}]': finished 2024-09-12T01:25:28.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:28 smithi053 ceph-mon[35158]: osdmap e75: 12 total, 11 up, 12 in 2024-09-12T01:25:28.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:28 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:25:28.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:28 smithi053 ceph-mon[35158]: from='client.? 172.21.15.104:0/910573473' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:25:29.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:29 smithi053 ceph-mon[35158]: pgmap v252: 1 pgs: 1 peering; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-12T01:25:29.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:29 smithi053 ceph-mon[35158]: osdmap e76: 12 total, 11 up, 12 in 2024-09-12T01:25:29.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:29 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:25:30.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:29 smithi104 ceph-mon[35219]: pgmap v252: 1 pgs: 1 peering; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-12T01:25:30.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:29 smithi104 ceph-mon[35219]: osdmap e76: 12 total, 11 up, 12 in 2024-09-12T01:25:30.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:29 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:25:30.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:29 smithi001 ceph-mon[30387]: pgmap v252: 1 pgs: 1 peering; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-12T01:25:30.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:29 smithi001 ceph-mon[30387]: osdmap e76: 12 total, 11 up, 12 in 2024-09-12T01:25:30.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:29 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:25:31.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:31 smithi053 ceph-mon[35158]: pgmap v254: 1 pgs: 1 peering; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-09-12T01:25:32.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:31 smithi104 ceph-mon[35219]: pgmap v254: 1 pgs: 1 peering; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-09-12T01:25:32.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:31 smithi001 ceph-mon[30387]: pgmap v254: 1 pgs: 1 peering; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-09-12T01:25:33.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:33 smithi104 ceph-mon[35219]: pgmap v255: 1 pgs: 1 peering; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T01:25:33.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:33 smithi053 ceph-mon[35158]: pgmap v255: 1 pgs: 1 peering; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T01:25:34.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:33 smithi001 ceph-mon[30387]: pgmap v255: 1 pgs: 1 peering; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T01:25:35.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:35 smithi104 ceph-mon[35219]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-12T01:25:35.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:35 smithi053 ceph-mon[35158]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-12T01:25:36.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:35 smithi001 ceph-mon[30387]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-12T01:25:36.962 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:36 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-12T01:25:36.962 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:36 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:25:36.962 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:36 smithi104 ceph-mon[35219]: Deploying daemon osd.11 on smithi104 2024-09-12T01:25:36.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:36 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-12T01:25:36.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:36 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:25:36.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:36 smithi053 ceph-mon[35158]: Deploying daemon osd.11 on smithi104 2024-09-12T01:25:37.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:36 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-12T01:25:37.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:36 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:25:37.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:36 smithi001 ceph-mon[30387]: Deploying daemon osd.11 on smithi104 2024-09-12T01:25:37.969 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:37 smithi104 ceph-mon[35219]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-12T01:25:37.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:37 smithi053 ceph-mon[35158]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-12T01:25:38.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:37 smithi001 ceph-mon[30387]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-12T01:25:39.976 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:39 smithi104 ceph-mon[35219]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:25:39.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:39 smithi053 ceph-mon[35158]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:25:40.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:39 smithi001 ceph-mon[30387]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:25:41.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:41 smithi053 ceph-mon[35158]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:25:41.994 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:41 smithi104 ceph-mon[35219]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:25:42.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:41 smithi001 ceph-mon[30387]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:25:42.756 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:42 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:25:42.756 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:42 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:42.756 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:42 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:42.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:42 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:25:42.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:42 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:42.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:42 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:43.079 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:42 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:25:43.079 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:42 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:43.079 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:42 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:44.300 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:44 smithi104 ceph-mon[35219]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:25:44.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:44 smithi001 ceph-mon[30387]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:25:44.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:44 smithi053 ceph-mon[35158]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:25:45.538 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:45.537+0000 7fde48821640 1 -- 172.21.15.104:0/3108062937 <== mgr.14150 v2:172.21.15.1:6800/1928662456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fde44062630 con 0x7fde2c0a2c30 2024-09-12T01:25:45.538 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 11 on host 'smithi104' 2024-09-12T01:25:45.540 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:45.540+0000 7fde49823640 1 -- 172.21.15.104:0/3108062937 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fde2c0a2c30 msgr2=0x7fde2c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:25:45.540 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:45.540+0000 7fde49823640 1 --2- 172.21.15.104:0/3108062937 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fde2c0a2c30 0x7fde2c0a50f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fde38005ab0 tx=0x7fde38000bc0 comp rx=0 tx=0).stop 2024-09-12T01:25:45.540 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:45.540+0000 7fde49823640 1 -- 172.21.15.104:0/3108062937 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde44102690 msgr2=0x7fde44197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:25:45.540 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:45.540+0000 7fde49823640 1 --2- 172.21.15.104:0/3108062937 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde44102690 0x7fde44197fc0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fde3400b4a0 tx=0x7fde3400b960 comp rx=0 tx=0).stop 2024-09-12T01:25:45.540 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:45.540+0000 7fde49823640 1 -- 172.21.15.104:0/3108062937 shutdown_connections 2024-09-12T01:25:45.541 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:45.540+0000 7fde49823640 1 --2- 172.21.15.104:0/3108062937 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde441078f0 0x7fde4419f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:45.541 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:45.540+0000 7fde49823640 1 --2- 172.21.15.104:0/3108062937 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde44104fc0 0x7fde44198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:45.541 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:45.540+0000 7fde49823640 1 --2- 172.21.15.104:0/3108062937 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fde2c0a2c30 0x7fde2c0a50f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:45.541 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:45.540+0000 7fde49823640 1 --2- 172.21.15.104:0/3108062937 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde44102690 0x7fde44197fc0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:45.541 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:45.540+0000 7fde49823640 1 -- 172.21.15.104:0/3108062937 >> 172.21.15.104:0/3108062937 conn(0x7fde440fc290 msgr2=0x7fde44103010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:25:45.541 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:45.540+0000 7fde49823640 1 -- 172.21.15.104:0/3108062937 shutdown_connections 2024-09-12T01:25:45.541 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:45.540+0000 7fde49823640 1 -- 172.21.15.104:0/3108062937 wait complete. 2024-09-12T01:25:46.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:45 smithi001 ceph-mon[30387]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:25:46.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:45 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:46.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:45 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:46.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:45 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:25:46.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:45 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:25:46.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:45 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:46.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:45 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:25:46.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:45 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:46.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:45 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:46.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:45 smithi053 ceph-mon[35158]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:25:46.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:45 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:46.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:45 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:46.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:45 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:25:46.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:45 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:25:46.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:45 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:46.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:45 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:25:46.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:45 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:46.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:45 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:46.309 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:45 smithi104 ceph-mon[35219]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:25:46.310 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:45 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:46.310 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:45 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:46.310 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:45 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:25:46.310 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:45 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:25:46.310 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:45 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:46.310 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:45 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:25:46.310 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:45 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:46.310 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:45 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:46.336 DEBUG:teuthology.orchestra.run.smithi104:osd.11> sudo journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@osd.11.service 2024-09-12T01:25:46.339 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-12T01:25:46.340 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-12T01:25:46.728 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:25:47.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.337+0000 7f3daae00640 1 -- 172.21.15.1:0/109611925 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3da4074c50 msgr2=0x7f3da40750b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:25:47.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.337+0000 7f3daae00640 1 --2- 172.21.15.1:0/109611925 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3da4074c50 0x7f3da40750b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3d90009900 tx=0x7f3d9002a410 comp rx=0 tx=0).stop 2024-09-12T01:25:47.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.338+0000 7f3daae00640 1 -- 172.21.15.1:0/109611925 shutdown_connections 2024-09-12T01:25:47.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.338+0000 7f3daae00640 1 --2- 172.21.15.1:0/109611925 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3da410e1e0 0x7f3da41105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:47.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.338+0000 7f3daae00640 1 --2- 172.21.15.1:0/109611925 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3da4074c50 0x7f3da40750b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:47.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.338+0000 7f3daae00640 1 --2- 172.21.15.1:0/109611925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3da4076220 0x7f3da4074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:47.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.338+0000 7f3daae00640 1 -- 172.21.15.1:0/109611925 >> 172.21.15.1:0/109611925 conn(0x7f3da40fd390 msgr2=0x7f3da40ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:25:47.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.338+0000 7f3daae00640 1 -- 172.21.15.1:0/109611925 shutdown_connections 2024-09-12T01:25:47.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.338+0000 7f3daae00640 1 -- 172.21.15.1:0/109611925 wait complete. 2024-09-12T01:25:47.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.338+0000 7f3daae00640 1 Processor -- start 2024-09-12T01:25:47.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.339+0000 7f3daae00640 1 -- start start 2024-09-12T01:25:47.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.339+0000 7f3daae00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3da4074c50 0x7f3da419e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:25:47.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.339+0000 7f3daae00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3da4076220 0x7f3da419ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:25:47.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.339+0000 7f3daae00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3da410e1e0 0x7f3da41a5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:25:47.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.339+0000 7f3daae00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3da4113ea0 con 0x7f3da4074c50 2024-09-12T01:25:47.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.339+0000 7f3daae00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3da4113d20 con 0x7f3da410e1e0 2024-09-12T01:25:47.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.339+0000 7f3daae00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3da4114020 con 0x7f3da4076220 2024-09-12T01:25:47.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.339+0000 7f3da3e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3da4074c50 0x7f3da419e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:25:47.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.339+0000 7f3da3e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3da4074c50 0x7f3da419e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54802/0 (socket says 172.21.15.1:54802) 2024-09-12T01:25:47.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.339+0000 7f3da3e00640 1 -- 172.21.15.1:0/2470545723 learned_addr learned my addr 172.21.15.1:0/2470545723 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:25:47.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.339+0000 7f3da3400640 1 --2- 172.21.15.1:0/2470545723 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3da4076220 0x7f3da419ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:25:47.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.339+0000 7f3da8e00640 1 --2- 172.21.15.1:0/2470545723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3da410e1e0 0x7f3da41a5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:25:47.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.340+0000 7f3da3e00640 1 -- 172.21.15.1:0/2470545723 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3da4076220 msgr2=0x7f3da419ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:25:47.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.340+0000 7f3da3e00640 1 --2- 172.21.15.1:0/2470545723 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3da4076220 0x7f3da419ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:47.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.340+0000 7f3da3e00640 1 -- 172.21.15.1:0/2470545723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3da410e1e0 msgr2=0x7f3da41a5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:25:47.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.340+0000 7f3da3e00640 1 --2- 172.21.15.1:0/2470545723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3da410e1e0 0x7f3da41a5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:47.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.340+0000 7f3da3e00640 1 -- 172.21.15.1:0/2470545723 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da41a6180 con 0x7f3da4074c50 2024-09-12T01:25:47.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.340+0000 7f3da3400640 1 --2- 172.21.15.1:0/2470545723 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3da4076220 0x7f3da419ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:25:47.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.340+0000 7f3da3e00640 1 --2- 172.21.15.1:0/2470545723 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3da4074c50 0x7f3da419e550 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3d9400e9f0 tx=0x7f3d9400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:25:47.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.340+0000 7f3da0c00640 1 -- 172.21.15.1:0/2470545723 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9400ccc0 con 0x7f3da4074c50 2024-09-12T01:25:47.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.340+0000 7f3da0c00640 1 -- 172.21.15.1:0/2470545723 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d94005500 con 0x7f3da4074c50 2024-09-12T01:25:47.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.340+0000 7f3daae00640 1 -- 172.21.15.1:0/2470545723 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3da41a6470 con 0x7f3da4074c50 2024-09-12T01:25:47.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.340+0000 7f3da0c00640 1 -- 172.21.15.1:0/2470545723 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d94010690 con 0x7f3da4074c50 2024-09-12T01:25:47.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.340+0000 7f3daae00640 1 -- 172.21.15.1:0/2470545723 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3da41a6930 con 0x7f3da4074c50 2024-09-12T01:25:47.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.342+0000 7f3daae00640 1 -- 172.21.15.1:0/2470545723 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3da41043b0 con 0x7f3da4074c50 2024-09-12T01:25:47.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.345+0000 7f3da0c00640 1 -- 172.21.15.1:0/2470545723 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f3d940046c0 con 0x7f3da4074c50 2024-09-12T01:25:47.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.346+0000 7f3da0c00640 1 --2- 172.21.15.1:0/2470545723 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f3d880a2bb0 0x7f3d880a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:25:47.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.347+0000 7f3da0c00640 1 -- 172.21.15.1:0/2470545723 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6809+0+0 (secure 0 0 0) 0x7f3d940cefe0 con 0x7f3da4074c50 2024-09-12T01:25:47.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.347+0000 7f3da3400640 1 --2- 172.21.15.1:0/2470545723 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f3d880a2bb0 0x7f3d880a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:25:47.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.347+0000 7f3da3400640 1 --2- 172.21.15.1:0/2470545723 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f3d880a2bb0 0x7f3d880a5070 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3d900314e0 tx=0x7f3d90000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:25:47.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.349+0000 7f3da0c00640 1 -- 172.21.15.1:0/2470545723 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3d940541b0 con 0x7f3da4074c50 2024-09-12T01:25:47.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.535+0000 7f3daae00640 1 -- 172.21.15.1:0/2470545723 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f3da4062df0 con 0x7f3da4074c50 2024-09-12T01:25:47.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.536+0000 7f3da0c00640 1 -- 172.21.15.1:0/2470545723 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v76) ==== 74+0+133 (secure 0 0 0) 0x7f3d9409c4c0 con 0x7f3da4074c50 2024-09-12T01:25:47.536 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:25:47.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.538+0000 7f3daae00640 1 -- 172.21.15.1:0/2470545723 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f3d880a2bb0 msgr2=0x7f3d880a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:25:47.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.538+0000 7f3daae00640 1 --2- 172.21.15.1:0/2470545723 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f3d880a2bb0 0x7f3d880a5070 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3d900314e0 tx=0x7f3d90000bc0 comp rx=0 tx=0).stop 2024-09-12T01:25:47.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.538+0000 7f3daae00640 1 -- 172.21.15.1:0/2470545723 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3da4074c50 msgr2=0x7f3da419e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:25:47.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.538+0000 7f3daae00640 1 --2- 172.21.15.1:0/2470545723 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3da4074c50 0x7f3da419e550 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3d9400e9f0 tx=0x7f3d9400eeb0 comp rx=0 tx=0).stop 2024-09-12T01:25:47.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.539+0000 7f3daae00640 1 -- 172.21.15.1:0/2470545723 shutdown_connections 2024-09-12T01:25:47.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.539+0000 7f3daae00640 1 --2- 172.21.15.1:0/2470545723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3da410e1e0 0x7f3da41a5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:47.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.539+0000 7f3daae00640 1 --2- 172.21.15.1:0/2470545723 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f3d880a2bb0 0x7f3d880a5070 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:47.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.539+0000 7f3daae00640 1 --2- 172.21.15.1:0/2470545723 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3da4076220 0x7f3da419ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:47.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.539+0000 7f3daae00640 1 --2- 172.21.15.1:0/2470545723 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3da4074c50 0x7f3da419e550 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:47.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.539+0000 7f3daae00640 1 -- 172.21.15.1:0/2470545723 >> 172.21.15.1:0/2470545723 conn(0x7f3da40fd390 msgr2=0x7f3da4101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:25:47.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.539+0000 7f3daae00640 1 -- 172.21.15.1:0/2470545723 shutdown_connections 2024-09-12T01:25:47.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:47.539+0000 7f3daae00640 1 -- 172.21.15.1:0/2470545723 wait complete. 2024-09-12T01:25:48.038 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:47 smithi104 ceph-mon[35219]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:25:48.038 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:47 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/2470545723' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T01:25:48.038 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:47 smithi104 ceph-mon[35219]: from='osd.11 [v2:172.21.15.104:6824/3381427987,v1:172.21.15.104:6825/3381427987]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T01:25:48.038 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:47 smithi104 ceph-mon[35219]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T01:25:48.038 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:25:47 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:25:47.857+0000 7f19f5583640 -1 osd.11 0 log_to_monitors true 2024-09-12T01:25:48.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:47 smithi001 ceph-mon[30387]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:25:48.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:47 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/2470545723' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T01:25:48.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:47 smithi001 ceph-mon[30387]: from='osd.11 [v2:172.21.15.104:6824/3381427987,v1:172.21.15.104:6825/3381427987]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T01:25:48.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:47 smithi001 ceph-mon[30387]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T01:25:48.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:47 smithi053 ceph-mon[35158]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:25:48.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:47 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/2470545723' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T01:25:48.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:47 smithi053 ceph-mon[35158]: from='osd.11 [v2:172.21.15.104:6824/3381427987,v1:172.21.15.104:6825/3381427987]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T01:25:48.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:47 smithi053 ceph-mon[35158]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T01:25:48.510 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":76,"num_osds":12,"num_up_osds":11,"osd_up_since":1726104325,"num_in_osds":12,"osd_in_since":1726104327,"num_remapped_pgs":0} 2024-09-12T01:25:49.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:48 smithi001 ceph-mon[30387]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-12T01:25:49.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:48 smithi001 ceph-mon[30387]: osdmap e77: 12 total, 11 up, 12 in 2024-09-12T01:25:49.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:48 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:25:49.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:48 smithi001 ceph-mon[30387]: from='osd.11 [v2:172.21.15.104:6824/3381427987,v1:172.21.15.104:6825/3381427987]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-12T01:25:49.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:48 smithi001 ceph-mon[30387]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-12T01:25:49.217 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:48 smithi104 ceph-mon[35219]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-12T01:25:49.217 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:48 smithi104 ceph-mon[35219]: osdmap e77: 12 total, 11 up, 12 in 2024-09-12T01:25:49.217 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:48 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:25:49.217 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:48 smithi104 ceph-mon[35219]: from='osd.11 [v2:172.21.15.104:6824/3381427987,v1:172.21.15.104:6825/3381427987]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-12T01:25:49.217 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:48 smithi104 ceph-mon[35219]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-12T01:25:49.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:48 smithi053 ceph-mon[35158]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-12T01:25:49.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:48 smithi053 ceph-mon[35158]: osdmap e77: 12 total, 11 up, 12 in 2024-09-12T01:25:49.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:48 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:25:49.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:48 smithi053 ceph-mon[35158]: from='osd.11 [v2:172.21.15.104:6824/3381427987,v1:172.21.15.104:6825/3381427987]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-12T01:25:49.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:48 smithi053 ceph-mon[35158]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-12T01:25:49.512 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-12T01:25:49.891 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:25:49.926 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:25:49 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:25:49.597+0000 7f19f1506640 -1 osd.11 0 waiting for initial osdmap 2024-09-12T01:25:49.927 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:25:49 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:25:49.609+0000 7f19e9326640 -1 osd.11 78 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T01:25:49.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:49 smithi001 ceph-mon[30387]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:25:49.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:49 smithi001 ceph-mon[30387]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-12T01:25:49.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:49 smithi001 ceph-mon[30387]: osdmap e78: 12 total, 11 up, 12 in 2024-09-12T01:25:49.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:49 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:25:49.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:49 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:25:49.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:49 smithi001 ceph-mon[30387]: Detected new or changed devices on smithi104 2024-09-12T01:25:49.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:49 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:49.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:49 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:49.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:49 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:25:49.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:49 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:25:49.930 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:49 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:25:49.930 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:49 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:25:49.930 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:49 smithi001 ceph-mon[30387]: Adjusting osd_memory_target on smithi104 to 4272M 2024-09-12T01:25:49.930 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:49 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:49.930 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:49 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:25:49.930 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:49 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:25:49.930 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:49 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:49.930 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:49 smithi001 ceph-mon[30387]: from='osd.11 ' entity='osd.11' 2024-09-12T01:25:50.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:49 smithi053 ceph-mon[35158]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:25:50.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:49 smithi053 ceph-mon[35158]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-12T01:25:50.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:49 smithi053 ceph-mon[35158]: osdmap e78: 12 total, 11 up, 12 in 2024-09-12T01:25:50.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:49 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:25:50.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:49 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:25:50.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:49 smithi053 ceph-mon[35158]: Detected new or changed devices on smithi104 2024-09-12T01:25:50.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:49 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:50.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:49 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:50.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:49 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:25:50.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:49 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:25:50.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:49 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:25:50.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:49 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:25:50.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:49 smithi053 ceph-mon[35158]: Adjusting osd_memory_target on smithi104 to 4272M 2024-09-12T01:25:50.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:49 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:50.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:49 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:25:50.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:49 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:25:50.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:49 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:50.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:49 smithi053 ceph-mon[35158]: from='osd.11 ' entity='osd.11' 2024-09-12T01:25:50.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:49 smithi104 ceph-mon[35219]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:25:50.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:49 smithi104 ceph-mon[35219]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-12T01:25:50.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:49 smithi104 ceph-mon[35219]: osdmap e78: 12 total, 11 up, 12 in 2024-09-12T01:25:50.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:49 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:25:50.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:49 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:25:50.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:49 smithi104 ceph-mon[35219]: Detected new or changed devices on smithi104 2024-09-12T01:25:50.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:49 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:50.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:49 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:50.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:49 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:25:50.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:49 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:25:50.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:49 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:25:50.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:49 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:25:50.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:49 smithi104 ceph-mon[35219]: Adjusting osd_memory_target on smithi104 to 4272M 2024-09-12T01:25:50.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:49 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:50.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:49 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:25:50.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:49 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:25:50.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:49 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:50.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:49 smithi104 ceph-mon[35219]: from='osd.11 ' entity='osd.11' 2024-09-12T01:25:50.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.563+0000 7fb875a00640 1 -- 172.21.15.1:0/3405237477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb870107970 msgr2=0x7fb870109d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:25:50.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.563+0000 7fb875a00640 1 --2- 172.21.15.1:0/3405237477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb870107970 0x7fb870109d60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb868009ca0 tx=0x7fb86802a8c0 comp rx=0 tx=0).stop 2024-09-12T01:25:50.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.563+0000 7fb875a00640 1 -- 172.21.15.1:0/3405237477 shutdown_connections 2024-09-12T01:25:50.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.563+0000 7fb875a00640 1 --2- 172.21.15.1:0/3405237477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb870107970 0x7fb870109d60 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:50.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.563+0000 7fb875a00640 1 --2- 172.21.15.1:0/3405237477 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb870100230 0x7fb870107430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:50.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.563+0000 7fb875a00640 1 --2- 172.21.15.1:0/3405237477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8700ff860 0x7fb8700ffc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:50.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.563+0000 7fb875a00640 1 -- 172.21.15.1:0/3405237477 >> 172.21.15.1:0/3405237477 conn(0x7fb8700fb610 msgr2=0x7fb8700fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:25:50.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.563+0000 7fb875a00640 1 -- 172.21.15.1:0/3405237477 shutdown_connections 2024-09-12T01:25:50.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.563+0000 7fb875a00640 1 -- 172.21.15.1:0/3405237477 wait complete. 2024-09-12T01:25:50.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.564+0000 7fb875a00640 1 Processor -- start 2024-09-12T01:25:50.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.564+0000 7fb875a00640 1 -- start start 2024-09-12T01:25:50.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.565+0000 7fb875a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8700ff860 0x7fb87019a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:25:50.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.565+0000 7fb875a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb870100230 0x7fb87019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:25:50.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.565+0000 7fb875a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb870107970 0x7fb8701a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:25:50.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.565+0000 7fb875a00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb87010d880 con 0x7fb8700ff860 2024-09-12T01:25:50.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.565+0000 7fb875a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb87010d700 con 0x7fb870100230 2024-09-12T01:25:50.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.565+0000 7fb875a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb87010da00 con 0x7fb870107970 2024-09-12T01:25:50.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.565+0000 7fb86ea00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8700ff860 0x7fb87019a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:25:50.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.565+0000 7fb86e000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb870100230 0x7fb87019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:25:50.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.565+0000 7fb86ea00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8700ff860 0x7fb87019a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45200/0 (socket says 172.21.15.1:45200) 2024-09-12T01:25:50.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.565+0000 7fb86ea00640 1 -- 172.21.15.1:0/3489891172 learned_addr learned my addr 172.21.15.1:0/3489891172 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:25:50.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.565+0000 7fb86f400640 1 --2- 172.21.15.1:0/3489891172 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb870107970 0x7fb8701a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:25:50.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.565+0000 7fb86ea00640 1 -- 172.21.15.1:0/3489891172 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb870107970 msgr2=0x7fb8701a1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:25:50.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.565+0000 7fb86ea00640 1 --2- 172.21.15.1:0/3489891172 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb870107970 0x7fb8701a1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:50.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.565+0000 7fb86ea00640 1 -- 172.21.15.1:0/3489891172 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb870100230 msgr2=0x7fb87019a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:25:50.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.565+0000 7fb86ea00640 1 --2- 172.21.15.1:0/3489891172 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb870100230 0x7fb87019a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:50.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.565+0000 7fb86ea00640 1 -- 172.21.15.1:0/3489891172 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8701a1cb0 con 0x7fb8700ff860 2024-09-12T01:25:50.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.565+0000 7fb86f400640 1 --2- 172.21.15.1:0/3489891172 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb870107970 0x7fb8701a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:25:50.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.565+0000 7fb86e000640 1 --2- 172.21.15.1:0/3489891172 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb870100230 0x7fb87019a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:25:50.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.566+0000 7fb86ea00640 1 --2- 172.21.15.1:0/3489891172 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8700ff860 0x7fb87019a080 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb86000e9c0 tx=0x7fb86000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:25:50.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.566+0000 7fb867400640 1 -- 172.21.15.1:0/3489891172 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86000cd80 con 0x7fb8700ff860 2024-09-12T01:25:50.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.566+0000 7fb867400640 1 -- 172.21.15.1:0/3489891172 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8600035f0 con 0x7fb8700ff860 2024-09-12T01:25:50.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.566+0000 7fb875a00640 1 -- 172.21.15.1:0/3489891172 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8701a1fa0 con 0x7fb8700ff860 2024-09-12T01:25:50.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.566+0000 7fb867400640 1 -- 172.21.15.1:0/3489891172 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb860010690 con 0x7fb8700ff860 2024-09-12T01:25:50.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.566+0000 7fb875a00640 1 -- 172.21.15.1:0/3489891172 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8701a23f0 con 0x7fb8700ff860 2024-09-12T01:25:50.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.567+0000 7fb875a00640 1 -- 172.21.15.1:0/3489891172 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8701a26b0 con 0x7fb8700ff860 2024-09-12T01:25:50.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.569+0000 7fb867400640 1 -- 172.21.15.1:0/3489891172 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fb8600037a0 con 0x7fb8700ff860 2024-09-12T01:25:50.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.571+0000 7fb867400640 1 --2- 172.21.15.1:0/3489891172 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fb8540a2bb0 0x7fb8540a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:25:50.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.571+0000 7fb867400640 1 -- 172.21.15.1:0/3489891172 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6847+0+0 (secure 0 0 0) 0x7fb8600ceff0 con 0x7fb8700ff860 2024-09-12T01:25:50.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.571+0000 7fb86e000640 1 --2- 172.21.15.1:0/3489891172 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fb8540a2bb0 0x7fb8540a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:25:50.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.572+0000 7fb86e000640 1 --2- 172.21.15.1:0/3489891172 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fb8540a2bb0 0x7fb8540a5070 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb87019b5a0 tx=0x7fb85c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:25:50.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.573+0000 7fb867400640 1 -- 172.21.15.1:0/3489891172 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb86009c500 con 0x7fb8700ff860 2024-09-12T01:25:50.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.763+0000 7fb875a00640 1 -- 172.21.15.1:0/3489891172 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fb8701a2a40 con 0x7fb8700ff860 2024-09-12T01:25:50.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.764+0000 7fb867400640 1 -- 172.21.15.1:0/3489891172 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) ==== 74+0+133 (secure 0 0 0) 0x7fb86009be00 con 0x7fb8700ff860 2024-09-12T01:25:50.764 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:25:50.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.766+0000 7fb875a00640 1 -- 172.21.15.1:0/3489891172 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fb8540a2bb0 msgr2=0x7fb8540a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:25:50.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.766+0000 7fb875a00640 1 --2- 172.21.15.1:0/3489891172 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fb8540a2bb0 0x7fb8540a5070 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb87019b5a0 tx=0x7fb85c005f90 comp rx=0 tx=0).stop 2024-09-12T01:25:50.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.766+0000 7fb875a00640 1 -- 172.21.15.1:0/3489891172 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8700ff860 msgr2=0x7fb87019a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:25:50.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.766+0000 7fb875a00640 1 --2- 172.21.15.1:0/3489891172 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8700ff860 0x7fb87019a080 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb86000e9c0 tx=0x7fb86000ee80 comp rx=0 tx=0).stop 2024-09-12T01:25:50.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.766+0000 7fb875a00640 1 -- 172.21.15.1:0/3489891172 shutdown_connections 2024-09-12T01:25:50.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.766+0000 7fb875a00640 1 --2- 172.21.15.1:0/3489891172 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fb8540a2bb0 0x7fb8540a5070 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:50.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.766+0000 7fb875a00640 1 --2- 172.21.15.1:0/3489891172 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb870107970 0x7fb8701a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:50.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.766+0000 7fb875a00640 1 --2- 172.21.15.1:0/3489891172 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb870100230 0x7fb87019a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:50.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.766+0000 7fb875a00640 1 --2- 172.21.15.1:0/3489891172 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8700ff860 0x7fb87019a080 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:50.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.766+0000 7fb875a00640 1 -- 172.21.15.1:0/3489891172 >> 172.21.15.1:0/3489891172 conn(0x7fb8700fb610 msgr2=0x7fb8700fb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:25:50.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.766+0000 7fb875a00640 1 -- 172.21.15.1:0/3489891172 shutdown_connections 2024-09-12T01:25:50.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:50.767+0000 7fb875a00640 1 -- 172.21.15.1:0/3489891172 wait complete. 2024-09-12T01:25:51.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:50 smithi001 ceph-mon[30387]: purged_snaps scrub starts 2024-09-12T01:25:51.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:50 smithi001 ceph-mon[30387]: purged_snaps scrub ok 2024-09-12T01:25:51.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:50 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:25:51.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:50 smithi001 ceph-mon[30387]: osd.11 [v2:172.21.15.104:6824/3381427987,v1:172.21.15.104:6825/3381427987] boot 2024-09-12T01:25:51.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:50 smithi001 ceph-mon[30387]: osdmap e79: 12 total, 12 up, 12 in 2024-09-12T01:25:51.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:50 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:25:51.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:50 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/3489891172' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T01:25:51.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:50 smithi053 ceph-mon[35158]: purged_snaps scrub starts 2024-09-12T01:25:51.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:50 smithi053 ceph-mon[35158]: purged_snaps scrub ok 2024-09-12T01:25:51.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:50 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:25:51.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:50 smithi053 ceph-mon[35158]: osd.11 [v2:172.21.15.104:6824/3381427987,v1:172.21.15.104:6825/3381427987] boot 2024-09-12T01:25:51.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:50 smithi053 ceph-mon[35158]: osdmap e79: 12 total, 12 up, 12 in 2024-09-12T01:25:51.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:50 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:25:51.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:50 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/3489891172' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T01:25:51.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:50 smithi104 ceph-mon[35219]: purged_snaps scrub starts 2024-09-12T01:25:51.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:50 smithi104 ceph-mon[35219]: purged_snaps scrub ok 2024-09-12T01:25:51.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:50 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:25:51.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:50 smithi104 ceph-mon[35219]: osd.11 [v2:172.21.15.104:6824/3381427987,v1:172.21.15.104:6825/3381427987] boot 2024-09-12T01:25:51.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:50 smithi104 ceph-mon[35219]: osdmap e79: 12 total, 12 up, 12 in 2024-09-12T01:25:51.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:50 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:25:51.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:50 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/3489891172' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T01:25:51.669 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":79,"num_osds":12,"num_up_osds":12,"osd_up_since":1726104350,"num_in_osds":12,"osd_in_since":1726104327,"num_remapped_pgs":0} 2024-09-12T01:25:51.670 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T01:25:52.046 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:25:52.185 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:51 smithi001 ceph-mon[30387]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:25:52.185 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:51 smithi001 ceph-mon[30387]: osdmap e80: 12 total, 12 up, 12 in 2024-09-12T01:25:52.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:51 smithi053 ceph-mon[35158]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:25:52.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:51 smithi053 ceph-mon[35158]: osdmap e80: 12 total, 12 up, 12 in 2024-09-12T01:25:52.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:51 smithi104 ceph-mon[35219]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:25:52.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:51 smithi104 ceph-mon[35219]: osdmap e80: 12 total, 12 up, 12 in 2024-09-12T01:25:52.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.629+0000 7ff2ff000640 1 -- 172.21.15.1:0/1806307396 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2f8109120 msgr2=0x7ff2f8110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:25:52.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.629+0000 7ff2ff000640 1 --2- 172.21.15.1:0/1806307396 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2f8109120 0x7ff2f8110ca0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0009ca0 tx=0x7ff2f002a8c0 comp rx=0 tx=0).stop 2024-09-12T01:25:52.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.629+0000 7ff2ff000640 1 -- 172.21.15.1:0/1806307396 shutdown_connections 2024-09-12T01:25:52.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.629+0000 7ff2ff000640 1 --2- 172.21.15.1:0/1806307396 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2f8109120 0x7ff2f8110ca0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:52.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.629+0000 7ff2ff000640 1 --2- 172.21.15.1:0/1806307396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2f81049a0 0x7ff2f81089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:52.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.629+0000 7ff2ff000640 1 --2- 172.21.15.1:0/1806307396 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2f8103ff0 0x7ff2f81043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:52.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.629+0000 7ff2ff000640 1 -- 172.21.15.1:0/1806307396 >> 172.21.15.1:0/1806307396 conn(0x7ff2f80ffa90 msgr2=0x7ff2f8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:25:52.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.630+0000 7ff2ff000640 1 -- 172.21.15.1:0/1806307396 shutdown_connections 2024-09-12T01:25:52.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.630+0000 7ff2ff000640 1 -- 172.21.15.1:0/1806307396 wait complete. 2024-09-12T01:25:52.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.631+0000 7ff2ff000640 1 Processor -- start 2024-09-12T01:25:52.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.631+0000 7ff2ff000640 1 -- start start 2024-09-12T01:25:52.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.631+0000 7ff2ff000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2f8103ff0 0x7ff2f819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:25:52.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.631+0000 7ff2ff000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2f81049a0 0x7ff2f819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:25:52.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.631+0000 7ff2ff000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2f8109120 0x7ff2f819ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:25:52.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.631+0000 7ff2ff000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f8114130 con 0x7ff2f81049a0 2024-09-12T01:25:52.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.631+0000 7ff2ff000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f8113fb0 con 0x7ff2f8103ff0 2024-09-12T01:25:52.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.631+0000 7ff2ff000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f81142b0 con 0x7ff2f8109120 2024-09-12T01:25:52.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.631+0000 7ff2ef400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2f81049a0 0x7ff2f819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:25:52.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.632+0000 7ff2ef400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2f81049a0 0x7ff2f819aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45218/0 (socket says 172.21.15.1:45218) 2024-09-12T01:25:52.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.632+0000 7ff2ef400640 1 -- 172.21.15.1:0/4201373069 learned_addr learned my addr 172.21.15.1:0/4201373069 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:25:52.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.632+0000 7ff2f7e00640 1 --2- 172.21.15.1:0/4201373069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2f8103ff0 0x7ff2f819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:25:52.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.632+0000 7ff2fd000640 1 --2- 172.21.15.1:0/4201373069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2f8109120 0x7ff2f819ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:25:52.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.632+0000 7ff2ef400640 1 -- 172.21.15.1:0/4201373069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2f8109120 msgr2=0x7ff2f819ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:25:52.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.632+0000 7ff2ef400640 1 --2- 172.21.15.1:0/4201373069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2f8109120 0x7ff2f819ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:52.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.632+0000 7ff2ef400640 1 -- 172.21.15.1:0/4201373069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2f8103ff0 msgr2=0x7ff2f819a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:25:52.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.632+0000 7ff2ef400640 1 --2- 172.21.15.1:0/4201373069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2f8103ff0 0x7ff2f819a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:52.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.632+0000 7ff2ef400640 1 -- 172.21.15.1:0/4201373069 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2f819f540 con 0x7ff2f81049a0 2024-09-12T01:25:52.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.632+0000 7ff2ef400640 1 --2- 172.21.15.1:0/4201373069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2f81049a0 0x7ff2f819aad0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff2e000e9f0 tx=0x7ff2e000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:25:52.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.633+0000 7ff2f5600640 1 -- 172.21.15.1:0/4201373069 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e000cd80 con 0x7ff2f81049a0 2024-09-12T01:25:52.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.633+0000 7ff2ff000640 1 -- 172.21.15.1:0/4201373069 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2f819f830 con 0x7ff2f81049a0 2024-09-12T01:25:52.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.633+0000 7ff2f5600640 1 -- 172.21.15.1:0/4201373069 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2e0010910 con 0x7ff2f81049a0 2024-09-12T01:25:52.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.633+0000 7ff2f5600640 1 -- 172.21.15.1:0/4201373069 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e00188f0 con 0x7ff2f81049a0 2024-09-12T01:25:52.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.633+0000 7ff2ff000640 1 -- 172.21.15.1:0/4201373069 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2f81a7110 con 0x7ff2f81049a0 2024-09-12T01:25:52.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.634+0000 7ff2ff000640 1 -- 172.21.15.1:0/4201373069 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2f81043d0 con 0x7ff2f81049a0 2024-09-12T01:25:52.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.636+0000 7ff2f5600640 1 -- 172.21.15.1:0/4201373069 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7ff2e0010ac0 con 0x7ff2f81049a0 2024-09-12T01:25:52.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.637+0000 7ff2f5600640 1 --2- 172.21.15.1:0/4201373069 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff2dc0a2b80 0x7ff2dc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:25:52.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.637+0000 7ff2f5600640 1 -- 172.21.15.1:0/4201373069 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7ff2e00cef50 con 0x7ff2f81049a0 2024-09-12T01:25:52.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.637+0000 7ff2f7e00640 1 --2- 172.21.15.1:0/4201373069 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff2dc0a2b80 0x7ff2dc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:25:52.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.638+0000 7ff2f7e00640 1 --2- 172.21.15.1:0/4201373069 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff2dc0a2b80 0x7ff2dc0a5040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff2e800bbe0 tx=0x7ff2e800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:25:52.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.641+0000 7ff2f5600640 1 -- 172.21.15.1:0/4201373069 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff2e009bc60 con 0x7ff2f81049a0 2024-09-12T01:25:52.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.831+0000 7ff2ff000640 1 -- 172.21.15.1:0/4201373069 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff2f81a7440 con 0x7ff2f81049a0 2024-09-12T01:25:52.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.832+0000 7ff2f5600640 1 -- 172.21.15.1:0/4201373069 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+18386 (secure 0 0 0) 0x7ff2e009ff60 con 0x7ff2f81049a0 2024-09-12T01:25:52.832 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:25:52.832 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":80,"fsid":"c0e4f180-70a4-11ef-bcea-c7b262605968","created":"2024-09-12T01:18:19.337406+0000","modified":"2024-09-12T01:25:51.603600+0000","last_up_change":"2024-09-12T01:25:50.601477+0000","last_in_change":"2024-09-12T01:25:27.674600+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:21:41.204242+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b9a396ae-bf97-462f-9ef9-b10ef6b9d337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6803","nonce":1125335136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6805","nonce":1125335136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6809","nonce":1125335136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6807","nonce":1125335136}]},"public_addr":"172.21.15.1:6803/1125335136","cluster_addr":"172.21.15.1:6805/1125335136","heartbeat_back_addr":"172.21.15.1:6809/1125335136","heartbeat_front_addr":"172.21.15.1:6807/1125335136","state":["exists","up"]},{"osd":1,"uuid":"dfe7a8a3-3700-4789-ac64-88660de9e0d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6801","nonce":3131049165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6803","nonce":3131049165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6807","nonce":3131049165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6805","nonce":3131049165}]},"public_addr":"172.21.15.53:6801/3131049165","cluster_addr":"172.21.15.53:6803/3131049165","heartbeat_back_addr":"172.21.15.53:6807/3131049165","heartbeat_front_addr":"172.21.15.53:6805/3131049165","state":["exists","up"]},{"osd":2,"uuid":"5d9d3603-ba7c-4115-9267-12abcb20d55b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6801","nonce":1133285636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6803","nonce":1133285636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6807","nonce":1133285636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6805","nonce":1133285636}]},"public_addr":"172.21.15.104:6801/1133285636","cluster_addr":"172.21.15.104:6803/1133285636","heartbeat_back_addr":"172.21.15.104:6807/1133285636","heartbeat_front_addr":"172.21.15.104:6805/1133285636","state":["exists","up"]},{"osd":3,"uuid":"1e063c66-41b4-4db1-a964-28e122ce628e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6811","nonce":1247468803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6813","nonce":1247468803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6817","nonce":1247468803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6815","nonce":1247468803}]},"public_addr":"172.21.15.1:6811/1247468803","cluster_addr":"172.21.15.1:6813/1247468803","heartbeat_back_addr":"172.21.15.1:6817/1247468803","heartbeat_front_addr":"172.21.15.1:6815/1247468803","state":["exists","up"]},{"osd":4,"uuid":"fe89ece0-2171-45d5-a06d-e400f77233e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6809","nonce":2673413639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6811","nonce":2673413639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6815","nonce":2673413639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6813","nonce":2673413639}]},"public_addr":"172.21.15.53:6809/2673413639","cluster_addr":"172.21.15.53:6811/2673413639","heartbeat_back_addr":"172.21.15.53:6815/2673413639","heartbeat_front_addr":"172.21.15.53:6813/2673413639","state":["exists","up"]},{"osd":5,"uuid":"daab795c-9942-4d16-aa34-8bba30a4b99b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6809","nonce":3131498790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6811","nonce":3131498790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6815","nonce":3131498790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6813","nonce":3131498790}]},"public_addr":"172.21.15.104:6809/3131498790","cluster_addr":"172.21.15.104:6811/3131498790","heartbeat_back_addr":"172.21.15.104:6815/3131498790","heartbeat_front_addr":"172.21.15.104:6813/3131498790","state":["exists","up"]},{"osd":6,"uuid":"f31d8633-e435-499b-9d46-0aa11a2481cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6819","nonce":3413067918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6821","nonce":3413067918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6825","nonce":3413067918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6823","nonce":3413067918}]},"public_addr":"172.21.15.1:6819/3413067918","cluster_addr":"172.21.15.1:6821/3413067918","heartbeat_back_addr":"172.21.15.1:6825/3413067918","heartbeat_front_addr":"172.21.15.1:6823/3413067918","state":["exists","up"]},{"osd":7,"uuid":"e64da49e-d31e-4801-b6cc-634a233b4761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6817","nonce":2841484136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6819","nonce":2841484136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6823","nonce":2841484136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6821","nonce":2841484136}]},"public_addr":"172.21.15.53:6817/2841484136","cluster_addr":"172.21.15.53:6819/2841484136","heartbeat_back_addr":"172.21.15.53:6823/2841484136","heartbeat_front_addr":"172.21.15.53:6821/2841484136","state":["exists","up"]},{"osd":8,"uuid":"fdfb5a42-dca7-4874-b479-176b4a191e21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6817","nonce":941453547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6819","nonce":941453547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6823","nonce":941453547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6821","nonce":941453547}]},"public_addr":"172.21.15.104:6817/941453547","cluster_addr":"172.21.15.104:6819/941453547","heartbeat_back_addr":"172.21.15.104:6823/941453547","heartbeat_front_addr":"172.21.15.104:6821/941453547","state":["exists","up"]},{"osd":9,"uuid":"7f81537a-cae5-4bb8-aec2-4eb07deb7958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6827","nonce":958287586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6829","nonce":958287586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6833","nonce":958287586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6831","nonce":958287586}]},"public_addr":"172.21.15.1:6827/958287586","cluster_addr":"172.21.15.1:6829/958287586","heartbeat_back_addr":"172.21.15.1:6833/958287586","heartbeat_front_addr":"172.21.15.1:6831/958287586","state":["exists","up"]},{"osd":10,"uuid":"e3c9c37d-46b9-4585-990d-69c447453bd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6825","nonce":3357909498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6827","nonce":3357909498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6831","nonce":3357909498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6829","nonce":3357909498}]},"public_addr":"172.21.15.53:6825/3357909498","cluster_addr":"172.21.15.53:6827/3357909498","heartbeat_back_addr":"172.21.15.53:6831/3357909498","heartbeat_front_addr":"172.21.15.53:6829/3357909498","state":["exists","up"]},{"osd":11,"uuid":"474b5345-f2ae-4e0e-a197-53ab11f58c04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6825","nonce":3381427987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6827","nonce":3381427987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6831","nonce":3381427987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6829","nonce":3381427987}]},"public_addr":"172.21.15.104:6825/3381427987","cluster_addr":"172.21.15.104:6827/3381427987","heartbeat_back_addr":"172.21.15.104:6831/3381427987","heartbeat_front_addr":"172.21.15.104:6829/3381427987","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:20:46.903181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:21:12.735556+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:21:37.868739+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:22:09.141228+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:22:36.438563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:23:02.220948+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:23:33.505881+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:24:00.982176+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:24:26.010444+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:24:56.711121+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:25:23.846495+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:25:48.815427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1373984726":"2024-09-13T01:19:14.580637+0000","172.21.15.1:0/1429963718":"2024-09-13T01:19:14.580637+0000","172.21.15.1:6801/4101281615":"2024-09-13T01:19:14.580637+0000","172.21.15.1:6800/4101281615":"2024-09-13T01:19:14.580637+0000","172.21.15.1:0/2564759586":"2024-09-13T01:19:14.580637+0000","172.21.15.1:6801/2935968453":"2024-09-13T01:18:45.882682+0000","172.21.15.1:6800/2935968453":"2024-09-13T01:18:45.882682+0000","172.21.15.1:0/1459848813":"2024-09-13T01:18:45.882682+0000","172.21.15.1:0/578605922":"2024-09-13T01:18:45.882682+0000","172.21.15.1:0/738035035":"2024-09-13T01:18:45.882682+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:25:52.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.834+0000 7ff2ff000640 1 -- 172.21.15.1:0/4201373069 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff2dc0a2b80 msgr2=0x7ff2dc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:25:52.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.834+0000 7ff2ff000640 1 --2- 172.21.15.1:0/4201373069 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff2dc0a2b80 0x7ff2dc0a5040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff2e800bbe0 tx=0x7ff2e800b390 comp rx=0 tx=0).stop 2024-09-12T01:25:52.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.834+0000 7ff2ff000640 1 -- 172.21.15.1:0/4201373069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2f81049a0 msgr2=0x7ff2f819aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:25:52.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.834+0000 7ff2ff000640 1 --2- 172.21.15.1:0/4201373069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2f81049a0 0x7ff2f819aad0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff2e000e9f0 tx=0x7ff2e000eeb0 comp rx=0 tx=0).stop 2024-09-12T01:25:52.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.834+0000 7ff2ff000640 1 -- 172.21.15.1:0/4201373069 shutdown_connections 2024-09-12T01:25:52.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.834+0000 7ff2ff000640 1 --2- 172.21.15.1:0/4201373069 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff2dc0a2b80 0x7ff2dc0a5040 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:52.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.834+0000 7ff2ff000640 1 --2- 172.21.15.1:0/4201373069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2f8109120 0x7ff2f819ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:52.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.834+0000 7ff2ff000640 1 --2- 172.21.15.1:0/4201373069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2f81049a0 0x7ff2f819aad0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:52.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.834+0000 7ff2ff000640 1 --2- 172.21.15.1:0/4201373069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2f8103ff0 0x7ff2f819a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:52.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.834+0000 7ff2ff000640 1 -- 172.21.15.1:0/4201373069 >> 172.21.15.1:0/4201373069 conn(0x7ff2f80ffa90 msgr2=0x7ff2f8101ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:25:52.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.835+0000 7ff2ff000640 1 -- 172.21.15.1:0/4201373069 shutdown_connections 2024-09-12T01:25:52.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:52.835+0000 7ff2ff000640 1 -- 172.21.15.1:0/4201373069 wait complete. 2024-09-12T01:25:53.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:52 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/4201373069' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:25:53.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:52 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/4201373069' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:25:53.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:52 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/4201373069' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:25:53.637 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T01:21:41.204242+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T01:25:53.637 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-12T01:25:54.012 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:25:54.188 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:53 smithi001 ceph-mon[30387]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:25:54.226 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:53 smithi104 ceph-mon[35219]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:25:54.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:53 smithi053 ceph-mon[35158]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:25:54.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.574+0000 7fdec8e00640 1 -- 172.21.15.1:0/2075279252 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdec4103ff0 msgr2=0x7fdec41043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:25:54.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.574+0000 7fdec8e00640 1 --2- 172.21.15.1:0/2075279252 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdec4103ff0 0x7fdec41043d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4009ca0 tx=0x7fdeb402a8c0 comp rx=0 tx=0).stop 2024-09-12T01:25:54.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.574+0000 7fdec8e00640 1 -- 172.21.15.1:0/2075279252 shutdown_connections 2024-09-12T01:25:54.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.574+0000 7fdec8e00640 1 --2- 172.21.15.1:0/2075279252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdec4109120 0x7fdec4110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:54.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.574+0000 7fdec8e00640 1 --2- 172.21.15.1:0/2075279252 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdec41049a0 0x7fdec41089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:54.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.574+0000 7fdec8e00640 1 --2- 172.21.15.1:0/2075279252 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdec4103ff0 0x7fdec41043d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:54.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.574+0000 7fdec8e00640 1 -- 172.21.15.1:0/2075279252 >> 172.21.15.1:0/2075279252 conn(0x7fdec40ffa90 msgr2=0x7fdec4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:25:54.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.574+0000 7fdec8e00640 1 -- 172.21.15.1:0/2075279252 shutdown_connections 2024-09-12T01:25:54.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.574+0000 7fdec8e00640 1 -- 172.21.15.1:0/2075279252 wait complete. 2024-09-12T01:25:54.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.575+0000 7fdec8e00640 1 Processor -- start 2024-09-12T01:25:54.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.575+0000 7fdec8e00640 1 -- start start 2024-09-12T01:25:54.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.575+0000 7fdec8e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdec4103ff0 0x7fdec419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:25:54.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.575+0000 7fdec8e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdec41049a0 0x7fdec419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:25:54.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.576+0000 7fdec8e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdec4109120 0x7fdec419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:25:54.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.576+0000 7fdec8e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdec4114190 con 0x7fdec41049a0 2024-09-12T01:25:54.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.576+0000 7fdec8e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdec4114010 con 0x7fdec4109120 2024-09-12T01:25:54.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.576+0000 7fdec8e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdec4114310 con 0x7fdec4103ff0 2024-09-12T01:25:54.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.576+0000 7fdeb9400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdec41049a0 0x7fdec419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:25:54.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.576+0000 7fdec1e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdec4103ff0 0x7fdec419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:25:54.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.576+0000 7fdeb9400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdec41049a0 0x7fdec419abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45244/0 (socket says 172.21.15.1:45244) 2024-09-12T01:25:54.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.576+0000 7fdec1e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdec4103ff0 0x7fdec419a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:36662/0 (socket says 172.21.15.1:36662) 2024-09-12T01:25:54.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.576+0000 7fdeb9400640 1 -- 172.21.15.1:0/2783298449 learned_addr learned my addr 172.21.15.1:0/2783298449 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:25:54.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.576+0000 7fdec2800640 1 --2- 172.21.15.1:0/2783298449 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdec4109120 0x7fdec419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:25:54.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.576+0000 7fdec2800640 1 -- 172.21.15.1:0/2783298449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdec4103ff0 msgr2=0x7fdec419a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:25:54.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.576+0000 7fdec2800640 1 --2- 172.21.15.1:0/2783298449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdec4103ff0 0x7fdec419a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:54.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.576+0000 7fdec2800640 1 -- 172.21.15.1:0/2783298449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdec41049a0 msgr2=0x7fdec419abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:25:54.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.576+0000 7fdec2800640 1 --2- 172.21.15.1:0/2783298449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdec41049a0 0x7fdec419abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:54.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.576+0000 7fdec2800640 1 -- 172.21.15.1:0/2783298449 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec419f640 con 0x7fdec4109120 2024-09-12T01:25:54.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.577+0000 7fdec2800640 1 --2- 172.21.15.1:0/2783298449 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdec4109120 0x7fdec419ef60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdebc00e970 tx=0x7fdebc00ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:25:54.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.577+0000 7fdebb400640 1 -- 172.21.15.1:0/2783298449 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdebc00ccc0 con 0x7fdec4109120 2024-09-12T01:25:54.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.577+0000 7fdec8e00640 1 -- 172.21.15.1:0/2783298449 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdec419f930 con 0x7fdec4109120 2024-09-12T01:25:54.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.577+0000 7fdebb400640 1 -- 172.21.15.1:0/2783298449 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdebc005020 con 0x7fdec4109120 2024-09-12T01:25:54.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.577+0000 7fdebb400640 1 -- 172.21.15.1:0/2783298449 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdebc004a00 con 0x7fdec4109120 2024-09-12T01:25:54.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.577+0000 7fdec8e00640 1 -- 172.21.15.1:0/2783298449 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdec41a7210 con 0x7fdec4109120 2024-09-12T01:25:54.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.578+0000 7fdec8e00640 1 -- 172.21.15.1:0/2783298449 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdec41043d0 con 0x7fdec4109120 2024-09-12T01:25:54.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.580+0000 7fdebb400640 1 -- 172.21.15.1:0/2783298449 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fdebc0051d0 con 0x7fdec4109120 2024-09-12T01:25:54.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.581+0000 7fdebb400640 1 --2- 172.21.15.1:0/2783298449 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fdea80a2b80 0x7fdea80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:25:54.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.581+0000 7fdebb400640 1 -- 172.21.15.1:0/2783298449 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7fdebc0ceb70 con 0x7fdec4109120 2024-09-12T01:25:54.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.582+0000 7fdec1e00640 1 --2- 172.21.15.1:0/2783298449 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fdea80a2b80 0x7fdea80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:25:54.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.582+0000 7fdec1e00640 1 --2- 172.21.15.1:0/2783298449 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fdea80a2b80 0x7fdea80a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4006010 tx=0x7fdeb4009820 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:25:54.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.585+0000 7fdebb400640 1 -- 172.21.15.1:0/2783298449 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdebc09bed0 con 0x7fdec4109120 2024-09-12T01:25:54.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.771+0000 7fdec8e00640 1 -- 172.21.15.1:0/2783298449 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdec402d7c0 con 0x7fdec4109120 2024-09-12T01:25:54.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.771+0000 7fdebb400640 1 -- 172.21.15.1:0/2783298449 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) ==== 93+0+10 (secure 0 0 0) 0x7fdebc09b7d0 con 0x7fdec4109120 2024-09-12T01:25:54.771 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-09-12T01:25:54.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.773+0000 7fdec8e00640 1 -- 172.21.15.1:0/2783298449 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fdea80a2b80 msgr2=0x7fdea80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:25:54.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.773+0000 7fdec8e00640 1 --2- 172.21.15.1:0/2783298449 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fdea80a2b80 0x7fdea80a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4006010 tx=0x7fdeb4009820 comp rx=0 tx=0).stop 2024-09-12T01:25:54.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.773+0000 7fdec8e00640 1 -- 172.21.15.1:0/2783298449 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdec4109120 msgr2=0x7fdec419ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:25:54.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.773+0000 7fdec8e00640 1 --2- 172.21.15.1:0/2783298449 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdec4109120 0x7fdec419ef60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdebc00e970 tx=0x7fdebc00ee30 comp rx=0 tx=0).stop 2024-09-12T01:25:54.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.773+0000 7fdec8e00640 1 -- 172.21.15.1:0/2783298449 shutdown_connections 2024-09-12T01:25:54.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.773+0000 7fdec8e00640 1 --2- 172.21.15.1:0/2783298449 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdec4109120 0x7fdec419ef60 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:54.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.773+0000 7fdec8e00640 1 --2- 172.21.15.1:0/2783298449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdec41049a0 0x7fdec419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:54.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.773+0000 7fdec8e00640 1 --2- 172.21.15.1:0/2783298449 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fdea80a2b80 0x7fdea80a5040 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:54.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.773+0000 7fdec8e00640 1 --2- 172.21.15.1:0/2783298449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdec4103ff0 0x7fdec419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:54.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.773+0000 7fdec8e00640 1 -- 172.21.15.1:0/2783298449 >> 172.21.15.1:0/2783298449 conn(0x7fdec40ffa90 msgr2=0x7fdec4100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:25:54.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.773+0000 7fdec8e00640 1 -- 172.21.15.1:0/2783298449 shutdown_connections 2024-09-12T01:25:54.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:54.774+0000 7fdec8e00640 1 -- 172.21.15.1:0/2783298449 wait complete. 2024-09-12T01:25:55.176 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:54 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/2783298449' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T01:25:55.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:54 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/2783298449' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T01:25:55.241 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:54 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/2783298449' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T01:25:55.550 INFO:tasks.cephadm:Adding mds.a on smithi001 2024-09-12T01:25:55.551 INFO:tasks.cephadm:Adding mds.d on smithi001 2024-09-12T01:25:55.551 INFO:tasks.cephadm:Adding mds.g on smithi001 2024-09-12T01:25:55.551 INFO:tasks.cephadm:Adding mds.j on smithi001 2024-09-12T01:25:55.551 INFO:tasks.cephadm:Adding mds.b on smithi053 2024-09-12T01:25:55.551 INFO:tasks.cephadm:Adding mds.e on smithi053 2024-09-12T01:25:55.551 INFO:tasks.cephadm:Adding mds.h on smithi053 2024-09-12T01:25:55.551 INFO:tasks.cephadm:Adding mds.k on smithi053 2024-09-12T01:25:55.551 INFO:tasks.cephadm:Adding mds.c on smithi104 2024-09-12T01:25:55.551 INFO:tasks.cephadm:Adding mds.f on smithi104 2024-09-12T01:25:55.551 INFO:tasks.cephadm:Adding mds.i on smithi104 2024-09-12T01:25:55.551 INFO:tasks.cephadm:Adding mds.l on smithi104 2024-09-12T01:25:55.552 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph orch apply mds all '12;smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi104=c;smithi104=f;smithi104=i;smithi104=l' 2024-09-12T01:25:55.806 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:25:56.194 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:55 smithi104 ceph-mon[35219]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:25:56.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:55 smithi053 ceph-mon[35158]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:25:56.285 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.277+0000 7fb755a8c640 1 -- 172.21.15.104:0/2250790293 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb750100ad0 msgr2=0x7fb750100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:25:56.286 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.277+0000 7fb755a8c640 1 --2- 172.21.15.104:0/2250790293 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb750100ad0 0x7fb750100ed0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb744009900 tx=0x7fb74402a410 comp rx=0 tx=0).stop 2024-09-12T01:25:56.286 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.277+0000 7fb755a8c640 1 -- 172.21.15.104:0/2250790293 shutdown_connections 2024-09-12T01:25:56.286 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.277+0000 7fb755a8c640 1 --2- 172.21.15.104:0/2250790293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb750102690 0x7fb750108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:56.286 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.277+0000 7fb755a8c640 1 --2- 172.21.15.104:0/2250790293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb750101cd0 0x7fb750102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:56.286 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.277+0000 7fb755a8c640 1 --2- 172.21.15.104:0/2250790293 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb750100ad0 0x7fb750100ed0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:56.286 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.277+0000 7fb755a8c640 1 -- 172.21.15.104:0/2250790293 >> 172.21.15.104:0/2250790293 conn(0x7fb7500fc280 msgr2=0x7fb7500fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:25:56.286 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.277+0000 7fb755a8c640 1 -- 172.21.15.104:0/2250790293 shutdown_connections 2024-09-12T01:25:56.286 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.277+0000 7fb755a8c640 1 -- 172.21.15.104:0/2250790293 wait complete. 2024-09-12T01:25:56.286 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.278+0000 7fb755a8c640 1 Processor -- start 2024-09-12T01:25:56.286 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.278+0000 7fb755a8c640 1 -- start start 2024-09-12T01:25:56.286 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.278+0000 7fb755a8c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb750100ad0 0x7fb75006ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:25:56.286 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.278+0000 7fb755a8c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb750101cd0 0x7fb75019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:25:56.287 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.278+0000 7fb755a8c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb750102690 0x7fb75019f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:25:56.287 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.278+0000 7fb755a8c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb75010c7d0 con 0x7fb750101cd0 2024-09-12T01:25:56.287 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.278+0000 7fb755a8c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb75010c650 con 0x7fb750102690 2024-09-12T01:25:56.287 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.278+0000 7fb755a8c640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb75010c950 con 0x7fb750100ad0 2024-09-12T01:25:56.287 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.278+0000 7fb74f7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb750100ad0 0x7fb75006ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:25:56.287 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.278+0000 7fb74f7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb750100ad0 0x7fb75006ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35826/0 (socket says 172.21.15.104:35826) 2024-09-12T01:25:56.287 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.278+0000 7fb74f7fe640 1 -- 172.21.15.104:0/1712479078 learned_addr learned my addr 172.21.15.104:0/1712479078 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-09-12T01:25:56.287 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.279+0000 7fb74effd640 1 --2- 172.21.15.104:0/1712479078 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb750101cd0 0x7fb75019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:25:56.287 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.279+0000 7fb74f7fe640 1 -- 172.21.15.104:0/1712479078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb750102690 msgr2=0x7fb75019f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:25:56.287 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.279+0000 7fb74ffff640 1 --2- 172.21.15.104:0/1712479078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb750102690 0x7fb75019f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:25:56.287 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.279+0000 7fb74f7fe640 1 --2- 172.21.15.104:0/1712479078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb750102690 0x7fb75019f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:56.287 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.279+0000 7fb74f7fe640 1 -- 172.21.15.104:0/1712479078 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb750101cd0 msgr2=0x7fb75019cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:25:56.288 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.279+0000 7fb74f7fe640 1 --2- 172.21.15.104:0/1712479078 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb750101cd0 0x7fb75019cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:56.288 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.279+0000 7fb74f7fe640 1 -- 172.21.15.104:0/1712479078 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb75019fba0 con 0x7fb750100ad0 2024-09-12T01:25:56.288 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.279+0000 7fb74effd640 1 --2- 172.21.15.104:0/1712479078 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb750101cd0 0x7fb75019cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:25:56.288 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.279+0000 7fb74ffff640 1 --2- 172.21.15.104:0/1712479078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb750102690 0x7fb75019f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:25:56.288 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.279+0000 7fb74f7fe640 1 --2- 172.21.15.104:0/1712479078 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb750100ad0 0x7fb75006ab10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb744031610 tx=0x7fb7440043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:25:56.288 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.279+0000 7fb74cff9640 1 -- 172.21.15.104:0/1712479078 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74403f030 con 0x7fb750100ad0 2024-09-12T01:25:56.288 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.279+0000 7fb755a8c640 1 -- 172.21.15.104:0/1712479078 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb75019fe30 con 0x7fb750100ad0 2024-09-12T01:25:56.288 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.279+0000 7fb74cff9640 1 -- 172.21.15.104:0/1712479078 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb744003ce0 con 0x7fb750100ad0 2024-09-12T01:25:56.288 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.279+0000 7fb74cff9640 1 -- 172.21.15.104:0/1712479078 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb744035070 con 0x7fb750100ad0 2024-09-12T01:25:56.288 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.279+0000 7fb755a8c640 1 -- 172.21.15.104:0/1712479078 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7501a0310 con 0x7fb750100ad0 2024-09-12T01:25:56.288 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.280+0000 7fb755a8c640 1 -- 172.21.15.104:0/1712479078 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb750107ed0 con 0x7fb750100ad0 2024-09-12T01:25:56.289 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.281+0000 7fb74cff9640 1 -- 172.21.15.104:0/1712479078 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fb744042030 con 0x7fb750100ad0 2024-09-12T01:25:56.289 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.282+0000 7fb74cff9640 1 --2- 172.21.15.104:0/1712479078 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fb7380a2b80 0x7fb7380a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:25:56.289 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.282+0000 7fb74cff9640 1 -- 172.21.15.104:0/1712479078 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7fb7440ef210 con 0x7fb750100ad0 2024-09-12T01:25:56.289 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.282+0000 7fb74effd640 1 --2- 172.21.15.104:0/1712479078 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fb7380a2b80 0x7fb7380a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:25:56.289 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.283+0000 7fb74effd640 1 --2- 172.21.15.104:0/1712479078 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fb7380a2b80 0x7fb7380a5040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb75006a890 tx=0x7fb740005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:25:56.289 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.286+0000 7fb74cff9640 1 -- 172.21.15.104:0/1712479078 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb7440bc610 con 0x7fb750100ad0 2024-09-12T01:25:56.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:55 smithi001 ceph-mon[30387]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:25:56.468 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.468+0000 7fb755a8c640 1 -- 172.21.15.104:0/1712479078 --> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi104=c;smithi104=f;smithi104=i;smithi104=l", "target": ["mon-mgr", ""]}) -- 0x7fb7500627f0 con 0x7fb7380a2b80 2024-09-12T01:25:56.477 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.476+0000 7fb74cff9640 1 -- 172.21.15.104:0/1712479078 <== mgr.14150 v2:172.21.15.1:6800/1928662456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fb7500627f0 con 0x7fb7380a2b80 2024-09-12T01:25:56.477 INFO:teuthology.orchestra.run.smithi104.stdout:Scheduled mds.all update... 2024-09-12T01:25:56.479 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.479+0000 7fb755a8c640 1 -- 172.21.15.104:0/1712479078 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fb7380a2b80 msgr2=0x7fb7380a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:25:56.479 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.479+0000 7fb755a8c640 1 --2- 172.21.15.104:0/1712479078 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fb7380a2b80 0x7fb7380a5040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb75006a890 tx=0x7fb740005e50 comp rx=0 tx=0).stop 2024-09-12T01:25:56.479 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.479+0000 7fb755a8c640 1 -- 172.21.15.104:0/1712479078 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb750100ad0 msgr2=0x7fb75006ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:25:56.480 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.479+0000 7fb755a8c640 1 --2- 172.21.15.104:0/1712479078 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb750100ad0 0x7fb75006ab10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb744031610 tx=0x7fb7440043b0 comp rx=0 tx=0).stop 2024-09-12T01:25:56.480 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.479+0000 7fb755a8c640 1 -- 172.21.15.104:0/1712479078 shutdown_connections 2024-09-12T01:25:56.480 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.479+0000 7fb755a8c640 1 --2- 172.21.15.104:0/1712479078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb750102690 0x7fb75019f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:56.480 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.479+0000 7fb755a8c640 1 --2- 172.21.15.104:0/1712479078 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb750101cd0 0x7fb75019cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:56.480 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.479+0000 7fb755a8c640 1 --2- 172.21.15.104:0/1712479078 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fb7380a2b80 0x7fb7380a5040 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:56.480 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.479+0000 7fb755a8c640 1 --2- 172.21.15.104:0/1712479078 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb750100ad0 0x7fb75006ab10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:56.480 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.479+0000 7fb755a8c640 1 -- 172.21.15.104:0/1712479078 >> 172.21.15.104:0/1712479078 conn(0x7fb7500fc280 msgr2=0x7fb7501050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:25:56.480 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.479+0000 7fb755a8c640 1 -- 172.21.15.104:0/1712479078 shutdown_connections 2024-09-12T01:25:56.480 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-12T01:25:56.479+0000 7fb755a8c640 1 -- 172.21.15.104:0/1712479078 wait complete. 2024-09-12T01:25:57.481 DEBUG:teuthology.orchestra.run.smithi001:mds.a> sudo journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mds.a.service 2024-09-12T01:25:57.486 DEBUG:teuthology.orchestra.run.smithi001:mds.d> sudo journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mds.d.service 2024-09-12T01:25:57.532 DEBUG:teuthology.orchestra.run.smithi001:mds.g> sudo journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mds.g.service 2024-09-12T01:25:57.577 DEBUG:teuthology.orchestra.run.smithi001:mds.j> sudo journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mds.j.service 2024-09-12T01:25:57.622 DEBUG:teuthology.orchestra.run.smithi053:mds.b> sudo journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mds.b.service 2024-09-12T01:25:57.624 DEBUG:teuthology.orchestra.run.smithi053:mds.e> sudo journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mds.e.service 2024-09-12T01:25:57.669 DEBUG:teuthology.orchestra.run.smithi053:mds.h> sudo journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mds.h.service 2024-09-12T01:25:57.712 DEBUG:teuthology.orchestra.run.smithi053:mds.k> sudo journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mds.k.service 2024-09-12T01:25:57.733 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:57 smithi053 ceph-mon[35158]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi104=c;smithi104=f;smithi104=i;smithi104=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:25:57.733 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:57 smithi053 ceph-mon[35158]: Saving service mds.all spec with placement smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi104=c;smithi104=f;smithi104=i;smithi104=l;count:12 2024-09-12T01:25:57.733 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:57 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:57.733 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:57 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:25:57.734 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:57 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:25:57.734 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:57 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:25:57.734 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:57 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:57.734 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:57 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:25:57.734 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:57 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:25:57.734 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:57 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:25:57.735 DEBUG:teuthology.orchestra.run.smithi104:mds.c> sudo journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mds.c.service 2024-09-12T01:25:57.737 DEBUG:teuthology.orchestra.run.smithi104:mds.f> sudo journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mds.f.service 2024-09-12T01:25:57.757 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:57 smithi104 ceph-mon[35219]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi104=c;smithi104=f;smithi104=i;smithi104=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:25:57.757 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:57 smithi104 ceph-mon[35219]: Saving service mds.all spec with placement smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi104=c;smithi104=f;smithi104=i;smithi104=l;count:12 2024-09-12T01:25:57.757 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:57 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:57.757 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:57 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:25:57.758 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:57 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:25:57.758 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:57 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:25:57.758 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:57 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:57.758 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:57 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:25:57.758 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:57 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:25:57.758 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:57 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:25:57.759 DEBUG:teuthology.orchestra.run.smithi104:mds.i> sudo journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mds.i.service 2024-09-12T01:25:57.802 DEBUG:teuthology.orchestra.run.smithi104:mds.l> sudo journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mds.l.service 2024-09-12T01:25:57.845 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-12T01:25:57.845 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:25:57.893 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:57 smithi001 ceph-mon[30387]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi104=c;smithi104=f;smithi104=i;smithi104=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:25:57.893 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:57 smithi001 ceph-mon[30387]: Saving service mds.all spec with placement smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi104=c;smithi104=f;smithi104=i;smithi104=l;count:12 2024-09-12T01:25:57.893 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:57 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:57.893 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:57 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:25:57.894 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:57 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:25:57.894 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:57 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:25:57.894 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:57 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:25:57.894 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:57 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:25:57.894 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:57 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:25:57.894 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:57 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:25:58.214 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:25:58.667 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:58 smithi001 ceph-mon[30387]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:25:58.667 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:58 smithi001 ceph-mon[30387]: Deploying daemon mds.l on smithi104 2024-09-12T01:25:58.713 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:58 smithi104 ceph-mon[35219]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:25:58.713 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:58 smithi104 ceph-mon[35219]: Deploying daemon mds.l on smithi104 2024-09-12T01:25:58.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:58 smithi053 ceph-mon[35158]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:25:58.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:58 smithi053 ceph-mon[35158]: Deploying daemon mds.l on smithi104 2024-09-12T01:25:58.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:58.780+0000 7f5448400640 1 -- 172.21.15.1:0/2336245439 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5440103ff0 msgr2=0x7f54401043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:25:58.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:58.780+0000 7f5448400640 1 --2- 172.21.15.1:0/2336245439 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5440103ff0 0x7f54401043d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5434009900 tx=0x7f543402a410 comp rx=0 tx=0).stop 2024-09-12T01:25:58.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:58.780+0000 7f5448400640 1 -- 172.21.15.1:0/2336245439 shutdown_connections 2024-09-12T01:25:58.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:58.780+0000 7f5448400640 1 --2- 172.21.15.1:0/2336245439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5440109120 0x7f5440110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:58.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:58.780+0000 7f5448400640 1 --2- 172.21.15.1:0/2336245439 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54401049a0 0x7f54401089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:58.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:58.780+0000 7f5448400640 1 --2- 172.21.15.1:0/2336245439 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5440103ff0 0x7f54401043d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:58.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:58.780+0000 7f5448400640 1 -- 172.21.15.1:0/2336245439 >> 172.21.15.1:0/2336245439 conn(0x7f54400ffa90 msgr2=0x7f5440101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:25:58.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:58.780+0000 7f5448400640 1 -- 172.21.15.1:0/2336245439 shutdown_connections 2024-09-12T01:25:58.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:58.780+0000 7f5448400640 1 -- 172.21.15.1:0/2336245439 wait complete. 2024-09-12T01:25:58.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:58.781+0000 7f5448400640 1 Processor -- start 2024-09-12T01:25:58.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:58.781+0000 7f5448400640 1 -- start start 2024-09-12T01:25:58.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:58.782+0000 7f5448400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5440103ff0 0x7f544019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:25:58.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:58.782+0000 7f5448400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54401049a0 0x7f544019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:25:58.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:58.782+0000 7f5448400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5440109120 0x7f544019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:25:58.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:58.782+0000 7f5448400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5440114170 con 0x7f54401049a0 2024-09-12T01:25:58.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:58.782+0000 7f5448400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5440113ff0 con 0x7f5440103ff0 2024-09-12T01:25:58.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:58.782+0000 7f5448400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f54401142f0 con 0x7f5440109120 2024-09-12T01:25:58.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:58.782+0000 7f5445000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54401049a0 0x7f544019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:25:58.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:58.782+0000 7f5446400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5440109120 0x7f544019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:25:58.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:58.782+0000 7f5446400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5440109120 0x7f544019ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:36674/0 (socket says 172.21.15.1:36674) 2024-09-12T01:25:58.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:58.782+0000 7f5446400640 1 -- 172.21.15.1:0/752936340 learned_addr learned my addr 172.21.15.1:0/752936340 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:25:58.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:58.782+0000 7f5445a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5440103ff0 0x7f544019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:25:58.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:58.783+0000 7f5446400640 1 -- 172.21.15.1:0/752936340 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5440103ff0 msgr2=0x7f544019a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:25:58.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:58.783+0000 7f5446400640 1 --2- 172.21.15.1:0/752936340 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5440103ff0 0x7f544019a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:58.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:58.783+0000 7f5446400640 1 -- 172.21.15.1:0/752936340 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54401049a0 msgr2=0x7f544019abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:25:58.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:58.783+0000 7f5446400640 1 --2- 172.21.15.1:0/752936340 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54401049a0 0x7f544019abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:58.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:58.783+0000 7f5446400640 1 -- 172.21.15.1:0/752936340 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f544019f620 con 0x7f5440109120 2024-09-12T01:25:58.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:58.783+0000 7f5446400640 1 --2- 172.21.15.1:0/752936340 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5440109120 0x7f544019ef40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f543800eb10 tx=0x7f543800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:25:58.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:58.783+0000 7f543e000640 1 -- 172.21.15.1:0/752936340 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5438004b20 con 0x7f5440109120 2024-09-12T01:25:58.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:58.783+0000 7f543e000640 1 -- 172.21.15.1:0/752936340 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5438004cc0 con 0x7f5440109120 2024-09-12T01:25:58.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:58.783+0000 7f5448400640 1 -- 172.21.15.1:0/752936340 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f544019f910 con 0x7f5440109120 2024-09-12T01:25:58.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:58.783+0000 7f543e000640 1 -- 172.21.15.1:0/752936340 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5438010760 con 0x7f5440109120 2024-09-12T01:25:58.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:58.783+0000 7f5448400640 1 -- 172.21.15.1:0/752936340 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54401a71f0 con 0x7f5440109120 2024-09-12T01:25:58.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:58.785+0000 7f5448400640 1 -- 172.21.15.1:0/752936340 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54401043d0 con 0x7f5440109120 2024-09-12T01:25:58.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:58.786+0000 7f543e000640 1 -- 172.21.15.1:0/752936340 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f5438005020 con 0x7f5440109120 2024-09-12T01:25:58.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:58.788+0000 7f543e000640 1 --2- 172.21.15.1:0/752936340 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f54280a2c00 0x7f54280a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:25:58.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:58.788+0000 7f543e000640 1 -- 172.21.15.1:0/752936340 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7f54380cf0d0 con 0x7f5440109120 2024-09-12T01:25:58.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:58.788+0000 7f5445a00640 1 --2- 172.21.15.1:0/752936340 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f54280a2c00 0x7f54280a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:25:58.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:58.789+0000 7f5445a00640 1 --2- 172.21.15.1:0/752936340 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f54280a2c00 0x7f54280a50c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5434000c00 tx=0x7f5434005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:25:58.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:58.792+0000 7f543e000640 1 -- 172.21.15.1:0/752936340 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f543809c430 con 0x7f5440109120 2024-09-12T01:25:59.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:59.016+0000 7f5448400640 1 -- 172.21.15.1:0/752936340 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f544010c300 con 0x7f5440109120 2024-09-12T01:25:59.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:59.017+0000 7f543e000640 1 -- 172.21.15.1:0/752936340 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f543809bd30 con 0x7f5440109120 2024-09-12T01:25:59.017 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:25:59.017 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":1,"btime":"2024-09-12T01:18:19:337176+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-12T01:25:59.017 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 1 2024-09-12T01:25:59.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:59.020+0000 7f5448400640 1 -- 172.21.15.1:0/752936340 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f54280a2c00 msgr2=0x7f54280a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:25:59.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:59.020+0000 7f5448400640 1 --2- 172.21.15.1:0/752936340 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f54280a2c00 0x7f54280a50c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5434000c00 tx=0x7f5434005b50 comp rx=0 tx=0).stop 2024-09-12T01:25:59.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:59.020+0000 7f5448400640 1 -- 172.21.15.1:0/752936340 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5440109120 msgr2=0x7f544019ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:25:59.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:59.020+0000 7f5448400640 1 --2- 172.21.15.1:0/752936340 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5440109120 0x7f544019ef40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f543800eb10 tx=0x7f543800c2d0 comp rx=0 tx=0).stop 2024-09-12T01:25:59.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:59.020+0000 7f5448400640 1 -- 172.21.15.1:0/752936340 shutdown_connections 2024-09-12T01:25:59.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:59.020+0000 7f5448400640 1 --2- 172.21.15.1:0/752936340 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f54280a2c00 0x7f54280a50c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:59.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:59.020+0000 7f5448400640 1 --2- 172.21.15.1:0/752936340 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5440109120 0x7f544019ef40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:59.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:59.020+0000 7f5448400640 1 --2- 172.21.15.1:0/752936340 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54401049a0 0x7f544019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:59.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:59.020+0000 7f5448400640 1 --2- 172.21.15.1:0/752936340 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5440103ff0 0x7f544019a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:25:59.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:59.020+0000 7f5448400640 1 -- 172.21.15.1:0/752936340 >> 172.21.15.1:0/752936340 conn(0x7f54400ffa90 msgr2=0x7f5440100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:25:59.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:59.020+0000 7f5448400640 1 -- 172.21.15.1:0/752936340 shutdown_connections 2024-09-12T01:25:59.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:25:59.020+0000 7f5448400640 1 -- 172.21.15.1:0/752936340 wait complete. 2024-09-12T01:25:59.324 INFO:journalctl@ceph.mds.l.smithi104.stdout:Sep 12 01:25:59 smithi104 systemd[1]: Starting Ceph mds.l for c0e4f180-70a4-11ef-bcea-c7b262605968... 2024-09-12T01:25:59.635 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:25:59 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/752936340' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:25:59.636 INFO:journalctl@ceph.mds.l.smithi104.stdout:Sep 12 01:25:59 smithi104 podman[64602]: 2024-09-12 01:25:59.374998701 +0000 UTC m=+0.106108030 container create 53a28893b85e8d057fc1c1d89ec9d0ff54aaf2830560fbca9bea04e6feb740e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-l, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-12T01:25:59.636 INFO:journalctl@ceph.mds.l.smithi104.stdout:Sep 12 01:25:59 smithi104 podman[64602]: 2024-09-12 01:25:59.284908468 +0000 UTC m=+0.016017805 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:25:59.636 INFO:journalctl@ceph.mds.l.smithi104.stdout:Sep 12 01:25:59 smithi104 podman[64602]: 2024-09-12 01:25:59.611288141 +0000 UTC m=+0.342397475 container init 53a28893b85e8d057fc1c1d89ec9d0ff54aaf2830560fbca9bea04e6feb740e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-l, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-12T01:25:59.636 INFO:journalctl@ceph.mds.l.smithi104.stdout:Sep 12 01:25:59 smithi104 podman[64602]: 2024-09-12 01:25:59.614113895 +0000 UTC m=+0.345223224 container start 53a28893b85e8d057fc1c1d89ec9d0ff54aaf2830560fbca9bea04e6feb740e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-l, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-12T01:25:59.636 INFO:journalctl@ceph.mds.l.smithi104.stdout:Sep 12 01:25:59 smithi104 ceph-mds[64621]: -- 172.21.15.104:0/2978619368 <== mon.1 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563319baf860 con 0x563319b8d400 2024-09-12T01:25:59.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:25:59 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/752936340' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:25:59.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:25:59 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/752936340' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:26:00.074 INFO:journalctl@ceph.mds.l.smithi104.stdout:Sep 12 01:25:59 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-l[64617]: starting mds.l at 2024-09-12T01:26:00.075 INFO:journalctl@ceph.mds.l.smithi104.stdout:Sep 12 01:25:59 smithi104 bash[64602]: 53a28893b85e8d057fc1c1d89ec9d0ff54aaf2830560fbca9bea04e6feb740e1 2024-09-12T01:26:00.075 INFO:journalctl@ceph.mds.l.smithi104.stdout:Sep 12 01:25:59 smithi104 systemd[1]: Started Ceph mds.l for c0e4f180-70a4-11ef-bcea-c7b262605968. 2024-09-12T01:26:00.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:00 smithi053 ceph-mon[35158]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:00.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:00 smithi053 ceph-mon[35158]: mds.? [v2:172.21.15.104:6832/2071825633,v1:172.21.15.104:6833/2071825633] up:boot 2024-09-12T01:26:00.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:00 smithi053 ceph-mon[35158]: fsmap 1 up:standby 2024-09-12T01:26:00.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:00 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T01:26:00.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:00 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:00.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:00 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:00.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:00 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:00.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:00 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:26:00.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:00 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:26:00.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:00 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:26:00.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:00 smithi001 ceph-mon[30387]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:00.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:00 smithi001 ceph-mon[30387]: mds.? [v2:172.21.15.104:6832/2071825633,v1:172.21.15.104:6833/2071825633] up:boot 2024-09-12T01:26:00.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:00 smithi001 ceph-mon[30387]: fsmap 1 up:standby 2024-09-12T01:26:00.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:00 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T01:26:00.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:00 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:00.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:00 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:00.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:00 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:00.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:00 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:26:00.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:00 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:26:00.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:00 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:26:00.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:00 smithi104 ceph-mon[35219]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:00.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:00 smithi104 ceph-mon[35219]: mds.? [v2:172.21.15.104:6832/2071825633,v1:172.21.15.104:6833/2071825633] up:boot 2024-09-12T01:26:00.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:00 smithi104 ceph-mon[35219]: fsmap 1 up:standby 2024-09-12T01:26:00.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:00 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T01:26:00.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:00 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:00.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:00 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:00.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:00 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:00.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:00 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:26:00.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:00 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:26:00.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:00 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:26:01.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:01 smithi053 ceph-mon[35158]: Deploying daemon mds.k on smithi053 2024-09-12T01:26:01.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:01 smithi104 ceph-mon[35219]: Deploying daemon mds.k on smithi053 2024-09-12T01:26:01.829 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:26:01.885 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:01 smithi001 ceph-mon[30387]: Deploying daemon mds.k on smithi053 2024-09-12T01:26:02.148 INFO:journalctl@ceph.mds.k.smithi053.stdout:Sep 12 01:26:01 smithi053 systemd[1]: Starting Ceph mds.k for c0e4f180-70a4-11ef-bcea-c7b262605968... 2024-09-12T01:26:02.218 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:26:02.453 INFO:journalctl@ceph.mds.k.smithi053.stdout:Sep 12 01:26:02 smithi053 podman[64554]: 2024-09-12 01:26:02.051952538 +0000 UTC m=+0.015966107 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:26:02.454 INFO:journalctl@ceph.mds.k.smithi053.stdout:Sep 12 01:26:02 smithi053 podman[64554]: 2024-09-12 01:26:02.17755031 +0000 UTC m=+0.141563913 container create cc65a2bf2316dce33bd40d309b2bd67232e80e947fd0ad9cfe4c0e820739629e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-k, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T01:26:02.454 INFO:journalctl@ceph.mds.k.smithi053.stdout:Sep 12 01:26:02 smithi053 podman[64554]: 2024-09-12 01:26:02.45527111 +0000 UTC m=+0.419284668 container init cc65a2bf2316dce33bd40d309b2bd67232e80e947fd0ad9cfe4c0e820739629e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-k, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-12T01:26:02.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:02 smithi001 ceph-mon[30387]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:02.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:02 smithi053 ceph-mon[35158]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:02.740 INFO:journalctl@ceph.mds.k.smithi053.stdout:Sep 12 01:26:02 smithi053 podman[64554]: 2024-09-12 01:26:02.45817271 +0000 UTC m=+0.422186268 container start cc65a2bf2316dce33bd40d309b2bd67232e80e947fd0ad9cfe4c0e820739629e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-k, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3) 2024-09-12T01:26:02.740 INFO:journalctl@ceph.mds.k.smithi053.stdout:Sep 12 01:26:02 smithi053 ceph-mds[64574]: -- 172.21.15.53:0/1073515319 <== mon.2 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557bceb41860 con 0x557bceb1f400 2024-09-12T01:26:02.740 INFO:journalctl@ceph.mds.k.smithi053.stdout:Sep 12 01:26:02 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-k[64570]: starting mds.k at 2024-09-12T01:26:02.740 INFO:journalctl@ceph.mds.k.smithi053.stdout:Sep 12 01:26:02 smithi053 bash[64554]: cc65a2bf2316dce33bd40d309b2bd67232e80e947fd0ad9cfe4c0e820739629e 2024-09-12T01:26:02.741 INFO:journalctl@ceph.mds.k.smithi053.stdout:Sep 12 01:26:02 smithi053 systemd[1]: Started Ceph mds.k for c0e4f180-70a4-11ef-bcea-c7b262605968. 2024-09-12T01:26:02.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:02.816+0000 7f1c33a00640 1 -- 172.21.15.1:0/2138206042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c2c074ee0 msgr2=0x7f1c2c10def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:02.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:02.816+0000 7f1c33a00640 1 --2- 172.21.15.1:0/2138206042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c2c074ee0 0x7f1c2c10def0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1c18009860 tx=0x7f1c1802a3d0 comp rx=0 tx=0).stop 2024-09-12T01:26:02.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:02.816+0000 7f1c33a00640 1 -- 172.21.15.1:0/2138206042 shutdown_connections 2024-09-12T01:26:02.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:02.816+0000 7f1c33a00640 1 --2- 172.21.15.1:0/2138206042 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c2c10e430 0x7f1c2c110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:02.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:02.816+0000 7f1c33a00640 1 --2- 172.21.15.1:0/2138206042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c2c074ee0 0x7f1c2c10def0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:02.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:02.816+0000 7f1c33a00640 1 --2- 172.21.15.1:0/2138206042 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c2c0745c0 0x7f1c2c0749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:02.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:02.816+0000 7f1c33a00640 1 -- 172.21.15.1:0/2138206042 >> 172.21.15.1:0/2138206042 conn(0x7f1c2c0fd8b0 msgr2=0x7f1c2c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:26:02.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:02.817+0000 7f1c33a00640 1 -- 172.21.15.1:0/2138206042 shutdown_connections 2024-09-12T01:26:02.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:02.817+0000 7f1c33a00640 1 -- 172.21.15.1:0/2138206042 wait complete. 2024-09-12T01:26:02.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:02.818+0000 7f1c33a00640 1 Processor -- start 2024-09-12T01:26:02.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:02.818+0000 7f1c33a00640 1 -- start start 2024-09-12T01:26:02.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:02.818+0000 7f1c33a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c2c0745c0 0x7f1c2c19a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:02.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:02.818+0000 7f1c33a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c2c074ee0 0x7f1c2c19abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:02.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:02.818+0000 7f1c33a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c2c10e430 0x7f1c2c19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:02.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:02.818+0000 7f1c33a00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2c1141c0 con 0x7f1c2c10e430 2024-09-12T01:26:02.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:02.818+0000 7f1c33a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2c114040 con 0x7f1c2c0745c0 2024-09-12T01:26:02.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:02.818+0000 7f1c33a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2c114340 con 0x7f1c2c074ee0 2024-09-12T01:26:02.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:02.818+0000 7f1c31000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c2c0745c0 0x7f1c2c19a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:02.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:02.818+0000 7f1c31a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c2c10e430 0x7f1c2c19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:02.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:02.818+0000 7f1c31000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c2c0745c0 0x7f1c2c19a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:47124/0 (socket says 172.21.15.1:47124) 2024-09-12T01:26:02.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:02.818+0000 7f1c31000640 1 -- 172.21.15.1:0/3760554862 learned_addr learned my addr 172.21.15.1:0/3760554862 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:26:02.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:02.819+0000 7f1c2be00640 1 --2- 172.21.15.1:0/3760554862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c2c074ee0 0x7f1c2c19abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:02.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:02.819+0000 7f1c31000640 1 -- 172.21.15.1:0/3760554862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c2c074ee0 msgr2=0x7f1c2c19abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:02.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:02.819+0000 7f1c31000640 1 --2- 172.21.15.1:0/3760554862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c2c074ee0 0x7f1c2c19abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:02.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:02.819+0000 7f1c31000640 1 -- 172.21.15.1:0/3760554862 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c2c10e430 msgr2=0x7f1c2c19ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:02.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:02.819+0000 7f1c31000640 1 --2- 172.21.15.1:0/3760554862 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c2c10e430 0x7f1c2c19ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:02.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:02.819+0000 7f1c31000640 1 -- 172.21.15.1:0/3760554862 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c2c19f630 con 0x7f1c2c0745c0 2024-09-12T01:26:02.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:02.819+0000 7f1c31000640 1 --2- 172.21.15.1:0/3760554862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c2c0745c0 0x7f1c2c19a680 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1c2000e9f0 tx=0x7f1c2000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:26:02.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:02.820+0000 7f1c29600640 1 -- 172.21.15.1:0/3760554862 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2000cd80 con 0x7f1c2c0745c0 2024-09-12T01:26:02.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:02.820+0000 7f1c33a00640 1 -- 172.21.15.1:0/3760554862 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c2c19f920 con 0x7f1c2c0745c0 2024-09-12T01:26:02.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:02.820+0000 7f1c33a00640 1 -- 172.21.15.1:0/3760554862 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c2c1a7200 con 0x7f1c2c0745c0 2024-09-12T01:26:02.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:02.820+0000 7f1c29600640 1 -- 172.21.15.1:0/3760554862 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c20010910 con 0x7f1c2c0745c0 2024-09-12T01:26:02.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:02.820+0000 7f1c29600640 1 -- 172.21.15.1:0/3760554862 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c20018980 con 0x7f1c2c0745c0 2024-09-12T01:26:02.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:02.821+0000 7f1c33a00640 1 -- 172.21.15.1:0/3760554862 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c2c075c70 con 0x7f1c2c0745c0 2024-09-12T01:26:02.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:02.823+0000 7f1c29600640 1 -- 172.21.15.1:0/3760554862 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f1c20010ac0 con 0x7f1c2c0745c0 2024-09-12T01:26:02.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:02.824+0000 7f1c29600640 1 --2- 172.21.15.1:0/3760554862 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1c140a2b80 0x7f1c140a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:02.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:02.824+0000 7f1c29600640 1 -- 172.21.15.1:0/3760554862 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7f1c200cf310 con 0x7f1c2c0745c0 2024-09-12T01:26:02.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:02.824+0000 7f1c2be00640 1 --2- 172.21.15.1:0/3760554862 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1c140a2b80 0x7f1c140a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:02.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:02 smithi104 ceph-mon[35219]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:02.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:02.825+0000 7f1c2be00640 1 --2- 172.21.15.1:0/3760554862 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1c140a2b80 0x7f1c140a5040 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c0ffe40 tx=0x7f1c18005c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:26:02.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:02.826+0000 7f1c29600640 1 -- 172.21.15.1:0/3760554862 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1c2009c020 con 0x7f1c2c0745c0 2024-09-12T01:26:03.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:03.060+0000 7f1c33a00640 1 -- 172.21.15.1:0/3760554862 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c2c1095b0 con 0x7f1c2c0745c0 2024-09-12T01:26:03.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:03.060+0000 7f1c29600640 1 -- 172.21.15.1:0/3760554862 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2263 (secure 0 0 0) 0x7f1c200a0320 con 0x7f1c2c0745c0 2024-09-12T01:26:03.060 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:26:03.060 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":3,"btime":"2024-09-12T01:26:02:495700+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/2071825633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2071825633},{"type":"v1","addr":"172.21.15.104:6833","nonce":2071825633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2930310711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2930310711},{"type":"v1","addr":"172.21.15.53:6833","nonce":2930310711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-12T01:26:03.060 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 3 2024-09-12T01:26:03.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:03.063+0000 7f1c33a00640 1 -- 172.21.15.1:0/3760554862 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1c140a2b80 msgr2=0x7f1c140a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:03.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:03.063+0000 7f1c33a00640 1 --2- 172.21.15.1:0/3760554862 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1c140a2b80 0x7f1c140a5040 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c0ffe40 tx=0x7f1c18005c90 comp rx=0 tx=0).stop 2024-09-12T01:26:03.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:03.063+0000 7f1c33a00640 1 -- 172.21.15.1:0/3760554862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c2c0745c0 msgr2=0x7f1c2c19a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:03.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:03.063+0000 7f1c33a00640 1 --2- 172.21.15.1:0/3760554862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c2c0745c0 0x7f1c2c19a680 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1c2000e9f0 tx=0x7f1c2000eeb0 comp rx=0 tx=0).stop 2024-09-12T01:26:03.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:03.063+0000 7f1c33a00640 1 -- 172.21.15.1:0/3760554862 shutdown_connections 2024-09-12T01:26:03.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:03.063+0000 7f1c33a00640 1 --2- 172.21.15.1:0/3760554862 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c2c10e430 0x7f1c2c19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:03.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:03.063+0000 7f1c33a00640 1 --2- 172.21.15.1:0/3760554862 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1c140a2b80 0x7f1c140a5040 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:03.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:03.063+0000 7f1c33a00640 1 --2- 172.21.15.1:0/3760554862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c2c074ee0 0x7f1c2c19abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:03.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:03.063+0000 7f1c33a00640 1 --2- 172.21.15.1:0/3760554862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c2c0745c0 0x7f1c2c19a680 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:03.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:03.063+0000 7f1c33a00640 1 -- 172.21.15.1:0/3760554862 >> 172.21.15.1:0/3760554862 conn(0x7f1c2c0fd8b0 msgr2=0x7f1c2c0fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:26:03.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:03.063+0000 7f1c33a00640 1 -- 172.21.15.1:0/3760554862 shutdown_connections 2024-09-12T01:26:03.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:03.063+0000 7f1c33a00640 1 -- 172.21.15.1:0/3760554862 wait complete. 2024-09-12T01:26:03.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:03 smithi104 ceph-mon[35219]: mds.? [v2:172.21.15.53:6832/2930310711,v1:172.21.15.53:6833/2930310711] up:boot 2024-09-12T01:26:03.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:03 smithi104 ceph-mon[35219]: fsmap 2 up:standby 2024-09-12T01:26:03.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:03 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T01:26:03.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:03 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:03.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:03 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:03.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:03 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:03.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:03 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:26:03.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:03 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:26:03.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:03 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:26:03.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:03 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/3760554862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:26:03.882 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:03 smithi001 ceph-mon[30387]: mds.? [v2:172.21.15.53:6832/2930310711,v1:172.21.15.53:6833/2930310711] up:boot 2024-09-12T01:26:03.883 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:03 smithi001 ceph-mon[30387]: fsmap 2 up:standby 2024-09-12T01:26:03.883 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:03 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T01:26:03.883 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:03 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:03.883 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:03 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:03.883 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:03 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:03.883 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:03 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:26:03.883 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:03 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:26:03.883 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:03 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:26:03.883 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:03 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/3760554862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:26:03.904 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:03 smithi053 ceph-mon[35158]: mds.? [v2:172.21.15.53:6832/2930310711,v1:172.21.15.53:6833/2930310711] up:boot 2024-09-12T01:26:03.905 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:03 smithi053 ceph-mon[35158]: fsmap 2 up:standby 2024-09-12T01:26:03.905 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:03 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T01:26:03.905 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:03 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:03.905 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:03 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:03.905 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:03 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:03.905 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:03 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:26:03.906 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:03 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:26:03.906 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:03 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:26:03.906 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:03 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/3760554862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:26:04.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:04 smithi001 ceph-mon[30387]: Deploying daemon mds.j on smithi001 2024-09-12T01:26:04.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:04 smithi001 ceph-mon[30387]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:04.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:04 smithi104 ceph-mon[35219]: Deploying daemon mds.j on smithi001 2024-09-12T01:26:04.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:04 smithi104 ceph-mon[35219]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:04.880 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:04 smithi053 ceph-mon[35158]: Deploying daemon mds.j on smithi001 2024-09-12T01:26:04.881 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:04 smithi053 ceph-mon[35158]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:05.925 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:26:06.562 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:06 smithi001 ceph-mon[30387]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:06.786 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:26:06.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:06 smithi104 ceph-mon[35219]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:06.927 INFO:journalctl@ceph.mds.j.smithi001.stdout:Sep 12 01:26:06 smithi001 systemd[1]: Started Ceph mds.j for c0e4f180-70a4-11ef-bcea-c7b262605968. 2024-09-12T01:26:06.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:06 smithi053 ceph-mon[35158]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:07.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.417+0000 7fcfce400640 1 -- 172.21.15.1:0/4163824407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfc81012a0 msgr2=0x7fcfc8105770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:07.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.417+0000 7fcfce400640 1 --2- 172.21.15.1:0/4163824407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfc81012a0 0x7fcfc8105770 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4009900 tx=0x7fcfb402a470 comp rx=0 tx=0).stop 2024-09-12T01:26:07.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.417+0000 7fcfce400640 1 -- 172.21.15.1:0/4163824407 shutdown_connections 2024-09-12T01:26:07.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.417+0000 7fcfce400640 1 --2- 172.21.15.1:0/4163824407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfc8105d90 0x7fcfc8106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:07.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.417+0000 7fcfce400640 1 --2- 172.21.15.1:0/4163824407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfc81012a0 0x7fcfc8105770 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:07.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.417+0000 7fcfce400640 1 --2- 172.21.15.1:0/4163824407 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcfc8100980 0x7fcfc8100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:07.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.417+0000 7fcfce400640 1 -- 172.21.15.1:0/4163824407 >> 172.21.15.1:0/4163824407 conn(0x7fcfc80fc610 msgr2=0x7fcfc80fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:26:07.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.417+0000 7fcfce400640 1 -- 172.21.15.1:0/4163824407 shutdown_connections 2024-09-12T01:26:07.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.417+0000 7fcfce400640 1 -- 172.21.15.1:0/4163824407 wait complete. 2024-09-12T01:26:07.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.418+0000 7fcfce400640 1 Processor -- start 2024-09-12T01:26:07.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.418+0000 7fcfce400640 1 -- start start 2024-09-12T01:26:07.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.418+0000 7fcfce400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfc8100980 0x7fcfc81a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:07.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.418+0000 7fcfce400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfc81012a0 0x7fcfc81a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:07.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.419+0000 7fcfce400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcfc8105d90 0x7fcfc819e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:07.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.419+0000 7fcfce400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc81183e0 con 0x7fcfc8100980 2024-09-12T01:26:07.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.419+0000 7fcfce400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc8118260 con 0x7fcfc81012a0 2024-09-12T01:26:07.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.419+0000 7fcfce400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc8118560 con 0x7fcfc8105d90 2024-09-12T01:26:07.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.419+0000 7fcfcd000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfc8100980 0x7fcfc81a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:07.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.419+0000 7fcfc7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfc81012a0 0x7fcfc81a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:07.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.419+0000 7fcfc7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfc81012a0 0x7fcfc81a4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:47164/0 (socket says 172.21.15.1:47164) 2024-09-12T01:26:07.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.419+0000 7fcfcd000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfc8100980 0x7fcfc81a4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33744/0 (socket says 172.21.15.1:33744) 2024-09-12T01:26:07.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.419+0000 7fcfc7e00640 1 -- 172.21.15.1:0/1127628659 learned_addr learned my addr 172.21.15.1:0/1127628659 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:26:07.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.419+0000 7fcfcda00640 1 --2- 172.21.15.1:0/1127628659 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcfc8105d90 0x7fcfc819e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:07.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.419+0000 7fcfcd000640 1 -- 172.21.15.1:0/1127628659 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcfc8105d90 msgr2=0x7fcfc819e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:07.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.419+0000 7fcfcd000640 1 --2- 172.21.15.1:0/1127628659 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcfc8105d90 0x7fcfc819e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:07.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.419+0000 7fcfcd000640 1 -- 172.21.15.1:0/1127628659 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfc81012a0 msgr2=0x7fcfc81a4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:07.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.419+0000 7fcfcd000640 1 --2- 172.21.15.1:0/1127628659 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfc81012a0 0x7fcfc81a4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:07.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.419+0000 7fcfcd000640 1 -- 172.21.15.1:0/1127628659 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc819efa0 con 0x7fcfc8100980 2024-09-12T01:26:07.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.420+0000 7fcfcd000640 1 --2- 172.21.15.1:0/1127628659 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfc8100980 0x7fcfc81a4660 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc00d6d0 tx=0x7fcfbc00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:26:07.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.420+0000 7fcfc5600640 1 -- 172.21.15.1:0/1127628659 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc017070 con 0x7fcfc8100980 2024-09-12T01:26:07.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.420+0000 7fcfce400640 1 -- 172.21.15.1:0/1127628659 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfc819f290 con 0x7fcfc8100980 2024-09-12T01:26:07.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.420+0000 7fcfc5600640 1 -- 172.21.15.1:0/1127628659 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfbc009820 con 0x7fcfc8100980 2024-09-12T01:26:07.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.420+0000 7fcfc5600640 1 -- 172.21.15.1:0/1127628659 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc004cd0 con 0x7fcfc8100980 2024-09-12T01:26:07.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.420+0000 7fcfce400640 1 -- 172.21.15.1:0/1127628659 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfc81ab5e0 con 0x7fcfc8100980 2024-09-12T01:26:07.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.421+0000 7fcfce400640 1 -- 172.21.15.1:0/1127628659 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfc806b070 con 0x7fcfc8100980 2024-09-12T01:26:07.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.423+0000 7fcfc5600640 1 -- 172.21.15.1:0/1127628659 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fcfbc0099d0 con 0x7fcfc8100980 2024-09-12T01:26:07.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.424+0000 7fcfc5600640 1 --2- 172.21.15.1:0/1127628659 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fcfb00a2b80 0x7fcfb00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:07.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.425+0000 7fcfc5600640 1 -- 172.21.15.1:0/1127628659 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7fcfbc0ceb50 con 0x7fcfc8100980 2024-09-12T01:26:07.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.425+0000 7fcfc7e00640 1 --2- 172.21.15.1:0/1127628659 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fcfb00a2b80 0x7fcfb00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:07.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.426+0000 7fcfc7e00640 1 --2- 172.21.15.1:0/1127628659 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fcfb00a2b80 0x7fcfb00a5040 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcfb40096c0 tx=0x7fcfb40046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:26:07.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.427+0000 7fcfc5600640 1 -- 172.21.15.1:0/1127628659 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcfbc09b860 con 0x7fcfc8100980 2024-09-12T01:26:07.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.684+0000 7fcfce400640 1 -- 172.21.15.1:0/1127628659 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcfc819fc60 con 0x7fcfc8100980 2024-09-12T01:26:07.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.685+0000 7fcfc5600640 1 -- 172.21.15.1:0/1127628659 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3082 (secure 0 0 0) 0x7fcfbc09fb60 con 0x7fcfc8100980 2024-09-12T01:26:07.685 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:26:07.685 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":4,"btime":"2024-09-12T01:26:07:506957+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14430,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/572192306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":572192306},{"type":"v1","addr":"172.21.15.1:6835","nonce":572192306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/2071825633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2071825633},{"type":"v1","addr":"172.21.15.104:6833","nonce":2071825633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2930310711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2930310711},{"type":"v1","addr":"172.21.15.53:6833","nonce":2930310711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-12T01:26:07.685 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 4 2024-09-12T01:26:07.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.687+0000 7fcfce400640 1 -- 172.21.15.1:0/1127628659 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fcfb00a2b80 msgr2=0x7fcfb00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:07.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.687+0000 7fcfce400640 1 --2- 172.21.15.1:0/1127628659 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fcfb00a2b80 0x7fcfb00a5040 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcfb40096c0 tx=0x7fcfb40046c0 comp rx=0 tx=0).stop 2024-09-12T01:26:07.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.687+0000 7fcfce400640 1 -- 172.21.15.1:0/1127628659 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfc8100980 msgr2=0x7fcfc81a4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:07.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.687+0000 7fcfce400640 1 --2- 172.21.15.1:0/1127628659 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfc8100980 0x7fcfc81a4660 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc00d6d0 tx=0x7fcfbc00db90 comp rx=0 tx=0).stop 2024-09-12T01:26:07.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.688+0000 7fcfce400640 1 -- 172.21.15.1:0/1127628659 shutdown_connections 2024-09-12T01:26:07.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.688+0000 7fcfce400640 1 --2- 172.21.15.1:0/1127628659 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fcfb00a2b80 0x7fcfb00a5040 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:07.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.688+0000 7fcfce400640 1 --2- 172.21.15.1:0/1127628659 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcfc8105d90 0x7fcfc819e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:07.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.688+0000 7fcfce400640 1 --2- 172.21.15.1:0/1127628659 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfc81012a0 0x7fcfc81a4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:07.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.688+0000 7fcfce400640 1 --2- 172.21.15.1:0/1127628659 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfc8100980 0x7fcfc81a4660 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:07.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.688+0000 7fcfce400640 1 -- 172.21.15.1:0/1127628659 >> 172.21.15.1:0/1127628659 conn(0x7fcfc80fc610 msgr2=0x7fcfc8103710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:26:07.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.688+0000 7fcfce400640 1 -- 172.21.15.1:0/1127628659 shutdown_connections 2024-09-12T01:26:07.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:07.688+0000 7fcfce400640 1 -- 172.21.15.1:0/1127628659 wait complete. 2024-09-12T01:26:07.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:07 smithi001 ceph-mon[30387]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:07.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:07 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:07.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:07 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:07.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:07 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:07.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:07 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:26:07.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:07 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:26:07.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:07 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:26:07.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:07 smithi001 ceph-mon[30387]: Deploying daemon mds.d on smithi001 2024-09-12T01:26:07.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:07 smithi001 ceph-mon[30387]: mds.? [v2:172.21.15.1:6834/572192306,v1:172.21.15.1:6835/572192306] up:boot 2024-09-12T01:26:07.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:07 smithi001 ceph-mon[30387]: fsmap 3 up:standby 2024-09-12T01:26:07.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:07 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T01:26:07.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:07 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/1127628659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:26:08.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:07 smithi104 ceph-mon[35219]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:08.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:07 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:08.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:07 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:08.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:07 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:08.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:07 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:26:08.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:07 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:26:08.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:07 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:26:08.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:07 smithi104 ceph-mon[35219]: Deploying daemon mds.d on smithi001 2024-09-12T01:26:08.076 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:07 smithi104 ceph-mon[35219]: mds.? [v2:172.21.15.1:6834/572192306,v1:172.21.15.1:6835/572192306] up:boot 2024-09-12T01:26:08.076 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:07 smithi104 ceph-mon[35219]: fsmap 3 up:standby 2024-09-12T01:26:08.076 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:07 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T01:26:08.076 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:07 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/1127628659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:26:08.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:07 smithi053 ceph-mon[35158]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:08.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:07 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:08.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:07 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:08.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:07 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:08.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:07 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:26:08.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:07 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:26:08.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:07 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:26:08.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:07 smithi053 ceph-mon[35158]: Deploying daemon mds.d on smithi001 2024-09-12T01:26:08.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:07 smithi053 ceph-mon[35158]: mds.? [v2:172.21.15.1:6834/572192306,v1:172.21.15.1:6835/572192306] up:boot 2024-09-12T01:26:08.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:07 smithi053 ceph-mon[35158]: fsmap 3 up:standby 2024-09-12T01:26:08.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:07 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T01:26:08.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:07 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/1127628659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:26:10.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:09 smithi104 ceph-mon[35219]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:10.174 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:09 smithi001 ceph-mon[30387]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:10.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:09 smithi053 ceph-mon[35158]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:10.755 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 12 01:26:10 smithi001 systemd[1]: Starting Ceph mds.d for c0e4f180-70a4-11ef-bcea-c7b262605968... 2024-09-12T01:26:11.044 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 12 01:26:10 smithi001 podman[70445]: 2024-09-12 01:26:10.661225263 +0000 UTC m=+0.027944933 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:26:11.044 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 12 01:26:10 smithi001 podman[70445]: 2024-09-12 01:26:10.78334248 +0000 UTC m=+0.150062125 container create d1561eab7856fc786a147f1877d513844b637834fc303ef308e48eea9cf82292 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-d, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T01:26:11.044 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 12 01:26:11 smithi001 podman[70445]: 2024-09-12 01:26:11.044897047 +0000 UTC m=+0.411616695 container init d1561eab7856fc786a147f1877d513844b637834fc303ef308e48eea9cf82292 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-d, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-12T01:26:11.168 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:26:11.427 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 12 01:26:11 smithi001 podman[70445]: 2024-09-12 01:26:11.051561346 +0000 UTC m=+0.418281000 container start d1561eab7856fc786a147f1877d513844b637834fc303ef308e48eea9cf82292 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-d, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-12T01:26:11.428 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 12 01:26:11 smithi001 ceph-mds[70464]: -- 172.21.15.1:0/3473203516 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x558ef5dfe680 con 0x558ef5dd2800 2024-09-12T01:26:11.428 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 12 01:26:11 smithi001 ceph-mds[70464]: -- 172.21.15.1:0/3473203516 <== mon.0 v2:172.21.15.1:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558ef5df5860 con 0x558ef5dd2800 2024-09-12T01:26:11.428 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 12 01:26:11 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-d[70460]: starting mds.d at 2024-09-12T01:26:11.428 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 12 01:26:11 smithi001 bash[70445]: d1561eab7856fc786a147f1877d513844b637834fc303ef308e48eea9cf82292 2024-09-12T01:26:11.428 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 12 01:26:11 smithi001 systemd[1]: Started Ceph mds.d for c0e4f180-70a4-11ef-bcea-c7b262605968. 2024-09-12T01:26:11.559 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:26:12.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.144+0000 7fc314600640 1 -- 172.21.15.1:0/2605411592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc30c076720 msgr2=0x7fc30c076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:12.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.144+0000 7fc314600640 1 --2- 172.21.15.1:0/2605411592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc30c076720 0x7fc30c076b00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc300009900 tx=0x7fc30002a400 comp rx=0 tx=0).stop 2024-09-12T01:26:12.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.145+0000 7fc314600640 1 -- 172.21.15.1:0/2605411592 shutdown_connections 2024-09-12T01:26:12.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.145+0000 7fc314600640 1 --2- 172.21.15.1:0/2605411592 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc30c112a30 0x7fc30c114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:12.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.145+0000 7fc314600640 1 --2- 172.21.15.1:0/2605411592 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc30c077040 0x7fc30c1124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:12.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.145+0000 7fc314600640 1 --2- 172.21.15.1:0/2605411592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc30c076720 0x7fc30c076b00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:12.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.145+0000 7fc314600640 1 -- 172.21.15.1:0/2605411592 >> 172.21.15.1:0/2605411592 conn(0x7fc30c0ffa90 msgr2=0x7fc30c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:26:12.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.145+0000 7fc314600640 1 -- 172.21.15.1:0/2605411592 shutdown_connections 2024-09-12T01:26:12.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.145+0000 7fc314600640 1 -- 172.21.15.1:0/2605411592 wait complete. 2024-09-12T01:26:12.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.146+0000 7fc314600640 1 Processor -- start 2024-09-12T01:26:12.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.146+0000 7fc314600640 1 -- start start 2024-09-12T01:26:12.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.146+0000 7fc314600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc30c076720 0x7fc30c1a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:12.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.146+0000 7fc314600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc30c077040 0x7fc30c1a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:12.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.146+0000 7fc314600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc30c112a30 0x7fc30c19e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:12.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.146+0000 7fc314600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc30c1185f0 con 0x7fc30c112a30 2024-09-12T01:26:12.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.146+0000 7fc314600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc30c118470 con 0x7fc30c076720 2024-09-12T01:26:12.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.146+0000 7fc314600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc30c118770 con 0x7fc30c077040 2024-09-12T01:26:12.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.147+0000 7fc312600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc30c112a30 0x7fc30c19e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:12.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.147+0000 7fc312600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc30c112a30 0x7fc30c19e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55986/0 (socket says 172.21.15.1:55986) 2024-09-12T01:26:12.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.147+0000 7fc312600640 1 -- 172.21.15.1:0/337194641 learned_addr learned my addr 172.21.15.1:0/337194641 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:26:12.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.147+0000 7fc311200640 1 --2- 172.21.15.1:0/337194641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc30c077040 0x7fc30c1a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:12.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.147+0000 7fc311c00640 1 --2- 172.21.15.1:0/337194641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc30c076720 0x7fc30c1a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:12.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.147+0000 7fc312600640 1 -- 172.21.15.1:0/337194641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc30c077040 msgr2=0x7fc30c1a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:12.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.147+0000 7fc312600640 1 --2- 172.21.15.1:0/337194641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc30c077040 0x7fc30c1a4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:12.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.147+0000 7fc312600640 1 -- 172.21.15.1:0/337194641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc30c076720 msgr2=0x7fc30c1a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:12.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.147+0000 7fc312600640 1 --2- 172.21.15.1:0/337194641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc30c076720 0x7fc30c1a46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:12.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.147+0000 7fc312600640 1 -- 172.21.15.1:0/337194641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc30c19efa0 con 0x7fc30c112a30 2024-09-12T01:26:12.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.147+0000 7fc312600640 1 --2- 172.21.15.1:0/337194641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc30c112a30 0x7fc30c19e870 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc304009fd0 tx=0x7fc30400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:26:12.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.148+0000 7fc30a000640 1 -- 172.21.15.1:0/337194641 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc304004d20 con 0x7fc30c112a30 2024-09-12T01:26:12.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.148+0000 7fc30a000640 1 -- 172.21.15.1:0/337194641 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc304005020 con 0x7fc30c112a30 2024-09-12T01:26:12.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.148+0000 7fc314600640 1 -- 172.21.15.1:0/337194641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc30c19f290 con 0x7fc30c112a30 2024-09-12T01:26:12.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.148+0000 7fc30a000640 1 -- 172.21.15.1:0/337194641 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3040116d0 con 0x7fc30c112a30 2024-09-12T01:26:12.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.148+0000 7fc314600640 1 -- 172.21.15.1:0/337194641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc30c1ab580 con 0x7fc30c112a30 2024-09-12T01:26:12.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.149+0000 7fc314600640 1 -- 172.21.15.1:0/337194641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc30c076b00 con 0x7fc30c112a30 2024-09-12T01:26:12.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.151+0000 7fc30a000640 1 -- 172.21.15.1:0/337194641 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fc3040051d0 con 0x7fc30c112a30 2024-09-12T01:26:12.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.152+0000 7fc30a000640 1 --2- 172.21.15.1:0/337194641 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fc2f40a2b80 0x7fc2f40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:12.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.152+0000 7fc30a000640 1 -- 172.21.15.1:0/337194641 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7fc3040cf0f0 con 0x7fc30c112a30 2024-09-12T01:26:12.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.153+0000 7fc311c00640 1 --2- 172.21.15.1:0/337194641 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fc2f40a2b80 0x7fc2f40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:12.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.153+0000 7fc311c00640 1 --2- 172.21.15.1:0/337194641 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fc2f40a2b80 0x7fc2f40a5040 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc300006010 tx=0x7fc300000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:26:12.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.156+0000 7fc30a000640 1 -- 172.21.15.1:0/337194641 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc304016030 con 0x7fc30c112a30 2024-09-12T01:26:12.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.379+0000 7fc314600640 1 -- 172.21.15.1:0/337194641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc30c02d7c0 con 0x7fc30c112a30 2024-09-12T01:26:12.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:12 smithi001 ceph-mon[30387]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:12.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:12 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:12.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:12 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:12.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:12 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:12.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:12 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:26:12.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:12 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:26:12.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:12 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:26:12.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:12 smithi001 ceph-mon[30387]: Deploying daemon mds.e on smithi053 2024-09-12T01:26:12.381 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:12 smithi001 ceph-mon[30387]: mds.? [v2:172.21.15.1:6836/1050734277,v1:172.21.15.1:6837/1050734277] up:boot 2024-09-12T01:26:12.381 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:12 smithi001 ceph-mon[30387]: fsmap 4 up:standby 2024-09-12T01:26:12.381 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:12 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T01:26:12.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.380+0000 7fc30a000640 1 -- 172.21.15.1:0/337194641 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3904 (secure 0 0 0) 0x7fc30409be00 con 0x7fc30c112a30 2024-09-12T01:26:12.381 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:26:12.381 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":5,"btime":"2024-09-12T01:26:11:764439+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14430,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/572192306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":572192306},{"type":"v1","addr":"172.21.15.1:6835","nonce":572192306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14442,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/1050734277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":1050734277},{"type":"v1","addr":"172.21.15.1:6837","nonce":1050734277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/2071825633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2071825633},{"type":"v1","addr":"172.21.15.104:6833","nonce":2071825633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2930310711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2930310711},{"type":"v1","addr":"172.21.15.53:6833","nonce":2930310711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-12T01:26:12.382 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 5 2024-09-12T01:26:12.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.382+0000 7fc314600640 1 -- 172.21.15.1:0/337194641 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fc2f40a2b80 msgr2=0x7fc2f40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:12.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.382+0000 7fc314600640 1 --2- 172.21.15.1:0/337194641 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fc2f40a2b80 0x7fc2f40a5040 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc300006010 tx=0x7fc300000bc0 comp rx=0 tx=0).stop 2024-09-12T01:26:12.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.383+0000 7fc314600640 1 -- 172.21.15.1:0/337194641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc30c112a30 msgr2=0x7fc30c19e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:12.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.383+0000 7fc314600640 1 --2- 172.21.15.1:0/337194641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc30c112a30 0x7fc30c19e870 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc304009fd0 tx=0x7fc30400ee60 comp rx=0 tx=0).stop 2024-09-12T01:26:12.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.383+0000 7fc314600640 1 -- 172.21.15.1:0/337194641 shutdown_connections 2024-09-12T01:26:12.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.383+0000 7fc314600640 1 --2- 172.21.15.1:0/337194641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc30c112a30 0x7fc30c19e870 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:12.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.383+0000 7fc314600640 1 --2- 172.21.15.1:0/337194641 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fc2f40a2b80 0x7fc2f40a5040 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:12.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.383+0000 7fc314600640 1 --2- 172.21.15.1:0/337194641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc30c077040 0x7fc30c1a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:12.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.383+0000 7fc314600640 1 --2- 172.21.15.1:0/337194641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc30c076720 0x7fc30c1a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:12.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.383+0000 7fc314600640 1 -- 172.21.15.1:0/337194641 >> 172.21.15.1:0/337194641 conn(0x7fc30c0ffa90 msgr2=0x7fc30c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:26:12.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.383+0000 7fc314600640 1 -- 172.21.15.1:0/337194641 shutdown_connections 2024-09-12T01:26:12.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:12.383+0000 7fc314600640 1 -- 172.21.15.1:0/337194641 wait complete. 2024-09-12T01:26:12.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:12 smithi053 ceph-mon[35158]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:12.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:12 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:12.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:12 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:12.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:12 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:12.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:12 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:26:12.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:12 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:26:12.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:12 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:26:12.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:12 smithi053 ceph-mon[35158]: Deploying daemon mds.e on smithi053 2024-09-12T01:26:12.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:12 smithi053 ceph-mon[35158]: mds.? [v2:172.21.15.1:6836/1050734277,v1:172.21.15.1:6837/1050734277] up:boot 2024-09-12T01:26:12.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:12 smithi053 ceph-mon[35158]: fsmap 4 up:standby 2024-09-12T01:26:12.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:12 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T01:26:12.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:12 smithi104 ceph-mon[35219]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:12.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:12 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:12.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:12 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:12.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:12 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:12.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:12 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:26:12.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:12 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:26:12.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:12 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:26:12.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:12 smithi104 ceph-mon[35219]: Deploying daemon mds.e on smithi053 2024-09-12T01:26:12.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:12 smithi104 ceph-mon[35219]: mds.? [v2:172.21.15.1:6836/1050734277,v1:172.21.15.1:6837/1050734277] up:boot 2024-09-12T01:26:12.576 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:12 smithi104 ceph-mon[35219]: fsmap 4 up:standby 2024-09-12T01:26:12.576 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:12 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T01:26:13.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:13 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/337194641' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:26:13.478 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:13 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/337194641' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:26:13.479 INFO:journalctl@ceph.mds.e.smithi053.stdout:Sep 12 01:26:13 smithi053 systemd[1]: Starting Ceph mds.e for c0e4f180-70a4-11ef-bcea-c7b262605968... 2024-09-12T01:26:13.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:13 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/337194641' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:26:13.740 INFO:journalctl@ceph.mds.e.smithi053.stdout:Sep 12 01:26:13 smithi053 podman[64867]: 2024-09-12 01:26:13.382080866 +0000 UTC m=+0.015653731 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:26:13.740 INFO:journalctl@ceph.mds.e.smithi053.stdout:Sep 12 01:26:13 smithi053 podman[64867]: 2024-09-12 01:26:13.499310277 +0000 UTC m=+0.132883186 container create a74684ef297f069a2e7e78492019653b137c7ca071217a4df3ad11a4db81cff8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-e, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-12T01:26:14.223 INFO:journalctl@ceph.mds.e.smithi053.stdout:Sep 12 01:26:13 smithi053 podman[64867]: 2024-09-12 01:26:13.76052824 +0000 UTC m=+0.394101090 container init a74684ef297f069a2e7e78492019653b137c7ca071217a4df3ad11a4db81cff8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-e, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-12T01:26:14.224 INFO:journalctl@ceph.mds.e.smithi053.stdout:Sep 12 01:26:13 smithi053 podman[64867]: 2024-09-12 01:26:13.763440292 +0000 UTC m=+0.397013143 container start a74684ef297f069a2e7e78492019653b137c7ca071217a4df3ad11a4db81cff8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-e, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-12T01:26:14.224 INFO:journalctl@ceph.mds.e.smithi053.stdout:Sep 12 01:26:13 smithi053 ceph-mds[64886]: -- 172.21.15.53:0/54100774 <== mon.2 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f28c7bf860 con 0x55f28c79d400 2024-09-12T01:26:14.224 INFO:journalctl@ceph.mds.e.smithi053.stdout:Sep 12 01:26:13 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-e[64882]: starting mds.e at 2024-09-12T01:26:14.224 INFO:journalctl@ceph.mds.e.smithi053.stdout:Sep 12 01:26:13 smithi053 bash[64867]: a74684ef297f069a2e7e78492019653b137c7ca071217a4df3ad11a4db81cff8 2024-09-12T01:26:14.224 INFO:journalctl@ceph.mds.e.smithi053.stdout:Sep 12 01:26:13 smithi053 systemd[1]: Started Ceph mds.e for c0e4f180-70a4-11ef-bcea-c7b262605968. 2024-09-12T01:26:14.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:14 smithi001 ceph-mon[30387]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:14.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:14 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:14.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:14 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:14.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:14 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:14.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:14 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:26:14.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:14 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:26:14.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:14 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:26:14.477 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:14 smithi104 ceph-mon[35219]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:14.478 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:14 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:14.478 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:14 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:14.478 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:14 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:14.478 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:14 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:26:14.478 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:14 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:26:14.478 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:14 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:26:14.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:14 smithi053 ceph-mon[35158]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:14.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:14 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:14.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:14 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:14.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:14 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:14.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:14 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:26:14.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:14 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:26:14.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:14 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:26:15.149 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:26:15.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:15 smithi053 ceph-mon[35158]: Deploying daemon mds.f on smithi104 2024-09-12T01:26:15.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:15 smithi053 ceph-mon[35158]: mds.? [v2:172.21.15.53:6834/2032381725,v1:172.21.15.53:6835/2032381725] up:boot 2024-09-12T01:26:15.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:15 smithi053 ceph-mon[35158]: fsmap 5 up:standby 2024-09-12T01:26:15.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:15 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T01:26:15.533 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:26:15.549 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:15 smithi104 ceph-mon[35219]: Deploying daemon mds.f on smithi104 2024-09-12T01:26:15.549 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:15 smithi104 ceph-mon[35219]: mds.? [v2:172.21.15.53:6834/2032381725,v1:172.21.15.53:6835/2032381725] up:boot 2024-09-12T01:26:15.549 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:15 smithi104 ceph-mon[35219]: fsmap 5 up:standby 2024-09-12T01:26:15.550 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:15 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T01:26:15.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:15 smithi001 ceph-mon[30387]: Deploying daemon mds.f on smithi104 2024-09-12T01:26:15.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:15 smithi001 ceph-mon[30387]: mds.? [v2:172.21.15.53:6834/2032381725,v1:172.21.15.53:6835/2032381725] up:boot 2024-09-12T01:26:15.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:15 smithi001 ceph-mon[30387]: fsmap 5 up:standby 2024-09-12T01:26:15.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:15 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T01:26:16.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.095+0000 7fd326600640 1 -- 172.21.15.1:0/3872354423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd320103ff0 msgr2=0x7fd3201043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:16.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.095+0000 7fd326600640 1 --2- 172.21.15.1:0/3872354423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd320103ff0 0x7fd3201043d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd310009900 tx=0x7fd31002a3f0 comp rx=0 tx=0).stop 2024-09-12T01:26:16.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.095+0000 7fd326600640 1 -- 172.21.15.1:0/3872354423 shutdown_connections 2024-09-12T01:26:16.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.095+0000 7fd326600640 1 --2- 172.21.15.1:0/3872354423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd320109120 0x7fd320110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:16.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.095+0000 7fd326600640 1 --2- 172.21.15.1:0/3872354423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3201049a0 0x7fd3201089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:16.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.095+0000 7fd326600640 1 --2- 172.21.15.1:0/3872354423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd320103ff0 0x7fd3201043d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:16.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.095+0000 7fd326600640 1 -- 172.21.15.1:0/3872354423 >> 172.21.15.1:0/3872354423 conn(0x7fd3200ffa90 msgr2=0x7fd320101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:26:16.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.095+0000 7fd326600640 1 -- 172.21.15.1:0/3872354423 shutdown_connections 2024-09-12T01:26:16.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.095+0000 7fd326600640 1 -- 172.21.15.1:0/3872354423 wait complete. 2024-09-12T01:26:16.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.096+0000 7fd326600640 1 Processor -- start 2024-09-12T01:26:16.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.096+0000 7fd326600640 1 -- start start 2024-09-12T01:26:16.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.097+0000 7fd326600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd320103ff0 0x7fd32019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:16.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.097+0000 7fd326600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3201049a0 0x7fd32019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:16.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.097+0000 7fd326600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd320109120 0x7fd32019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:16.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.097+0000 7fd326600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd320114150 con 0x7fd320109120 2024-09-12T01:26:16.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.097+0000 7fd326600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd320113fd0 con 0x7fd3201049a0 2024-09-12T01:26:16.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.097+0000 7fd326600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd3201142d0 con 0x7fd320103ff0 2024-09-12T01:26:16.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.097+0000 7fd31f400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd320103ff0 0x7fd32019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:16.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.097+0000 7fd31f400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd320103ff0 0x7fd32019a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:48046/0 (socket says 172.21.15.1:48046) 2024-09-12T01:26:16.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.097+0000 7fd31f400640 1 -- 172.21.15.1:0/1859296668 learned_addr learned my addr 172.21.15.1:0/1859296668 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:26:16.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.097+0000 7fd317e00640 1 --2- 172.21.15.1:0/1859296668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3201049a0 0x7fd32019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:16.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.097+0000 7fd31fe00640 1 --2- 172.21.15.1:0/1859296668 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd320109120 0x7fd32019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:16.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.097+0000 7fd31f400640 1 -- 172.21.15.1:0/1859296668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3201049a0 msgr2=0x7fd32019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:16.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.097+0000 7fd31f400640 1 --2- 172.21.15.1:0/1859296668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3201049a0 0x7fd32019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:16.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.097+0000 7fd31f400640 1 -- 172.21.15.1:0/1859296668 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd320109120 msgr2=0x7fd32019eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:16.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.097+0000 7fd31f400640 1 --2- 172.21.15.1:0/1859296668 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd320109120 0x7fd32019eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:16.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.097+0000 7fd31f400640 1 -- 172.21.15.1:0/1859296668 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd32019f5a0 con 0x7fd320103ff0 2024-09-12T01:26:16.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.098+0000 7fd31f400640 1 --2- 172.21.15.1:0/1859296668 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd320103ff0 0x7fd32019a5f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd310006010 tx=0x7fd310004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:26:16.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.098+0000 7fd31cc00640 1 -- 172.21.15.1:0/1859296668 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31003f030 con 0x7fd320103ff0 2024-09-12T01:26:16.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.098+0000 7fd31cc00640 1 -- 172.21.15.1:0/1859296668 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd310004440 con 0x7fd320103ff0 2024-09-12T01:26:16.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.098+0000 7fd326600640 1 -- 172.21.15.1:0/1859296668 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd32019f830 con 0x7fd320103ff0 2024-09-12T01:26:16.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.098+0000 7fd31cc00640 1 -- 172.21.15.1:0/1859296668 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd310035070 con 0x7fd320103ff0 2024-09-12T01:26:16.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.098+0000 7fd326600640 1 -- 172.21.15.1:0/1859296668 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3201a7170 con 0x7fd320103ff0 2024-09-12T01:26:16.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.099+0000 7fd326600640 1 -- 172.21.15.1:0/1859296668 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3201054d0 con 0x7fd320103ff0 2024-09-12T01:26:16.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.101+0000 7fd31cc00640 1 -- 172.21.15.1:0/1859296668 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fd310042030 con 0x7fd320103ff0 2024-09-12T01:26:16.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.102+0000 7fd31cc00640 1 --2- 172.21.15.1:0/1859296668 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fd3040a2b80 0x7fd3040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:16.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.102+0000 7fd31cc00640 1 -- 172.21.15.1:0/1859296668 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7fd3100eef90 con 0x7fd320103ff0 2024-09-12T01:26:16.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.102+0000 7fd317e00640 1 --2- 172.21.15.1:0/1859296668 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fd3040a2b80 0x7fd3040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:16.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.103+0000 7fd317e00640 1 --2- 172.21.15.1:0/1859296668 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fd3040a2b80 0x7fd3040a5040 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd3200ffeb0 tx=0x7fd308005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:26:16.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.106+0000 7fd31cc00640 1 -- 172.21.15.1:0/1859296668 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd3100bbca0 con 0x7fd320103ff0 2024-09-12T01:26:16.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.332+0000 7fd326600640 1 -- 172.21.15.1:0/1859296668 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd3201043d0 con 0x7fd320103ff0 2024-09-12T01:26:16.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.333+0000 7fd31cc00640 1 -- 172.21.15.1:0/1859296668 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4729 (secure 0 0 0) 0x7fd3100bffa0 con 0x7fd320103ff0 2024-09-12T01:26:16.333 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:26:16.333 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":6,"btime":"2024-09-12T01:26:14:223875+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14430,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/572192306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":572192306},{"type":"v1","addr":"172.21.15.1:6835","nonce":572192306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14442,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/1050734277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":1050734277},{"type":"v1","addr":"172.21.15.1:6837","nonce":1050734277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/2071825633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2071825633},{"type":"v1","addr":"172.21.15.104:6833","nonce":2071825633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2930310711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2930310711},{"type":"v1","addr":"172.21.15.53:6833","nonce":2930310711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2032381725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2032381725},{"type":"v1","addr":"172.21.15.53:6835","nonce":2032381725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-12T01:26:16.333 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 6 2024-09-12T01:26:16.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.336+0000 7fd326600640 1 -- 172.21.15.1:0/1859296668 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fd3040a2b80 msgr2=0x7fd3040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:16.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.336+0000 7fd326600640 1 --2- 172.21.15.1:0/1859296668 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fd3040a2b80 0x7fd3040a5040 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd3200ffeb0 tx=0x7fd308005f90 comp rx=0 tx=0).stop 2024-09-12T01:26:16.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.336+0000 7fd326600640 1 -- 172.21.15.1:0/1859296668 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd320103ff0 msgr2=0x7fd32019a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:16.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.336+0000 7fd326600640 1 --2- 172.21.15.1:0/1859296668 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd320103ff0 0x7fd32019a5f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd310006010 tx=0x7fd310004040 comp rx=0 tx=0).stop 2024-09-12T01:26:16.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.336+0000 7fd326600640 1 -- 172.21.15.1:0/1859296668 shutdown_connections 2024-09-12T01:26:16.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.336+0000 7fd326600640 1 --2- 172.21.15.1:0/1859296668 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd320109120 0x7fd32019eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:16.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.336+0000 7fd326600640 1 --2- 172.21.15.1:0/1859296668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3201049a0 0x7fd32019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:16.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.336+0000 7fd326600640 1 --2- 172.21.15.1:0/1859296668 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fd3040a2b80 0x7fd3040a5040 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:16.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.336+0000 7fd326600640 1 --2- 172.21.15.1:0/1859296668 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd320103ff0 0x7fd32019a5f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:16.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.336+0000 7fd326600640 1 -- 172.21.15.1:0/1859296668 >> 172.21.15.1:0/1859296668 conn(0x7fd3200ffa90 msgr2=0x7fd320100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:26:16.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.336+0000 7fd326600640 1 -- 172.21.15.1:0/1859296668 shutdown_connections 2024-09-12T01:26:16.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:16.336+0000 7fd326600640 1 -- 172.21.15.1:0/1859296668 wait complete. 2024-09-12T01:26:16.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:16 smithi053 ceph-mon[35158]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:16.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:16 smithi104 ceph-mon[35219]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:16.575 INFO:journalctl@ceph.mds.f.smithi104.stdout:Sep 12 01:26:16 smithi104 systemd[1]: Started Ceph mds.f for c0e4f180-70a4-11ef-bcea-c7b262605968. 2024-09-12T01:26:16.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:16 smithi001 ceph-mon[30387]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:17.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:17 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/1859296668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:26:17.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:17 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:17.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:17 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:17.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:17 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:17.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:17 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:26:17.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:17 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:26:17.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:17 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:26:17.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:17 smithi001 ceph-mon[30387]: Deploying daemon mds.c on smithi104 2024-09-12T01:26:17.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:17 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/1859296668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:26:17.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:17 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:17.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:17 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:17.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:17 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:17.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:17 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:26:17.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:17 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:26:17.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:17 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:26:17.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:17 smithi104 ceph-mon[35219]: Deploying daemon mds.c on smithi104 2024-09-12T01:26:17.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:17 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/1859296668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:26:17.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:17 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:17.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:17 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:17.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:17 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:17.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:17 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:26:17.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:17 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:26:17.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:17 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:26:17.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:17 smithi053 ceph-mon[35158]: Deploying daemon mds.c on smithi104 2024-09-12T01:26:18.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:18 smithi104 ceph-mon[35219]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:18.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:18 smithi104 ceph-mon[35219]: mds.? [v2:172.21.15.104:6834/1547038540,v1:172.21.15.104:6835/1547038540] up:boot 2024-09-12T01:26:18.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:18 smithi104 ceph-mon[35219]: fsmap 6 up:standby 2024-09-12T01:26:18.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:18 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T01:26:18.575 INFO:journalctl@ceph.mds.c.smithi104.stdout:Sep 12 01:26:18 smithi104 systemd[1]: Starting Ceph mds.c for c0e4f180-70a4-11ef-bcea-c7b262605968... 2024-09-12T01:26:18.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:18 smithi001 ceph-mon[30387]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:18.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:18 smithi001 ceph-mon[30387]: mds.? [v2:172.21.15.104:6834/1547038540,v1:172.21.15.104:6835/1547038540] up:boot 2024-09-12T01:26:18.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:18 smithi001 ceph-mon[30387]: fsmap 6 up:standby 2024-09-12T01:26:18.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:18 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T01:26:18.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:18 smithi053 ceph-mon[35158]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:18.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:18 smithi053 ceph-mon[35158]: mds.? [v2:172.21.15.104:6834/1547038540,v1:172.21.15.104:6835/1547038540] up:boot 2024-09-12T01:26:18.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:18 smithi053 ceph-mon[35158]: fsmap 6 up:standby 2024-09-12T01:26:18.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:18 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T01:26:18.892 INFO:journalctl@ceph.mds.c.smithi104.stdout:Sep 12 01:26:18 smithi104 podman[65199]: 2024-09-12 01:26:18.515554771 +0000 UTC m=+0.015822579 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:26:18.892 INFO:journalctl@ceph.mds.c.smithi104.stdout:Sep 12 01:26:18 smithi104 podman[65199]: 2024-09-12 01:26:18.649149372 +0000 UTC m=+0.149417156 container create fdc976a557b2baea7786bfbdf2461f4240448ea036e530e2a1df439f2bc93d9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-12T01:26:18.892 INFO:journalctl@ceph.mds.c.smithi104.stdout:Sep 12 01:26:18 smithi104 podman[65199]: 2024-09-12 01:26:18.893546813 +0000 UTC m=+0.393814598 container init fdc976a557b2baea7786bfbdf2461f4240448ea036e530e2a1df439f2bc93d9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-c, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-12T01:26:19.150 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:26:19.325 INFO:journalctl@ceph.mds.c.smithi104.stdout:Sep 12 01:26:18 smithi104 podman[65199]: 2024-09-12 01:26:18.896555925 +0000 UTC m=+0.396823718 container start fdc976a557b2baea7786bfbdf2461f4240448ea036e530e2a1df439f2bc93d9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-c, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-12T01:26:19.325 INFO:journalctl@ceph.mds.c.smithi104.stdout:Sep 12 01:26:18 smithi104 ceph-mds[65218]: -- 172.21.15.104:0/2227879209 <== mon.1 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ce74439860 con 0x55ce74417000 2024-09-12T01:26:19.325 INFO:journalctl@ceph.mds.c.smithi104.stdout:Sep 12 01:26:18 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-c[65214]: starting mds.c at 2024-09-12T01:26:19.325 INFO:journalctl@ceph.mds.c.smithi104.stdout:Sep 12 01:26:18 smithi104 bash[65199]: fdc976a557b2baea7786bfbdf2461f4240448ea036e530e2a1df439f2bc93d9f 2024-09-12T01:26:19.325 INFO:journalctl@ceph.mds.c.smithi104.stdout:Sep 12 01:26:18 smithi104 systemd[1]: Started Ceph mds.c for c0e4f180-70a4-11ef-bcea-c7b262605968. 2024-09-12T01:26:19.539 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:26:20.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.119+0000 7ff2dc200640 1 -- 172.21.15.1:0/1645260537 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2d4112a50 msgr2=0x7ff2d4114e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:20.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.119+0000 7ff2dc200640 1 --2- 172.21.15.1:0/1645260537 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2d4112a50 0x7ff2d4114e40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc009ca0 tx=0x7ff2cc02a8c0 comp rx=0 tx=0).stop 2024-09-12T01:26:20.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.119+0000 7ff2dc200640 1 -- 172.21.15.1:0/1645260537 shutdown_connections 2024-09-12T01:26:20.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.119+0000 7ff2dc200640 1 --2- 172.21.15.1:0/1645260537 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2d4112a50 0x7ff2d4114e40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:20.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.119+0000 7ff2dc200640 1 --2- 172.21.15.1:0/1645260537 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2d4077040 0x7ff2d4112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:20.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.119+0000 7ff2dc200640 1 --2- 172.21.15.1:0/1645260537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2d4076720 0x7ff2d4076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:20.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.119+0000 7ff2dc200640 1 -- 172.21.15.1:0/1645260537 >> 172.21.15.1:0/1645260537 conn(0x7ff2d40ffa50 msgr2=0x7ff2d4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:26:20.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.119+0000 7ff2dc200640 1 -- 172.21.15.1:0/1645260537 shutdown_connections 2024-09-12T01:26:20.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.119+0000 7ff2dc200640 1 -- 172.21.15.1:0/1645260537 wait complete. 2024-09-12T01:26:20.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.120+0000 7ff2dc200640 1 Processor -- start 2024-09-12T01:26:20.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.120+0000 7ff2dc200640 1 -- start start 2024-09-12T01:26:20.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.121+0000 7ff2dc200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2d4076720 0x7ff2d41a4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:20.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.121+0000 7ff2dc200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2d4077040 0x7ff2d41a4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:20.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.121+0000 7ff2dc200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2d4112a50 0x7ff2d419e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:20.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.121+0000 7ff2dc200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d4118590 con 0x7ff2d4076720 2024-09-12T01:26:20.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.121+0000 7ff2dc200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d4118410 con 0x7ff2d4112a50 2024-09-12T01:26:20.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.121+0000 7ff2dc200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d4118710 con 0x7ff2d4077040 2024-09-12T01:26:20.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.121+0000 7ff2d9800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2d4076720 0x7ff2d41a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:20.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.121+0000 7ff2da200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2d4112a50 0x7ff2d419e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:20.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.121+0000 7ff2d9800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2d4076720 0x7ff2d41a4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45938/0 (socket says 172.21.15.1:45938) 2024-09-12T01:26:20.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.121+0000 7ff2da200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2d4112a50 0x7ff2d419e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:34146/0 (socket says 172.21.15.1:34146) 2024-09-12T01:26:20.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.121+0000 7ff2d9800640 1 -- 172.21.15.1:0/2956294235 learned_addr learned my addr 172.21.15.1:0/2956294235 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:26:20.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.121+0000 7ff2d8e00640 1 --2- 172.21.15.1:0/2956294235 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2d4077040 0x7ff2d41a4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:20.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.121+0000 7ff2da200640 1 -- 172.21.15.1:0/2956294235 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2d4077040 msgr2=0x7ff2d41a4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:20.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.121+0000 7ff2da200640 1 --2- 172.21.15.1:0/2956294235 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2d4077040 0x7ff2d41a4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:20.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.121+0000 7ff2da200640 1 -- 172.21.15.1:0/2956294235 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2d4076720 msgr2=0x7ff2d41a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:20.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.121+0000 7ff2da200640 1 --2- 172.21.15.1:0/2956294235 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2d4076720 0x7ff2d41a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:20.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.121+0000 7ff2da200640 1 -- 172.21.15.1:0/2956294235 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2d419f030 con 0x7ff2d4112a50 2024-09-12T01:26:20.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.122+0000 7ff2da200640 1 --2- 172.21.15.1:0/2956294235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2d4112a50 0x7ff2d419e900 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc02ade0 tx=0x7ff2cc0319f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:26:20.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.122+0000 7ff2d2000640 1 -- 172.21.15.1:0/2956294235 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc003b70 con 0x7ff2d4112a50 2024-09-12T01:26:20.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.122+0000 7ff2d2000640 1 -- 172.21.15.1:0/2956294235 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2cc003d10 con 0x7ff2d4112a50 2024-09-12T01:26:20.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.122+0000 7ff2d2000640 1 -- 172.21.15.1:0/2956294235 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc02fc80 con 0x7ff2d4112a50 2024-09-12T01:26:20.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.122+0000 7ff2dc200640 1 -- 172.21.15.1:0/2956294235 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2d419f2c0 con 0x7ff2d4112a50 2024-09-12T01:26:20.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.122+0000 7ff2dc200640 1 -- 172.21.15.1:0/2956294235 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2d41ab610 con 0x7ff2d4112a50 2024-09-12T01:26:20.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.123+0000 7ff2dc200640 1 -- 172.21.15.1:0/2956294235 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2d4076b00 con 0x7ff2d4112a50 2024-09-12T01:26:20.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.125+0000 7ff2d2000640 1 -- 172.21.15.1:0/2956294235 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7ff2cc004050 con 0x7ff2d4112a50 2024-09-12T01:26:20.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.126+0000 7ff2d2000640 1 --2- 172.21.15.1:0/2956294235 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff2bc0a2c00 0x7ff2bc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:20.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.126+0000 7ff2d2000640 1 -- 172.21.15.1:0/2956294235 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7ff2cc0efe40 con 0x7ff2d4112a50 2024-09-12T01:26:20.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.126+0000 7ff2d9800640 1 --2- 172.21.15.1:0/2956294235 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff2bc0a2c00 0x7ff2bc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:20.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.127+0000 7ff2d9800640 1 --2- 172.21.15.1:0/2956294235 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff2bc0a2c00 0x7ff2bc0a50c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff2d419fa40 tx=0x7ff2c8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:26:20.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.130+0000 7ff2d2000640 1 -- 172.21.15.1:0/2956294235 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff2cc0bd2c0 con 0x7ff2d4112a50 2024-09-12T01:26:20.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:20 smithi053 ceph-mon[35158]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:20.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:20 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:20.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:20 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:20.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:20 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:20.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:20 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:26:20.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:20 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:26:20.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:20 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:26:20.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:20 smithi053 ceph-mon[35158]: Deploying daemon mds.h on smithi053 2024-09-12T01:26:20.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:20 smithi053 ceph-mon[35158]: mds.? [v2:172.21.15.104:6836/464720247,v1:172.21.15.104:6837/464720247] up:boot 2024-09-12T01:26:20.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:20 smithi053 ceph-mon[35158]: fsmap 7 up:standby 2024-09-12T01:26:20.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:20 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T01:26:20.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:20 smithi104 ceph-mon[35219]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:20.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:20 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:20.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:20 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:20.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:20 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:20.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:20 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:26:20.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:20 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:26:20.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:20 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:26:20.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:20 smithi104 ceph-mon[35219]: Deploying daemon mds.h on smithi053 2024-09-12T01:26:20.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:20 smithi104 ceph-mon[35219]: mds.? [v2:172.21.15.104:6836/464720247,v1:172.21.15.104:6837/464720247] up:boot 2024-09-12T01:26:20.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:20 smithi104 ceph-mon[35219]: fsmap 7 up:standby 2024-09-12T01:26:20.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:20 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T01:26:20.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.352+0000 7ff2dc200640 1 -- 172.21.15.1:0/2956294235 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2d410dbe0 con 0x7ff2d4112a50 2024-09-12T01:26:20.352 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:20 smithi001 ceph-mon[30387]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:20.352 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:20 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:20.352 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:20 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:20.352 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:20 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:20.352 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:20 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:26:20.352 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:20 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:26:20.352 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:20 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:26:20.353 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:20 smithi001 ceph-mon[30387]: Deploying daemon mds.h on smithi053 2024-09-12T01:26:20.353 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:20 smithi001 ceph-mon[30387]: mds.? [v2:172.21.15.104:6836/464720247,v1:172.21.15.104:6837/464720247] up:boot 2024-09-12T01:26:20.353 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:20 smithi001 ceph-mon[30387]: fsmap 7 up:standby 2024-09-12T01:26:20.353 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:20 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T01:26:20.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.353+0000 7ff2d2000640 1 -- 172.21.15.1:0/2956294235 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6382 (secure 0 0 0) 0x7ff2cc0bcbc0 con 0x7ff2d4112a50 2024-09-12T01:26:20.354 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:26:20.354 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":8,"btime":"2024-09-12T01:26:19:247852+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14430,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/572192306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":572192306},{"type":"v1","addr":"172.21.15.1:6835","nonce":572192306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14442,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/1050734277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":1050734277},{"type":"v1","addr":"172.21.15.1:6837","nonce":1050734277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/2071825633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2071825633},{"type":"v1","addr":"172.21.15.104:6833","nonce":2071825633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2930310711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2930310711},{"type":"v1","addr":"172.21.15.53:6833","nonce":2930310711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2032381725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2032381725},{"type":"v1","addr":"172.21.15.53:6835","nonce":2032381725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/1547038540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1547038540},{"type":"v1","addr":"172.21.15.104:6835","nonce":1547038540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24493,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/464720247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":464720247},{"type":"v1","addr":"172.21.15.104:6837","nonce":464720247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-12T01:26:20.354 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 8 2024-09-12T01:26:20.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.355+0000 7ff2dc200640 1 -- 172.21.15.1:0/2956294235 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff2bc0a2c00 msgr2=0x7ff2bc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:20.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.355+0000 7ff2dc200640 1 --2- 172.21.15.1:0/2956294235 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff2bc0a2c00 0x7ff2bc0a50c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff2d419fa40 tx=0x7ff2c8005f90 comp rx=0 tx=0).stop 2024-09-12T01:26:20.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.356+0000 7ff2dc200640 1 -- 172.21.15.1:0/2956294235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2d4112a50 msgr2=0x7ff2d419e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:20.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.356+0000 7ff2dc200640 1 --2- 172.21.15.1:0/2956294235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2d4112a50 0x7ff2d419e900 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc02ade0 tx=0x7ff2cc0319f0 comp rx=0 tx=0).stop 2024-09-12T01:26:20.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.356+0000 7ff2dc200640 1 -- 172.21.15.1:0/2956294235 shutdown_connections 2024-09-12T01:26:20.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.356+0000 7ff2dc200640 1 --2- 172.21.15.1:0/2956294235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2d4112a50 0x7ff2d419e900 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:20.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.356+0000 7ff2dc200640 1 --2- 172.21.15.1:0/2956294235 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff2bc0a2c00 0x7ff2bc0a50c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:20.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.356+0000 7ff2dc200640 1 --2- 172.21.15.1:0/2956294235 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2d4077040 0x7ff2d41a4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:20.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.356+0000 7ff2dc200640 1 --2- 172.21.15.1:0/2956294235 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2d4076720 0x7ff2d41a4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:20.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.356+0000 7ff2dc200640 1 -- 172.21.15.1:0/2956294235 >> 172.21.15.1:0/2956294235 conn(0x7ff2d40ffa50 msgr2=0x7ff2d4101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:26:20.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.356+0000 7ff2dc200640 1 -- 172.21.15.1:0/2956294235 shutdown_connections 2024-09-12T01:26:20.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:20.356+0000 7ff2dc200640 1 -- 172.21.15.1:0/2956294235 wait complete. 2024-09-12T01:26:21.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:21 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/2956294235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:26:21.382 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:21 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/2956294235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:26:21.382 INFO:journalctl@ceph.mds.h.smithi053.stdout:Sep 12 01:26:21 smithi053 systemd[1]: Starting Ceph mds.h for c0e4f180-70a4-11ef-bcea-c7b262605968... 2024-09-12T01:26:21.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:21 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/2956294235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:26:21.665 INFO:journalctl@ceph.mds.h.smithi053.stdout:Sep 12 01:26:21 smithi053 podman[65167]: 2024-09-12 01:26:21.285434612 +0000 UTC m=+0.016064696 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:26:21.665 INFO:journalctl@ceph.mds.h.smithi053.stdout:Sep 12 01:26:21 smithi053 podman[65167]: 2024-09-12 01:26:21.396803447 +0000 UTC m=+0.127433517 container create c5a1c644c4e5956f91f4d2fe71cb138c9c5e9cc036a18b08f4305fbab8d4048e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-h, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-12T01:26:21.665 INFO:journalctl@ceph.mds.h.smithi053.stdout:Sep 12 01:26:21 smithi053 podman[65167]: 2024-09-12 01:26:21.666137883 +0000 UTC m=+0.396767970 container init c5a1c644c4e5956f91f4d2fe71cb138c9c5e9cc036a18b08f4305fbab8d4048e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-h, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T01:26:21.989 INFO:journalctl@ceph.mds.h.smithi053.stdout:Sep 12 01:26:21 smithi053 podman[65167]: 2024-09-12 01:26:21.669116318 +0000 UTC m=+0.399746395 container start c5a1c644c4e5956f91f4d2fe71cb138c9c5e9cc036a18b08f4305fbab8d4048e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-h, GIT_CLEAN=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0) 2024-09-12T01:26:21.990 INFO:journalctl@ceph.mds.h.smithi053.stdout:Sep 12 01:26:21 smithi053 ceph-mds[65186]: -- 172.21.15.53:0/2294753967 <== mon.2 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558b7ff1f860 con 0x558b7fefd400 2024-09-12T01:26:21.990 INFO:journalctl@ceph.mds.h.smithi053.stdout:Sep 12 01:26:21 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-h[65182]: starting mds.h at 2024-09-12T01:26:21.990 INFO:journalctl@ceph.mds.h.smithi053.stdout:Sep 12 01:26:21 smithi053 bash[65167]: c5a1c644c4e5956f91f4d2fe71cb138c9c5e9cc036a18b08f4305fbab8d4048e 2024-09-12T01:26:21.990 INFO:journalctl@ceph.mds.h.smithi053.stdout:Sep 12 01:26:21 smithi053 systemd[1]: Started Ceph mds.h for c0e4f180-70a4-11ef-bcea-c7b262605968. 2024-09-12T01:26:22.529 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:22 smithi104 ceph-mon[35219]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:22.530 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:22 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:22.530 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:22 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:22.530 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:22 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:22.530 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:22 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:26:22.530 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:22 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:26:22.531 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:22 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:26:22.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:22 smithi001 ceph-mon[30387]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:22.679 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:22 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:22.679 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:22 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:22.679 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:22 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:22.679 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:22 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:26:22.680 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:22 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:26:22.680 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:22 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:26:22.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:22 smithi053 ceph-mon[35158]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:22.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:22 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:22.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:22 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:22.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:22 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:22.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:22 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:26:22.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:22 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:26:22.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:22 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:26:23.198 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:26:23.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:23 smithi104 ceph-mon[35219]: Deploying daemon mds.i on smithi104 2024-09-12T01:26:23.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:23 smithi104 ceph-mon[35219]: mds.? [v2:172.21.15.53:6836/2848743754,v1:172.21.15.53:6837/2848743754] up:boot 2024-09-12T01:26:23.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:23 smithi104 ceph-mon[35219]: fsmap 8 up:standby 2024-09-12T01:26:23.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:23 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T01:26:23.585 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:26:23.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:23 smithi001 ceph-mon[30387]: Deploying daemon mds.i on smithi104 2024-09-12T01:26:23.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:23 smithi001 ceph-mon[30387]: mds.? [v2:172.21.15.53:6836/2848743754,v1:172.21.15.53:6837/2848743754] up:boot 2024-09-12T01:26:23.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:23 smithi001 ceph-mon[30387]: fsmap 8 up:standby 2024-09-12T01:26:23.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:23 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T01:26:23.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:23 smithi053 ceph-mon[35158]: Deploying daemon mds.i on smithi104 2024-09-12T01:26:23.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:23 smithi053 ceph-mon[35158]: mds.? [v2:172.21.15.53:6836/2848743754,v1:172.21.15.53:6837/2848743754] up:boot 2024-09-12T01:26:23.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:23 smithi053 ceph-mon[35158]: fsmap 8 up:standby 2024-09-12T01:26:23.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:23 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T01:26:23.986 INFO:journalctl@ceph.mds.i.smithi104.stdout:Sep 12 01:26:23 smithi104 systemd[1]: Starting Ceph mds.i for c0e4f180-70a4-11ef-bcea-c7b262605968... 2024-09-12T01:26:24.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.152+0000 7fa066c00640 1 -- 172.21.15.1:0/3422486801 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0601049a0 msgr2=0x7fa0601089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:24.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.152+0000 7fa066c00640 1 --2- 172.21.15.1:0/3422486801 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0601049a0 0x7fa0601089f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa048009900 tx=0x7fa04802a410 comp rx=0 tx=0).stop 2024-09-12T01:26:24.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.152+0000 7fa066c00640 1 -- 172.21.15.1:0/3422486801 shutdown_connections 2024-09-12T01:26:24.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.152+0000 7fa066c00640 1 --2- 172.21.15.1:0/3422486801 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa060109120 0x7fa060110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:24.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.152+0000 7fa066c00640 1 --2- 172.21.15.1:0/3422486801 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0601049a0 0x7fa0601089f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:24.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.152+0000 7fa066c00640 1 --2- 172.21.15.1:0/3422486801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa060103ff0 0x7fa0601043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:24.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.152+0000 7fa066c00640 1 -- 172.21.15.1:0/3422486801 >> 172.21.15.1:0/3422486801 conn(0x7fa0600ffa90 msgr2=0x7fa060101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:26:24.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.152+0000 7fa066c00640 1 -- 172.21.15.1:0/3422486801 shutdown_connections 2024-09-12T01:26:24.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.152+0000 7fa066c00640 1 -- 172.21.15.1:0/3422486801 wait complete. 2024-09-12T01:26:24.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.153+0000 7fa066c00640 1 Processor -- start 2024-09-12T01:26:24.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.153+0000 7fa066c00640 1 -- start start 2024-09-12T01:26:24.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.153+0000 7fa066c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa060103ff0 0x7fa06019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:24.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.153+0000 7fa066c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0601049a0 0x7fa06019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:24.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.153+0000 7fa066c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa060109120 0x7fa06019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:24.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.153+0000 7fa066c00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa060114120 con 0x7fa0601049a0 2024-09-12T01:26:24.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.153+0000 7fa066c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa060113fa0 con 0x7fa060109120 2024-09-12T01:26:24.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.153+0000 7fa066c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa0601142a0 con 0x7fa060103ff0 2024-09-12T01:26:24.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.154+0000 7fa057e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0601049a0 0x7fa06019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:24.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.154+0000 7fa057e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0601049a0 0x7fa06019abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45962/0 (socket says 172.21.15.1:45962) 2024-09-12T01:26:24.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.154+0000 7fa057e00640 1 -- 172.21.15.1:0/3578126574 learned_addr learned my addr 172.21.15.1:0/3578126574 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:26:24.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.154+0000 7fa064c00640 1 --2- 172.21.15.1:0/3578126574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa060109120 0x7fa06019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:24.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.154+0000 7fa057e00640 1 -- 172.21.15.1:0/3578126574 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa060103ff0 msgr2=0x7fa06019a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:24.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.154+0000 7fa057e00640 1 --2- 172.21.15.1:0/3578126574 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa060103ff0 0x7fa06019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:24.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.154+0000 7fa057e00640 1 -- 172.21.15.1:0/3578126574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa060109120 msgr2=0x7fa06019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:24.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.154+0000 7fa057e00640 1 --2- 172.21.15.1:0/3578126574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa060109120 0x7fa06019ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:24.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.154+0000 7fa057e00640 1 -- 172.21.15.1:0/3578126574 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa06019f650 con 0x7fa0601049a0 2024-09-12T01:26:24.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.154+0000 7fa064c00640 1 --2- 172.21.15.1:0/3578126574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa060109120 0x7fa06019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:26:24.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.155+0000 7fa057e00640 1 --2- 172.21.15.1:0/3578126574 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0601049a0 0x7fa06019abe0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa048006010 tx=0x7fa048004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:26:24.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.155+0000 7fa05d600640 1 -- 172.21.15.1:0/3578126574 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04803f030 con 0x7fa0601049a0 2024-09-12T01:26:24.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.155+0000 7fa05d600640 1 -- 172.21.15.1:0/3578126574 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa048004440 con 0x7fa0601049a0 2024-09-12T01:26:24.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.155+0000 7fa05d600640 1 -- 172.21.15.1:0/3578126574 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa048035070 con 0x7fa0601049a0 2024-09-12T01:26:24.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.155+0000 7fa066c00640 1 -- 172.21.15.1:0/3578126574 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa06019f8e0 con 0x7fa0601049a0 2024-09-12T01:26:24.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.155+0000 7fa066c00640 1 -- 172.21.15.1:0/3578126574 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0601a7220 con 0x7fa0601049a0 2024-09-12T01:26:24.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.156+0000 7fa066c00640 1 -- 172.21.15.1:0/3578126574 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0601043d0 con 0x7fa0601049a0 2024-09-12T01:26:24.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.158+0000 7fa05d600640 1 -- 172.21.15.1:0/3578126574 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fa048042030 con 0x7fa0601049a0 2024-09-12T01:26:24.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.160+0000 7fa05d600640 1 --2- 172.21.15.1:0/3578126574 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fa0440a2b80 0x7fa0440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:24.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.160+0000 7fa05d600640 1 -- 172.21.15.1:0/3578126574 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7fa0480eefe0 con 0x7fa0601049a0 2024-09-12T01:26:24.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.160+0000 7fa05fe00640 1 --2- 172.21.15.1:0/3578126574 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fa0440a2b80 0x7fa0440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:24.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.161+0000 7fa05fe00640 1 --2- 172.21.15.1:0/3578126574 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fa0440a2b80 0x7fa0440a5040 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa0500058c0 tx=0x7fa050005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:26:24.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.165+0000 7fa05d600640 1 -- 172.21.15.1:0/3578126574 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa0480bc3a0 con 0x7fa0601049a0 2024-09-12T01:26:24.256 INFO:journalctl@ceph.mds.i.smithi104.stdout:Sep 12 01:26:23 smithi104 podman[65499]: 2024-09-12 01:26:23.889387857 +0000 UTC m=+0.015521508 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:26:24.256 INFO:journalctl@ceph.mds.i.smithi104.stdout:Sep 12 01:26:24 smithi104 podman[65499]: 2024-09-12 01:26:24.012507266 +0000 UTC m=+0.138640910 container create ee23f250750b6cc74e5969642ad3af70c874ed32e4c93c0175e44377f9bf53e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-i, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-12T01:26:24.256 INFO:journalctl@ceph.mds.i.smithi104.stdout:Sep 12 01:26:24 smithi104 podman[65499]: 2024-09-12 01:26:24.257069882 +0000 UTC m=+0.383203527 container init ee23f250750b6cc74e5969642ad3af70c874ed32e4c93c0175e44377f9bf53e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-i, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T01:26:24.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.387+0000 7fa066c00640 1 -- 172.21.15.1:0/3578126574 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa06010c360 con 0x7fa0601049a0 2024-09-12T01:26:24.387 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:24 smithi001 ceph-mon[30387]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:24.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.388+0000 7fa05d600640 1 -- 172.21.15.1:0/3578126574 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7207 (secure 0 0 0) 0x7fa0480bbca0 con 0x7fa0601049a0 2024-09-12T01:26:24.388 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:26:24.388 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":9,"btime":"2024-09-12T01:26:22:270140+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14430,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/572192306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":572192306},{"type":"v1","addr":"172.21.15.1:6835","nonce":572192306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14442,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/1050734277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":1050734277},{"type":"v1","addr":"172.21.15.1:6837","nonce":1050734277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/2071825633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2071825633},{"type":"v1","addr":"172.21.15.104:6833","nonce":2071825633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2930310711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2930310711},{"type":"v1","addr":"172.21.15.53:6833","nonce":2930310711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2032381725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2032381725},{"type":"v1","addr":"172.21.15.53:6835","nonce":2032381725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/1547038540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1547038540},{"type":"v1","addr":"172.21.15.104:6835","nonce":1547038540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24493,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/464720247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":464720247},{"type":"v1","addr":"172.21.15.104:6837","nonce":464720247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/2848743754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2848743754},{"type":"v1","addr":"172.21.15.53:6837","nonce":2848743754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-12T01:26:24.388 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 9 2024-09-12T01:26:24.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.390+0000 7fa066c00640 1 -- 172.21.15.1:0/3578126574 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fa0440a2b80 msgr2=0x7fa0440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:24.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.390+0000 7fa066c00640 1 --2- 172.21.15.1:0/3578126574 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fa0440a2b80 0x7fa0440a5040 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa0500058c0 tx=0x7fa050005850 comp rx=0 tx=0).stop 2024-09-12T01:26:24.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.390+0000 7fa066c00640 1 -- 172.21.15.1:0/3578126574 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0601049a0 msgr2=0x7fa06019abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:24.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.390+0000 7fa066c00640 1 --2- 172.21.15.1:0/3578126574 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0601049a0 0x7fa06019abe0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa048006010 tx=0x7fa048004040 comp rx=0 tx=0).stop 2024-09-12T01:26:24.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.390+0000 7fa066c00640 1 -- 172.21.15.1:0/3578126574 shutdown_connections 2024-09-12T01:26:24.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.390+0000 7fa066c00640 1 --2- 172.21.15.1:0/3578126574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa060109120 0x7fa06019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:24.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.390+0000 7fa066c00640 1 --2- 172.21.15.1:0/3578126574 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0601049a0 0x7fa06019abe0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:24.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.390+0000 7fa066c00640 1 --2- 172.21.15.1:0/3578126574 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fa0440a2b80 0x7fa0440a5040 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:24.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.390+0000 7fa066c00640 1 --2- 172.21.15.1:0/3578126574 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa060103ff0 0x7fa06019a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:24.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.390+0000 7fa066c00640 1 -- 172.21.15.1:0/3578126574 >> 172.21.15.1:0/3578126574 conn(0x7fa0600ffa90 msgr2=0x7fa060100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:26:24.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.391+0000 7fa066c00640 1 -- 172.21.15.1:0/3578126574 shutdown_connections 2024-09-12T01:26:24.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:24.391+0000 7fa066c00640 1 -- 172.21.15.1:0/3578126574 wait complete. 2024-09-12T01:26:24.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:24 smithi104 ceph-mon[35219]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:24.575 INFO:journalctl@ceph.mds.i.smithi104.stdout:Sep 12 01:26:24 smithi104 podman[65499]: 2024-09-12 01:26:24.260235544 +0000 UTC m=+0.386369197 container start ee23f250750b6cc74e5969642ad3af70c874ed32e4c93c0175e44377f9bf53e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-i, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-12T01:26:24.575 INFO:journalctl@ceph.mds.i.smithi104.stdout:Sep 12 01:26:24 smithi104 ceph-mds[65518]: -- 172.21.15.104:0/2386173484 <== mon.1 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c4e8c75860 con 0x55c4e8c53000 2024-09-12T01:26:24.575 INFO:journalctl@ceph.mds.i.smithi104.stdout:Sep 12 01:26:24 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-i[65514]: starting mds.i at 2024-09-12T01:26:24.575 INFO:journalctl@ceph.mds.i.smithi104.stdout:Sep 12 01:26:24 smithi104 bash[65499]: ee23f250750b6cc74e5969642ad3af70c874ed32e4c93c0175e44377f9bf53e3 2024-09-12T01:26:24.575 INFO:journalctl@ceph.mds.i.smithi104.stdout:Sep 12 01:26:24 smithi104 systemd[1]: Started Ceph mds.i for c0e4f180-70a4-11ef-bcea-c7b262605968. 2024-09-12T01:26:24.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:24 smithi053 ceph-mon[35158]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:25.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:25 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/3578126574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:26:25.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:25 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:25.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:25 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:25.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:25 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:25.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:25 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:26:25.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:25 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:26:25.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:25 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:26:25.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:25 smithi001 ceph-mon[30387]: Deploying daemon mds.g on smithi001 2024-09-12T01:26:25.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:25 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/3578126574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:26:25.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:25 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:25.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:25 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:25.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:25 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:25.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:25 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:26:25.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:25 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:26:25.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:25 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:26:25.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:25 smithi104 ceph-mon[35219]: Deploying daemon mds.g on smithi001 2024-09-12T01:26:25.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:25 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/3578126574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:26:25.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:25 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:25.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:25 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:25.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:25 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:25.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:25 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:26:25.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:25 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:26:25.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:25 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:26:25.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:25 smithi053 ceph-mon[35158]: Deploying daemon mds.g on smithi001 2024-09-12T01:26:26.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:26 smithi104 ceph-mon[35219]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:26.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:26 smithi104 ceph-mon[35219]: mds.? [v2:172.21.15.104:6838/1459976041,v1:172.21.15.104:6839/1459976041] up:boot 2024-09-12T01:26:26.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:26 smithi104 ceph-mon[35219]: fsmap 9 up:standby 2024-09-12T01:26:26.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:26 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T01:26:26.635 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:26 smithi001 ceph-mon[30387]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:26.635 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:26 smithi001 ceph-mon[30387]: mds.? [v2:172.21.15.104:6838/1459976041,v1:172.21.15.104:6839/1459976041] up:boot 2024-09-12T01:26:26.635 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:26 smithi001 ceph-mon[30387]: fsmap 9 up:standby 2024-09-12T01:26:26.635 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:26 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T01:26:26.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:26 smithi053 ceph-mon[35158]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:26.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:26 smithi053 ceph-mon[35158]: mds.? [v2:172.21.15.104:6838/1459976041,v1:172.21.15.104:6839/1459976041] up:boot 2024-09-12T01:26:26.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:26 smithi053 ceph-mon[35158]: fsmap 9 up:standby 2024-09-12T01:26:26.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:26 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T01:26:27.221 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:26:27.674 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 12 01:26:27 smithi001 systemd[1]: Starting Ceph mds.g for c0e4f180-70a4-11ef-bcea-c7b262605968... 2024-09-12T01:26:27.928 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 12 01:26:27 smithi001 podman[71233]: 2024-09-12 01:26:27.58126556 +0000 UTC m=+0.035093269 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:26:27.928 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 12 01:26:27 smithi001 podman[71233]: 2024-09-12 01:26:27.697139774 +0000 UTC m=+0.150967448 container create 801be67e9eb3ad63d9b2e957ae148b23f887b7228fba5bba5115bca9d0a3a435 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-g, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_CLEAN=True) 2024-09-12T01:26:28.080 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:26:28.228 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 12 01:26:27 smithi001 podman[71233]: 2024-09-12 01:26:27.941779243 +0000 UTC m=+0.395606947 container init 801be67e9eb3ad63d9b2e957ae148b23f887b7228fba5bba5115bca9d0a3a435 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-g, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T01:26:28.229 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 12 01:26:27 smithi001 podman[71233]: 2024-09-12 01:26:27.948061773 +0000 UTC m=+0.401889447 container start 801be67e9eb3ad63d9b2e957ae148b23f887b7228fba5bba5115bca9d0a3a435 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-g, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0) 2024-09-12T01:26:28.229 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 12 01:26:27 smithi001 ceph-mds[71277]: -- 172.21.15.1:0/915385424 <== mon.0 v2:172.21.15.1:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561dd8955860 con 0x561dd8932800 2024-09-12T01:26:28.229 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 12 01:26:27 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-g[71269]: starting mds.g at 2024-09-12T01:26:28.229 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 12 01:26:28 smithi001 bash[71233]: 801be67e9eb3ad63d9b2e957ae148b23f887b7228fba5bba5115bca9d0a3a435 2024-09-12T01:26:28.229 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 12 01:26:28 smithi001 systemd[1]: Started Ceph mds.g for c0e4f180-70a4-11ef-bcea-c7b262605968. 2024-09-12T01:26:28.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:28 smithi104 ceph-mon[35219]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:28.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:28 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:28.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:28 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:28.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:28 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:28.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:28 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:26:28.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:28 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:26:28.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:28 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:26:28.577 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:28 smithi001 ceph-mon[30387]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:28.577 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:28 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:28.578 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:28 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:28.578 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:28 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:28.578 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:28 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:26:28.578 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:28 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:26:28.578 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:28 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:26:28.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.692+0000 7fe789800640 1 -- 172.21.15.1:0/1725323849 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe784076720 msgr2=0x7fe784076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:28.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.692+0000 7fe789800640 1 --2- 172.21.15.1:0/1725323849 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe784076720 0x7fe784076b00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe774009900 tx=0x7fe77402a410 comp rx=0 tx=0).stop 2024-09-12T01:26:28.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.692+0000 7fe789800640 1 -- 172.21.15.1:0/1725323849 shutdown_connections 2024-09-12T01:26:28.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.692+0000 7fe789800640 1 --2- 172.21.15.1:0/1725323849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe784112870 0x7fe784114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:28.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.692+0000 7fe789800640 1 --2- 172.21.15.1:0/1725323849 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe784077040 0x7fe784112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:28.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.692+0000 7fe789800640 1 --2- 172.21.15.1:0/1725323849 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe784076720 0x7fe784076b00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:28.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.692+0000 7fe789800640 1 -- 172.21.15.1:0/1725323849 >> 172.21.15.1:0/1725323849 conn(0x7fe7840ff910 msgr2=0x7fe784101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:26:28.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.692+0000 7fe789800640 1 -- 172.21.15.1:0/1725323849 shutdown_connections 2024-09-12T01:26:28.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.692+0000 7fe789800640 1 -- 172.21.15.1:0/1725323849 wait complete. 2024-09-12T01:26:28.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.693+0000 7fe789800640 1 Processor -- start 2024-09-12T01:26:28.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.693+0000 7fe789800640 1 -- start start 2024-09-12T01:26:28.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.694+0000 7fe789800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe784076720 0x7fe7841a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:28.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.694+0000 7fe789800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe784077040 0x7fe7841a4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:28.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.694+0000 7fe789800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe784112870 0x7fe78419e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:28.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.694+0000 7fe789800640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe784118400 con 0x7fe784112870 2024-09-12T01:26:28.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.694+0000 7fe789800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe784118280 con 0x7fe784077040 2024-09-12T01:26:28.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.694+0000 7fe789800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe784118580 con 0x7fe784076720 2024-09-12T01:26:28.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.694+0000 7fe783e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe784076720 0x7fe7841a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:28.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.694+0000 7fe788e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe784112870 0x7fe78419e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:28.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.694+0000 7fe783e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe784076720 0x7fe7841a4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:39348/0 (socket says 172.21.15.1:39348) 2024-09-12T01:26:28.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.694+0000 7fe783e00640 1 -- 172.21.15.1:0/2750471130 learned_addr learned my addr 172.21.15.1:0/2750471130 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:26:28.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.694+0000 7fe783400640 1 --2- 172.21.15.1:0/2750471130 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe784077040 0x7fe7841a4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:28.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.695+0000 7fe783e00640 1 -- 172.21.15.1:0/2750471130 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe784077040 msgr2=0x7fe7841a4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:28.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.695+0000 7fe783e00640 1 --2- 172.21.15.1:0/2750471130 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe784077040 0x7fe7841a4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:28.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.695+0000 7fe783e00640 1 -- 172.21.15.1:0/2750471130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe784112870 msgr2=0x7fe78419e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:28.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.695+0000 7fe783e00640 1 --2- 172.21.15.1:0/2750471130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe784112870 0x7fe78419e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:28.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.695+0000 7fe783e00640 1 -- 172.21.15.1:0/2750471130 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe78419ef40 con 0x7fe784076720 2024-09-12T01:26:28.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.695+0000 7fe783e00640 1 --2- 172.21.15.1:0/2750471130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe784076720 0x7fe7841a4620 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe774006010 tx=0x7fe774004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:26:28.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.695+0000 7fe780c00640 1 -- 172.21.15.1:0/2750471130 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77403f030 con 0x7fe784076720 2024-09-12T01:26:28.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.695+0000 7fe789800640 1 -- 172.21.15.1:0/2750471130 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe78419f1d0 con 0x7fe784076720 2024-09-12T01:26:28.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.695+0000 7fe789800640 1 -- 172.21.15.1:0/2750471130 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7841ab480 con 0x7fe784076720 2024-09-12T01:26:28.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.696+0000 7fe780c00640 1 -- 172.21.15.1:0/2750471130 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe774004440 con 0x7fe784076720 2024-09-12T01:26:28.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.696+0000 7fe780c00640 1 -- 172.21.15.1:0/2750471130 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe774049060 con 0x7fe784076720 2024-09-12T01:26:28.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.697+0000 7fe789800640 1 -- 172.21.15.1:0/2750471130 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe784077e60 con 0x7fe784076720 2024-09-12T01:26:28.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.699+0000 7fe780c00640 1 -- 172.21.15.1:0/2750471130 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fe774042030 con 0x7fe784076720 2024-09-12T01:26:28.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.701+0000 7fe780c00640 1 --2- 172.21.15.1:0/2750471130 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe7680a2c00 0x7fe7680a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:28.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.701+0000 7fe780c00640 1 -- 172.21.15.1:0/2750471130 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7fe7740eeff0 con 0x7fe784076720 2024-09-12T01:26:28.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.701+0000 7fe783400640 1 --2- 172.21.15.1:0/2750471130 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe7680a2c00 0x7fe7680a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:28.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.701+0000 7fe783400640 1 --2- 172.21.15.1:0/2750471130 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe7680a2c00 0x7fe7680a50c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe7840793e0 tx=0x7fe770005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:26:28.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.705+0000 7fe780c00640 1 -- 172.21.15.1:0/2750471130 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe7740bbd20 con 0x7fe784076720 2024-09-12T01:26:28.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:28 smithi053 ceph-mon[35158]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:28.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:28 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:28.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:28 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:28.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:28 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:28.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:28 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:26:28.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:28 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:26:28.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:28 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:26:28.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.949+0000 7fe789800640 1 -- 172.21.15.1:0/2750471130 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe78410da00 con 0x7fe784076720 2024-09-12T01:26:28.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.950+0000 7fe780c00640 1 -- 172.21.15.1:0/2750471130 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8860 (secure 0 0 0) 0x7fe7740c0020 con 0x7fe784076720 2024-09-12T01:26:28.950 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:26:28.950 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":11,"btime":"2024-09-12T01:26:28:291199+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14430,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/572192306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":572192306},{"type":"v1","addr":"172.21.15.1:6835","nonce":572192306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14442,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/1050734277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":1050734277},{"type":"v1","addr":"172.21.15.1:6837","nonce":1050734277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14481,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/3651849993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":3651849993},{"type":"v1","addr":"172.21.15.1:6839","nonce":3651849993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/2071825633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2071825633},{"type":"v1","addr":"172.21.15.104:6833","nonce":2071825633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2930310711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2930310711},{"type":"v1","addr":"172.21.15.53:6833","nonce":2930310711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2032381725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2032381725},{"type":"v1","addr":"172.21.15.53:6835","nonce":2032381725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/1547038540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1547038540},{"type":"v1","addr":"172.21.15.104:6835","nonce":1547038540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24493,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/464720247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":464720247},{"type":"v1","addr":"172.21.15.104:6837","nonce":464720247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/2848743754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2848743754},{"type":"v1","addr":"172.21.15.53:6837","nonce":2848743754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6839/1459976041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1459976041},{"type":"v1","addr":"172.21.15.104:6839","nonce":1459976041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-12T01:26:28.950 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 11 2024-09-12T01:26:28.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.953+0000 7fe789800640 1 -- 172.21.15.1:0/2750471130 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe7680a2c00 msgr2=0x7fe7680a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:28.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.953+0000 7fe789800640 1 --2- 172.21.15.1:0/2750471130 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe7680a2c00 0x7fe7680a50c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe7840793e0 tx=0x7fe770005f90 comp rx=0 tx=0).stop 2024-09-12T01:26:28.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.953+0000 7fe789800640 1 -- 172.21.15.1:0/2750471130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe784076720 msgr2=0x7fe7841a4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:28.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.953+0000 7fe789800640 1 --2- 172.21.15.1:0/2750471130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe784076720 0x7fe7841a4620 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe774006010 tx=0x7fe774004040 comp rx=0 tx=0).stop 2024-09-12T01:26:28.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.953+0000 7fe789800640 1 -- 172.21.15.1:0/2750471130 shutdown_connections 2024-09-12T01:26:28.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.953+0000 7fe789800640 1 --2- 172.21.15.1:0/2750471130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe784112870 0x7fe78419e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:28.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.953+0000 7fe789800640 1 --2- 172.21.15.1:0/2750471130 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe784077040 0x7fe7841a4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:28.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.953+0000 7fe789800640 1 --2- 172.21.15.1:0/2750471130 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe7680a2c00 0x7fe7680a50c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:28.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.953+0000 7fe789800640 1 --2- 172.21.15.1:0/2750471130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe784076720 0x7fe7841a4620 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:28.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.953+0000 7fe789800640 1 -- 172.21.15.1:0/2750471130 >> 172.21.15.1:0/2750471130 conn(0x7fe7840ff910 msgr2=0x7fe784101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:26:28.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.953+0000 7fe789800640 1 -- 172.21.15.1:0/2750471130 shutdown_connections 2024-09-12T01:26:28.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:28.953+0000 7fe789800640 1 -- 172.21.15.1:0/2750471130 wait complete. 2024-09-12T01:26:29.557 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:29 smithi001 ceph-mon[30387]: Deploying daemon mds.a on smithi001 2024-09-12T01:26:29.557 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:29 smithi001 ceph-mon[30387]: mds.? [v2:172.21.15.1:6838/3651849993,v1:172.21.15.1:6839/3651849993] up:boot 2024-09-12T01:26:29.558 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:29 smithi001 ceph-mon[30387]: fsmap 10 up:standby 2024-09-12T01:26:29.558 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:29 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T01:26:29.558 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:29 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/2750471130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:26:29.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:29 smithi104 ceph-mon[35219]: Deploying daemon mds.a on smithi001 2024-09-12T01:26:29.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:29 smithi104 ceph-mon[35219]: mds.? [v2:172.21.15.1:6838/3651849993,v1:172.21.15.1:6839/3651849993] up:boot 2024-09-12T01:26:29.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:29 smithi104 ceph-mon[35219]: fsmap 10 up:standby 2024-09-12T01:26:29.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:29 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T01:26:29.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:29 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/2750471130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:26:29.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:29 smithi053 ceph-mon[35158]: Deploying daemon mds.a on smithi001 2024-09-12T01:26:29.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:29 smithi053 ceph-mon[35158]: mds.? [v2:172.21.15.1:6838/3651849993,v1:172.21.15.1:6839/3651849993] up:boot 2024-09-12T01:26:29.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:29 smithi053 ceph-mon[35158]: fsmap 10 up:standby 2024-09-12T01:26:29.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:29 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T01:26:29.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:29 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/2750471130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:26:30.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:30 smithi104 ceph-mon[35219]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:30.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:30 smithi001 ceph-mon[30387]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:30.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:30 smithi053 ceph-mon[35158]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:31.902 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 12 01:26:31 smithi001 systemd[1]: Starting Ceph mds.a for c0e4f180-70a4-11ef-bcea-c7b262605968... 2024-09-12T01:26:32.177 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 12 01:26:31 smithi001 podman[71614]: 2024-09-12 01:26:31.807128609 +0000 UTC m=+0.027416451 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:26:32.178 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 12 01:26:31 smithi001 podman[71614]: 2024-09-12 01:26:31.937159501 +0000 UTC m=+0.157447317 container create 7f31d72433303eb83f41f3ce88a9b7abdfd908912ef44b6f8d7125dfff41ab25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-12T01:26:32.371 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:26:32.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:32 smithi104 ceph-mon[35219]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:32.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:32 smithi001 ceph-mon[30387]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:32.678 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 12 01:26:32 smithi001 podman[71614]: 2024-09-12 01:26:32.214754208 +0000 UTC m=+0.435042055 container init 7f31d72433303eb83f41f3ce88a9b7abdfd908912ef44b6f8d7125dfff41ab25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-a, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-12T01:26:32.678 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 12 01:26:32 smithi001 podman[71614]: 2024-09-12 01:26:32.223809108 +0000 UTC m=+0.444096925 container start 7f31d72433303eb83f41f3ce88a9b7abdfd908912ef44b6f8d7125dfff41ab25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2) 2024-09-12T01:26:32.678 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 12 01:26:32 smithi001 ceph-mds[71632]: -- 172.21.15.1:0/1953996898 <== mon.0 v2:172.21.15.1:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555b04577860 con 0x555b04555400 2024-09-12T01:26:32.678 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 12 01:26:32 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-a[71628]: starting mds.a at 2024-09-12T01:26:32.678 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 12 01:26:32 smithi001 bash[71614]: 7f31d72433303eb83f41f3ce88a9b7abdfd908912ef44b6f8d7125dfff41ab25 2024-09-12T01:26:32.678 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 12 01:26:32 smithi001 systemd[1]: Started Ceph mds.a for c0e4f180-70a4-11ef-bcea-c7b262605968. 2024-09-12T01:26:32.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:32 smithi053 ceph-mon[35158]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:32.757 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:26:33.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.383+0000 7ff2fb200640 1 -- 172.21.15.1:0/1339875660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2fc112870 msgr2=0x7ff2fc114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:33.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.383+0000 7ff2fb200640 1 --2- 172.21.15.1:0/1339875660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2fc112870 0x7ff2fc114c60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009ca0 tx=0x7ff2f402a8c0 comp rx=0 tx=0).stop 2024-09-12T01:26:33.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.383+0000 7ff2fb200640 1 -- 172.21.15.1:0/1339875660 shutdown_connections 2024-09-12T01:26:33.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.383+0000 7ff2fb200640 1 --2- 172.21.15.1:0/1339875660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2fc112870 0x7ff2fc114c60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:33.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.383+0000 7ff2fb200640 1 --2- 172.21.15.1:0/1339875660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2fc077040 0x7ff2fc112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:33.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.383+0000 7ff2fb200640 1 --2- 172.21.15.1:0/1339875660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2fc076720 0x7ff2fc076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:33.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.383+0000 7ff2fb200640 1 -- 172.21.15.1:0/1339875660 >> 172.21.15.1:0/1339875660 conn(0x7ff2fc0ff8d0 msgr2=0x7ff2fc101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:26:33.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.383+0000 7ff2fb200640 1 -- 172.21.15.1:0/1339875660 shutdown_connections 2024-09-12T01:26:33.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.383+0000 7ff2fb200640 1 -- 172.21.15.1:0/1339875660 wait complete. 2024-09-12T01:26:33.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.384+0000 7ff2fb200640 1 Processor -- start 2024-09-12T01:26:33.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.384+0000 7ff2fb200640 1 -- start start 2024-09-12T01:26:33.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.385+0000 7ff2fb200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2fc076720 0x7ff2fc1a45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:33.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.385+0000 7ff2fb200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2fc077040 0x7ff2fc1a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:33.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.385+0000 7ff2fb200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2fc112870 0x7ff2fc19e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:33.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.385+0000 7ff2fb200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc118420 con 0x7ff2fc077040 2024-09-12T01:26:33.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.385+0000 7ff2fb200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc1182a0 con 0x7ff2fc112870 2024-09-12T01:26:33.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.385+0000 7ff2fb200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc1185a0 con 0x7ff2fc076720 2024-09-12T01:26:33.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.385+0000 7ff2fa800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2fc112870 0x7ff2fc19e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:33.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.385+0000 7ff2f3e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2fc077040 0x7ff2fc1a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:33.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.385+0000 7ff2fa800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2fc112870 0x7ff2fc19e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:39962/0 (socket says 172.21.15.1:39962) 2024-09-12T01:26:33.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.385+0000 7ff2fa800640 1 -- 172.21.15.1:0/1667370201 learned_addr learned my addr 172.21.15.1:0/1667370201 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:26:33.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.385+0000 7ff2f9e00640 1 --2- 172.21.15.1:0/1667370201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2fc076720 0x7ff2fc1a45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:33.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.385+0000 7ff2fa800640 1 -- 172.21.15.1:0/1667370201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2fc076720 msgr2=0x7ff2fc1a45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:33.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.385+0000 7ff2fa800640 1 --2- 172.21.15.1:0/1667370201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2fc076720 0x7ff2fc1a45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:33.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.385+0000 7ff2fa800640 1 -- 172.21.15.1:0/1667370201 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2fc077040 msgr2=0x7ff2fc1a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:33.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.385+0000 7ff2fa800640 1 --2- 172.21.15.1:0/1667370201 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2fc077040 0x7ff2fc1a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:33.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.385+0000 7ff2fa800640 1 -- 172.21.15.1:0/1667370201 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2fc19ef00 con 0x7ff2fc112870 2024-09-12T01:26:33.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.386+0000 7ff2fa800640 1 --2- 172.21.15.1:0/1667370201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2fc112870 0x7ff2fc19e690 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4005e30 tx=0x7ff2f4003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:26:33.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.386+0000 7ff2f2a00640 1 -- 172.21.15.1:0/1667370201 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4031650 con 0x7ff2fc112870 2024-09-12T01:26:33.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.386+0000 7ff2f2a00640 1 -- 172.21.15.1:0/1667370201 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2f40317f0 con 0x7ff2fc112870 2024-09-12T01:26:33.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.386+0000 7ff2fb200640 1 -- 172.21.15.1:0/1667370201 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2fc19f190 con 0x7ff2fc112870 2024-09-12T01:26:33.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.386+0000 7ff2f2a00640 1 -- 172.21.15.1:0/1667370201 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f402fa10 con 0x7ff2fc112870 2024-09-12T01:26:33.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.386+0000 7ff2fb200640 1 -- 172.21.15.1:0/1667370201 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2fc1ab540 con 0x7ff2fc112870 2024-09-12T01:26:33.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.387+0000 7ff2fb200640 1 -- 172.21.15.1:0/1667370201 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2fc19f550 con 0x7ff2fc112870 2024-09-12T01:26:33.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.389+0000 7ff2f2a00640 1 -- 172.21.15.1:0/1667370201 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7ff2f402fbd0 con 0x7ff2fc112870 2024-09-12T01:26:33.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.390+0000 7ff2f2a00640 1 --2- 172.21.15.1:0/1667370201 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff2e00a2c00 0x7ff2e00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:33.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.390+0000 7ff2f2a00640 1 -- 172.21.15.1:0/1667370201 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7ff2f40f0040 con 0x7ff2fc112870 2024-09-12T01:26:33.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.391+0000 7ff2f9e00640 1 --2- 172.21.15.1:0/1667370201 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff2e00a2c00 0x7ff2e00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:33.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.391+0000 7ff2f9e00640 1 --2- 172.21.15.1:0/1667370201 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff2e00a2c00 0x7ff2e00a50c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec009bd0 tx=0x7ff2ec005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:26:33.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.392+0000 7ff2f2a00640 1 -- 172.21.15.1:0/1667370201 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff2f40bd3a0 con 0x7ff2fc112870 2024-09-12T01:26:33.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:33 smithi053 ceph-mon[35158]: mds.? [v2:172.21.15.1:6840/2464781973,v1:172.21.15.1:6841/2464781973] up:boot 2024-09-12T01:26:33.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:33 smithi053 ceph-mon[35158]: fsmap 11 up:standby 2024-09-12T01:26:33.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:33 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T01:26:33.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:33 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:33.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:33 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:33.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:33 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:33.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:33 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:26:33.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:33 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:26:33.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:33 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:26:33.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:33 smithi053 ceph-mon[35158]: Deploying daemon mds.b on smithi053 2024-09-12T01:26:33.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:33 smithi104 ceph-mon[35219]: mds.? [v2:172.21.15.1:6840/2464781973,v1:172.21.15.1:6841/2464781973] up:boot 2024-09-12T01:26:33.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:33 smithi104 ceph-mon[35219]: fsmap 11 up:standby 2024-09-12T01:26:33.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:33 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T01:26:33.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:33 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:33.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:33 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:33.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:33 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:33.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:33 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:26:33.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:33 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:26:33.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:33 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:26:33.576 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:33 smithi104 ceph-mon[35219]: Deploying daemon mds.b on smithi053 2024-09-12T01:26:33.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.612+0000 7ff2fb200640 1 -- 172.21.15.1:0/1667370201 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2fc077e60 con 0x7ff2fc112870 2024-09-12T01:26:33.613 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:33 smithi001 ceph-mon[30387]: mds.? [v2:172.21.15.1:6840/2464781973,v1:172.21.15.1:6841/2464781973] up:boot 2024-09-12T01:26:33.613 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:33 smithi001 ceph-mon[30387]: fsmap 11 up:standby 2024-09-12T01:26:33.614 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:33 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T01:26:33.614 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:33 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:33.614 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:33 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:33.614 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:33 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:33.614 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:33 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:26:33.614 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:33 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:26:33.614 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:33 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:26:33.614 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:33 smithi001 ceph-mon[30387]: Deploying daemon mds.b on smithi053 2024-09-12T01:26:33.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.613+0000 7ff2f2a00640 1 -- 172.21.15.1:0/1667370201 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9683 (secure 0 0 0) 0x7ff2f40319a0 con 0x7ff2fc112870 2024-09-12T01:26:33.615 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:26:33.615 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":12,"btime":"2024-09-12T01:26:32:307575+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14430,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/572192306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":572192306},{"type":"v1","addr":"172.21.15.1:6835","nonce":572192306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14442,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/1050734277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":1050734277},{"type":"v1","addr":"172.21.15.1:6837","nonce":1050734277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14481,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/3651849993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":3651849993},{"type":"v1","addr":"172.21.15.1:6839","nonce":3651849993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14493,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/2464781973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2464781973},{"type":"v1","addr":"172.21.15.1:6841","nonce":2464781973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/2071825633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2071825633},{"type":"v1","addr":"172.21.15.104:6833","nonce":2071825633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2930310711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2930310711},{"type":"v1","addr":"172.21.15.53:6833","nonce":2930310711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2032381725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2032381725},{"type":"v1","addr":"172.21.15.53:6835","nonce":2032381725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/1547038540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1547038540},{"type":"v1","addr":"172.21.15.104:6835","nonce":1547038540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24493,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/464720247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":464720247},{"type":"v1","addr":"172.21.15.104:6837","nonce":464720247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/2848743754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2848743754},{"type":"v1","addr":"172.21.15.53:6837","nonce":2848743754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6839/1459976041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1459976041},{"type":"v1","addr":"172.21.15.104:6839","nonce":1459976041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-12T01:26:33.616 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 12 2024-09-12T01:26:33.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.615+0000 7ff2fb200640 1 -- 172.21.15.1:0/1667370201 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff2e00a2c00 msgr2=0x7ff2e00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:33.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.615+0000 7ff2fb200640 1 --2- 172.21.15.1:0/1667370201 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff2e00a2c00 0x7ff2e00a50c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec009bd0 tx=0x7ff2ec005f90 comp rx=0 tx=0).stop 2024-09-12T01:26:33.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.615+0000 7ff2fb200640 1 -- 172.21.15.1:0/1667370201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2fc112870 msgr2=0x7ff2fc19e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:33.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.615+0000 7ff2fb200640 1 --2- 172.21.15.1:0/1667370201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2fc112870 0x7ff2fc19e690 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4005e30 tx=0x7ff2f4003f10 comp rx=0 tx=0).stop 2024-09-12T01:26:33.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.615+0000 7ff2fb200640 1 -- 172.21.15.1:0/1667370201 shutdown_connections 2024-09-12T01:26:33.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.615+0000 7ff2fb200640 1 --2- 172.21.15.1:0/1667370201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2fc112870 0x7ff2fc19e690 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:33.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.615+0000 7ff2fb200640 1 --2- 172.21.15.1:0/1667370201 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2fc077040 0x7ff2fc1a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:33.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.615+0000 7ff2fb200640 1 --2- 172.21.15.1:0/1667370201 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff2e00a2c00 0x7ff2e00a50c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:33.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.615+0000 7ff2fb200640 1 --2- 172.21.15.1:0/1667370201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2fc076720 0x7ff2fc1a45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:33.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.615+0000 7ff2fb200640 1 -- 172.21.15.1:0/1667370201 >> 172.21.15.1:0/1667370201 conn(0x7ff2fc0ff8d0 msgr2=0x7ff2fc101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:26:33.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.616+0000 7ff2fb200640 1 -- 172.21.15.1:0/1667370201 shutdown_connections 2024-09-12T01:26:33.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:33.616+0000 7ff2fb200640 1 -- 172.21.15.1:0/1667370201 wait complete. 2024-09-12T01:26:34.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:34 smithi053 ceph-mon[35158]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:34.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:34 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/1667370201' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:26:34.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:34 smithi001 ceph-mon[30387]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:34.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:34 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/1667370201' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:26:34.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:34 smithi104 ceph-mon[35219]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:34.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:34 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/1667370201' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:26:35.239 INFO:journalctl@ceph.mds.b.smithi053.stdout:Sep 12 01:26:34 smithi053 systemd[1]: Started Ceph mds.b for c0e4f180-70a4-11ef-bcea-c7b262605968. 2024-09-12T01:26:36.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:35 smithi104 ceph-mon[35219]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:36.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:35 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:36.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:35 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:36.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:35 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:36.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:35 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:36.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:35 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:36.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:35 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:26:36.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:35 smithi104 ceph-mon[35219]: mds.? [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] up:boot 2024-09-12T01:26:36.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:35 smithi104 ceph-mon[35219]: fsmap 12 up:standby 2024-09-12T01:26:36.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:35 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T01:26:36.149 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:35 smithi053 ceph-mon[35158]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:36.150 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:35 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:36.150 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:35 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:36.150 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:35 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:36.150 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:35 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:36.150 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:35 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:36.150 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:35 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:26:36.150 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:35 smithi053 ceph-mon[35158]: mds.? [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] up:boot 2024-09-12T01:26:36.150 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:35 smithi053 ceph-mon[35158]: fsmap 12 up:standby 2024-09-12T01:26:36.150 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:35 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T01:26:36.283 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:35 smithi001 ceph-mon[30387]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:36.283 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:35 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:36.284 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:35 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:36.284 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:35 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:36.284 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:35 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:36.284 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:35 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:36.284 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:35 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:26:36.284 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:35 smithi001 ceph-mon[30387]: mds.? [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] up:boot 2024-09-12T01:26:36.284 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:35 smithi001 ceph-mon[30387]: fsmap 12 up:standby 2024-09-12T01:26:36.284 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:35 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T01:26:36.411 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:26:36.825 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:26:37.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.387+0000 7f9ed7400640 1 -- 172.21.15.1:0/1036502088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ed0074590 msgr2=0x7f9ed0074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:37.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.387+0000 7f9ed7400640 1 --2- 172.21.15.1:0/1036502088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ed0074590 0x7f9ed0074970 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4009900 tx=0x7f9ec402a480 comp rx=0 tx=0).stop 2024-09-12T01:26:37.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.387+0000 7f9ed7400640 1 -- 172.21.15.1:0/1036502088 shutdown_connections 2024-09-12T01:26:37.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.387+0000 7f9ed7400640 1 --2- 172.21.15.1:0/1036502088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ed0112b90 0x7f9ed0114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:37.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.387+0000 7f9ed7400640 1 --2- 172.21.15.1:0/1036502088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ed0074eb0 0x7f9ed0112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:37.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.387+0000 7f9ed7400640 1 --2- 172.21.15.1:0/1036502088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ed0074590 0x7f9ed0074970 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:37.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.387+0000 7f9ed7400640 1 -- 172.21.15.1:0/1036502088 >> 172.21.15.1:0/1036502088 conn(0x7f9ed00fd8c0 msgr2=0x7f9ed00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:26:37.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.387+0000 7f9ed7400640 1 -- 172.21.15.1:0/1036502088 shutdown_connections 2024-09-12T01:26:37.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.387+0000 7f9ed7400640 1 -- 172.21.15.1:0/1036502088 wait complete. 2024-09-12T01:26:37.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.388+0000 7f9ed7400640 1 Processor -- start 2024-09-12T01:26:37.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.388+0000 7f9ed7400640 1 -- start start 2024-09-12T01:26:37.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.389+0000 7f9ed7400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ed0074590 0x7f9ed01a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:37.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.389+0000 7f9ed7400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ed0074eb0 0x7f9ed01a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:37.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.389+0000 7f9ed7400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ed0112b90 0x7f9ed019e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:37.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.389+0000 7f9ed7400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed0118570 con 0x7f9ed0074eb0 2024-09-12T01:26:37.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.389+0000 7f9ed7400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed01183f0 con 0x7f9ed0112b90 2024-09-12T01:26:37.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.389+0000 7f9ed7400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed01186f0 con 0x7f9ed0074590 2024-09-12T01:26:37.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.389+0000 7f9ecfe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ed0074eb0 0x7f9ed01a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:37.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.389+0000 7f9ed4a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ed0074590 0x7f9ed01a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:37.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.389+0000 7f9ed4a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ed0074590 0x7f9ed01a4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:38024/0 (socket says 172.21.15.1:38024) 2024-09-12T01:26:37.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.389+0000 7f9ed4a00640 1 -- 172.21.15.1:0/818273031 learned_addr learned my addr 172.21.15.1:0/818273031 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:26:37.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.389+0000 7f9ed5400640 1 --2- 172.21.15.1:0/818273031 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ed0112b90 0x7f9ed019e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:37.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.390+0000 7f9ed4a00640 1 -- 172.21.15.1:0/818273031 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ed0112b90 msgr2=0x7f9ed019e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:37.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.390+0000 7f9ed4a00640 1 --2- 172.21.15.1:0/818273031 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ed0112b90 0x7f9ed019e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:37.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.390+0000 7f9ed4a00640 1 -- 172.21.15.1:0/818273031 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ed0074eb0 msgr2=0x7f9ed01a4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:37.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.390+0000 7f9ed4a00640 1 --2- 172.21.15.1:0/818273031 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ed0074eb0 0x7f9ed01a4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:37.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.390+0000 7f9ed4a00640 1 -- 172.21.15.1:0/818273031 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ed019f000 con 0x7f9ed0074590 2024-09-12T01:26:37.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.390+0000 7f9ed4a00640 1 --2- 172.21.15.1:0/818273031 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ed0074590 0x7f9ed01a4710 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4006010 tx=0x7f9ec4004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:26:37.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.390+0000 7f9ecd600640 1 -- 172.21.15.1:0/818273031 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec403f030 con 0x7f9ed0074590 2024-09-12T01:26:37.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.390+0000 7f9ecd600640 1 -- 172.21.15.1:0/818273031 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ec4004440 con 0x7f9ed0074590 2024-09-12T01:26:37.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.390+0000 7f9ed7400640 1 -- 172.21.15.1:0/818273031 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ed019f290 con 0x7f9ed0074590 2024-09-12T01:26:37.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.390+0000 7f9ecd600640 1 -- 172.21.15.1:0/818273031 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4035070 con 0x7f9ed0074590 2024-09-12T01:26:37.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.390+0000 7f9ed7400640 1 -- 172.21.15.1:0/818273031 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ed01ab5e0 con 0x7f9ed0074590 2024-09-12T01:26:37.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.392+0000 7f9ed7400640 1 -- 172.21.15.1:0/818273031 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ed0074970 con 0x7f9ed0074590 2024-09-12T01:26:37.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.393+0000 7f9ecd600640 1 -- 172.21.15.1:0/818273031 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f9ec4042030 con 0x7f9ed0074590 2024-09-12T01:26:37.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.395+0000 7f9ecd600640 1 --2- 172.21.15.1:0/818273031 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f9eb80a2c00 0x7f9eb80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:37.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.395+0000 7f9ecd600640 1 -- 172.21.15.1:0/818273031 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7f9ec40eef10 con 0x7f9ed0074590 2024-09-12T01:26:37.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.395+0000 7f9ecfe00640 1 --2- 172.21.15.1:0/818273031 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f9eb80a2c00 0x7f9eb80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:37.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.395+0000 7f9ecfe00640 1 --2- 172.21.15.1:0/818273031 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f9eb80a2c00 0x7f9eb80a50c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9ed00ffec0 tx=0x7f9ebc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:26:37.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.399+0000 7f9ecd600640 1 -- 172.21.15.1:0/818273031 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9ec40bbc20 con 0x7f9ed0074590 2024-09-12T01:26:37.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.625+0000 7f9ed7400640 1 -- 172.21.15.1:0/818273031 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ed019fe00 con 0x7f9ed0074590 2024-09-12T01:26:37.625 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:26:37.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.626+0000 7f9ecd600640 1 -- 172.21.15.1:0/818273031 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10509 (secure 0 0 0) 0x7f9ec40bff20 con 0x7f9ed0074590 2024-09-12T01:26:37.626 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":13,"btime":"2024-09-12T01:26:35:413210+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14430,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/572192306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":572192306},{"type":"v1","addr":"172.21.15.1:6835","nonce":572192306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14442,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/1050734277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":1050734277},{"type":"v1","addr":"172.21.15.1:6837","nonce":1050734277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14481,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/3651849993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":3651849993},{"type":"v1","addr":"172.21.15.1:6839","nonce":3651849993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14493,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/2464781973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2464781973},{"type":"v1","addr":"172.21.15.1:6841","nonce":2464781973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/2071825633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2071825633},{"type":"v1","addr":"172.21.15.104:6833","nonce":2071825633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2930310711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2930310711},{"type":"v1","addr":"172.21.15.53:6833","nonce":2930310711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2032381725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2032381725},{"type":"v1","addr":"172.21.15.53:6835","nonce":2032381725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/1547038540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1547038540},{"type":"v1","addr":"172.21.15.104:6835","nonce":1547038540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24493,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/464720247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":464720247},{"type":"v1","addr":"172.21.15.104:6837","nonce":464720247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/2848743754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2848743754},{"type":"v1","addr":"172.21.15.53:6837","nonce":2848743754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6839/1459976041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1459976041},{"type":"v1","addr":"172.21.15.104:6839","nonce":1459976041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24545,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6839/4019389301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":4019389301},{"type":"v1","addr":"172.21.15.53:6839","nonce":4019389301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-12T01:26:37.626 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 13 2024-09-12T01:26:37.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.628+0000 7f9ed7400640 1 -- 172.21.15.1:0/818273031 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f9eb80a2c00 msgr2=0x7f9eb80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:37.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.628+0000 7f9ed7400640 1 --2- 172.21.15.1:0/818273031 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f9eb80a2c00 0x7f9eb80a50c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9ed00ffec0 tx=0x7f9ebc005f90 comp rx=0 tx=0).stop 2024-09-12T01:26:37.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.628+0000 7f9ed7400640 1 -- 172.21.15.1:0/818273031 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ed0074590 msgr2=0x7f9ed01a4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:37.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.628+0000 7f9ed7400640 1 --2- 172.21.15.1:0/818273031 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ed0074590 0x7f9ed01a4710 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4006010 tx=0x7f9ec4004040 comp rx=0 tx=0).stop 2024-09-12T01:26:37.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.628+0000 7f9ed7400640 1 -- 172.21.15.1:0/818273031 shutdown_connections 2024-09-12T01:26:37.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.628+0000 7f9ed7400640 1 --2- 172.21.15.1:0/818273031 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ed0112b90 0x7f9ed019e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:37.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.628+0000 7f9ed7400640 1 --2- 172.21.15.1:0/818273031 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ed0074eb0 0x7f9ed01a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:37.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.628+0000 7f9ed7400640 1 --2- 172.21.15.1:0/818273031 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f9eb80a2c00 0x7f9eb80a50c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:37.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.628+0000 7f9ed7400640 1 --2- 172.21.15.1:0/818273031 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ed0074590 0x7f9ed01a4710 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:37.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.628+0000 7f9ed7400640 1 -- 172.21.15.1:0/818273031 >> 172.21.15.1:0/818273031 conn(0x7f9ed00fd8c0 msgr2=0x7f9ed00ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:26:37.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.628+0000 7f9ed7400640 1 -- 172.21.15.1:0/818273031 shutdown_connections 2024-09-12T01:26:37.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:37.629+0000 7f9ed7400640 1 -- 172.21.15.1:0/818273031 wait complete. 2024-09-12T01:26:37.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:37 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:37.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:37 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:37.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:37 smithi001 ceph-mon[30387]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:37.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:37 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:37.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:37 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:37.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:37 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/818273031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:26:37.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:37 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:37.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:37 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:37.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:37 smithi053 ceph-mon[35158]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:37.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:37 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:37.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:37 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:37.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:37 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/818273031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:26:38.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:37 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:38.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:37 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:38.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:37 smithi104 ceph-mon[35219]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:38.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:37 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:38.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:37 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:38.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:37 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/818273031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:26:38.593 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-12T01:26:38.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:38 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:38.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:38 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:38.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:38 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:26:38.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:38 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:26:38.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:38 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:38.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:38 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:26:38.986 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:26:38.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:38 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:38.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:38 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:38.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:38 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:26:38.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:38 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:26:38.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:38 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:38.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:38 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:26:39.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:38 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:39.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:38 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:39.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:38 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:26:39.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:38 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:26:39.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:38 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:39.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:38 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:26:39.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.627+0000 7fed6f400640 1 -- 172.21.15.1:0/678826023 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed68101de0 msgr2=0x7fed681021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:39.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.627+0000 7fed6f400640 1 --2- 172.21.15.1:0/678826023 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed68101de0 0x7fed681021c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fed5c009900 tx=0x7fed5c02a3f0 comp rx=0 tx=0).stop 2024-09-12T01:26:39.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.627+0000 7fed6f400640 1 -- 172.21.15.1:0/678826023 shutdown_connections 2024-09-12T01:26:39.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.627+0000 7fed6f400640 1 --2- 172.21.15.1:0/678826023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed68106f10 0x7fed6810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:39.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.627+0000 7fed6f400640 1 --2- 172.21.15.1:0/678826023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed68102790 0x7fed681067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:39.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.627+0000 7fed6f400640 1 --2- 172.21.15.1:0/678826023 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed68101de0 0x7fed681021c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:39.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.627+0000 7fed6f400640 1 -- 172.21.15.1:0/678826023 >> 172.21.15.1:0/678826023 conn(0x7fed680fd8c0 msgr2=0x7fed680ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:26:39.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.627+0000 7fed6f400640 1 -- 172.21.15.1:0/678826023 shutdown_connections 2024-09-12T01:26:39.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.628+0000 7fed6f400640 1 -- 172.21.15.1:0/678826023 wait complete. 2024-09-12T01:26:39.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.628+0000 7fed6f400640 1 Processor -- start 2024-09-12T01:26:39.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.629+0000 7fed6f400640 1 -- start start 2024-09-12T01:26:39.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.629+0000 7fed6f400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed68101de0 0x7fed6819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:39.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.629+0000 7fed6f400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed68102790 0x7fed6819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:39.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.629+0000 7fed6f400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed68106f10 0x7fed6819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:39.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.629+0000 7fed6f400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fed68111f70 con 0x7fed68102790 2024-09-12T01:26:39.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.629+0000 7fed6f400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fed68111df0 con 0x7fed68101de0 2024-09-12T01:26:39.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.629+0000 7fed6f400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fed681120f0 con 0x7fed68106f10 2024-09-12T01:26:39.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.629+0000 7fed6ca00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed68101de0 0x7fed6819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:39.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.629+0000 7fed67e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed68102790 0x7fed6819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:39.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.629+0000 7fed6ca00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed68101de0 0x7fed6819a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:59232/0 (socket says 172.21.15.1:59232) 2024-09-12T01:26:39.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.629+0000 7fed67e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed68102790 0x7fed6819ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52194/0 (socket says 172.21.15.1:52194) 2024-09-12T01:26:39.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.629+0000 7fed6ca00640 1 -- 172.21.15.1:0/1050566619 learned_addr learned my addr 172.21.15.1:0/1050566619 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:26:39.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.629+0000 7fed6d400640 1 --2- 172.21.15.1:0/1050566619 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed68106f10 0x7fed6819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:39.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.630+0000 7fed6ca00640 1 -- 172.21.15.1:0/1050566619 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed68106f10 msgr2=0x7fed6819ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:39.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.630+0000 7fed6ca00640 1 --2- 172.21.15.1:0/1050566619 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed68106f10 0x7fed6819ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:39.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.630+0000 7fed6ca00640 1 -- 172.21.15.1:0/1050566619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed68102790 msgr2=0x7fed6819ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:39.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.630+0000 7fed6ca00640 1 --2- 172.21.15.1:0/1050566619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed68102790 0x7fed6819ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:39.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.630+0000 7fed6ca00640 1 -- 172.21.15.1:0/1050566619 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed6819f600 con 0x7fed68101de0 2024-09-12T01:26:39.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.630+0000 7fed6ca00640 1 --2- 172.21.15.1:0/1050566619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed68101de0 0x7fed6819a650 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fed5c006010 tx=0x7fed5c004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:26:39.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.630+0000 7fed65600640 1 -- 172.21.15.1:0/1050566619 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5c03f030 con 0x7fed68101de0 2024-09-12T01:26:39.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.630+0000 7fed6f400640 1 -- 172.21.15.1:0/1050566619 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed6819f890 con 0x7fed68101de0 2024-09-12T01:26:39.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.630+0000 7fed65600640 1 -- 172.21.15.1:0/1050566619 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed5c004440 con 0x7fed68101de0 2024-09-12T01:26:39.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.630+0000 7fed65600640 1 -- 172.21.15.1:0/1050566619 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5c035070 con 0x7fed68101de0 2024-09-12T01:26:39.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.630+0000 7fed6f400640 1 -- 172.21.15.1:0/1050566619 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed681a71d0 con 0x7fed68101de0 2024-09-12T01:26:39.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.631+0000 7fed6f400640 1 -- 172.21.15.1:0/1050566619 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed681032c0 con 0x7fed68101de0 2024-09-12T01:26:39.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.633+0000 7fed65600640 1 -- 172.21.15.1:0/1050566619 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fed5c042030 con 0x7fed68101de0 2024-09-12T01:26:39.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.634+0000 7fed65600640 1 --2- 172.21.15.1:0/1050566619 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fed500a2b80 0x7fed500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:39.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.635+0000 7fed65600640 1 -- 172.21.15.1:0/1050566619 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7fed5c0ef0f0 con 0x7fed68101de0 2024-09-12T01:26:39.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.635+0000 7fed67e00640 1 --2- 172.21.15.1:0/1050566619 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fed500a2b80 0x7fed500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:39.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.635+0000 7fed67e00640 1 --2- 172.21.15.1:0/1050566619 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fed500a2b80 0x7fed500a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fed68076480 tx=0x7fed54005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:26:39.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.637+0000 7fed65600640 1 -- 172.21.15.1:0/1050566619 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fed5c0bc450 con 0x7fed68101de0 2024-09-12T01:26:39.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.835+0000 7fed6f400640 1 -- 172.21.15.1:0/1050566619 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fed681a73a0 con 0x7fed68101de0 2024-09-12T01:26:39.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.835+0000 7fed65600640 1 -- 172.21.15.1:0/1050566619 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v80) ==== 84+0+65 (secure 0 0 0) 0x7fed5c0bbd50 con 0x7fed68101de0 2024-09-12T01:26:39.835 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:26:39.835 INFO:teuthology.orchestra.run.smithi001.stdout:[ 2024-09-12T01:26:39.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:26:39.835 INFO:teuthology.orchestra.run.smithi001.stdout: "poolnum": 1, 2024-09-12T01:26:39.835 INFO:teuthology.orchestra.run.smithi001.stdout: "poolname": ".mgr" 2024-09-12T01:26:39.835 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:26:39.835 INFO:teuthology.orchestra.run.smithi001.stdout:] 2024-09-12T01:26:39.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.838+0000 7fed6f400640 1 -- 172.21.15.1:0/1050566619 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fed500a2b80 msgr2=0x7fed500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:39.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.838+0000 7fed6f400640 1 --2- 172.21.15.1:0/1050566619 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fed500a2b80 0x7fed500a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fed68076480 tx=0x7fed54005f90 comp rx=0 tx=0).stop 2024-09-12T01:26:39.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.838+0000 7fed6f400640 1 -- 172.21.15.1:0/1050566619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed68101de0 msgr2=0x7fed6819a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:39.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.838+0000 7fed6f400640 1 --2- 172.21.15.1:0/1050566619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed68101de0 0x7fed6819a650 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fed5c006010 tx=0x7fed5c004040 comp rx=0 tx=0).stop 2024-09-12T01:26:39.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.839+0000 7fed6f400640 1 -- 172.21.15.1:0/1050566619 shutdown_connections 2024-09-12T01:26:39.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.839+0000 7fed6f400640 1 --2- 172.21.15.1:0/1050566619 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fed500a2b80 0x7fed500a5040 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:39.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.839+0000 7fed6f400640 1 --2- 172.21.15.1:0/1050566619 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed68106f10 0x7fed6819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:39.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.839+0000 7fed6f400640 1 --2- 172.21.15.1:0/1050566619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed68102790 0x7fed6819ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:39.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.839+0000 7fed6f400640 1 --2- 172.21.15.1:0/1050566619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed68101de0 0x7fed6819a650 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:39.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.839+0000 7fed6f400640 1 -- 172.21.15.1:0/1050566619 >> 172.21.15.1:0/1050566619 conn(0x7fed680fd8c0 msgr2=0x7fed680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:26:39.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.839+0000 7fed6f400640 1 -- 172.21.15.1:0/1050566619 shutdown_connections 2024-09-12T01:26:39.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:39.839+0000 7fed6f400640 1 -- 172.21.15.1:0/1050566619 wait complete. 2024-09-12T01:26:40.596 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:40 smithi001 ceph-mon[30387]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:40.596 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:40 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:40.597 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:40 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/1050566619' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-12T01:26:40.597 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:40 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:40.597 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:40 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:40.597 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:40 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:40.597 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:40 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:40.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:40 smithi053 ceph-mon[35158]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:40.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:40 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:40.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:40 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/1050566619' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-12T01:26:40.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:40 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:40.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:40 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:40.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:40 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:40.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:40 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:40.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:40 smithi104 ceph-mon[35219]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:40.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:40 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:40.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:40 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/1050566619' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-12T01:26:40.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:40 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:40.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:40 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:40.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:40 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:40.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:40 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:41.127 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-12T01:26:41.128 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-12T01:26:41.659 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:26:42.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:42.279+0000 7fd379c00640 1 -- 172.21.15.1:0/391785788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3741019c0 msgr2=0x7fd374101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:42.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:42.279+0000 7fd379c00640 1 --2- 172.21.15.1:0/391785788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3741019c0 0x7fd374101e20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd35c009900 tx=0x7fd35c02a400 comp rx=0 tx=0).stop 2024-09-12T01:26:42.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:42.279+0000 7fd379c00640 1 -- 172.21.15.1:0/391785788 shutdown_connections 2024-09-12T01:26:42.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:42.279+0000 7fd379c00640 1 --2- 172.21.15.1:0/391785788 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd374102360 0x7fd37410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:42.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:42.279+0000 7fd379c00640 1 --2- 172.21.15.1:0/391785788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3741019c0 0x7fd374101e20 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:42.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:42.279+0000 7fd379c00640 1 --2- 172.21.15.1:0/391785788 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3741079c0 0x7fd374107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:42.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:42.279+0000 7fd379c00640 1 -- 172.21.15.1:0/391785788 >> 172.21.15.1:0/391785788 conn(0x7fd3740fd740 msgr2=0x7fd3740ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:26:42.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:42.279+0000 7fd379c00640 1 -- 172.21.15.1:0/391785788 shutdown_connections 2024-09-12T01:26:42.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:42.279+0000 7fd379c00640 1 -- 172.21.15.1:0/391785788 wait complete. 2024-09-12T01:26:42.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:42.280+0000 7fd379c00640 1 Processor -- start 2024-09-12T01:26:42.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:42.280+0000 7fd379c00640 1 -- start start 2024-09-12T01:26:42.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:42.281+0000 7fd379c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3741019c0 0x7fd37419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:42.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:42.281+0000 7fd379c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd374102360 0x7fd37419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:42.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:42.281+0000 7fd379c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3741079c0 0x7fd37419ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:42.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:42.281+0000 7fd379c00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd374111dc0 con 0x7fd374102360 2024-09-12T01:26:42.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:42.281+0000 7fd379c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd374111c40 con 0x7fd3741079c0 2024-09-12T01:26:42.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:42.281+0000 7fd379c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd374111f40 con 0x7fd3741019c0 2024-09-12T01:26:42.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:42.281+0000 7fd36be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd374102360 0x7fd37419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:42.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:42.281+0000 7fd373e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3741019c0 0x7fd37419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:42.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:42.281+0000 7fd373e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3741019c0 0x7fd37419a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:57762/0 (socket says 172.21.15.1:57762) 2024-09-12T01:26:42.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:42.281+0000 7fd373e00640 1 -- 172.21.15.1:0/2184252359 learned_addr learned my addr 172.21.15.1:0/2184252359 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:26:42.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:42.281+0000 7fd379200640 1 --2- 172.21.15.1:0/2184252359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3741079c0 0x7fd37419ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:42.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:42.281+0000 7fd36be00640 1 -- 172.21.15.1:0/2184252359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3741019c0 msgr2=0x7fd37419a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:42.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:42.281+0000 7fd36be00640 1 --2- 172.21.15.1:0/2184252359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3741019c0 0x7fd37419a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:42.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:42.281+0000 7fd36be00640 1 -- 172.21.15.1:0/2184252359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3741079c0 msgr2=0x7fd37419ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:42.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:42.281+0000 7fd36be00640 1 --2- 172.21.15.1:0/2184252359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3741079c0 0x7fd37419ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:42.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:42.281+0000 7fd36be00640 1 -- 172.21.15.1:0/2184252359 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd37419f4f0 con 0x7fd374102360 2024-09-12T01:26:42.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:42.282+0000 7fd36be00640 1 --2- 172.21.15.1:0/2184252359 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd374102360 0x7fd37419aa80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd35c005940 tx=0x7fd35c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:26:42.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:42.282+0000 7fd371600640 1 -- 172.21.15.1:0/2184252359 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c03f070 con 0x7fd374102360 2024-09-12T01:26:42.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:42.282+0000 7fd371600640 1 -- 172.21.15.1:0/2184252359 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd35c031c30 con 0x7fd374102360 2024-09-12T01:26:42.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:42.282+0000 7fd379c00640 1 -- 172.21.15.1:0/2184252359 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd37419f780 con 0x7fd374102360 2024-09-12T01:26:42.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:42.282+0000 7fd371600640 1 -- 172.21.15.1:0/2184252359 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c035030 con 0x7fd374102360 2024-09-12T01:26:42.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:42.282+0000 7fd379c00640 1 -- 172.21.15.1:0/2184252359 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3741a70c0 con 0x7fd374102360 2024-09-12T01:26:42.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:42.283+0000 7fd379c00640 1 -- 172.21.15.1:0/2184252359 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd374103100 con 0x7fd374102360 2024-09-12T01:26:42.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:42.285+0000 7fd371600640 1 -- 172.21.15.1:0/2184252359 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fd35c042070 con 0x7fd374102360 2024-09-12T01:26:42.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:42.287+0000 7fd371600640 1 --2- 172.21.15.1:0/2184252359 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fd3580a2b80 0x7fd3580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:42.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:42.287+0000 7fd371600640 1 -- 172.21.15.1:0/2184252359 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7fd35c0ef160 con 0x7fd374102360 2024-09-12T01:26:42.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:42.287+0000 7fd373e00640 1 --2- 172.21.15.1:0/2184252359 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fd3580a2b80 0x7fd3580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:42.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:42.288+0000 7fd373e00640 1 --2- 172.21.15.1:0/2184252359 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fd3580a2b80 0x7fd3580a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd364009c50 tx=0x7fd364005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:26:42.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:42.290+0000 7fd371600640 1 -- 172.21.15.1:0/2184252359 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd35c0bc4c0 con 0x7fd374102360 2024-09-12T01:26:42.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:42.507+0000 7fd379c00640 1 -- 172.21.15.1:0/2184252359 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fd3741a7290 con 0x7fd374102360 2024-09-12T01:26:42.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:42 smithi001 ceph-mon[30387]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:42.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:42 smithi053 ceph-mon[35158]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:42.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:42 smithi104 ceph-mon[35219]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:43.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:43.425+0000 7fd371600640 1 -- 172.21.15.1:0/2184252359 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v81) ==== 138+0+0 (secure 0 0 0) 0x7fd35c0bbdc0 con 0x7fd374102360 2024-09-12T01:26:43.425 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_metadata' created 2024-09-12T01:26:43.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:43.428+0000 7fd379c00640 1 -- 172.21.15.1:0/2184252359 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fd3580a2b80 msgr2=0x7fd3580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:43.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:43.428+0000 7fd379c00640 1 --2- 172.21.15.1:0/2184252359 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fd3580a2b80 0x7fd3580a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd364009c50 tx=0x7fd364005f90 comp rx=0 tx=0).stop 2024-09-12T01:26:43.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:43.428+0000 7fd379c00640 1 -- 172.21.15.1:0/2184252359 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd374102360 msgr2=0x7fd37419aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:43.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:43.428+0000 7fd379c00640 1 --2- 172.21.15.1:0/2184252359 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd374102360 0x7fd37419aa80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd35c005940 tx=0x7fd35c003c10 comp rx=0 tx=0).stop 2024-09-12T01:26:43.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:43.429+0000 7fd379c00640 1 -- 172.21.15.1:0/2184252359 shutdown_connections 2024-09-12T01:26:43.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:43.429+0000 7fd379c00640 1 --2- 172.21.15.1:0/2184252359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3741079c0 0x7fd37419ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:43.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:43.429+0000 7fd379c00640 1 --2- 172.21.15.1:0/2184252359 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd374102360 0x7fd37419aa80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:43.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:43.429+0000 7fd379c00640 1 --2- 172.21.15.1:0/2184252359 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fd3580a2b80 0x7fd3580a5040 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:43.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:43.429+0000 7fd379c00640 1 --2- 172.21.15.1:0/2184252359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3741019c0 0x7fd37419a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:43.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:43.429+0000 7fd379c00640 1 -- 172.21.15.1:0/2184252359 >> 172.21.15.1:0/2184252359 conn(0x7fd3740fd740 msgr2=0x7fd37410d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:26:43.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:43.429+0000 7fd379c00640 1 -- 172.21.15.1:0/2184252359 shutdown_connections 2024-09-12T01:26:43.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:43.431+0000 7fd379c00640 1 -- 172.21.15.1:0/2184252359 wait complete. 2024-09-12T01:26:43.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:43 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/2184252359' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T01:26:43.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:43 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:43.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:43 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:43.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:43 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:26:43.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:43 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:26:43.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:43 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:43.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:43 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/2184252359' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T01:26:43.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:43 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:43.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:43 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:43.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:43 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:26:43.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:43 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:26:43.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:43 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:43.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:43 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/2184252359' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T01:26:43.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:43 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:43.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:43 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:43.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:43 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:26:43.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:43 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:26:43.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:43 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:26:44.367 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-12T01:26:44.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:44 smithi001 ceph-mon[30387]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:44.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:44 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/2184252359' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-12T01:26:44.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:44 smithi001 ceph-mon[30387]: osdmap e81: 12 total, 12 up, 12 in 2024-09-12T01:26:44.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:44 smithi053 ceph-mon[35158]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:44.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:44 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/2184252359' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-12T01:26:44.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:44 smithi053 ceph-mon[35158]: osdmap e81: 12 total, 12 up, 12 in 2024-09-12T01:26:44.758 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:26:44.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:44 smithi104 ceph-mon[35219]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:44.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:44 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/2184252359' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-12T01:26:44.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:44 smithi104 ceph-mon[35219]: osdmap e81: 12 total, 12 up, 12 in 2024-09-12T01:26:45.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:45.375+0000 7fe884000640 1 -- 172.21.15.1:0/2641393517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe87c074510 msgr2=0x7fe87c0748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:45.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:45.375+0000 7fe884000640 1 --2- 172.21.15.1:0/2641393517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe87c074510 0x7fe87c0748f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe870009860 tx=0x7fe87002a3d0 comp rx=0 tx=0).stop 2024-09-12T01:26:45.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:45.375+0000 7fe884000640 1 -- 172.21.15.1:0/2641393517 shutdown_connections 2024-09-12T01:26:45.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:45.375+0000 7fe884000640 1 --2- 172.21.15.1:0/2641393517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe87c10e450 0x7fe87c110840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:45.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:45.375+0000 7fe884000640 1 --2- 172.21.15.1:0/2641393517 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe87c074e30 0x7fe87c10df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:45.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:45.375+0000 7fe884000640 1 --2- 172.21.15.1:0/2641393517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe87c074510 0x7fe87c0748f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:45.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:45.375+0000 7fe884000640 1 -- 172.21.15.1:0/2641393517 >> 172.21.15.1:0/2641393517 conn(0x7fe87c0fd8f0 msgr2=0x7fe87c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:26:45.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:45.375+0000 7fe884000640 1 -- 172.21.15.1:0/2641393517 shutdown_connections 2024-09-12T01:26:45.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:45.375+0000 7fe884000640 1 -- 172.21.15.1:0/2641393517 wait complete. 2024-09-12T01:26:45.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:45.376+0000 7fe884000640 1 Processor -- start 2024-09-12T01:26:45.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:45.376+0000 7fe884000640 1 -- start start 2024-09-12T01:26:45.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:45.376+0000 7fe884000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe87c074510 0x7fe87c19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:45.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:45.376+0000 7fe884000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe87c074e30 0x7fe87c19ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:45.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:45.377+0000 7fe884000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe87c10e450 0x7fe87c19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:45.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:45.377+0000 7fe884000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe87c1141d0 con 0x7fe87c074510 2024-09-12T01:26:45.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:45.377+0000 7fe884000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe87c114050 con 0x7fe87c10e450 2024-09-12T01:26:45.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:45.377+0000 7fe884000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe87c114350 con 0x7fe87c074e30 2024-09-12T01:26:45.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:45.377+0000 7fe881600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe87c074510 0x7fe87c19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:45.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:45.377+0000 7fe882000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe87c10e450 0x7fe87c19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:45.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:45.377+0000 7fe880c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe87c074e30 0x7fe87c19ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:45.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:45.377+0000 7fe882000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe87c10e450 0x7fe87c19efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:59256/0 (socket says 172.21.15.1:59256) 2024-09-12T01:26:45.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:45.377+0000 7fe882000640 1 -- 172.21.15.1:0/2458865415 learned_addr learned my addr 172.21.15.1:0/2458865415 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:26:45.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:45.377+0000 7fe880c00640 1 -- 172.21.15.1:0/2458865415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe87c10e450 msgr2=0x7fe87c19efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:45.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:45.377+0000 7fe880c00640 1 --2- 172.21.15.1:0/2458865415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe87c10e450 0x7fe87c19efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:45.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:45.377+0000 7fe880c00640 1 -- 172.21.15.1:0/2458865415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe87c074510 msgr2=0x7fe87c19a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:45.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:45.377+0000 7fe880c00640 1 --2- 172.21.15.1:0/2458865415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe87c074510 0x7fe87c19a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:45.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:45.377+0000 7fe880c00640 1 -- 172.21.15.1:0/2458865415 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe87c19f6a0 con 0x7fe87c074e30 2024-09-12T01:26:45.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:45.378+0000 7fe880c00640 1 --2- 172.21.15.1:0/2458865415 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe87c074e30 0x7fe87c19ac30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe86800e9f0 tx=0x7fe86800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:26:45.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:45.378+0000 7fe87a000640 1 -- 172.21.15.1:0/2458865415 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86800ccc0 con 0x7fe87c074e30 2024-09-12T01:26:45.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:45.378+0000 7fe87a000640 1 -- 172.21.15.1:0/2458865415 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe868005020 con 0x7fe87c074e30 2024-09-12T01:26:45.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:45.378+0000 7fe87a000640 1 -- 172.21.15.1:0/2458865415 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8680049e0 con 0x7fe87c074e30 2024-09-12T01:26:45.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:45.378+0000 7fe884000640 1 -- 172.21.15.1:0/2458865415 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe87c19f990 con 0x7fe87c074e30 2024-09-12T01:26:45.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:45.378+0000 7fe884000640 1 -- 172.21.15.1:0/2458865415 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe87c1a7270 con 0x7fe87c074e30 2024-09-12T01:26:45.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:45.379+0000 7fe884000640 1 -- 172.21.15.1:0/2458865415 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe87c0748f0 con 0x7fe87c074e30 2024-09-12T01:26:45.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:45.381+0000 7fe87a000640 1 -- 172.21.15.1:0/2458865415 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fe868005560 con 0x7fe87c074e30 2024-09-12T01:26:45.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:45.383+0000 7fe87a000640 1 --2- 172.21.15.1:0/2458865415 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe8640a2b80 0x7fe8640a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:45.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:45.384+0000 7fe87a000640 1 -- 172.21.15.1:0/2458865415 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7497+0+0 (secure 0 0 0) 0x7fe8680cedd0 con 0x7fe87c074e30 2024-09-12T01:26:45.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:45.384+0000 7fe881600640 1 --2- 172.21.15.1:0/2458865415 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe8640a2b80 0x7fe8640a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:45.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:45.384+0000 7fe881600640 1 --2- 172.21.15.1:0/2458865415 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe8640a2b80 0x7fe8640a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe87002a9c0 tx=0x7fe870005c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:26:45.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:45.385+0000 7fe87a000640 1 -- 172.21.15.1:0/2458865415 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe8680051d0 con 0x7fe87c074e30 2024-09-12T01:26:45.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:45.576+0000 7fe884000640 1 -- 172.21.15.1:0/2458865415 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fe87c19ba50 con 0x7fe87c074e30 2024-09-12T01:26:45.576 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:45 smithi001 ceph-mon[30387]: osdmap e82: 12 total, 12 up, 12 in 2024-09-12T01:26:45.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:45 smithi053 ceph-mon[35158]: osdmap e82: 12 total, 12 up, 12 in 2024-09-12T01:26:45.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:45 smithi104 ceph-mon[35219]: osdmap e82: 12 total, 12 up, 12 in 2024-09-12T01:26:46.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:46.453+0000 7fe87a000640 1 -- 172.21.15.1:0/2458865415 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v84) ==== 171+0+0 (secure 0 0 0) 0x7fe86809c050 con 0x7fe87c074e30 2024-09-12T01:26:46.454 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_data' created 2024-09-12T01:26:46.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:46.457+0000 7fe884000640 1 -- 172.21.15.1:0/2458865415 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe8640a2b80 msgr2=0x7fe8640a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:46.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:46.457+0000 7fe884000640 1 --2- 172.21.15.1:0/2458865415 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe8640a2b80 0x7fe8640a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe87002a9c0 tx=0x7fe870005c90 comp rx=0 tx=0).stop 2024-09-12T01:26:46.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:46.457+0000 7fe884000640 1 -- 172.21.15.1:0/2458865415 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe87c074e30 msgr2=0x7fe87c19ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:46.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:46.457+0000 7fe884000640 1 --2- 172.21.15.1:0/2458865415 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe87c074e30 0x7fe87c19ac30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe86800e9f0 tx=0x7fe86800eeb0 comp rx=0 tx=0).stop 2024-09-12T01:26:46.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:46.458+0000 7fe884000640 1 -- 172.21.15.1:0/2458865415 shutdown_connections 2024-09-12T01:26:46.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:46.458+0000 7fe884000640 1 --2- 172.21.15.1:0/2458865415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe87c10e450 0x7fe87c19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:46.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:46.458+0000 7fe884000640 1 --2- 172.21.15.1:0/2458865415 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe8640a2b80 0x7fe8640a5040 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:46.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:46.458+0000 7fe884000640 1 --2- 172.21.15.1:0/2458865415 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe87c074e30 0x7fe87c19ac30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:46.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:46.458+0000 7fe884000640 1 --2- 172.21.15.1:0/2458865415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe87c074510 0x7fe87c19a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:46.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:46.458+0000 7fe884000640 1 -- 172.21.15.1:0/2458865415 >> 172.21.15.1:0/2458865415 conn(0x7fe87c0fd8f0 msgr2=0x7fe87c0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:26:46.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:46.458+0000 7fe884000640 1 -- 172.21.15.1:0/2458865415 shutdown_connections 2024-09-12T01:26:46.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:46.459+0000 7fe884000640 1 -- 172.21.15.1:0/2458865415 wait complete. 2024-09-12T01:26:46.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:46 smithi053 ceph-mon[35158]: pgmap v297: 33 pgs: 3 creating+peering, 26 unknown, 4 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:46.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:46 smithi053 ceph-mon[35158]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T01:26:46.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:46 smithi053 ceph-mon[35158]: osdmap e83: 12 total, 12 up, 12 in 2024-09-12T01:26:46.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:46 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/2458865415' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-12T01:26:46.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:46 smithi053 ceph-mon[35158]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-12T01:26:46.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:46 smithi104 ceph-mon[35219]: pgmap v297: 33 pgs: 3 creating+peering, 26 unknown, 4 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:46.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:46 smithi104 ceph-mon[35219]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T01:26:46.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:46 smithi104 ceph-mon[35219]: osdmap e83: 12 total, 12 up, 12 in 2024-09-12T01:26:46.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:46 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/2458865415' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-12T01:26:46.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:46 smithi104 ceph-mon[35219]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-12T01:26:46.876 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:46 smithi001 ceph-mon[30387]: pgmap v297: 33 pgs: 3 creating+peering, 26 unknown, 4 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:46.877 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:46 smithi001 ceph-mon[30387]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T01:26:46.877 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:46 smithi001 ceph-mon[30387]: osdmap e83: 12 total, 12 up, 12 in 2024-09-12T01:26:46.877 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:46 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/2458865415' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-12T01:26:46.877 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:46 smithi001 ceph-mon[30387]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-12T01:26:47.297 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-12T01:26:47.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:47 smithi001 ceph-mon[30387]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-12T01:26:47.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:47 smithi001 ceph-mon[30387]: osdmap e84: 12 total, 12 up, 12 in 2024-09-12T01:26:47.698 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:26:47.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:47 smithi053 ceph-mon[35158]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-12T01:26:47.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:47 smithi053 ceph-mon[35158]: osdmap e84: 12 total, 12 up, 12 in 2024-09-12T01:26:47.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:47 smithi104 ceph-mon[35219]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-12T01:26:47.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:47 smithi104 ceph-mon[35219]: osdmap e84: 12 total, 12 up, 12 in 2024-09-12T01:26:48.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:48.315+0000 7f45b1e00640 1 -- 172.21.15.1:0/2130779676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45ac106f10 msgr2=0x7f45ac10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:48.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:48.315+0000 7f45b1e00640 1 --2- 172.21.15.1:0/2130779676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45ac106f10 0x7f45ac10ea90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f45a4009ca0 tx=0x7f45a402a8c0 comp rx=0 tx=0).stop 2024-09-12T01:26:48.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:48.315+0000 7f45b1e00640 1 -- 172.21.15.1:0/2130779676 shutdown_connections 2024-09-12T01:26:48.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:48.315+0000 7f45b1e00640 1 --2- 172.21.15.1:0/2130779676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45ac106f10 0x7f45ac10ea90 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:48.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:48.315+0000 7f45b1e00640 1 --2- 172.21.15.1:0/2130779676 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45ac102790 0x7f45ac1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:48.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:48.315+0000 7f45b1e00640 1 --2- 172.21.15.1:0/2130779676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45ac101de0 0x7f45ac1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:48.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:48.315+0000 7f45b1e00640 1 -- 172.21.15.1:0/2130779676 >> 172.21.15.1:0/2130779676 conn(0x7f45ac0fd8c0 msgr2=0x7f45ac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:26:48.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:48.315+0000 7f45b1e00640 1 -- 172.21.15.1:0/2130779676 shutdown_connections 2024-09-12T01:26:48.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:48.315+0000 7f45b1e00640 1 -- 172.21.15.1:0/2130779676 wait complete. 2024-09-12T01:26:48.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:48.316+0000 7f45b1e00640 1 Processor -- start 2024-09-12T01:26:48.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:48.316+0000 7f45b1e00640 1 -- start start 2024-09-12T01:26:48.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:48.317+0000 7f45b1e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45ac101de0 0x7f45ac0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:48.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:48.317+0000 7f45b1e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45ac102790 0x7f45ac074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:48.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:48.317+0000 7f45b1e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45ac106f10 0x7f45ac074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:48.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:48.317+0000 7f45b1e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac111f70 con 0x7f45ac102790 2024-09-12T01:26:48.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:48.317+0000 7f45b1e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac111df0 con 0x7f45ac106f10 2024-09-12T01:26:48.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:48.317+0000 7f45b1e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac1120f0 con 0x7f45ac101de0 2024-09-12T01:26:48.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:48.317+0000 7f45a2000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45ac102790 0x7f45ac074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:48.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:48.317+0000 7f45aaa00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45ac101de0 0x7f45ac0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:48.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:48.317+0000 7f45a2000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45ac102790 0x7f45ac074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52256/0 (socket says 172.21.15.1:52256) 2024-09-12T01:26:48.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:48.317+0000 7f45a2000640 1 -- 172.21.15.1:0/165305942 learned_addr learned my addr 172.21.15.1:0/165305942 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:26:48.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:48.317+0000 7f45ab400640 1 --2- 172.21.15.1:0/165305942 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45ac106f10 0x7f45ac074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:48.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:48.317+0000 7f45a2000640 1 -- 172.21.15.1:0/165305942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45ac101de0 msgr2=0x7f45ac0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:48.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:48.317+0000 7f45a2000640 1 --2- 172.21.15.1:0/165305942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45ac101de0 0x7f45ac0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:48.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:48.317+0000 7f45a2000640 1 -- 172.21.15.1:0/165305942 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45ac106f10 msgr2=0x7f45ac074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:48.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:48.317+0000 7f45a2000640 1 --2- 172.21.15.1:0/165305942 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45ac106f10 0x7f45ac074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:48.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:48.317+0000 7f45a2000640 1 -- 172.21.15.1:0/165305942 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45ac1a6870 con 0x7f45ac102790 2024-09-12T01:26:48.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:48.317+0000 7f45aaa00640 1 --2- 172.21.15.1:0/165305942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45ac101de0 0x7f45ac0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:26:48.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:48.318+0000 7f45a2000640 1 --2- 172.21.15.1:0/165305942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45ac102790 0x7f45ac074a30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f459400e9f0 tx=0x7f459400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:26:48.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:48.318+0000 7f45b0a00640 1 -- 172.21.15.1:0/165305942 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459400cd80 con 0x7f45ac102790 2024-09-12T01:26:48.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:48.318+0000 7f45b0a00640 1 -- 172.21.15.1:0/165305942 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4594005500 con 0x7f45ac102790 2024-09-12T01:26:48.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:48.318+0000 7f45b1e00640 1 -- 172.21.15.1:0/165305942 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45ac1a6b60 con 0x7f45ac102790 2024-09-12T01:26:48.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:48.318+0000 7f45b0a00640 1 -- 172.21.15.1:0/165305942 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4594010690 con 0x7f45ac102790 2024-09-12T01:26:48.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:48.318+0000 7f45b1e00640 1 -- 172.21.15.1:0/165305942 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45ac1a70a0 con 0x7f45ac102790 2024-09-12T01:26:48.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:48.319+0000 7f45b1e00640 1 -- 172.21.15.1:0/165305942 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45ac1021c0 con 0x7f45ac102790 2024-09-12T01:26:48.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:48.323+0000 7f45b0a00640 1 -- 172.21.15.1:0/165305942 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f45940046c0 con 0x7f45ac102790 2024-09-12T01:26:48.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:48.324+0000 7f45b0a00640 1 --2- 172.21.15.1:0/165305942 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f45900a2c00 0x7f45900a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:48.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:48.324+0000 7f45b0a00640 1 -- 172.21.15.1:0/165305942 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7879+0+0 (secure 0 0 0) 0x7f45940cf440 con 0x7f45ac102790 2024-09-12T01:26:48.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:48.324+0000 7f45aaa00640 1 --2- 172.21.15.1:0/165305942 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f45900a2c00 0x7f45900a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:48.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:48.325+0000 7f45aaa00640 1 --2- 172.21.15.1:0/165305942 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f45900a2c00 0x7f45900a50c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f459c005d00 tx=0x7f459c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:26:48.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:48.327+0000 7f45b0a00640 1 -- 172.21.15.1:0/165305942 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f45940541b0 con 0x7f45ac102790 2024-09-12T01:26:48.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:48.554+0000 7f45b1e00640 1 -- 172.21.15.1:0/165305942 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f45ac075a20 con 0x7f45ac102790 2024-09-12T01:26:48.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:48 smithi053 ceph-mon[35158]: pgmap v300: 97 pgs: 7 creating+peering, 69 unknown, 21 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:48.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:48 smithi053 ceph-mon[35158]: osdmap e85: 12 total, 12 up, 12 in 2024-09-12T01:26:48.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:48 smithi104 ceph-mon[35219]: pgmap v300: 97 pgs: 7 creating+peering, 69 unknown, 21 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:48.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:48 smithi104 ceph-mon[35219]: osdmap e85: 12 total, 12 up, 12 in 2024-09-12T01:26:48.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:48 smithi001 ceph-mon[30387]: pgmap v300: 97 pgs: 7 creating+peering, 69 unknown, 21 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:48.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:48 smithi001 ceph-mon[30387]: osdmap e85: 12 total, 12 up, 12 in 2024-09-12T01:26:49.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:49.469+0000 7f45b0a00640 1 -- 172.21.15.1:0/165305942 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f45ac075a20 con 0x7f45ac102790 2024-09-12T01:26:49.469 INFO:teuthology.orchestra.run.smithi001.stderr:new fs with metadata pool 2 and data pool 3 2024-09-12T01:26:49.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:49.473+0000 7f45b1e00640 1 -- 172.21.15.1:0/165305942 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f45900a2c00 msgr2=0x7f45900a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:49.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:49.473+0000 7f45b1e00640 1 --2- 172.21.15.1:0/165305942 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f45900a2c00 0x7f45900a50c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f459c005d00 tx=0x7f459c00b410 comp rx=0 tx=0).stop 2024-09-12T01:26:49.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:49.473+0000 7f45b1e00640 1 -- 172.21.15.1:0/165305942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45ac102790 msgr2=0x7f45ac074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:49.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:49.473+0000 7f45b1e00640 1 --2- 172.21.15.1:0/165305942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45ac102790 0x7f45ac074a30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f459400e9f0 tx=0x7f459400edf0 comp rx=0 tx=0).stop 2024-09-12T01:26:49.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:49.473+0000 7f45b1e00640 1 -- 172.21.15.1:0/165305942 shutdown_connections 2024-09-12T01:26:49.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:49.473+0000 7f45b1e00640 1 --2- 172.21.15.1:0/165305942 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45ac106f10 0x7f45ac074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:49.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:49.473+0000 7f45b1e00640 1 --2- 172.21.15.1:0/165305942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45ac102790 0x7f45ac074a30 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:49.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:49.473+0000 7f45b1e00640 1 --2- 172.21.15.1:0/165305942 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f45900a2c00 0x7f45900a50c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:49.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:49.473+0000 7f45b1e00640 1 --2- 172.21.15.1:0/165305942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45ac101de0 0x7f45ac0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:49.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:49.473+0000 7f45b1e00640 1 -- 172.21.15.1:0/165305942 >> 172.21.15.1:0/165305942 conn(0x7f45ac0fd8c0 msgr2=0x7f45ac0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:26:49.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:49.473+0000 7f45b1e00640 1 -- 172.21.15.1:0/165305942 shutdown_connections 2024-09-12T01:26:49.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:49.473+0000 7f45b1e00640 1 -- 172.21.15.1:0/165305942 wait complete. 2024-09-12T01:26:49.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:49 smithi053 ceph-mon[35158]: osdmap e86: 12 total, 12 up, 12 in 2024-09-12T01:26:49.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:49 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/165305942' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-12T01:26:49.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:49 smithi053 ceph-mon[35158]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-12T01:26:49.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:49 smithi104 ceph-mon[35219]: osdmap e86: 12 total, 12 up, 12 in 2024-09-12T01:26:49.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:49 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/165305942' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-12T01:26:49.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:49 smithi104 ceph-mon[35219]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-12T01:26:49.861 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:49 smithi001 ceph-mon[30387]: osdmap e86: 12 total, 12 up, 12 in 2024-09-12T01:26:49.861 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:49 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/165305942' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-12T01:26:49.861 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:49 smithi001 ceph-mon[30387]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-12T01:26:50.365 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-12T01:26:50.365 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-12T01:26:50.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:50 smithi001 ceph-mon[30387]: pgmap v303: 97 pgs: 7 creating+peering, 48 unknown, 42 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:50.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:50 smithi001 ceph-mon[30387]: osdmap e87: 12 total, 12 up, 12 in 2024-09-12T01:26:50.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:50 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/165305942' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-12T01:26:50.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:50 smithi001 ceph-mon[30387]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-12T01:26:50.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:50 smithi001 ceph-mon[30387]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-12T01:26:50.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:50 smithi053 ceph-mon[35158]: pgmap v303: 97 pgs: 7 creating+peering, 48 unknown, 42 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:50.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:50 smithi053 ceph-mon[35158]: osdmap e87: 12 total, 12 up, 12 in 2024-09-12T01:26:50.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:50 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/165305942' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-12T01:26:50.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:50 smithi053 ceph-mon[35158]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-12T01:26:50.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:50 smithi053 ceph-mon[35158]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-12T01:26:50.750 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:26:50.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:50 smithi104 ceph-mon[35219]: pgmap v303: 97 pgs: 7 creating+peering, 48 unknown, 42 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:26:50.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:50 smithi104 ceph-mon[35219]: osdmap e87: 12 total, 12 up, 12 in 2024-09-12T01:26:50.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:50 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/165305942' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-12T01:26:50.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:50 smithi104 ceph-mon[35219]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-12T01:26:50.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:50 smithi104 ceph-mon[35219]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-12T01:26:51.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:51.368+0000 7f8dacc00640 1 -- 172.21.15.1:0/664414050 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8da8112870 msgr2=0x7f8da8114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:51.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:51.368+0000 7f8dacc00640 1 --2- 172.21.15.1:0/664414050 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8da8112870 0x7f8da8114c60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8da0009ca0 tx=0x7f8da002a8c0 comp rx=0 tx=0).stop 2024-09-12T01:26:51.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:51.368+0000 7f8dacc00640 1 -- 172.21.15.1:0/664414050 shutdown_connections 2024-09-12T01:26:51.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:51.368+0000 7f8dacc00640 1 --2- 172.21.15.1:0/664414050 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8da8112870 0x7f8da8114c60 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:51.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:51.368+0000 7f8dacc00640 1 --2- 172.21.15.1:0/664414050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8da8077040 0x7f8da8112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:51.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:51.368+0000 7f8dacc00640 1 --2- 172.21.15.1:0/664414050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8da8076720 0x7f8da8076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:51.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:51.368+0000 7f8dacc00640 1 -- 172.21.15.1:0/664414050 >> 172.21.15.1:0/664414050 conn(0x7f8da80ff8d0 msgr2=0x7f8da8101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:26:51.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:51.368+0000 7f8dacc00640 1 -- 172.21.15.1:0/664414050 shutdown_connections 2024-09-12T01:26:51.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:51.368+0000 7f8dacc00640 1 -- 172.21.15.1:0/664414050 wait complete. 2024-09-12T01:26:51.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:51.369+0000 7f8dacc00640 1 Processor -- start 2024-09-12T01:26:51.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:51.369+0000 7f8dacc00640 1 -- start start 2024-09-12T01:26:51.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:51.369+0000 7f8dacc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8da8076720 0x7f8da81a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:51.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:51.369+0000 7f8dacc00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8da8077040 0x7f8da81a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:51.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:51.369+0000 7f8dacc00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8da8112870 0x7f8da819e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:51.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:51.369+0000 7f8dacc00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8da81183c0 con 0x7f8da8112870 2024-09-12T01:26:51.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:51.369+0000 7f8dacc00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8da8118240 con 0x7f8da8076720 2024-09-12T01:26:51.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:51.369+0000 7f8dacc00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8da8118540 con 0x7f8da8077040 2024-09-12T01:26:51.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:51.370+0000 7f8da7400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8da8076720 0x7f8da81a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:51.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:51.370+0000 7f8da7e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8da8112870 0x7f8da819e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:51.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:51.370+0000 7f8da7400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8da8076720 0x7f8da81a4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:53292/0 (socket says 172.21.15.1:53292) 2024-09-12T01:26:51.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:51.370+0000 7f8da7400640 1 -- 172.21.15.1:0/1643047682 learned_addr learned my addr 172.21.15.1:0/1643047682 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:26:51.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:51.370+0000 7f8d9fe00640 1 --2- 172.21.15.1:0/1643047682 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8da8077040 0x7f8da81a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:51.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:51.370+0000 7f8da7e00640 1 -- 172.21.15.1:0/1643047682 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8da8077040 msgr2=0x7f8da81a4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:51.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:51.370+0000 7f8da7e00640 1 --2- 172.21.15.1:0/1643047682 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8da8077040 0x7f8da81a4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:51.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:51.370+0000 7f8da7e00640 1 -- 172.21.15.1:0/1643047682 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8da8076720 msgr2=0x7f8da81a4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:51.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:51.370+0000 7f8da7e00640 1 --2- 172.21.15.1:0/1643047682 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8da8076720 0x7f8da81a4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:51.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:51.370+0000 7f8da7e00640 1 -- 172.21.15.1:0/1643047682 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8da819ef40 con 0x7f8da8112870 2024-09-12T01:26:51.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:51.370+0000 7f8da7400640 1 --2- 172.21.15.1:0/1643047682 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8da8076720 0x7f8da81a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:26:51.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:51.370+0000 7f8da7e00640 1 --2- 172.21.15.1:0/1643047682 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8da8112870 0x7f8da819e6d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8da002ade0 tx=0x7f8da00319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:26:51.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:51.371+0000 7f8da4c00640 1 -- 172.21.15.1:0/1643047682 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da0003b70 con 0x7f8da8112870 2024-09-12T01:26:51.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:51.371+0000 7f8da4c00640 1 -- 172.21.15.1:0/1643047682 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8da0003d10 con 0x7f8da8112870 2024-09-12T01:26:51.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:51.371+0000 7f8da4c00640 1 -- 172.21.15.1:0/1643047682 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da002fc10 con 0x7f8da8112870 2024-09-12T01:26:51.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:51.371+0000 7f8dacc00640 1 -- 172.21.15.1:0/1643047682 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8da819f1d0 con 0x7f8da8112870 2024-09-12T01:26:51.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:51.371+0000 7f8dacc00640 1 -- 172.21.15.1:0/1643047682 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8da81ab580 con 0x7f8da8112870 2024-09-12T01:26:51.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:51.372+0000 7f8dacc00640 1 -- 172.21.15.1:0/1643047682 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8da819f590 con 0x7f8da8112870 2024-09-12T01:26:51.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:51.374+0000 7f8da4c00640 1 -- 172.21.15.1:0/1643047682 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f8da0004050 con 0x7f8da8112870 2024-09-12T01:26:51.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:51.375+0000 7f8da4c00640 1 --2- 172.21.15.1:0/1643047682 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f8d8c0a2c00 0x7f8d8c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:51.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:51.375+0000 7f8da4c00640 1 -- 172.21.15.1:0/1643047682 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f8da00eff40 con 0x7f8da8112870 2024-09-12T01:26:51.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:51.375+0000 7f8da7400640 1 --2- 172.21.15.1:0/1643047682 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f8d8c0a2c00 0x7f8d8c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:51.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:51.376+0000 7f8da7400640 1 --2- 172.21.15.1:0/1643047682 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f8d8c0a2c00 0x7f8d8c0a50c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8d98011500 tx=0x7f8d98005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:26:51.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:51.379+0000 7f8da4c00640 1 -- 172.21.15.1:0/1643047682 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8da00bd070 con 0x7f8da8112870 2024-09-12T01:26:51.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:51.575+0000 7f8dacc00640 1 -- 172.21.15.1:0/1643047682 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f8da819fc00 con 0x7f8da8112870 2024-09-12T01:26:51.574 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:51 smithi001 ceph-mon[30387]: mds.? [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] up:active 2024-09-12T01:26:51.575 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:51 smithi001 ceph-mon[30387]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T01:26:51.575 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:51 smithi001 ceph-mon[30387]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T01:26:51.575 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:51 smithi001 ceph-mon[30387]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T01:26:51.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:51 smithi053 ceph-mon[35158]: mds.? [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] up:active 2024-09-12T01:26:51.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:51 smithi053 ceph-mon[35158]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T01:26:51.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:51 smithi053 ceph-mon[35158]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T01:26:51.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:51 smithi053 ceph-mon[35158]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T01:26:51.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:51 smithi104 ceph-mon[35219]: mds.? [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] up:active 2024-09-12T01:26:51.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:51 smithi104 ceph-mon[35219]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T01:26:51.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:51 smithi104 ceph-mon[35219]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T01:26:51.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:51 smithi104 ceph-mon[35219]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T01:26:52.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:52.488+0000 7f8da4c00640 1 -- 172.21.15.1:0/1643047682 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v88) ==== 157+0+0 (secure 0 0 0) 0x7f8da00bc970 con 0x7f8da8112870 2024-09-12T01:26:52.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:52.493+0000 7f8dacc00640 1 -- 172.21.15.1:0/1643047682 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f8d8c0a2c00 msgr2=0x7f8d8c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:52.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:52.493+0000 7f8dacc00640 1 --2- 172.21.15.1:0/1643047682 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f8d8c0a2c00 0x7f8d8c0a50c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8d98011500 tx=0x7f8d98005f90 comp rx=0 tx=0).stop 2024-09-12T01:26:52.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:52.493+0000 7f8dacc00640 1 -- 172.21.15.1:0/1643047682 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8da8112870 msgr2=0x7f8da819e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:52.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:52.493+0000 7f8dacc00640 1 --2- 172.21.15.1:0/1643047682 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8da8112870 0x7f8da819e6d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8da002ade0 tx=0x7f8da00319f0 comp rx=0 tx=0).stop 2024-09-12T01:26:52.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:52.493+0000 7f8dacc00640 1 -- 172.21.15.1:0/1643047682 shutdown_connections 2024-09-12T01:26:52.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:52.493+0000 7f8dacc00640 1 --2- 172.21.15.1:0/1643047682 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8da8112870 0x7f8da819e6d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:52.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:52.493+0000 7f8dacc00640 1 --2- 172.21.15.1:0/1643047682 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f8d8c0a2c00 0x7f8d8c0a50c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:52.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:52.493+0000 7f8dacc00640 1 --2- 172.21.15.1:0/1643047682 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8da8077040 0x7f8da81a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:52.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:52.493+0000 7f8dacc00640 1 --2- 172.21.15.1:0/1643047682 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8da8076720 0x7f8da81a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:52.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:52.493+0000 7f8dacc00640 1 -- 172.21.15.1:0/1643047682 >> 172.21.15.1:0/1643047682 conn(0x7f8da80ff8d0 msgr2=0x7f8da8101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:26:52.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:52.493+0000 7f8dacc00640 1 -- 172.21.15.1:0/1643047682 shutdown_connections 2024-09-12T01:26:52.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:52.493+0000 7f8dacc00640 1 -- 172.21.15.1:0/1643047682 wait complete. 2024-09-12T01:26:52.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:52 smithi053 ceph-mon[35158]: pgmap v305: 97 pgs: 4 creating+peering, 13 unknown, 80 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 2 op/s 2024-09-12T01:26:52.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:52 smithi053 ceph-mon[35158]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-12T01:26:52.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:52 smithi053 ceph-mon[35158]: Cluster is now healthy 2024-09-12T01:26:52.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:52 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/1643047682' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-12T01:26:52.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:52 smithi104 ceph-mon[35219]: pgmap v305: 97 pgs: 4 creating+peering, 13 unknown, 80 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 2 op/s 2024-09-12T01:26:52.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:52 smithi104 ceph-mon[35219]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-12T01:26:52.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:52 smithi104 ceph-mon[35219]: Cluster is now healthy 2024-09-12T01:26:52.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:52 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/1643047682' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-12T01:26:52.866 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:52 smithi001 ceph-mon[30387]: pgmap v305: 97 pgs: 4 creating+peering, 13 unknown, 80 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 2 op/s 2024-09-12T01:26:52.866 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:52 smithi001 ceph-mon[30387]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-12T01:26:52.866 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:52 smithi001 ceph-mon[30387]: Cluster is now healthy 2024-09-12T01:26:52.866 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:52 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/1643047682' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-12T01:26:53.425 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-12T01:26:53.808 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:26:54.163 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:53 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/1643047682' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-12T01:26:54.164 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:53 smithi053 ceph-mon[35158]: osdmap e88: 12 total, 12 up, 12 in 2024-09-12T01:26:54.164 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:53 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/1643047682' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-12T01:26:54.164 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:53 smithi104 ceph-mon[35219]: osdmap e88: 12 total, 12 up, 12 in 2024-09-12T01:26:54.164 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:53 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/1643047682' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-12T01:26:54.164 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:53 smithi001 ceph-mon[30387]: osdmap e88: 12 total, 12 up, 12 in 2024-09-12T01:26:54.516 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:54 smithi001 ceph-mon[30387]: pgmap v307: 97 pgs: 97 active+clean; 579 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 4 op/s 2024-09-12T01:26:54.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:54.516+0000 7fe54d600640 1 -- 172.21.15.1:0/2455061175 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe548103ff0 msgr2=0x7fe5481043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:54.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:54.516+0000 7fe54d600640 1 --2- 172.21.15.1:0/2455061175 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe548103ff0 0x7fe5481043d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe538009900 tx=0x7fe53802a410 comp rx=0 tx=0).stop 2024-09-12T01:26:54.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:54.516+0000 7fe54d600640 1 -- 172.21.15.1:0/2455061175 shutdown_connections 2024-09-12T01:26:54.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:54.516+0000 7fe54d600640 1 --2- 172.21.15.1:0/2455061175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe548109120 0x7fe548110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:54.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:54.516+0000 7fe54d600640 1 --2- 172.21.15.1:0/2455061175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5481049a0 0x7fe5481089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:54.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:54.516+0000 7fe54d600640 1 --2- 172.21.15.1:0/2455061175 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe548103ff0 0x7fe5481043d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:54.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:54.516+0000 7fe54d600640 1 -- 172.21.15.1:0/2455061175 >> 172.21.15.1:0/2455061175 conn(0x7fe5480ffa90 msgr2=0x7fe548101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:26:54.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:54.516+0000 7fe54d600640 1 -- 172.21.15.1:0/2455061175 shutdown_connections 2024-09-12T01:26:54.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:54.516+0000 7fe54d600640 1 -- 172.21.15.1:0/2455061175 wait complete. 2024-09-12T01:26:54.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:54.517+0000 7fe54d600640 1 Processor -- start 2024-09-12T01:26:54.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:54.517+0000 7fe54d600640 1 -- start start 2024-09-12T01:26:54.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:54.518+0000 7fe54d600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe548103ff0 0x7fe54819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:54.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:54.518+0000 7fe54d600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5481049a0 0x7fe54819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:54.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:54.518+0000 7fe54d600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe548109120 0x7fe54819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:54.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:54.518+0000 7fe54d600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe548114150 con 0x7fe548103ff0 2024-09-12T01:26:54.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:54.518+0000 7fe54d600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe548113fd0 con 0x7fe548109120 2024-09-12T01:26:54.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:54.518+0000 7fe54d600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe5481142d0 con 0x7fe5481049a0 2024-09-12T01:26:54.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:54.518+0000 7fe546a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe548103ff0 0x7fe54819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:54.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:54.518+0000 7fe53e000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5481049a0 0x7fe54819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:54.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:54.518+0000 7fe546a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe548103ff0 0x7fe54819a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52132/0 (socket says 172.21.15.1:52132) 2024-09-12T01:26:54.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:54.518+0000 7fe546a00640 1 -- 172.21.15.1:0/3935764815 learned_addr learned my addr 172.21.15.1:0/3935764815 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:26:54.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:54.518+0000 7fe547400640 1 --2- 172.21.15.1:0/3935764815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe548109120 0x7fe54819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:54.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:54.518+0000 7fe53e000640 1 -- 172.21.15.1:0/3935764815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe548109120 msgr2=0x7fe54819eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:54.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:54.518+0000 7fe53e000640 1 --2- 172.21.15.1:0/3935764815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe548109120 0x7fe54819eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:54.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:54.518+0000 7fe53e000640 1 -- 172.21.15.1:0/3935764815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe548103ff0 msgr2=0x7fe54819a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:54.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:54.518+0000 7fe53e000640 1 --2- 172.21.15.1:0/3935764815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe548103ff0 0x7fe54819a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:54.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:54.518+0000 7fe53e000640 1 -- 172.21.15.1:0/3935764815 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe54819f5a0 con 0x7fe5481049a0 2024-09-12T01:26:54.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:54.519+0000 7fe547400640 1 --2- 172.21.15.1:0/3935764815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe548109120 0x7fe54819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:26:54.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:54.519+0000 7fe53e000640 1 --2- 172.21.15.1:0/3935764815 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5481049a0 0x7fe54819ab30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe530005fd0 tx=0x7fe53000dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:26:54.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:54.519+0000 7fe53fe00640 1 -- 172.21.15.1:0/3935764815 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe530009c30 con 0x7fe5481049a0 2024-09-12T01:26:54.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:54.519+0000 7fe53fe00640 1 -- 172.21.15.1:0/3935764815 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe530009dd0 con 0x7fe5481049a0 2024-09-12T01:26:54.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:54.519+0000 7fe54d600640 1 -- 172.21.15.1:0/3935764815 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe54819f890 con 0x7fe5481049a0 2024-09-12T01:26:54.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:54.519+0000 7fe53fe00640 1 -- 172.21.15.1:0/3935764815 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe530010460 con 0x7fe5481049a0 2024-09-12T01:26:54.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:54.519+0000 7fe54d600640 1 -- 172.21.15.1:0/3935764815 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5481a7170 con 0x7fe5481049a0 2024-09-12T01:26:54.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:54.520+0000 7fe54d600640 1 -- 172.21.15.1:0/3935764815 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5481043d0 con 0x7fe5481049a0 2024-09-12T01:26:54.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:54.523+0000 7fe53fe00640 1 -- 172.21.15.1:0/3935764815 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fe530003ef0 con 0x7fe5481049a0 2024-09-12T01:26:54.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:54.524+0000 7fe53fe00640 1 --2- 172.21.15.1:0/3935764815 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe52c0a2c00 0x7fe52c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:54.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:54.524+0000 7fe53fe00640 1 -- 172.21.15.1:0/3935764815 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8277+0+0 (secure 0 0 0) 0x7fe5300ceb60 con 0x7fe5481049a0 2024-09-12T01:26:54.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:54.524+0000 7fe546a00640 1 --2- 172.21.15.1:0/3935764815 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe52c0a2c00 0x7fe52c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:54.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:54.525+0000 7fe546a00640 1 --2- 172.21.15.1:0/3935764815 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe52c0a2c00 0x7fe52c0a50c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe5380096c0 tx=0x7fe538005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:26:54.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:54.528+0000 7fe53fe00640 1 -- 172.21.15.1:0/3935764815 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe530017020 con 0x7fe5481049a0 2024-09-12T01:26:54.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:54.713+0000 7fe54d600640 1 -- 172.21.15.1:0/3935764815 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe54802d7c0 con 0x7fe5481049a0 2024-09-12T01:26:54.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:54 smithi104 ceph-mon[35219]: pgmap v307: 97 pgs: 97 active+clean; 579 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 4 op/s 2024-09-12T01:26:54.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:54 smithi053 ceph-mon[35158]: pgmap v307: 97 pgs: 97 active+clean; 579 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 4 op/s 2024-09-12T01:26:55.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:55 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/3935764815' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T01:26:55.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:55 smithi053 ceph-mon[35158]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T01:26:55.740 INFO:journalctl@ceph.mds.b.smithi053.stdout:Sep 12 01:26:55 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-b[65497]: 2024-09-12T01:26:55.426+0000 7fcfc64e1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T01:26:55.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:55 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/3935764815' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T01:26:55.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:55 smithi104 ceph-mon[35219]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T01:26:55.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:55 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/3935764815' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T01:26:55.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:55 smithi001 ceph-mon[30387]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T01:26:56.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:56.507+0000 7fe53fe00640 1 -- 172.21.15.1:0/3935764815 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v90) ==== 228+0+0 (secure 0 0 0) 0x7fe53009b3e0 con 0x7fe5481049a0 2024-09-12T01:26:56.507 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_data_ec' created 2024-09-12T01:26:56.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:56.513+0000 7fe54d600640 1 -- 172.21.15.1:0/3935764815 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe52c0a2c00 msgr2=0x7fe52c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:56.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:56.513+0000 7fe54d600640 1 --2- 172.21.15.1:0/3935764815 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe52c0a2c00 0x7fe52c0a50c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe5380096c0 tx=0x7fe538005a20 comp rx=0 tx=0).stop 2024-09-12T01:26:56.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:56.513+0000 7fe54d600640 1 -- 172.21.15.1:0/3935764815 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5481049a0 msgr2=0x7fe54819ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:56.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:56.513+0000 7fe54d600640 1 --2- 172.21.15.1:0/3935764815 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5481049a0 0x7fe54819ab30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe530005fd0 tx=0x7fe53000dde0 comp rx=0 tx=0).stop 2024-09-12T01:26:56.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:56.513+0000 7fe54d600640 1 -- 172.21.15.1:0/3935764815 shutdown_connections 2024-09-12T01:26:56.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:56.513+0000 7fe54d600640 1 --2- 172.21.15.1:0/3935764815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe548109120 0x7fe54819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:56.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:56.513+0000 7fe54d600640 1 --2- 172.21.15.1:0/3935764815 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe52c0a2c00 0x7fe52c0a50c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:56.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:56.513+0000 7fe54d600640 1 --2- 172.21.15.1:0/3935764815 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5481049a0 0x7fe54819ab30 secure :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe530005fd0 tx=0x7fe53000dde0 comp rx=0 tx=0).stop 2024-09-12T01:26:56.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:56.513+0000 7fe54d600640 1 --2- 172.21.15.1:0/3935764815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe548103ff0 0x7fe54819a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:56.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:56.513+0000 7fe54d600640 1 -- 172.21.15.1:0/3935764815 >> 172.21.15.1:0/3935764815 conn(0x7fe5480ffa90 msgr2=0x7fe548100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:26:56.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:56.514+0000 7fe54d600640 1 -- 172.21.15.1:0/3935764815 shutdown_connections 2024-09-12T01:26:56.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:56.515+0000 7fe54d600640 1 -- 172.21.15.1:0/3935764815 wait complete. 2024-09-12T01:26:56.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:56 smithi104 ceph-mon[35219]: pgmap v308: 97 pgs: 97 active+clean; 579 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-12T01:26:56.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:56 smithi104 ceph-mon[35219]: osdmap e89: 12 total, 12 up, 12 in 2024-09-12T01:26:56.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:56 smithi001 ceph-mon[30387]: pgmap v308: 97 pgs: 97 active+clean; 579 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-12T01:26:56.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:56 smithi001 ceph-mon[30387]: osdmap e89: 12 total, 12 up, 12 in 2024-09-12T01:26:56.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:56 smithi053 ceph-mon[35158]: pgmap v308: 97 pgs: 97 active+clean; 579 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-12T01:26:56.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:56 smithi053 ceph-mon[35158]: osdmap e89: 12 total, 12 up, 12 in 2024-09-12T01:26:57.389 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-12T01:26:57.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:57 smithi001 ceph-mon[30387]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-12T01:26:57.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:57 smithi001 ceph-mon[30387]: osdmap e90: 12 total, 12 up, 12 in 2024-09-12T01:26:57.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:57 smithi001 ceph-mon[30387]: osdmap e91: 12 total, 12 up, 12 in 2024-09-12T01:26:57.785 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:26:57.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:57 smithi104 ceph-mon[35219]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-12T01:26:57.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:57 smithi104 ceph-mon[35219]: osdmap e90: 12 total, 12 up, 12 in 2024-09-12T01:26:57.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:57 smithi104 ceph-mon[35219]: osdmap e91: 12 total, 12 up, 12 in 2024-09-12T01:26:57.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:57 smithi053 ceph-mon[35158]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-12T01:26:57.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:57 smithi053 ceph-mon[35158]: osdmap e90: 12 total, 12 up, 12 in 2024-09-12T01:26:57.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:57 smithi053 ceph-mon[35158]: osdmap e91: 12 total, 12 up, 12 in 2024-09-12T01:26:58.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:58.392+0000 7fc9efe00640 1 -- 172.21.15.1:0/1806728756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9e8101de0 msgr2=0x7fc9e81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:58.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:58.392+0000 7fc9efe00640 1 --2- 172.21.15.1:0/1806728756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9e8101de0 0x7fc9e81021c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc009900 tx=0x7fc9dc02a3f0 comp rx=0 tx=0).stop 2024-09-12T01:26:58.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:58.392+0000 7fc9efe00640 1 -- 172.21.15.1:0/1806728756 shutdown_connections 2024-09-12T01:26:58.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:58.392+0000 7fc9efe00640 1 --2- 172.21.15.1:0/1806728756 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9e8106f10 0x7fc9e810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:58.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:58.392+0000 7fc9efe00640 1 --2- 172.21.15.1:0/1806728756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9e8102790 0x7fc9e81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:58.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:58.392+0000 7fc9efe00640 1 --2- 172.21.15.1:0/1806728756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9e8101de0 0x7fc9e81021c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:58.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:58.392+0000 7fc9efe00640 1 -- 172.21.15.1:0/1806728756 >> 172.21.15.1:0/1806728756 conn(0x7fc9e80fd8c0 msgr2=0x7fc9e80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:26:58.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:58.392+0000 7fc9efe00640 1 -- 172.21.15.1:0/1806728756 shutdown_connections 2024-09-12T01:26:58.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:58.392+0000 7fc9efe00640 1 -- 172.21.15.1:0/1806728756 wait complete. 2024-09-12T01:26:58.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:58.393+0000 7fc9efe00640 1 Processor -- start 2024-09-12T01:26:58.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:58.393+0000 7fc9efe00640 1 -- start start 2024-09-12T01:26:58.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:58.393+0000 7fc9efe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9e8101de0 0x7fc9e80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:58.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:58.393+0000 7fc9efe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9e8102790 0x7fc9e8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:58.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:58.393+0000 7fc9efe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9e8106f10 0x7fc9e8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:58.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:58.393+0000 7fc9efe00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e8111f80 con 0x7fc9e8102790 2024-09-12T01:26:58.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:58.393+0000 7fc9efe00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e8111e00 con 0x7fc9e8101de0 2024-09-12T01:26:58.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:58.393+0000 7fc9efe00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e8112100 con 0x7fc9e8106f10 2024-09-12T01:26:58.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:58.394+0000 7fc9ed400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9e8101de0 0x7fc9e80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:58.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:58.394+0000 7fc9ed400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9e8101de0 0x7fc9e80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:53310/0 (socket says 172.21.15.1:53310) 2024-09-12T01:26:58.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:58.394+0000 7fc9ed400640 1 -- 172.21.15.1:0/2663649280 learned_addr learned my addr 172.21.15.1:0/2663649280 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:26:58.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:58.394+0000 7fc9eca00640 1 --2- 172.21.15.1:0/2663649280 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9e8102790 0x7fc9e8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:58.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:58.394+0000 7fc9ede00640 1 --2- 172.21.15.1:0/2663649280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9e8106f10 0x7fc9e8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:58.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:58.394+0000 7fc9ed400640 1 -- 172.21.15.1:0/2663649280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9e8106f10 msgr2=0x7fc9e8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:58.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:58.394+0000 7fc9ed400640 1 --2- 172.21.15.1:0/2663649280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9e8106f10 0x7fc9e8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:58.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:58.394+0000 7fc9ed400640 1 -- 172.21.15.1:0/2663649280 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9e8102790 msgr2=0x7fc9e8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:58.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:58.394+0000 7fc9ed400640 1 --2- 172.21.15.1:0/2663649280 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9e8102790 0x7fc9e8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:58.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:58.394+0000 7fc9ed400640 1 -- 172.21.15.1:0/2663649280 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e81a6920 con 0x7fc9e8101de0 2024-09-12T01:26:58.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:58.394+0000 7fc9ed400640 1 --2- 172.21.15.1:0/2663649280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9e8101de0 0x7fc9e80744f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc004770 tx=0x7fc9dc003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:26:58.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:58.395+0000 7fc9e6000640 1 -- 172.21.15.1:0/2663649280 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc03f070 con 0x7fc9e8101de0 2024-09-12T01:26:58.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:58.395+0000 7fc9e6000640 1 -- 172.21.15.1:0/2663649280 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9dc031ca0 con 0x7fc9e8101de0 2024-09-12T01:26:58.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:58.395+0000 7fc9e6000640 1 -- 172.21.15.1:0/2663649280 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc035030 con 0x7fc9e8101de0 2024-09-12T01:26:58.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:58.395+0000 7fc9efe00640 1 -- 172.21.15.1:0/2663649280 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9e81a6bb0 con 0x7fc9e8101de0 2024-09-12T01:26:58.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:58.395+0000 7fc9efe00640 1 -- 172.21.15.1:0/2663649280 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9e81a7040 con 0x7fc9e8101de0 2024-09-12T01:26:58.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:58.396+0000 7fc9efe00640 1 -- 172.21.15.1:0/2663649280 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9e81021c0 con 0x7fc9e8101de0 2024-09-12T01:26:58.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:58.398+0000 7fc9e6000640 1 -- 172.21.15.1:0/2663649280 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fc9dc042070 con 0x7fc9e8101de0 2024-09-12T01:26:58.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:58.399+0000 7fc9e6000640 1 --2- 172.21.15.1:0/2663649280 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fc9d00a2c00 0x7fc9d00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:26:58.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:58.400+0000 7fc9e6000640 1 -- 172.21.15.1:0/2663649280 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8770+0+0 (secure 0 0 0) 0x7fc9dc0efda0 con 0x7fc9e8101de0 2024-09-12T01:26:58.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:58.400+0000 7fc9eca00640 1 --2- 172.21.15.1:0/2663649280 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fc9d00a2c00 0x7fc9d00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:26:58.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:58.400+0000 7fc9eca00640 1 --2- 172.21.15.1:0/2663649280 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fc9d00a2c00 0x7fc9d00a50c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc9d40095f0 tx=0x7fc9d4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:26:58.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:58.402+0000 7fc9e6000640 1 -- 172.21.15.1:0/2663649280 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc9dc0bc4f0 con 0x7fc9e8101de0 2024-09-12T01:26:58.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:58.591+0000 7fc9efe00640 1 -- 172.21.15.1:0/2663649280 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fc9e802d7c0 con 0x7fc9e8101de0 2024-09-12T01:26:58.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:58 smithi104 ceph-mon[35219]: pgmap v311: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 5 op/s 2024-09-12T01:26:58.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:58 smithi104 ceph-mon[35219]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T01:26:58.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:58 smithi104 ceph-mon[35219]: osdmap e92: 12 total, 12 up, 12 in 2024-09-12T01:26:58.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:58 smithi001 ceph-mon[30387]: pgmap v311: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 5 op/s 2024-09-12T01:26:58.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:58 smithi001 ceph-mon[30387]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T01:26:58.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:58 smithi001 ceph-mon[30387]: osdmap e92: 12 total, 12 up, 12 in 2024-09-12T01:26:58.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:58 smithi053 ceph-mon[35158]: pgmap v311: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 5 op/s 2024-09-12T01:26:58.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:58 smithi053 ceph-mon[35158]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T01:26:58.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:58 smithi053 ceph-mon[35158]: osdmap e92: 12 total, 12 up, 12 in 2024-09-12T01:26:59.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:59.541+0000 7fc9e6000640 1 -- 172.21.15.1:0/2663649280 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v93) ==== 169+0+0 (secure 0 0 0) 0x7fc9dc0c07f0 con 0x7fc9e8101de0 2024-09-12T01:26:59.541 INFO:teuthology.orchestra.run.smithi001.stderr:set pool 4 allow_ec_overwrites to true 2024-09-12T01:26:59.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:59.544+0000 7fc9efe00640 1 -- 172.21.15.1:0/2663649280 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fc9d00a2c00 msgr2=0x7fc9d00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:59.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:59.544+0000 7fc9efe00640 1 --2- 172.21.15.1:0/2663649280 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fc9d00a2c00 0x7fc9d00a50c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc9d40095f0 tx=0x7fc9d4005f90 comp rx=0 tx=0).stop 2024-09-12T01:26:59.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:59.545+0000 7fc9efe00640 1 -- 172.21.15.1:0/2663649280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9e8101de0 msgr2=0x7fc9e80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:26:59.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:59.545+0000 7fc9efe00640 1 --2- 172.21.15.1:0/2663649280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9e8101de0 0x7fc9e80744f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc004770 tx=0x7fc9dc003c10 comp rx=0 tx=0).stop 2024-09-12T01:26:59.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:59.545+0000 7fc9efe00640 1 -- 172.21.15.1:0/2663649280 shutdown_connections 2024-09-12T01:26:59.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:59.545+0000 7fc9efe00640 1 --2- 172.21.15.1:0/2663649280 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fc9d00a2c00 0x7fc9d00a50c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:59.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:59.545+0000 7fc9efe00640 1 --2- 172.21.15.1:0/2663649280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9e8106f10 0x7fc9e8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:59.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:59.545+0000 7fc9efe00640 1 --2- 172.21.15.1:0/2663649280 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9e8102790 0x7fc9e8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:59.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:59.545+0000 7fc9efe00640 1 --2- 172.21.15.1:0/2663649280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9e8101de0 0x7fc9e80744f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:26:59.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:59.545+0000 7fc9efe00640 1 -- 172.21.15.1:0/2663649280 >> 172.21.15.1:0/2663649280 conn(0x7fc9e80fd8c0 msgr2=0x7fc9e80febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:26:59.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:59.545+0000 7fc9efe00640 1 -- 172.21.15.1:0/2663649280 shutdown_connections 2024-09-12T01:26:59.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:26:59.545+0000 7fc9efe00640 1 -- 172.21.15.1:0/2663649280 wait complete. 2024-09-12T01:26:59.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:59 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/2663649280' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-12T01:26:59.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:26:59 smithi104 ceph-mon[35219]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-12T01:26:59.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:59 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/2663649280' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-12T01:26:59.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:26:59 smithi001 ceph-mon[30387]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-12T01:26:59.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:59 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/2663649280' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-12T01:26:59.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:26:59 smithi053 ceph-mon[35158]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-12T01:27:00.516 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-12T01:27:00.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:00 smithi104 ceph-mon[35219]: pgmap v314: 129 pgs: 30 unknown, 99 active+clean; 579 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:27:00.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:00 smithi104 ceph-mon[35219]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-12T01:27:00.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:00 smithi104 ceph-mon[35219]: osdmap e93: 12 total, 12 up, 12 in 2024-09-12T01:27:00.900 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:27:00.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:00 smithi001 ceph-mon[30387]: pgmap v314: 129 pgs: 30 unknown, 99 active+clean; 579 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:27:00.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:00 smithi001 ceph-mon[30387]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-12T01:27:00.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:00 smithi001 ceph-mon[30387]: osdmap e93: 12 total, 12 up, 12 in 2024-09-12T01:27:00.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:00 smithi053 ceph-mon[35158]: pgmap v314: 129 pgs: 30 unknown, 99 active+clean; 579 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:27:00.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:00 smithi053 ceph-mon[35158]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-12T01:27:00.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:00 smithi053 ceph-mon[35158]: osdmap e93: 12 total, 12 up, 12 in 2024-09-12T01:27:01.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:01.510+0000 7fe341400640 1 -- 172.21.15.1:0/3267462514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe33c108f60 msgr2=0x7fe33c110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:01.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:01.510+0000 7fe341400640 1 --2- 172.21.15.1:0/3267462514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe33c108f60 0x7fe33c110ae0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe334009be0 tx=0x7fe33402a6c0 comp rx=0 tx=0).stop 2024-09-12T01:27:01.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:01.510+0000 7fe341400640 1 -- 172.21.15.1:0/3267462514 shutdown_connections 2024-09-12T01:27:01.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:01.510+0000 7fe341400640 1 --2- 172.21.15.1:0/3267462514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe33c108f60 0x7fe33c110ae0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:01.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:01.510+0000 7fe341400640 1 --2- 172.21.15.1:0/3267462514 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe33c1047e0 0x7fe33c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:01.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:01.510+0000 7fe341400640 1 --2- 172.21.15.1:0/3267462514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe33c103e30 0x7fe33c104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:01.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:01.510+0000 7fe341400640 1 -- 172.21.15.1:0/3267462514 >> 172.21.15.1:0/3267462514 conn(0x7fe33c0ff8d0 msgr2=0x7fe33c101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:01.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:01.510+0000 7fe341400640 1 -- 172.21.15.1:0/3267462514 shutdown_connections 2024-09-12T01:27:01.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:01.510+0000 7fe341400640 1 -- 172.21.15.1:0/3267462514 wait complete. 2024-09-12T01:27:01.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:01.511+0000 7fe341400640 1 Processor -- start 2024-09-12T01:27:01.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:01.511+0000 7fe341400640 1 -- start start 2024-09-12T01:27:01.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:01.512+0000 7fe341400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe33c103e30 0x7fe33c19a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:01.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:01.512+0000 7fe341400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe33c1047e0 0x7fe33c19a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:01.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:01.512+0000 7fe341400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe33c108f60 0x7fe33c19ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:01.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:01.512+0000 7fe341400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c113f70 con 0x7fe33c103e30 2024-09-12T01:27:01.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:01.512+0000 7fe341400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c113df0 con 0x7fe33c1047e0 2024-09-12T01:27:01.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:01.512+0000 7fe341400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c1140f0 con 0x7fe33c108f60 2024-09-12T01:27:01.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:01.512+0000 7fe33be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe33c103e30 0x7fe33c19a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:01.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:01.512+0000 7fe33be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe33c103e30 0x7fe33c19a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47230/0 (socket says 172.21.15.1:47230) 2024-09-12T01:27:01.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:01.512+0000 7fe33be00640 1 -- 172.21.15.1:0/204247271 learned_addr learned my addr 172.21.15.1:0/204247271 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:27:01.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:01.512+0000 7fe33b400640 1 --2- 172.21.15.1:0/204247271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe33c1047e0 0x7fe33c19a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:01.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:01.512+0000 7fe340a00640 1 --2- 172.21.15.1:0/204247271 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe33c108f60 0x7fe33c19ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:01.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:01.513+0000 7fe33be00640 1 -- 172.21.15.1:0/204247271 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe33c108f60 msgr2=0x7fe33c19ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:01.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:01.513+0000 7fe33be00640 1 --2- 172.21.15.1:0/204247271 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe33c108f60 0x7fe33c19ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:01.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:01.513+0000 7fe33be00640 1 -- 172.21.15.1:0/204247271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe33c1047e0 msgr2=0x7fe33c19a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:01.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:01.513+0000 7fe33be00640 1 --2- 172.21.15.1:0/204247271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe33c1047e0 0x7fe33c19a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:01.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:01.513+0000 7fe33be00640 1 -- 172.21.15.1:0/204247271 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe33c19f3c0 con 0x7fe33c103e30 2024-09-12T01:27:01.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:01.513+0000 7fe33be00640 1 --2- 172.21.15.1:0/204247271 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe33c103e30 0x7fe33c19a410 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe32c00e9c0 tx=0x7fe32c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:01.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:01.513+0000 7fe338c00640 1 -- 172.21.15.1:0/204247271 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32c00cde0 con 0x7fe33c103e30 2024-09-12T01:27:01.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:01.513+0000 7fe338c00640 1 -- 172.21.15.1:0/204247271 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe32c0035f0 con 0x7fe33c103e30 2024-09-12T01:27:01.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:01.513+0000 7fe338c00640 1 -- 172.21.15.1:0/204247271 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32c010690 con 0x7fe33c103e30 2024-09-12T01:27:01.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:01.513+0000 7fe341400640 1 -- 172.21.15.1:0/204247271 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe33c19f6b0 con 0x7fe33c103e30 2024-09-12T01:27:01.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:01.513+0000 7fe341400640 1 -- 172.21.15.1:0/204247271 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe33c1a6f90 con 0x7fe33c103e30 2024-09-12T01:27:01.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:01.515+0000 7fe341400640 1 -- 172.21.15.1:0/204247271 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe33c104210 con 0x7fe33c103e30 2024-09-12T01:27:01.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:01.517+0000 7fe338c00640 1 -- 172.21.15.1:0/204247271 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fe32c0037a0 con 0x7fe33c103e30 2024-09-12T01:27:01.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:01.519+0000 7fe338c00640 1 --2- 172.21.15.1:0/204247271 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe3200a2c00 0x7fe3200a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:01.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:01.519+0000 7fe338c00640 1 -- 172.21.15.1:0/204247271 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8770+0+0 (secure 0 0 0) 0x7fe32c0cf630 con 0x7fe33c103e30 2024-09-12T01:27:01.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:01.519+0000 7fe33b400640 1 --2- 172.21.15.1:0/204247271 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe3200a2c00 0x7fe3200a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:01.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:01.520+0000 7fe33b400640 1 --2- 172.21.15.1:0/204247271 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe3200a2c00 0x7fe3200a50c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe33c0ffcf0 tx=0x7fe328005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:01.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:01.522+0000 7fe338c00640 1 -- 172.21.15.1:0/204247271 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe32c09c3a0 con 0x7fe33c103e30 2024-09-12T01:27:01.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:01.748+0000 7fe341400640 1 -- 172.21.15.1:0/204247271 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7fe33c02d7c0 con 0x7fe33c103e30 2024-09-12T01:27:02.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:02.544+0000 7fe338c00640 1 -- 172.21.15.1:0/204247271 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7fe32c09bca0 con 0x7fe33c103e30 2024-09-12T01:27:02.544 INFO:teuthology.orchestra.run.smithi001.stderr:added data pool 4 to fsmap 2024-09-12T01:27:02.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:02.548+0000 7fe341400640 1 -- 172.21.15.1:0/204247271 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe3200a2c00 msgr2=0x7fe3200a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:02.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:02.548+0000 7fe341400640 1 --2- 172.21.15.1:0/204247271 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe3200a2c00 0x7fe3200a50c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe33c0ffcf0 tx=0x7fe328005f90 comp rx=0 tx=0).stop 2024-09-12T01:27:02.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:02.548+0000 7fe341400640 1 -- 172.21.15.1:0/204247271 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe33c103e30 msgr2=0x7fe33c19a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:02.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:02.548+0000 7fe341400640 1 --2- 172.21.15.1:0/204247271 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe33c103e30 0x7fe33c19a410 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe32c00e9c0 tx=0x7fe32c00ee80 comp rx=0 tx=0).stop 2024-09-12T01:27:02.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:02.548+0000 7fe341400640 1 -- 172.21.15.1:0/204247271 shutdown_connections 2024-09-12T01:27:02.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:02.548+0000 7fe341400640 1 --2- 172.21.15.1:0/204247271 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe3200a2c00 0x7fe3200a50c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:02.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:02.548+0000 7fe341400640 1 --2- 172.21.15.1:0/204247271 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe33c108f60 0x7fe33c19ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:02.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:02.548+0000 7fe341400640 1 --2- 172.21.15.1:0/204247271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe33c1047e0 0x7fe33c19a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:02.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:02.548+0000 7fe341400640 1 --2- 172.21.15.1:0/204247271 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe33c103e30 0x7fe33c19a410 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:02.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:02.548+0000 7fe341400640 1 -- 172.21.15.1:0/204247271 >> 172.21.15.1:0/204247271 conn(0x7fe33c0ff8d0 msgr2=0x7fe33c100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:02.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:02.548+0000 7fe341400640 1 -- 172.21.15.1:0/204247271 shutdown_connections 2024-09-12T01:27:02.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:02.548+0000 7fe341400640 1 -- 172.21.15.1:0/204247271 wait complete. 2024-09-12T01:27:02.805 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:02 smithi053 ceph-mon[35158]: pgmap v316: 129 pgs: 6 unknown, 123 active+clean; 579 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:27:02.805 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:02 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/204247271' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-12T01:27:02.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:02 smithi104 ceph-mon[35219]: pgmap v316: 129 pgs: 6 unknown, 123 active+clean; 579 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:27:02.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:02 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/204247271' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-12T01:27:02.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:02 smithi001 ceph-mon[30387]: pgmap v316: 129 pgs: 6 unknown, 123 active+clean; 579 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:27:02.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:02 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/204247271' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-12T01:27:03.476 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:27:03.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:03 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/204247271' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-12T01:27:03.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:03 smithi104 ceph-mon[35219]: osdmap e94: 12 total, 12 up, 12 in 2024-09-12T01:27:03.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:03 smithi104 ceph-mon[35219]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T01:27:03.858 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:27:03.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:03 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/204247271' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-12T01:27:03.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:03 smithi001 ceph-mon[30387]: osdmap e94: 12 total, 12 up, 12 in 2024-09-12T01:27:03.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:03 smithi001 ceph-mon[30387]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T01:27:03.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:03 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/204247271' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-12T01:27:03.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:03 smithi053 ceph-mon[35158]: osdmap e94: 12 total, 12 up, 12 in 2024-09-12T01:27:03.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:03 smithi053 ceph-mon[35158]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T01:27:04.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.425+0000 7f98c2e00640 1 -- 172.21.15.1:0/3712773158 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98bc103e30 msgr2=0x7f98bc104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:04.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.425+0000 7f98c2e00640 1 --2- 172.21.15.1:0/3712773158 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98bc103e30 0x7f98bc104210 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f98b0009900 tx=0x7f98b002a3f0 comp rx=0 tx=0).stop 2024-09-12T01:27:04.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.425+0000 7f98c2e00640 1 -- 172.21.15.1:0/3712773158 shutdown_connections 2024-09-12T01:27:04.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.425+0000 7f98c2e00640 1 --2- 172.21.15.1:0/3712773158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98bc108f60 0x7f98bc110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:04.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.425+0000 7f98c2e00640 1 --2- 172.21.15.1:0/3712773158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98bc1047e0 0x7f98bc108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:04.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.425+0000 7f98c2e00640 1 --2- 172.21.15.1:0/3712773158 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98bc103e30 0x7f98bc104210 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:04.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.425+0000 7f98c2e00640 1 -- 172.21.15.1:0/3712773158 >> 172.21.15.1:0/3712773158 conn(0x7f98bc0ff8d0 msgr2=0x7f98bc101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:04.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.425+0000 7f98c2e00640 1 -- 172.21.15.1:0/3712773158 shutdown_connections 2024-09-12T01:27:04.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.425+0000 7f98c2e00640 1 -- 172.21.15.1:0/3712773158 wait complete. 2024-09-12T01:27:04.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.426+0000 7f98c2e00640 1 Processor -- start 2024-09-12T01:27:04.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.426+0000 7f98c2e00640 1 -- start start 2024-09-12T01:27:04.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.426+0000 7f98c2e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98bc103e30 0x7f98bc076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:04.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.426+0000 7f98c2e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98bc1047e0 0x7f98bc076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:04.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.426+0000 7f98c2e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98bc108f60 0x7f98bc077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:04.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.426+0000 7f98c2e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc113fe0 con 0x7f98bc1047e0 2024-09-12T01:27:04.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.426+0000 7f98c2e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc113e60 con 0x7f98bc103e30 2024-09-12T01:27:04.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.426+0000 7f98c2e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc114160 con 0x7f98bc108f60 2024-09-12T01:27:04.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.426+0000 7f98c1000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98bc1047e0 0x7f98bc076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:04.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.426+0000 7f98c1000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98bc1047e0 0x7f98bc076c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47240/0 (socket says 172.21.15.1:47240) 2024-09-12T01:27:04.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.426+0000 7f98c1a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98bc103e30 0x7f98bc076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:04.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.427+0000 7f98c1000640 1 -- 172.21.15.1:0/1619265107 learned_addr learned my addr 172.21.15.1:0/1619265107 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:27:04.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.426+0000 7f98c2400640 1 --2- 172.21.15.1:0/1619265107 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98bc108f60 0x7f98bc077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:04.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.427+0000 7f98c1000640 1 -- 172.21.15.1:0/1619265107 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98bc108f60 msgr2=0x7f98bc077180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:04.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.427+0000 7f98c1000640 1 --2- 172.21.15.1:0/1619265107 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98bc108f60 0x7f98bc077180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:04.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.427+0000 7f98c1000640 1 -- 172.21.15.1:0/1619265107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98bc103e30 msgr2=0x7f98bc076700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:04.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.427+0000 7f98c1000640 1 --2- 172.21.15.1:0/1619265107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98bc103e30 0x7f98bc076700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:04.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.427+0000 7f98c1000640 1 -- 172.21.15.1:0/1619265107 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98bc1aab20 con 0x7f98bc1047e0 2024-09-12T01:27:04.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.427+0000 7f98c1000640 1 --2- 172.21.15.1:0/1619265107 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98bc1047e0 0x7f98bc076c40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f98a800c950 tx=0x7f98a800ce10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:04.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.427+0000 7f98ba000640 1 -- 172.21.15.1:0/1619265107 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a8007c70 con 0x7f98bc1047e0 2024-09-12T01:27:04.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.427+0000 7f98ba000640 1 -- 172.21.15.1:0/1619265107 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98a8007e10 con 0x7f98bc1047e0 2024-09-12T01:27:04.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.427+0000 7f98ba000640 1 -- 172.21.15.1:0/1619265107 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a8018960 con 0x7f98bc1047e0 2024-09-12T01:27:04.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.427+0000 7f98c2e00640 1 -- 172.21.15.1:0/1619265107 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98bc1aae10 con 0x7f98bc1047e0 2024-09-12T01:27:04.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.428+0000 7f98c2e00640 1 -- 172.21.15.1:0/1619265107 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98bc1ab210 con 0x7f98bc1047e0 2024-09-12T01:27:04.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.429+0000 7f98c2e00640 1 -- 172.21.15.1:0/1619265107 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98bc105310 con 0x7f98bc1047e0 2024-09-12T01:27:04.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.431+0000 7f98ba000640 1 -- 172.21.15.1:0/1619265107 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f98a800fbc0 con 0x7f98bc1047e0 2024-09-12T01:27:04.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.432+0000 7f98ba000640 1 --2- 172.21.15.1:0/1619265107 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f98a40a2c00 0x7f98a40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:04.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.433+0000 7f98ba000640 1 -- 172.21.15.1:0/1619265107 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8802+0+0 (secure 0 0 0) 0x7f98a80cf200 con 0x7f98bc1047e0 2024-09-12T01:27:04.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.433+0000 7f98c1a00640 1 --2- 172.21.15.1:0/1619265107 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f98a40a2c00 0x7f98a40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:04.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.433+0000 7f98c1a00640 1 --2- 172.21.15.1:0/1619265107 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f98a40a2c00 0x7f98a40a50c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f98b0000c00 tx=0x7f98b00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:04.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.436+0000 7f98ba000640 1 -- 172.21.15.1:0/1619265107 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f98a8014030 con 0x7f98bc1047e0 2024-09-12T01:27:04.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.659+0000 7f98c2e00640 1 -- 172.21.15.1:0/1619265107 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98bc02d7c0 con 0x7f98bc1047e0 2024-09-12T01:27:04.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.660+0000 7f98ba000640 1 -- 172.21.15.1:0/1619265107 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11827 (secure 0 0 0) 0x7f98a809bec0 con 0x7f98bc1047e0 2024-09-12T01:27:04.660 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:27:04.660 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":17,"btime":"2024-09-12T01:27:02:542672+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14430,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/572192306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":572192306},{"type":"v1","addr":"172.21.15.1:6835","nonce":572192306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14442,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/1050734277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":1050734277},{"type":"v1","addr":"172.21.15.1:6837","nonce":1050734277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14481,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/3651849993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":3651849993},{"type":"v1","addr":"172.21.15.1:6839","nonce":3651849993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14493,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/2464781973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2464781973},{"type":"v1","addr":"172.21.15.1:6841","nonce":2464781973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/2071825633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2071825633},{"type":"v1","addr":"172.21.15.104:6833","nonce":2071825633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2930310711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2930310711},{"type":"v1","addr":"172.21.15.53:6833","nonce":2930310711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2032381725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2032381725},{"type":"v1","addr":"172.21.15.53:6835","nonce":2032381725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/1547038540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1547038540},{"type":"v1","addr":"172.21.15.104:6835","nonce":1547038540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24493,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/464720247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":464720247},{"type":"v1","addr":"172.21.15.104:6837","nonce":464720247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/2848743754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2848743754},{"type":"v1","addr":"172.21.15.53:6837","nonce":2848743754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6839/1459976041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1459976041},{"type":"v1","addr":"172.21.15.104:6839","nonce":1459976041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T01:26:48.555447+0000","modified":"2024-09-12T01:27:01.749504+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.53:6839/4019389301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":4019389301},{"type":"v1","addr":"172.21.15.53:6839","nonce":4019389301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-12T01:27:04.660 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 17 2024-09-12T01:27:04.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.662+0000 7f98c2e00640 1 -- 172.21.15.1:0/1619265107 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f98a40a2c00 msgr2=0x7f98a40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:04.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.662+0000 7f98c2e00640 1 --2- 172.21.15.1:0/1619265107 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f98a40a2c00 0x7f98a40a50c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f98b0000c00 tx=0x7f98b00058d0 comp rx=0 tx=0).stop 2024-09-12T01:27:04.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.662+0000 7f98c2e00640 1 -- 172.21.15.1:0/1619265107 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98bc1047e0 msgr2=0x7f98bc076c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:04.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.662+0000 7f98c2e00640 1 --2- 172.21.15.1:0/1619265107 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98bc1047e0 0x7f98bc076c40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f98a800c950 tx=0x7f98a800ce10 comp rx=0 tx=0).stop 2024-09-12T01:27:04.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.662+0000 7f98c2e00640 1 -- 172.21.15.1:0/1619265107 shutdown_connections 2024-09-12T01:27:04.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.662+0000 7f98c2e00640 1 --2- 172.21.15.1:0/1619265107 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f98a40a2c00 0x7f98a40a50c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:04.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.662+0000 7f98c2e00640 1 --2- 172.21.15.1:0/1619265107 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98bc108f60 0x7f98bc077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:04.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.662+0000 7f98c2e00640 1 --2- 172.21.15.1:0/1619265107 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98bc1047e0 0x7f98bc076c40 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:04.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.662+0000 7f98c2e00640 1 --2- 172.21.15.1:0/1619265107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98bc103e30 0x7f98bc076700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:04.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.662+0000 7f98c2e00640 1 -- 172.21.15.1:0/1619265107 >> 172.21.15.1:0/1619265107 conn(0x7f98bc0ff8d0 msgr2=0x7f98bc100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:04.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.662+0000 7f98c2e00640 1 -- 172.21.15.1:0/1619265107 shutdown_connections 2024-09-12T01:27:04.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:04.662+0000 7f98c2e00640 1 -- 172.21.15.1:0/1619265107 wait complete. 2024-09-12T01:27:04.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:04 smithi104 ceph-mon[35219]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:27:04.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:04 smithi104 ceph-mon[35219]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-12T01:27:04.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:04 smithi104 ceph-mon[35219]: Cluster is now healthy 2024-09-12T01:27:04.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:04 smithi001 ceph-mon[30387]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:27:04.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:04 smithi001 ceph-mon[30387]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-12T01:27:04.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:04 smithi001 ceph-mon[30387]: Cluster is now healthy 2024-09-12T01:27:04.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:04 smithi053 ceph-mon[35158]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:27:04.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:04 smithi053 ceph-mon[35158]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-12T01:27:04.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:04 smithi053 ceph-mon[35158]: Cluster is now healthy 2024-09-12T01:27:05.578 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T01:27:05.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:05 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/1619265107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:27:05.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:05 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/1619265107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:27:05.961 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:27:05.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:05 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/1619265107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:27:06.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.604+0000 7f16eee00640 1 -- 172.21.15.1:0/2271059228 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16e8101df0 msgr2=0x7f16e81021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:06.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.604+0000 7f16eee00640 1 --2- 172.21.15.1:0/2271059228 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16e8101df0 0x7f16e81021d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f16d8009ca0 tx=0x7f16d802a8c0 comp rx=0 tx=0).stop 2024-09-12T01:27:06.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.604+0000 7f16eee00640 1 -- 172.21.15.1:0/2271059228 shutdown_connections 2024-09-12T01:27:06.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.604+0000 7f16eee00640 1 --2- 172.21.15.1:0/2271059228 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16e8106f20 0x7f16e810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:06.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.604+0000 7f16eee00640 1 --2- 172.21.15.1:0/2271059228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16e81027a0 0x7f16e81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:06.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.604+0000 7f16eee00640 1 --2- 172.21.15.1:0/2271059228 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16e8101df0 0x7f16e81021d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:06.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.604+0000 7f16eee00640 1 -- 172.21.15.1:0/2271059228 >> 172.21.15.1:0/2271059228 conn(0x7f16e80fd8b0 msgr2=0x7f16e80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:06.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.604+0000 7f16eee00640 1 -- 172.21.15.1:0/2271059228 shutdown_connections 2024-09-12T01:27:06.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.604+0000 7f16eee00640 1 -- 172.21.15.1:0/2271059228 wait complete. 2024-09-12T01:27:06.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.605+0000 7f16eee00640 1 Processor -- start 2024-09-12T01:27:06.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.605+0000 7f16eee00640 1 -- start start 2024-09-12T01:27:06.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.605+0000 7f16eee00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16e8101df0 0x7f16e819a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:06.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.605+0000 7f16eee00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16e81027a0 0x7f16e819abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:06.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.606+0000 7f16eee00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16e8106f20 0x7f16e819eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:06.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.606+0000 7f16eee00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f16e8111f80 con 0x7f16e8106f20 2024-09-12T01:27:06.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.606+0000 7f16eee00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f16e8111e00 con 0x7f16e8101df0 2024-09-12T01:27:06.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.606+0000 7f16eee00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f16e8112100 con 0x7f16e81027a0 2024-09-12T01:27:06.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.606+0000 7f16ece00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16e8106f20 0x7f16e819eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:06.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.606+0000 7f16ece00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16e8106f20 0x7f16e819eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47262/0 (socket says 172.21.15.1:47262) 2024-09-12T01:27:06.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.606+0000 7f16ece00640 1 -- 172.21.15.1:0/4100545265 learned_addr learned my addr 172.21.15.1:0/4100545265 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:27:06.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.606+0000 7f16df400640 1 --2- 172.21.15.1:0/4100545265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16e81027a0 0x7f16e819abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:06.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.606+0000 7f16e7e00640 1 --2- 172.21.15.1:0/4100545265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16e8101df0 0x7f16e819a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:06.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.606+0000 7f16df400640 1 -- 172.21.15.1:0/4100545265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16e8101df0 msgr2=0x7f16e819a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:06.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.606+0000 7f16df400640 1 --2- 172.21.15.1:0/4100545265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16e8101df0 0x7f16e819a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:06.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.606+0000 7f16df400640 1 -- 172.21.15.1:0/4100545265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16e8106f20 msgr2=0x7f16e819eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:06.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.606+0000 7f16df400640 1 --2- 172.21.15.1:0/4100545265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16e8106f20 0x7f16e819eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:06.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.606+0000 7f16df400640 1 -- 172.21.15.1:0/4100545265 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e819f590 con 0x7f16e81027a0 2024-09-12T01:27:06.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.607+0000 7f16df400640 1 --2- 172.21.15.1:0/4100545265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16e81027a0 0x7f16e819abc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f16d000e9f0 tx=0x7f16d000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:06.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.607+0000 7f16e5600640 1 -- 172.21.15.1:0/4100545265 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d000ccc0 con 0x7f16e81027a0 2024-09-12T01:27:06.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.607+0000 7f16e5600640 1 -- 172.21.15.1:0/4100545265 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16d0010910 con 0x7f16e81027a0 2024-09-12T01:27:06.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.607+0000 7f16e5600640 1 -- 172.21.15.1:0/4100545265 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d0018960 con 0x7f16e81027a0 2024-09-12T01:27:06.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.607+0000 7f16eee00640 1 -- 172.21.15.1:0/4100545265 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16e819f880 con 0x7f16e81027a0 2024-09-12T01:27:06.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.607+0000 7f16eee00640 1 -- 172.21.15.1:0/4100545265 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16e81a7160 con 0x7f16e81027a0 2024-09-12T01:27:06.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.608+0000 7f16eee00640 1 -- 172.21.15.1:0/4100545265 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16e81021d0 con 0x7f16e81027a0 2024-09-12T01:27:06.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.610+0000 7f16e5600640 1 -- 172.21.15.1:0/4100545265 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f16d0010ac0 con 0x7f16e81027a0 2024-09-12T01:27:06.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.611+0000 7f16e5600640 1 --2- 172.21.15.1:0/4100545265 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f16cc0a2c00 0x7f16cc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:06.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.611+0000 7f16e5600640 1 -- 172.21.15.1:0/4100545265 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8802+0+0 (secure 0 0 0) 0x7f16d00cf5d0 con 0x7f16e81027a0 2024-09-12T01:27:06.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.611+0000 7f16e7e00640 1 --2- 172.21.15.1:0/4100545265 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f16cc0a2c00 0x7f16cc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:06.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.612+0000 7f16e7e00640 1 --2- 172.21.15.1:0/4100545265 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f16cc0a2c00 0x7f16cc0a50c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f16d8006010 tx=0x7f16d8009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:06.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.615+0000 7f16e5600640 1 -- 172.21.15.1:0/4100545265 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f16d009bc40 con 0x7f16e81027a0 2024-09-12T01:27:06.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.801+0000 7f16eee00640 1 -- 172.21.15.1:0/4100545265 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f16e81a7490 con 0x7f16e81027a0 2024-09-12T01:27:06.801 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:06 smithi001 ceph-mon[30387]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:27:06.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.803+0000 7f16e5600640 1 -- 172.21.15.1:0/4100545265 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24130 (secure 0 0 0) 0x7f16d009ff40 con 0x7f16e81027a0 2024-09-12T01:27:06.803 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:27:06.803 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":94,"fsid":"c0e4f180-70a4-11ef-bcea-c7b262605968","created":"2024-09-12T01:18:19.337406+0000","modified":"2024-09-12T01:27:01.749361+0000","last_up_change":"2024-09-12T01:25:50.601477+0000","last_in_change":"2024-09-12T01:25:27.674600+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:21:41.204242+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:26:43.066475+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:26:46.112771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T01:26:56.040130+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b9a396ae-bf97-462f-9ef9-b10ef6b9d337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6803","nonce":1125335136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6805","nonce":1125335136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6809","nonce":1125335136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6807","nonce":1125335136}]},"public_addr":"172.21.15.1:6803/1125335136","cluster_addr":"172.21.15.1:6805/1125335136","heartbeat_back_addr":"172.21.15.1:6809/1125335136","heartbeat_front_addr":"172.21.15.1:6807/1125335136","state":["exists","up"]},{"osd":1,"uuid":"dfe7a8a3-3700-4789-ac64-88660de9e0d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6801","nonce":3131049165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6803","nonce":3131049165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6807","nonce":3131049165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6805","nonce":3131049165}]},"public_addr":"172.21.15.53:6801/3131049165","cluster_addr":"172.21.15.53:6803/3131049165","heartbeat_back_addr":"172.21.15.53:6807/3131049165","heartbeat_front_addr":"172.21.15.53:6805/3131049165","state":["exists","up"]},{"osd":2,"uuid":"5d9d3603-ba7c-4115-9267-12abcb20d55b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6801","nonce":1133285636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6803","nonce":1133285636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6807","nonce":1133285636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6805","nonce":1133285636}]},"public_addr":"172.21.15.104:6801/1133285636","cluster_addr":"172.21.15.104:6803/1133285636","heartbeat_back_addr":"172.21.15.104:6807/1133285636","heartbeat_front_addr":"172.21.15.104:6805/1133285636","state":["exists","up"]},{"osd":3,"uuid":"1e063c66-41b4-4db1-a964-28e122ce628e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6811","nonce":1247468803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6813","nonce":1247468803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6817","nonce":1247468803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6815","nonce":1247468803}]},"public_addr":"172.21.15.1:6811/1247468803","cluster_addr":"172.21.15.1:6813/1247468803","heartbeat_back_addr":"172.21.15.1:6817/1247468803","heartbeat_front_addr":"172.21.15.1:6815/1247468803","state":["exists","up"]},{"osd":4,"uuid":"fe89ece0-2171-45d5-a06d-e400f77233e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6809","nonce":2673413639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6811","nonce":2673413639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6815","nonce":2673413639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6813","nonce":2673413639}]},"public_addr":"172.21.15.53:6809/2673413639","cluster_addr":"172.21.15.53:6811/2673413639","heartbeat_back_addr":"172.21.15.53:6815/2673413639","heartbeat_front_addr":"172.21.15.53:6813/2673413639","state":["exists","up"]},{"osd":5,"uuid":"daab795c-9942-4d16-aa34-8bba30a4b99b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6809","nonce":3131498790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6811","nonce":3131498790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6815","nonce":3131498790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6813","nonce":3131498790}]},"public_addr":"172.21.15.104:6809/3131498790","cluster_addr":"172.21.15.104:6811/3131498790","heartbeat_back_addr":"172.21.15.104:6815/3131498790","heartbeat_front_addr":"172.21.15.104:6813/3131498790","state":["exists","up"]},{"osd":6,"uuid":"f31d8633-e435-499b-9d46-0aa11a2481cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6819","nonce":3413067918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6821","nonce":3413067918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6825","nonce":3413067918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6823","nonce":3413067918}]},"public_addr":"172.21.15.1:6819/3413067918","cluster_addr":"172.21.15.1:6821/3413067918","heartbeat_back_addr":"172.21.15.1:6825/3413067918","heartbeat_front_addr":"172.21.15.1:6823/3413067918","state":["exists","up"]},{"osd":7,"uuid":"e64da49e-d31e-4801-b6cc-634a233b4761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6817","nonce":2841484136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6819","nonce":2841484136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6823","nonce":2841484136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6821","nonce":2841484136}]},"public_addr":"172.21.15.53:6817/2841484136","cluster_addr":"172.21.15.53:6819/2841484136","heartbeat_back_addr":"172.21.15.53:6823/2841484136","heartbeat_front_addr":"172.21.15.53:6821/2841484136","state":["exists","up"]},{"osd":8,"uuid":"fdfb5a42-dca7-4874-b479-176b4a191e21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6817","nonce":941453547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6819","nonce":941453547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6823","nonce":941453547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6821","nonce":941453547}]},"public_addr":"172.21.15.104:6817/941453547","cluster_addr":"172.21.15.104:6819/941453547","heartbeat_back_addr":"172.21.15.104:6823/941453547","heartbeat_front_addr":"172.21.15.104:6821/941453547","state":["exists","up"]},{"osd":9,"uuid":"7f81537a-cae5-4bb8-aec2-4eb07deb7958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6827","nonce":958287586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6829","nonce":958287586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6833","nonce":958287586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6831","nonce":958287586}]},"public_addr":"172.21.15.1:6827/958287586","cluster_addr":"172.21.15.1:6829/958287586","heartbeat_back_addr":"172.21.15.1:6833/958287586","heartbeat_front_addr":"172.21.15.1:6831/958287586","state":["exists","up"]},{"osd":10,"uuid":"e3c9c37d-46b9-4585-990d-69c447453bd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6825","nonce":3357909498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6827","nonce":3357909498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6831","nonce":3357909498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6829","nonce":3357909498}]},"public_addr":"172.21.15.53:6825/3357909498","cluster_addr":"172.21.15.53:6827/3357909498","heartbeat_back_addr":"172.21.15.53:6831/3357909498","heartbeat_front_addr":"172.21.15.53:6829/3357909498","state":["exists","up"]},{"osd":11,"uuid":"474b5345-f2ae-4e0e-a197-53ab11f58c04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6825","nonce":3381427987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6827","nonce":3381427987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6831","nonce":3381427987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6829","nonce":3381427987}]},"public_addr":"172.21.15.104:6825/3381427987","cluster_addr":"172.21.15.104:6827/3381427987","heartbeat_back_addr":"172.21.15.104:6831/3381427987","heartbeat_front_addr":"172.21.15.104:6829/3381427987","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:20:46.903181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:21:12.735556+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:21:37.868739+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:22:09.141228+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:22:36.438563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:23:02.220948+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:23:33.505881+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:24:00.982176+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:24:26.010444+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:24:56.711121+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:25:23.846495+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:25:48.815427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1373984726":"2024-09-13T01:19:14.580637+0000","172.21.15.1:0/1429963718":"2024-09-13T01:19:14.580637+0000","172.21.15.1:6801/4101281615":"2024-09-13T01:19:14.580637+0000","172.21.15.1:6800/4101281615":"2024-09-13T01:19:14.580637+0000","172.21.15.1:0/2564759586":"2024-09-13T01:19:14.580637+0000","172.21.15.1:6801/2935968453":"2024-09-13T01:18:45.882682+0000","172.21.15.1:6800/2935968453":"2024-09-13T01:18:45.882682+0000","172.21.15.1:0/1459848813":"2024-09-13T01:18:45.882682+0000","172.21.15.1:0/578605922":"2024-09-13T01:18:45.882682+0000","172.21.15.1:0/738035035":"2024-09-13T01:18:45.882682+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:27:06.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.806+0000 7f16eee00640 1 -- 172.21.15.1:0/4100545265 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f16cc0a2c00 msgr2=0x7f16cc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:06.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.806+0000 7f16eee00640 1 --2- 172.21.15.1:0/4100545265 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f16cc0a2c00 0x7f16cc0a50c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f16d8006010 tx=0x7f16d8009360 comp rx=0 tx=0).stop 2024-09-12T01:27:06.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.806+0000 7f16eee00640 1 -- 172.21.15.1:0/4100545265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16e81027a0 msgr2=0x7f16e819abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:06.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.806+0000 7f16eee00640 1 --2- 172.21.15.1:0/4100545265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16e81027a0 0x7f16e819abc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f16d000e9f0 tx=0x7f16d000eeb0 comp rx=0 tx=0).stop 2024-09-12T01:27:06.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.806+0000 7f16eee00640 1 -- 172.21.15.1:0/4100545265 shutdown_connections 2024-09-12T01:27:06.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.806+0000 7f16eee00640 1 --2- 172.21.15.1:0/4100545265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16e8106f20 0x7f16e819eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:06.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.806+0000 7f16eee00640 1 --2- 172.21.15.1:0/4100545265 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f16cc0a2c00 0x7f16cc0a50c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:06.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.806+0000 7f16eee00640 1 --2- 172.21.15.1:0/4100545265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16e81027a0 0x7f16e819abc0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:06.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.806+0000 7f16eee00640 1 --2- 172.21.15.1:0/4100545265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16e8101df0 0x7f16e819a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:06.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.806+0000 7f16eee00640 1 -- 172.21.15.1:0/4100545265 >> 172.21.15.1:0/4100545265 conn(0x7f16e80fd8b0 msgr2=0x7f16e80ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:06.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.806+0000 7f16eee00640 1 -- 172.21.15.1:0/4100545265 shutdown_connections 2024-09-12T01:27:06.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:06.806+0000 7f16eee00640 1 -- 172.21.15.1:0/4100545265 wait complete. 2024-09-12T01:27:06.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:06 smithi104 ceph-mon[35219]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:27:06.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:06 smithi053 ceph-mon[35158]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:27:07.693 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T01:27:07.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:07 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/4100545265' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:27:07.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:07 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/4100545265' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:27:07.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:07 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/4100545265' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:27:08.074 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:27:08.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.676+0000 7f4105800640 1 -- 172.21.15.1:0/1889071669 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f41001049c0 msgr2=0x7f4100108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:08.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.676+0000 7f4105800640 1 --2- 172.21.15.1:0/1889071669 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f41001049c0 0x7f4100108a10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f40ec009900 tx=0x7f40ec02a3f0 comp rx=0 tx=0).stop 2024-09-12T01:27:08.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.676+0000 7f4105800640 1 -- 172.21.15.1:0/1889071669 shutdown_connections 2024-09-12T01:27:08.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.676+0000 7f4105800640 1 --2- 172.21.15.1:0/1889071669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4100109140 0x7f4100110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:08.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.676+0000 7f4105800640 1 --2- 172.21.15.1:0/1889071669 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f41001049c0 0x7f4100108a10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:08.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.676+0000 7f4105800640 1 --2- 172.21.15.1:0/1889071669 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4100104010 0x7f41001043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:08.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.676+0000 7f4105800640 1 -- 172.21.15.1:0/1889071669 >> 172.21.15.1:0/1889071669 conn(0x7f41000ffa50 msgr2=0x7f4100101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:08.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.677+0000 7f4105800640 1 -- 172.21.15.1:0/1889071669 shutdown_connections 2024-09-12T01:27:08.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.677+0000 7f4105800640 1 -- 172.21.15.1:0/1889071669 wait complete. 2024-09-12T01:27:08.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.677+0000 7f4105800640 1 Processor -- start 2024-09-12T01:27:08.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.678+0000 7f4105800640 1 -- start start 2024-09-12T01:27:08.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.678+0000 7f4105800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4100104010 0x7f410019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:08.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.678+0000 7f4105800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f41001049c0 0x7f410019ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:08.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.678+0000 7f4105800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4100109140 0x7f410019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:08.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.678+0000 7f4105800640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f41001141b0 con 0x7f4100104010 2024-09-12T01:27:08.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.678+0000 7f4105800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4100114030 con 0x7f4100109140 2024-09-12T01:27:08.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.678+0000 7f4105800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4100114330 con 0x7f41001049c0 2024-09-12T01:27:08.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.678+0000 7f40fea00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4100104010 0x7f410019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:08.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.679+0000 7f40fe000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f41001049c0 0x7f410019ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:08.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.679+0000 7f40ff400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4100109140 0x7f410019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:08.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.679+0000 7f40fea00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4100104010 0x7f410019a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47266/0 (socket says 172.21.15.1:47266) 2024-09-12T01:27:08.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.679+0000 7f40fea00640 1 -- 172.21.15.1:0/1798384841 learned_addr learned my addr 172.21.15.1:0/1798384841 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:27:08.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.679+0000 7f40ff400640 1 -- 172.21.15.1:0/1798384841 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f41001049c0 msgr2=0x7f410019ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:08.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.679+0000 7f40ff400640 1 --2- 172.21.15.1:0/1798384841 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f41001049c0 0x7f410019ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:08.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.679+0000 7f40ff400640 1 -- 172.21.15.1:0/1798384841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4100104010 msgr2=0x7f410019a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:08.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.679+0000 7f40ff400640 1 --2- 172.21.15.1:0/1798384841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4100104010 0x7f410019a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:08.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.679+0000 7f40ff400640 1 -- 172.21.15.1:0/1798384841 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f410019f6c0 con 0x7f4100109140 2024-09-12T01:27:08.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.679+0000 7f40fe000640 1 --2- 172.21.15.1:0/1798384841 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f41001049c0 0x7f410019ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:27:08.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.679+0000 7f40fea00640 1 --2- 172.21.15.1:0/1798384841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4100104010 0x7f410019a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:27:08.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.679+0000 7f40ff400640 1 --2- 172.21.15.1:0/1798384841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4100109140 0x7f410019efe0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f40f8009fd0 tx=0x7f40f800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:08.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.680+0000 7f40f7400640 1 -- 172.21.15.1:0/1798384841 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f800ce20 con 0x7f4100109140 2024-09-12T01:27:08.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.680+0000 7f40f7400640 1 -- 172.21.15.1:0/1798384841 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40f8005020 con 0x7f4100109140 2024-09-12T01:27:08.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.680+0000 7f40f7400640 1 -- 172.21.15.1:0/1798384841 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f8010720 con 0x7f4100109140 2024-09-12T01:27:08.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.680+0000 7f4105800640 1 -- 172.21.15.1:0/1798384841 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f410019f9b0 con 0x7f4100109140 2024-09-12T01:27:08.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.680+0000 7f4105800640 1 -- 172.21.15.1:0/1798384841 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41001a7290 con 0x7f4100109140 2024-09-12T01:27:08.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.681+0000 7f4105800640 1 -- 172.21.15.1:0/1798384841 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41001043f0 con 0x7f4100109140 2024-09-12T01:27:08.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.683+0000 7f40f7400640 1 -- 172.21.15.1:0/1798384841 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f40f8005540 con 0x7f4100109140 2024-09-12T01:27:08.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.685+0000 7f40f7400640 1 --2- 172.21.15.1:0/1798384841 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f40e40a2c00 0x7f40e40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:08.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.685+0000 7f40f7400640 1 -- 172.21.15.1:0/1798384841 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8802+0+0 (secure 0 0 0) 0x7f40f80cfa00 con 0x7f4100109140 2024-09-12T01:27:08.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.685+0000 7f40fea00640 1 --2- 172.21.15.1:0/1798384841 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f40e40a2c00 0x7f40e40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:08.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.686+0000 7f40fea00640 1 --2- 172.21.15.1:0/1798384841 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f40e40a2c00 0x7f40e40a50c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f40f0011500 tx=0x7f40f0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:08.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.687+0000 7f40f7400640 1 -- 172.21.15.1:0/1798384841 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f40f809c6c0 con 0x7f4100109140 2024-09-12T01:27:08.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:08 smithi104 ceph-mon[35219]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:27:08.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.876+0000 7f4105800640 1 -- 172.21.15.1:0/1798384841 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f410019ba70 con 0x7f4100109140 2024-09-12T01:27:08.876 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:08 smithi001 ceph-mon[30387]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:27:08.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.878+0000 7f40f7400640 1 -- 172.21.15.1:0/1798384841 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24130 (secure 0 0 0) 0x7f410019ba70 con 0x7f4100109140 2024-09-12T01:27:08.878 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:27:08.878 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":94,"fsid":"c0e4f180-70a4-11ef-bcea-c7b262605968","created":"2024-09-12T01:18:19.337406+0000","modified":"2024-09-12T01:27:01.749361+0000","last_up_change":"2024-09-12T01:25:50.601477+0000","last_in_change":"2024-09-12T01:25:27.674600+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:21:41.204242+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:26:43.066475+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:26:46.112771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T01:26:56.040130+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b9a396ae-bf97-462f-9ef9-b10ef6b9d337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6803","nonce":1125335136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6805","nonce":1125335136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6809","nonce":1125335136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6807","nonce":1125335136}]},"public_addr":"172.21.15.1:6803/1125335136","cluster_addr":"172.21.15.1:6805/1125335136","heartbeat_back_addr":"172.21.15.1:6809/1125335136","heartbeat_front_addr":"172.21.15.1:6807/1125335136","state":["exists","up"]},{"osd":1,"uuid":"dfe7a8a3-3700-4789-ac64-88660de9e0d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6801","nonce":3131049165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6803","nonce":3131049165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6807","nonce":3131049165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6805","nonce":3131049165}]},"public_addr":"172.21.15.53:6801/3131049165","cluster_addr":"172.21.15.53:6803/3131049165","heartbeat_back_addr":"172.21.15.53:6807/3131049165","heartbeat_front_addr":"172.21.15.53:6805/3131049165","state":["exists","up"]},{"osd":2,"uuid":"5d9d3603-ba7c-4115-9267-12abcb20d55b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6801","nonce":1133285636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6803","nonce":1133285636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6807","nonce":1133285636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6805","nonce":1133285636}]},"public_addr":"172.21.15.104:6801/1133285636","cluster_addr":"172.21.15.104:6803/1133285636","heartbeat_back_addr":"172.21.15.104:6807/1133285636","heartbeat_front_addr":"172.21.15.104:6805/1133285636","state":["exists","up"]},{"osd":3,"uuid":"1e063c66-41b4-4db1-a964-28e122ce628e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6811","nonce":1247468803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6813","nonce":1247468803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6817","nonce":1247468803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6815","nonce":1247468803}]},"public_addr":"172.21.15.1:6811/1247468803","cluster_addr":"172.21.15.1:6813/1247468803","heartbeat_back_addr":"172.21.15.1:6817/1247468803","heartbeat_front_addr":"172.21.15.1:6815/1247468803","state":["exists","up"]},{"osd":4,"uuid":"fe89ece0-2171-45d5-a06d-e400f77233e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6809","nonce":2673413639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6811","nonce":2673413639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6815","nonce":2673413639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6813","nonce":2673413639}]},"public_addr":"172.21.15.53:6809/2673413639","cluster_addr":"172.21.15.53:6811/2673413639","heartbeat_back_addr":"172.21.15.53:6815/2673413639","heartbeat_front_addr":"172.21.15.53:6813/2673413639","state":["exists","up"]},{"osd":5,"uuid":"daab795c-9942-4d16-aa34-8bba30a4b99b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6809","nonce":3131498790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6811","nonce":3131498790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6815","nonce":3131498790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6813","nonce":3131498790}]},"public_addr":"172.21.15.104:6809/3131498790","cluster_addr":"172.21.15.104:6811/3131498790","heartbeat_back_addr":"172.21.15.104:6815/3131498790","heartbeat_front_addr":"172.21.15.104:6813/3131498790","state":["exists","up"]},{"osd":6,"uuid":"f31d8633-e435-499b-9d46-0aa11a2481cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6819","nonce":3413067918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6821","nonce":3413067918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6825","nonce":3413067918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6823","nonce":3413067918}]},"public_addr":"172.21.15.1:6819/3413067918","cluster_addr":"172.21.15.1:6821/3413067918","heartbeat_back_addr":"172.21.15.1:6825/3413067918","heartbeat_front_addr":"172.21.15.1:6823/3413067918","state":["exists","up"]},{"osd":7,"uuid":"e64da49e-d31e-4801-b6cc-634a233b4761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6817","nonce":2841484136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6819","nonce":2841484136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6823","nonce":2841484136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6821","nonce":2841484136}]},"public_addr":"172.21.15.53:6817/2841484136","cluster_addr":"172.21.15.53:6819/2841484136","heartbeat_back_addr":"172.21.15.53:6823/2841484136","heartbeat_front_addr":"172.21.15.53:6821/2841484136","state":["exists","up"]},{"osd":8,"uuid":"fdfb5a42-dca7-4874-b479-176b4a191e21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6817","nonce":941453547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6819","nonce":941453547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6823","nonce":941453547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6821","nonce":941453547}]},"public_addr":"172.21.15.104:6817/941453547","cluster_addr":"172.21.15.104:6819/941453547","heartbeat_back_addr":"172.21.15.104:6823/941453547","heartbeat_front_addr":"172.21.15.104:6821/941453547","state":["exists","up"]},{"osd":9,"uuid":"7f81537a-cae5-4bb8-aec2-4eb07deb7958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6827","nonce":958287586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6829","nonce":958287586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6833","nonce":958287586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6831","nonce":958287586}]},"public_addr":"172.21.15.1:6827/958287586","cluster_addr":"172.21.15.1:6829/958287586","heartbeat_back_addr":"172.21.15.1:6833/958287586","heartbeat_front_addr":"172.21.15.1:6831/958287586","state":["exists","up"]},{"osd":10,"uuid":"e3c9c37d-46b9-4585-990d-69c447453bd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6825","nonce":3357909498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6827","nonce":3357909498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6831","nonce":3357909498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6829","nonce":3357909498}]},"public_addr":"172.21.15.53:6825/3357909498","cluster_addr":"172.21.15.53:6827/3357909498","heartbeat_back_addr":"172.21.15.53:6831/3357909498","heartbeat_front_addr":"172.21.15.53:6829/3357909498","state":["exists","up"]},{"osd":11,"uuid":"474b5345-f2ae-4e0e-a197-53ab11f58c04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6825","nonce":3381427987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6827","nonce":3381427987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6831","nonce":3381427987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6829","nonce":3381427987}]},"public_addr":"172.21.15.104:6825/3381427987","cluster_addr":"172.21.15.104:6827/3381427987","heartbeat_back_addr":"172.21.15.104:6831/3381427987","heartbeat_front_addr":"172.21.15.104:6829/3381427987","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:20:46.903181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:21:12.735556+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:21:37.868739+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:22:09.141228+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:22:36.438563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:23:02.220948+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:23:33.505881+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:24:00.982176+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:24:26.010444+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:24:56.711121+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:25:23.846495+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:25:48.815427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1373984726":"2024-09-13T01:19:14.580637+0000","172.21.15.1:0/1429963718":"2024-09-13T01:19:14.580637+0000","172.21.15.1:6801/4101281615":"2024-09-13T01:19:14.580637+0000","172.21.15.1:6800/4101281615":"2024-09-13T01:19:14.580637+0000","172.21.15.1:0/2564759586":"2024-09-13T01:19:14.580637+0000","172.21.15.1:6801/2935968453":"2024-09-13T01:18:45.882682+0000","172.21.15.1:6800/2935968453":"2024-09-13T01:18:45.882682+0000","172.21.15.1:0/1459848813":"2024-09-13T01:18:45.882682+0000","172.21.15.1:0/578605922":"2024-09-13T01:18:45.882682+0000","172.21.15.1:0/738035035":"2024-09-13T01:18:45.882682+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:27:08.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.881+0000 7f4105800640 1 -- 172.21.15.1:0/1798384841 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f40e40a2c00 msgr2=0x7f40e40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:08.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.881+0000 7f4105800640 1 --2- 172.21.15.1:0/1798384841 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f40e40a2c00 0x7f40e40a50c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f40f0011500 tx=0x7f40f0005f90 comp rx=0 tx=0).stop 2024-09-12T01:27:08.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.881+0000 7f4105800640 1 -- 172.21.15.1:0/1798384841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4100109140 msgr2=0x7f410019efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:08.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.881+0000 7f4105800640 1 --2- 172.21.15.1:0/1798384841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4100109140 0x7f410019efe0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f40f8009fd0 tx=0x7f40f800ef10 comp rx=0 tx=0).stop 2024-09-12T01:27:08.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.881+0000 7f4105800640 1 -- 172.21.15.1:0/1798384841 shutdown_connections 2024-09-12T01:27:08.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.881+0000 7f4105800640 1 --2- 172.21.15.1:0/1798384841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4100109140 0x7f410019efe0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:08.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.881+0000 7f4105800640 1 --2- 172.21.15.1:0/1798384841 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f40e40a2c00 0x7f40e40a50c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:08.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.881+0000 7f4105800640 1 --2- 172.21.15.1:0/1798384841 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f41001049c0 0x7f410019ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:08.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.881+0000 7f4105800640 1 --2- 172.21.15.1:0/1798384841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4100104010 0x7f410019a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:08.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.881+0000 7f4105800640 1 -- 172.21.15.1:0/1798384841 >> 172.21.15.1:0/1798384841 conn(0x7f41000ffa50 msgr2=0x7f4100100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:08.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.882+0000 7f4105800640 1 -- 172.21.15.1:0/1798384841 shutdown_connections 2024-09-12T01:27:08.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:08.882+0000 7f4105800640 1 -- 172.21.15.1:0/1798384841 wait complete. 2024-09-12T01:27:08.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:08 smithi053 ceph-mon[35158]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:27:09.795 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:27:09.845 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:09 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/1798384841' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:27:09.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:09 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/1798384841' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:27:10.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:09 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/1798384841' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:27:10.174 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:27:10.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.753+0000 7fa254000640 1 -- 172.21.15.1:0/1045011508 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa24c101de0 msgr2=0x7fa24c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:10.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.753+0000 7fa254000640 1 --2- 172.21.15.1:0/1045011508 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa24c101de0 0x7fa24c1021c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa240009900 tx=0x7fa24002a410 comp rx=0 tx=0).stop 2024-09-12T01:27:10.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.753+0000 7fa254000640 1 -- 172.21.15.1:0/1045011508 shutdown_connections 2024-09-12T01:27:10.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.753+0000 7fa254000640 1 --2- 172.21.15.1:0/1045011508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa24c106f10 0x7fa24c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:10.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.753+0000 7fa254000640 1 --2- 172.21.15.1:0/1045011508 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa24c102790 0x7fa24c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:10.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.753+0000 7fa254000640 1 --2- 172.21.15.1:0/1045011508 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa24c101de0 0x7fa24c1021c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:10.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.753+0000 7fa254000640 1 -- 172.21.15.1:0/1045011508 >> 172.21.15.1:0/1045011508 conn(0x7fa24c0fd8c0 msgr2=0x7fa24c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:10.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.754+0000 7fa254000640 1 -- 172.21.15.1:0/1045011508 shutdown_connections 2024-09-12T01:27:10.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.754+0000 7fa254000640 1 -- 172.21.15.1:0/1045011508 wait complete. 2024-09-12T01:27:10.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.755+0000 7fa254000640 1 Processor -- start 2024-09-12T01:27:10.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.755+0000 7fa254000640 1 -- start start 2024-09-12T01:27:10.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.755+0000 7fa254000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa24c101de0 0x7fa24c1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:10.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.755+0000 7fa254000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa24c102790 0x7fa24c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:10.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.755+0000 7fa254000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa24c106f10 0x7fa24c19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:10.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.755+0000 7fa254000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa24c111fc0 con 0x7fa24c102790 2024-09-12T01:27:10.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.755+0000 7fa254000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa24c111e40 con 0x7fa24c106f10 2024-09-12T01:27:10.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.755+0000 7fa254000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa24c112140 con 0x7fa24c101de0 2024-09-12T01:27:10.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.755+0000 7fa250c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa24c102790 0x7fa24c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:10.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.755+0000 7fa250c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa24c102790 0x7fa24c198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56996/0 (socket says 172.21.15.1:56996) 2024-09-12T01:27:10.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.755+0000 7fa250c00640 1 -- 172.21.15.1:0/1859243481 learned_addr learned my addr 172.21.15.1:0/1859243481 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:27:10.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.755+0000 7fa252000640 1 --2- 172.21.15.1:0/1859243481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa24c106f10 0x7fa24c19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:10.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.756+0000 7fa251600640 1 --2- 172.21.15.1:0/1859243481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa24c101de0 0x7fa24c1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:10.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.756+0000 7fa250c00640 1 -- 172.21.15.1:0/1859243481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa24c101de0 msgr2=0x7fa24c1983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:10.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.756+0000 7fa250c00640 1 --2- 172.21.15.1:0/1859243481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa24c101de0 0x7fa24c1983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:10.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.756+0000 7fa250c00640 1 -- 172.21.15.1:0/1859243481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa24c106f10 msgr2=0x7fa24c19ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:10.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.756+0000 7fa250c00640 1 --2- 172.21.15.1:0/1859243481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa24c106f10 0x7fa24c19ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:10.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.756+0000 7fa250c00640 1 -- 172.21.15.1:0/1859243481 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa24c19d3a0 con 0x7fa24c102790 2024-09-12T01:27:10.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.756+0000 7fa252000640 1 --2- 172.21.15.1:0/1859243481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa24c106f10 0x7fa24c19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:27:10.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.756+0000 7fa250c00640 1 --2- 172.21.15.1:0/1859243481 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa24c102790 0x7fa24c198930 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa23800d950 tx=0x7fa23800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:10.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.756+0000 7fa24a000640 1 -- 172.21.15.1:0/1859243481 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa238017030 con 0x7fa24c102790 2024-09-12T01:27:10.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.757+0000 7fa254000640 1 -- 172.21.15.1:0/1859243481 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa24c19d690 con 0x7fa24c102790 2024-09-12T01:27:10.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.757+0000 7fa24a000640 1 -- 172.21.15.1:0/1859243481 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa238004020 con 0x7fa24c102790 2024-09-12T01:27:10.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.757+0000 7fa24a000640 1 -- 172.21.15.1:0/1859243481 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2380047a0 con 0x7fa24c102790 2024-09-12T01:27:10.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.757+0000 7fa254000640 1 -- 172.21.15.1:0/1859243481 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa24c1a4f70 con 0x7fa24c102790 2024-09-12T01:27:10.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.758+0000 7fa254000640 1 -- 172.21.15.1:0/1859243481 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa24c1021c0 con 0x7fa24c102790 2024-09-12T01:27:10.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.760+0000 7fa24a000640 1 -- 172.21.15.1:0/1859243481 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fa2380041d0 con 0x7fa24c102790 2024-09-12T01:27:10.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.761+0000 7fa24a000640 1 --2- 172.21.15.1:0/1859243481 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fa2340a2b80 0x7fa2340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:10.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.762+0000 7fa24a000640 1 -- 172.21.15.1:0/1859243481 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8802+0+0 (secure 0 0 0) 0x7fa2380cec10 con 0x7fa24c102790 2024-09-12T01:27:10.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.762+0000 7fa251600640 1 --2- 172.21.15.1:0/1859243481 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fa2340a2b80 0x7fa2340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:10.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.762+0000 7fa251600640 1 --2- 172.21.15.1:0/1859243481 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fa2340a2b80 0x7fa2340a5040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa240000c00 tx=0x7fa240005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:10.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.766+0000 7fa24a000640 1 -- 172.21.15.1:0/1859243481 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa23809b8d0 con 0x7fa24c102790 2024-09-12T01:27:10.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:10 smithi001 ceph-mon[30387]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:27:10.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.987+0000 7fa254000640 1 -- 172.21.15.1:0/1859243481 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa24c10a150 con 0x7fa24c102790 2024-09-12T01:27:10.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.988+0000 7fa24a000640 1 -- 172.21.15.1:0/1859243481 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11827 (secure 0 0 0) 0x7fa23809b1d0 con 0x7fa24c102790 2024-09-12T01:27:10.987 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:27:10.987 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":17,"btime":"2024-09-12T01:27:02:542672+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14430,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/572192306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":572192306},{"type":"v1","addr":"172.21.15.1:6835","nonce":572192306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14442,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/1050734277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":1050734277},{"type":"v1","addr":"172.21.15.1:6837","nonce":1050734277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14481,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/3651849993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":3651849993},{"type":"v1","addr":"172.21.15.1:6839","nonce":3651849993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14493,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/2464781973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2464781973},{"type":"v1","addr":"172.21.15.1:6841","nonce":2464781973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/2071825633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2071825633},{"type":"v1","addr":"172.21.15.104:6833","nonce":2071825633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2930310711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2930310711},{"type":"v1","addr":"172.21.15.53:6833","nonce":2930310711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2032381725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2032381725},{"type":"v1","addr":"172.21.15.53:6835","nonce":2032381725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/1547038540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1547038540},{"type":"v1","addr":"172.21.15.104:6835","nonce":1547038540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24493,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/464720247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":464720247},{"type":"v1","addr":"172.21.15.104:6837","nonce":464720247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/2848743754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2848743754},{"type":"v1","addr":"172.21.15.53:6837","nonce":2848743754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6839/1459976041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1459976041},{"type":"v1","addr":"172.21.15.104:6839","nonce":1459976041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T01:26:48.555447+0000","modified":"2024-09-12T01:27:01.749504+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.53:6839/4019389301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":4019389301},{"type":"v1","addr":"172.21.15.53:6839","nonce":4019389301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-12T01:27:10.988 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 17 2024-09-12T01:27:10.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:10 smithi053 ceph-mon[35158]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:27:10.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.990+0000 7fa254000640 1 -- 172.21.15.1:0/1859243481 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fa2340a2b80 msgr2=0x7fa2340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:10.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.990+0000 7fa254000640 1 --2- 172.21.15.1:0/1859243481 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fa2340a2b80 0x7fa2340a5040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa240000c00 tx=0x7fa240005b50 comp rx=0 tx=0).stop 2024-09-12T01:27:10.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.990+0000 7fa254000640 1 -- 172.21.15.1:0/1859243481 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa24c102790 msgr2=0x7fa24c198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:10.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.990+0000 7fa254000640 1 --2- 172.21.15.1:0/1859243481 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa24c102790 0x7fa24c198930 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa23800d950 tx=0x7fa23800de10 comp rx=0 tx=0).stop 2024-09-12T01:27:10.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.990+0000 7fa254000640 1 -- 172.21.15.1:0/1859243481 shutdown_connections 2024-09-12T01:27:10.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.990+0000 7fa254000640 1 --2- 172.21.15.1:0/1859243481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa24c106f10 0x7fa24c19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:10.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.990+0000 7fa254000640 1 --2- 172.21.15.1:0/1859243481 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa24c102790 0x7fa24c198930 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:10.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.990+0000 7fa254000640 1 --2- 172.21.15.1:0/1859243481 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fa2340a2b80 0x7fa2340a5040 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:10.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.990+0000 7fa254000640 1 --2- 172.21.15.1:0/1859243481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa24c101de0 0x7fa24c1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:10.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.990+0000 7fa254000640 1 -- 172.21.15.1:0/1859243481 >> 172.21.15.1:0/1859243481 conn(0x7fa24c0fd8c0 msgr2=0x7fa24c0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:10.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.990+0000 7fa254000640 1 -- 172.21.15.1:0/1859243481 shutdown_connections 2024-09-12T01:27:10.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:10.990+0000 7fa254000640 1 -- 172.21.15.1:0/1859243481 wait complete. 2024-09-12T01:27:11.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:10 smithi104 ceph-mon[35219]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:27:11.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:11 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/1859243481' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:27:11.974 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T01:27:11.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:11 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/1859243481' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:27:12.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:11 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/1859243481' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:27:12.356 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:27:12.815 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:12 smithi001 ceph-mon[30387]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:27:12.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:12.928+0000 7f3185c00640 1 -- 172.21.15.1:0/2363076883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f318010e330 msgr2=0x7f3180110720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:12.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:12.928+0000 7f3185c00640 1 --2- 172.21.15.1:0/2363076883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f318010e330 0x7f3180110720 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3178009ca0 tx=0x7f317802a8c0 comp rx=0 tx=0).stop 2024-09-12T01:27:12.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:12.928+0000 7f3185c00640 1 -- 172.21.15.1:0/2363076883 shutdown_connections 2024-09-12T01:27:12.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:12.928+0000 7f3185c00640 1 --2- 172.21.15.1:0/2363076883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f318010e330 0x7f3180110720 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:12.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:12.928+0000 7f3185c00640 1 --2- 172.21.15.1:0/2363076883 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3180101130 0x7f318010ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:12.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:12.928+0000 7f3185c00640 1 --2- 172.21.15.1:0/2363076883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3180100780 0x7f3180100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:12.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:12.928+0000 7f3185c00640 1 -- 172.21.15.1:0/2363076883 >> 172.21.15.1:0/2363076883 conn(0x7f31800fc610 msgr2=0x7f31800fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:12.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:12.928+0000 7f3185c00640 1 -- 172.21.15.1:0/2363076883 shutdown_connections 2024-09-12T01:27:12.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:12.928+0000 7f3185c00640 1 -- 172.21.15.1:0/2363076883 wait complete. 2024-09-12T01:27:12.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:12.929+0000 7f3185c00640 1 Processor -- start 2024-09-12T01:27:12.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:12.929+0000 7f3185c00640 1 -- start start 2024-09-12T01:27:12.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:12.930+0000 7f3185c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3180100780 0x7f318019a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:12.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:12.930+0000 7f3185c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3180101130 0x7f318019a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:12.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:12.930+0000 7f3185c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f318010e330 0x7f318019ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:12.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:12.930+0000 7f3185c00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3180114030 con 0x7f3180100780 2024-09-12T01:27:12.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:12.930+0000 7f3185c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3180113eb0 con 0x7f3180101130 2024-09-12T01:27:12.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:12.930+0000 7f3185c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f31801141b0 con 0x7f318010e330 2024-09-12T01:27:12.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:12.930+0000 7f317fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3180100780 0x7f318019a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:12.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:12.930+0000 7f317fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3180100780 0x7f318019a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57022/0 (socket says 172.21.15.1:57022) 2024-09-12T01:27:12.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:12.930+0000 7f317fe00640 1 -- 172.21.15.1:0/3686685738 learned_addr learned my addr 172.21.15.1:0/3686685738 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:27:12.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:12.930+0000 7f317f400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3180101130 0x7f318019a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:12.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:12.930+0000 7f3185200640 1 --2- 172.21.15.1:0/3686685738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f318010e330 0x7f318019ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:12.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:12.930+0000 7f317fe00640 1 -- 172.21.15.1:0/3686685738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f318010e330 msgr2=0x7f318019ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:12.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:12.930+0000 7f317fe00640 1 --2- 172.21.15.1:0/3686685738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f318010e330 0x7f318019ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:12.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:12.930+0000 7f317fe00640 1 -- 172.21.15.1:0/3686685738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3180101130 msgr2=0x7f318019a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:12.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:12.930+0000 7f317fe00640 1 --2- 172.21.15.1:0/3686685738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3180101130 0x7f318019a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:12.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:12.930+0000 7f317fe00640 1 -- 172.21.15.1:0/3686685738 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f318019f3e0 con 0x7f3180100780 2024-09-12T01:27:12.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:12.931+0000 7f317fe00640 1 --2- 172.21.15.1:0/3686685738 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3180100780 0x7f318019a430 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f317000e9f0 tx=0x7f317000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:12.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:12.931+0000 7f317cc00640 1 -- 172.21.15.1:0/3686685738 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317000cd80 con 0x7f3180100780 2024-09-12T01:27:12.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:12.931+0000 7f317cc00640 1 -- 172.21.15.1:0/3686685738 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3170010910 con 0x7f3180100780 2024-09-12T01:27:12.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:12.931+0000 7f3185c00640 1 -- 172.21.15.1:0/3686685738 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f318019f6d0 con 0x7f3180100780 2024-09-12T01:27:12.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:12.931+0000 7f317cc00640 1 -- 172.21.15.1:0/3686685738 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3170018990 con 0x7f3180100780 2024-09-12T01:27:12.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:12.931+0000 7f3185c00640 1 -- 172.21.15.1:0/3686685738 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31801a6fb0 con 0x7f3180100780 2024-09-12T01:27:12.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:12.932+0000 7f3185c00640 1 -- 172.21.15.1:0/3686685738 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3180100b60 con 0x7f3180100780 2024-09-12T01:27:12.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:12.936+0000 7f317cc00640 1 -- 172.21.15.1:0/3686685738 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f3170010ac0 con 0x7f3180100780 2024-09-12T01:27:12.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:12.937+0000 7f317cc00640 1 --2- 172.21.15.1:0/3686685738 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f31640a2c00 0x7f31640a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:12.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:12.937+0000 7f317cc00640 1 -- 172.21.15.1:0/3686685738 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8802+0+0 (secure 0 0 0) 0x7f31700cf8e0 con 0x7f3180100780 2024-09-12T01:27:12.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:12.937+0000 7f317f400640 1 --2- 172.21.15.1:0/3686685738 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f31640a2c00 0x7f31640a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:12.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:12.938+0000 7f317f400640 1 --2- 172.21.15.1:0/3686685738 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f31640a2c00 0x7f31640a50c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f318010ebe0 tx=0x7f316c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:12.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:12.939+0000 7f317cc00640 1 -- 172.21.15.1:0/3686685738 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f317009bfb0 con 0x7f3180100780 2024-09-12T01:27:12.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:12 smithi053 ceph-mon[35158]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:27:13.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:12 smithi104 ceph-mon[35219]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:27:13.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:13.127+0000 7f3185c00640 1 -- 172.21.15.1:0/3686685738 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f31801a7290 con 0x7f3180100780 2024-09-12T01:27:13.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:13.128+0000 7f317cc00640 1 -- 172.21.15.1:0/3686685738 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24130 (secure 0 0 0) 0x7f31700a02b0 con 0x7f3180100780 2024-09-12T01:27:13.128 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:27:13.128 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":94,"fsid":"c0e4f180-70a4-11ef-bcea-c7b262605968","created":"2024-09-12T01:18:19.337406+0000","modified":"2024-09-12T01:27:01.749361+0000","last_up_change":"2024-09-12T01:25:50.601477+0000","last_in_change":"2024-09-12T01:25:27.674600+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:21:41.204242+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:26:43.066475+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:26:46.112771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T01:26:56.040130+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b9a396ae-bf97-462f-9ef9-b10ef6b9d337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6803","nonce":1125335136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6805","nonce":1125335136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6809","nonce":1125335136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6807","nonce":1125335136}]},"public_addr":"172.21.15.1:6803/1125335136","cluster_addr":"172.21.15.1:6805/1125335136","heartbeat_back_addr":"172.21.15.1:6809/1125335136","heartbeat_front_addr":"172.21.15.1:6807/1125335136","state":["exists","up"]},{"osd":1,"uuid":"dfe7a8a3-3700-4789-ac64-88660de9e0d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6801","nonce":3131049165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6803","nonce":3131049165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6807","nonce":3131049165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6805","nonce":3131049165}]},"public_addr":"172.21.15.53:6801/3131049165","cluster_addr":"172.21.15.53:6803/3131049165","heartbeat_back_addr":"172.21.15.53:6807/3131049165","heartbeat_front_addr":"172.21.15.53:6805/3131049165","state":["exists","up"]},{"osd":2,"uuid":"5d9d3603-ba7c-4115-9267-12abcb20d55b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6801","nonce":1133285636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6803","nonce":1133285636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6807","nonce":1133285636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6805","nonce":1133285636}]},"public_addr":"172.21.15.104:6801/1133285636","cluster_addr":"172.21.15.104:6803/1133285636","heartbeat_back_addr":"172.21.15.104:6807/1133285636","heartbeat_front_addr":"172.21.15.104:6805/1133285636","state":["exists","up"]},{"osd":3,"uuid":"1e063c66-41b4-4db1-a964-28e122ce628e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6811","nonce":1247468803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6813","nonce":1247468803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6817","nonce":1247468803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6815","nonce":1247468803}]},"public_addr":"172.21.15.1:6811/1247468803","cluster_addr":"172.21.15.1:6813/1247468803","heartbeat_back_addr":"172.21.15.1:6817/1247468803","heartbeat_front_addr":"172.21.15.1:6815/1247468803","state":["exists","up"]},{"osd":4,"uuid":"fe89ece0-2171-45d5-a06d-e400f77233e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6809","nonce":2673413639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6811","nonce":2673413639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6815","nonce":2673413639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6813","nonce":2673413639}]},"public_addr":"172.21.15.53:6809/2673413639","cluster_addr":"172.21.15.53:6811/2673413639","heartbeat_back_addr":"172.21.15.53:6815/2673413639","heartbeat_front_addr":"172.21.15.53:6813/2673413639","state":["exists","up"]},{"osd":5,"uuid":"daab795c-9942-4d16-aa34-8bba30a4b99b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6809","nonce":3131498790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6811","nonce":3131498790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6815","nonce":3131498790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6813","nonce":3131498790}]},"public_addr":"172.21.15.104:6809/3131498790","cluster_addr":"172.21.15.104:6811/3131498790","heartbeat_back_addr":"172.21.15.104:6815/3131498790","heartbeat_front_addr":"172.21.15.104:6813/3131498790","state":["exists","up"]},{"osd":6,"uuid":"f31d8633-e435-499b-9d46-0aa11a2481cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6819","nonce":3413067918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6821","nonce":3413067918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6825","nonce":3413067918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6823","nonce":3413067918}]},"public_addr":"172.21.15.1:6819/3413067918","cluster_addr":"172.21.15.1:6821/3413067918","heartbeat_back_addr":"172.21.15.1:6825/3413067918","heartbeat_front_addr":"172.21.15.1:6823/3413067918","state":["exists","up"]},{"osd":7,"uuid":"e64da49e-d31e-4801-b6cc-634a233b4761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6817","nonce":2841484136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6819","nonce":2841484136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6823","nonce":2841484136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6821","nonce":2841484136}]},"public_addr":"172.21.15.53:6817/2841484136","cluster_addr":"172.21.15.53:6819/2841484136","heartbeat_back_addr":"172.21.15.53:6823/2841484136","heartbeat_front_addr":"172.21.15.53:6821/2841484136","state":["exists","up"]},{"osd":8,"uuid":"fdfb5a42-dca7-4874-b479-176b4a191e21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6817","nonce":941453547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6819","nonce":941453547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6823","nonce":941453547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6821","nonce":941453547}]},"public_addr":"172.21.15.104:6817/941453547","cluster_addr":"172.21.15.104:6819/941453547","heartbeat_back_addr":"172.21.15.104:6823/941453547","heartbeat_front_addr":"172.21.15.104:6821/941453547","state":["exists","up"]},{"osd":9,"uuid":"7f81537a-cae5-4bb8-aec2-4eb07deb7958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6827","nonce":958287586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6829","nonce":958287586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6833","nonce":958287586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6831","nonce":958287586}]},"public_addr":"172.21.15.1:6827/958287586","cluster_addr":"172.21.15.1:6829/958287586","heartbeat_back_addr":"172.21.15.1:6833/958287586","heartbeat_front_addr":"172.21.15.1:6831/958287586","state":["exists","up"]},{"osd":10,"uuid":"e3c9c37d-46b9-4585-990d-69c447453bd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6825","nonce":3357909498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6827","nonce":3357909498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6831","nonce":3357909498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6829","nonce":3357909498}]},"public_addr":"172.21.15.53:6825/3357909498","cluster_addr":"172.21.15.53:6827/3357909498","heartbeat_back_addr":"172.21.15.53:6831/3357909498","heartbeat_front_addr":"172.21.15.53:6829/3357909498","state":["exists","up"]},{"osd":11,"uuid":"474b5345-f2ae-4e0e-a197-53ab11f58c04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6825","nonce":3381427987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6827","nonce":3381427987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6831","nonce":3381427987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6829","nonce":3381427987}]},"public_addr":"172.21.15.104:6825/3381427987","cluster_addr":"172.21.15.104:6827/3381427987","heartbeat_back_addr":"172.21.15.104:6831/3381427987","heartbeat_front_addr":"172.21.15.104:6829/3381427987","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:20:46.903181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:21:12.735556+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:21:37.868739+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:22:09.141228+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:22:36.438563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:23:02.220948+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:23:33.505881+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:24:00.982176+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:24:26.010444+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:24:56.711121+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:25:23.846495+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:25:48.815427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1373984726":"2024-09-13T01:19:14.580637+0000","172.21.15.1:0/1429963718":"2024-09-13T01:19:14.580637+0000","172.21.15.1:6801/4101281615":"2024-09-13T01:19:14.580637+0000","172.21.15.1:6800/4101281615":"2024-09-13T01:19:14.580637+0000","172.21.15.1:0/2564759586":"2024-09-13T01:19:14.580637+0000","172.21.15.1:6801/2935968453":"2024-09-13T01:18:45.882682+0000","172.21.15.1:6800/2935968453":"2024-09-13T01:18:45.882682+0000","172.21.15.1:0/1459848813":"2024-09-13T01:18:45.882682+0000","172.21.15.1:0/578605922":"2024-09-13T01:18:45.882682+0000","172.21.15.1:0/738035035":"2024-09-13T01:18:45.882682+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:27:13.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:13.131+0000 7f3185c00640 1 -- 172.21.15.1:0/3686685738 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f31640a2c00 msgr2=0x7f31640a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:13.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:13.131+0000 7f3185c00640 1 --2- 172.21.15.1:0/3686685738 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f31640a2c00 0x7f31640a50c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f318010ebe0 tx=0x7f316c005f90 comp rx=0 tx=0).stop 2024-09-12T01:27:13.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:13.131+0000 7f3185c00640 1 -- 172.21.15.1:0/3686685738 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3180100780 msgr2=0x7f318019a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:13.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:13.131+0000 7f3185c00640 1 --2- 172.21.15.1:0/3686685738 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3180100780 0x7f318019a430 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f317000e9f0 tx=0x7f317000eeb0 comp rx=0 tx=0).stop 2024-09-12T01:27:13.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:13.131+0000 7f3185c00640 1 -- 172.21.15.1:0/3686685738 shutdown_connections 2024-09-12T01:27:13.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:13.131+0000 7f3185c00640 1 --2- 172.21.15.1:0/3686685738 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f31640a2c00 0x7f31640a50c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:13.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:13.131+0000 7f3185c00640 1 --2- 172.21.15.1:0/3686685738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f318010e330 0x7f318019ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:13.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:13.131+0000 7f3185c00640 1 --2- 172.21.15.1:0/3686685738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3180101130 0x7f318019a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:13.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:13.131+0000 7f3185c00640 1 --2- 172.21.15.1:0/3686685738 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3180100780 0x7f318019a430 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:13.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:13.131+0000 7f3185c00640 1 -- 172.21.15.1:0/3686685738 >> 172.21.15.1:0/3686685738 conn(0x7f31800fc610 msgr2=0x7f31800fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:13.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:13.131+0000 7f3185c00640 1 -- 172.21.15.1:0/3686685738 shutdown_connections 2024-09-12T01:27:13.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:13.132+0000 7f3185c00640 1 -- 172.21.15.1:0/3686685738 wait complete. 2024-09-12T01:27:13.811 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:13 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/3686685738' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:27:13.979 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-12T01:26:48.555447+0000', 'modified': '2024-09-12T01:27:01.749504+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24545}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24545': {'gid': 24545, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.53:6839/4019389301', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6838', 'nonce': 4019389301}, {'type': 'v1', 'addr': '172.21.15.53:6839', 'nonce': 4019389301}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24545, 'qdb_cluster': [24545]} 2024-09-12T01:27:13.979 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-12T01:27:13.979 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-12T01:27:13.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:13 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/3686685738' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:27:14.077 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:13 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/3686685738' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:27:14.365 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:27:14.842 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:14 smithi001 ceph-mon[30387]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:27:14.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.958+0000 7ff956c00640 1 -- 172.21.15.1:0/2479850958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9501047e0 msgr2=0x7ff950108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:14.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.958+0000 7ff956c00640 1 --2- 172.21.15.1:0/2479850958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9501047e0 0x7ff950108830 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff93c009860 tx=0x7ff93c02a340 comp rx=0 tx=0).stop 2024-09-12T01:27:14.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.958+0000 7ff956c00640 1 -- 172.21.15.1:0/2479850958 shutdown_connections 2024-09-12T01:27:14.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.958+0000 7ff956c00640 1 --2- 172.21.15.1:0/2479850958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff950108f60 0x7ff950110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:14.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.958+0000 7ff956c00640 1 --2- 172.21.15.1:0/2479850958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9501047e0 0x7ff950108830 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:14.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.958+0000 7ff956c00640 1 --2- 172.21.15.1:0/2479850958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff950103e30 0x7ff950104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:14.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.958+0000 7ff956c00640 1 -- 172.21.15.1:0/2479850958 >> 172.21.15.1:0/2479850958 conn(0x7ff9500ff8d0 msgr2=0x7ff950101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:14.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.958+0000 7ff956c00640 1 -- 172.21.15.1:0/2479850958 shutdown_connections 2024-09-12T01:27:14.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.958+0000 7ff956c00640 1 -- 172.21.15.1:0/2479850958 wait complete. 2024-09-12T01:27:14.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.959+0000 7ff956c00640 1 Processor -- start 2024-09-12T01:27:14.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.959+0000 7ff956c00640 1 -- start start 2024-09-12T01:27:14.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.960+0000 7ff956c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff950103e30 0x7ff95019a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:14.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.960+0000 7ff956c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9501047e0 0x7ff95019a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:14.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.960+0000 7ff956c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff950108f60 0x7ff95019ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:14.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.960+0000 7ff956c00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff950113f80 con 0x7ff9501047e0 2024-09-12T01:27:14.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.960+0000 7ff956c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff950113e00 con 0x7ff950108f60 2024-09-12T01:27:14.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.960+0000 7ff956c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff950114100 con 0x7ff950103e30 2024-09-12T01:27:14.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.960+0000 7ff954e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9501047e0 0x7ff95019a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:14.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.960+0000 7ff954e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9501047e0 0x7ff95019a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57036/0 (socket says 172.21.15.1:57036) 2024-09-12T01:27:14.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.960+0000 7ff954e00640 1 -- 172.21.15.1:0/3825037703 learned_addr learned my addr 172.21.15.1:0/3825037703 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:27:14.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.960+0000 7ff955800640 1 --2- 172.21.15.1:0/3825037703 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff950103e30 0x7ff95019a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:14.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.960+0000 7ff956200640 1 --2- 172.21.15.1:0/3825037703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff950108f60 0x7ff95019ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:14.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.961+0000 7ff954e00640 1 -- 172.21.15.1:0/3825037703 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff950103e30 msgr2=0x7ff95019a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:14.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.961+0000 7ff954e00640 1 --2- 172.21.15.1:0/3825037703 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff950103e30 0x7ff95019a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:14.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.961+0000 7ff954e00640 1 -- 172.21.15.1:0/3825037703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff950108f60 msgr2=0x7ff95019ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:14.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.961+0000 7ff954e00640 1 --2- 172.21.15.1:0/3825037703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff950108f60 0x7ff95019ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:14.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.961+0000 7ff954e00640 1 -- 172.21.15.1:0/3825037703 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff95019f400 con 0x7ff9501047e0 2024-09-12T01:27:14.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.961+0000 7ff956200640 1 --2- 172.21.15.1:0/3825037703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff950108f60 0x7ff95019ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:27:14.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.961+0000 7ff955800640 1 --2- 172.21.15.1:0/3825037703 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff950103e30 0x7ff95019a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:27:14.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.961+0000 7ff954e00640 1 --2- 172.21.15.1:0/3825037703 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9501047e0 0x7ff95019a990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff93c035040 tx=0x7ff93c033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:14.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.961+0000 7ff94e000640 1 -- 172.21.15.1:0/3825037703 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c03f020 con 0x7ff9501047e0 2024-09-12T01:27:14.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.961+0000 7ff94e000640 1 -- 172.21.15.1:0/3825037703 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff93c032770 con 0x7ff9501047e0 2024-09-12T01:27:14.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.961+0000 7ff956c00640 1 -- 172.21.15.1:0/3825037703 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff95019f690 con 0x7ff9501047e0 2024-09-12T01:27:14.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.961+0000 7ff94e000640 1 -- 172.21.15.1:0/3825037703 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c037040 con 0x7ff9501047e0 2024-09-12T01:27:14.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.962+0000 7ff956c00640 1 -- 172.21.15.1:0/3825037703 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9501a6fd0 con 0x7ff9501047e0 2024-09-12T01:27:14.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.965+0000 7ff94e000640 1 -- 172.21.15.1:0/3825037703 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7ff93c042020 con 0x7ff9501047e0 2024-09-12T01:27:14.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.965+0000 7ff95ac50740 1 Processor -- start 2024-09-12T01:27:14.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.965+0000 7ff95ac50740 1 -- start start 2024-09-12T01:27:14.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.966+0000 7ff95ac50740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2efd8a4e0 0x55b2efd8a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:14.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.966+0000 7ff95ac50740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2efd8ae00 0x55b2efd95180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:14.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.966+0000 7ff95ac50740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2efd956c0 0x55b2efd97ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:14.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.966+0000 7ff95ac50740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b2efcf00b0 con 0x55b2efd956c0 2024-09-12T01:27:14.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.966+0000 7ff95ac50740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b2efc97ab0 con 0x55b2efd8ae00 2024-09-12T01:27:14.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.966+0000 7ff95ac50740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b2efcec070 con 0x55b2efd8a4e0 2024-09-12T01:27:14.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.966+0000 7ff955800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2efd8ae00 0x55b2efd95180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:14.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.966+0000 7ff954e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2efd956c0 0x55b2efd97ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:14.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.966+0000 7ff955800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2efd8ae00 0x55b2efd95180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:55632/0 (socket says 172.21.15.1:55632) 2024-09-12T01:27:14.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.966+0000 7ff955800640 1 -- 172.21.15.1:0/4285883626 learned_addr learned my addr 172.21.15.1:0/4285883626 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:27:14.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.966+0000 7ff956200640 1 --2- 172.21.15.1:0/4285883626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2efd8a4e0 0x55b2efd8a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:14.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.967+0000 7ff955800640 1 -- 172.21.15.1:0/4285883626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2efd8a4e0 msgr2=0x55b2efd8a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:27:14.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.967+0000 7ff955800640 1 --2- 172.21.15.1:0/4285883626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2efd8a4e0 0x55b2efd8a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:14.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.967+0000 7ff955800640 1 -- 172.21.15.1:0/4285883626 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2efd956c0 msgr2=0x55b2efd97ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:27:14.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.967+0000 7ff955800640 1 --2- 172.21.15.1:0/4285883626 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2efd956c0 0x55b2efd97ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:14.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.967+0000 7ff955800640 1 -- 172.21.15.1:0/4285883626 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2efca44a0 con 0x55b2efd8ae00 2024-09-12T01:27:14.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.967+0000 7ff956200640 1 --2- 172.21.15.1:0/4285883626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2efd8a4e0 0x55b2efd8a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:27:14.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.967+0000 7ff94e000640 1 --2- 172.21.15.1:0/3825037703 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff9380a2c00 0x7ff9380a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:14.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.967+0000 7ff955800640 1 --2- 172.21.15.1:0/4285883626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2efd8ae00 0x55b2efd95180 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff9440114f0 tx=0x7ff9440119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5966a1ef5cb516a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:14.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.967+0000 7ff94e000640 1 -- 172.21.15.1:0/3825037703 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8802+0+0 (secure 0 0 0) 0x7ff93c0eebc0 con 0x7ff9501047e0 2024-09-12T01:27:14.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.967+0000 7ff956200640 1 --2- 172.21.15.1:0/3825037703 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff9380a2c00 0x7ff9380a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:14.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.967+0000 7ff942a00640 1 -- 172.21.15.1:0/4285883626 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff944005020 con 0x55b2efd8ae00 2024-09-12T01:27:14.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.967+0000 7ff942a00640 1 -- 172.21.15.1:0/4285883626 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9440047a0 con 0x55b2efd8ae00 2024-09-12T01:27:14.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.967+0000 7ff942a00640 1 -- 172.21.15.1:0/4285883626 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94400f9c0 con 0x55b2efd8ae00 2024-09-12T01:27:14.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.968+0000 7ff95ac50740 1 -- 172.21.15.1:0/4285883626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2efd8ae00 msgr2=0x55b2efd95180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:14.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.968+0000 7ff956200640 1 --2- 172.21.15.1:0/3825037703 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff9380a2c00 0x7ff9380a50c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff948005fd0 tx=0x7ff948009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:14.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.968+0000 7ff95ac50740 1 --2- 172.21.15.1:0/4285883626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2efd8ae00 0x55b2efd95180 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff9440114f0 tx=0x7ff9440119b0 comp rx=0 tx=0).stop 2024-09-12T01:27:14.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.968+0000 7ff95ac50740 1 -- 172.21.15.1:0/4285883626 shutdown_connections 2024-09-12T01:27:14.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.968+0000 7ff95ac50740 1 --2- 172.21.15.1:0/4285883626 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2efd956c0 0x55b2efd97ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:14.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.968+0000 7ff95ac50740 1 --2- 172.21.15.1:0/4285883626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2efd8ae00 0x55b2efd95180 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:14.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.968+0000 7ff95ac50740 1 --2- 172.21.15.1:0/4285883626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2efd8a4e0 0x55b2efd8a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:14.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.968+0000 7ff95ac50740 1 -- 172.21.15.1:0/4285883626 >> 172.21.15.1:0/4285883626 conn(0x55b2efd86a70 msgr2=0x55b2efd86e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:14.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.968+0000 7ff95ac50740 1 -- 172.21.15.1:0/4285883626 shutdown_connections 2024-09-12T01:27:14.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.969+0000 7ff95ac50740 1 -- 172.21.15.1:0/4285883626 wait complete. 2024-09-12T01:27:14.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.969+0000 7ff95ac50740 1 -- start start 2024-09-12T01:27:14.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.969+0000 7ff95ac50740 1 Processor -- start 2024-09-12T01:27:14.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.970+0000 7ff95ac50740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2efd8a4e0 0x55b2efd92580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:14.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.970+0000 7ff95ac50740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2efd8ae00 0x55b2efd92ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:14.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.970+0000 7ff95ac50740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2efd956c0 0x55b2efd93000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:14.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.970+0000 7ff95ac50740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b2efd98330 con 0x55b2efd956c0 2024-09-12T01:27:14.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.970+0000 7ff95ac50740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b2efc78920 con 0x55b2efd8ae00 2024-09-12T01:27:14.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.970+0000 7ff95ac50740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b2efd984b0 con 0x55b2efd8a4e0 2024-09-12T01:27:14.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.970+0000 7ff95ac50740 10 client.0 _ll_register_callbacks cb 0x55b2efc9c8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:27:14.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.970+0000 7ff956200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2efd956c0 0x55b2efd93000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:14.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.970+0000 7ff955800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2efd8a4e0 0x55b2efd92580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:14.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.970+0000 7ff954e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2efd8ae00 0x55b2efd92ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:14.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.970+0000 7ff955800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2efd8a4e0 0x55b2efd92580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:41288/0 (socket says 172.21.15.1:41288) 2024-09-12T01:27:14.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.970+0000 7ff955800640 1 -- 172.21.15.1:0/1723331458 learned_addr learned my addr 172.21.15.1:0/1723331458 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:27:14.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.970+0000 7ff955800640 1 -- 172.21.15.1:0/1723331458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2efd8ae00 msgr2=0x55b2efd92ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:27:14.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.970+0000 7ff955800640 1 --2- 172.21.15.1:0/1723331458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2efd8ae00 0x55b2efd92ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:14.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.970+0000 7ff955800640 1 -- 172.21.15.1:0/1723331458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2efd956c0 msgr2=0x55b2efd93000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:27:14.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.970+0000 7ff955800640 1 --2- 172.21.15.1:0/1723331458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2efd956c0 0x55b2efd93000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:14.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.970+0000 7ff955800640 1 -- 172.21.15.1:0/1723331458 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2efa612b0 con 0x55b2efd8a4e0 2024-09-12T01:27:14.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.971+0000 7ff955800640 1 --2- 172.21.15.1:0/1723331458 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2efd8a4e0 0x55b2efd92580 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff94401b040 tx=0x7ff944005200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=638d864e32a46ae8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:14.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.971+0000 7ff937400640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-12T01:27:14.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.971+0000 7ff937400640 1 -- 172.21.15.1:0/1723331458 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff944028070 con 0x55b2efd8a4e0 2024-09-12T01:27:14.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.971+0000 7ff95ac50740 1 -- 172.21.15.1:0/1723331458 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9440047a0 con 0x55b2efd8a4e0 2024-09-12T01:27:14.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.971+0000 7ff937400640 1 -- 172.21.15.1:0/1723331458 <== mon.1 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8802+0+0 (secure 0 0 0) 0x7ff94400f6c0 con 0x55b2efd8a4e0 2024-09-12T01:27:14.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.971+0000 7ff937400640 1 -- 172.21.15.1:0/1723331458 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7ff944034050 con 0x55b2efd8a4e0 2024-09-12T01:27:14.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.971+0000 7ff95ac50740 10 client.24628 fetch_fsmap learned FSMap version 17 2024-09-12T01:27:14.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.971+0000 7ff95ac50740 1 -- 172.21.15.1:0/1723331458 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55b2efda3fb0 con 0x55b2efd8a4e0 2024-09-12T01:27:14.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.972+0000 7ff937400640 1 -- 172.21.15.1:0/1723331458 <== mon.1 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9440047a0 con 0x55b2efd8a4e0 2024-09-12T01:27:14.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.972+0000 7ff937400640 1 -- 172.21.15.1:0/1723331458 <== mon.1 v2:172.21.15.104:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7ff9440246a0 con 0x55b2efd8a4e0 2024-09-12T01:27:14.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.972+0000 7ff95ac50740 10 client.24628 fetch_fsmap finished waiting for FSMap version 17 2024-09-12T01:27:14.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.972+0000 7ff95ac50740 10 client.24628 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T01:27:14.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.972+0000 7ff95ac50740 20 client.24628 populate_metadata read hostname 'smithi001' 2024-09-12T01:27:14.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.972+0000 7ff95ac50740 1 --2- 172.21.15.1:0/1723331458 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55b2efda6370 0x55b2efda77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:14.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.972+0000 7ff95ac50740 4 client.24628 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:27:14.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.972+0000 7ff95ac50740 1 -- 172.21.15.1:0/1723331458 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2efcfd620 con 0x55b2efda6370 2024-09-12T01:27:14.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.973+0000 7ff954e00640 1 --2- 172.21.15.1:0/1723331458 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55b2efda6370 0x55b2efda77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:14.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.973+0000 7ff954e00640 1 --2- 172.21.15.1:0/1723331458 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55b2efda6370 0x55b2efda77e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebfa4723c8c37ef3 server_cookie=28cc50da586f9a29 in_seq=0 out_seq=0 2024-09-12T01:27:14.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.973+0000 7ff937400640 10 client.24628 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:27:14.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.974+0000 7ff937400640 1 -- 172.21.15.1:0/1723331458 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b2efcfd620 con 0x55b2efda6370 2024-09-12T01:27:14.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.974+0000 7ff937400640 10 client.24628 handle_command_reply: tid=0 2024-09-12T01:27:14.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.974+0000 7ff95ac50740 1 client.24628 shutdown 2024-09-12T01:27:14.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.974+0000 7ff95ac50740 1 -- 172.21.15.1:0/1723331458 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2efd8a4e0 msgr2=0x55b2efd92580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:14.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.974+0000 7ff95ac50740 1 --2- 172.21.15.1:0/1723331458 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2efd8a4e0 0x55b2efd92580 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff94401b040 tx=0x7ff944005200 comp rx=0 tx=0).stop 2024-09-12T01:27:14.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.974+0000 7ff95ac50740 1 -- 172.21.15.1:0/1723331458 shutdown_connections 2024-09-12T01:27:14.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.974+0000 7ff95ac50740 1 --2- 172.21.15.1:0/1723331458 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55b2efda6370 0x55b2efda77e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:14.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.974+0000 7ff95ac50740 1 --2- 172.21.15.1:0/1723331458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2efd956c0 0x55b2efd93000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:14.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.974+0000 7ff95ac50740 1 --2- 172.21.15.1:0/1723331458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2efd8ae00 0x55b2efd92ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:14.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.974+0000 7ff95ac50740 1 --2- 172.21.15.1:0/1723331458 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2efd8a4e0 0x55b2efd92580 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:14.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.974+0000 7ff95ac50740 1 -- 172.21.15.1:0/1723331458 >> 172.21.15.1:0/1723331458 conn(0x55b2efd86a70 msgr2=0x55b2efd8c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:14.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.974+0000 7ff937400640 0 client.24628 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:27:14.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.974+0000 7ff95ac50740 1 -- 172.21.15.1:0/1723331458 shutdown_connections 2024-09-12T01:27:14.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.974+0000 7ff95ac50740 1 -- 172.21.15.1:0/1723331458 wait complete. 2024-09-12T01:27:14.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.974+0000 7ff95ac50740 20 client.24628 trim_cache size 0 max 16384 2024-09-12T01:27:14.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:14 smithi053 ceph-mon[35158]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:27:14.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.993+0000 7ff95ac50740 1 Processor -- start 2024-09-12T01:27:14.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.993+0000 7ff95ac50740 1 -- start start 2024-09-12T01:27:14.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.993+0000 7ff95ac50740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2efd8a4e0 0x55b2efd8cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:14.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.993+0000 7ff95ac50740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2efd8ae00 0x55b2efd92ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:14.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.993+0000 7ff95ac50740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2efd956c0 0x55b2efd93010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:14.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.993+0000 7ff95ac50740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b2efda5b20 con 0x55b2efd8ae00 2024-09-12T01:27:14.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.993+0000 7ff95ac50740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b2efda61f0 con 0x55b2efd956c0 2024-09-12T01:27:14.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.993+0000 7ff95ac50740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b2efda68c0 con 0x55b2efd8a4e0 2024-09-12T01:27:14.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.994+0000 7ff954e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2efd8ae00 0x55b2efd92ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:14.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.994+0000 7ff954e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2efd8ae00 0x55b2efd92ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57068/0 (socket says 172.21.15.1:57068) 2024-09-12T01:27:14.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.994+0000 7ff954e00640 1 -- 172.21.15.1:0/2227572850 learned_addr learned my addr 172.21.15.1:0/2227572850 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:27:14.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.994+0000 7ff956200640 1 --2- 172.21.15.1:0/2227572850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2efd956c0 0x55b2efd93010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:14.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.994+0000 7ff955800640 1 --2- 172.21.15.1:0/2227572850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2efd8a4e0 0x55b2efd8cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:14.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.994+0000 7ff954e00640 1 -- 172.21.15.1:0/2227572850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2efd8a4e0 msgr2=0x55b2efd8cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:27:14.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.994+0000 7ff954e00640 1 --2- 172.21.15.1:0/2227572850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2efd8a4e0 0x55b2efd8cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:14.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.994+0000 7ff954e00640 1 -- 172.21.15.1:0/2227572850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2efd956c0 msgr2=0x55b2efd93010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:27:14.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.994+0000 7ff954e00640 1 --2- 172.21.15.1:0/2227572850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2efd956c0 0x55b2efd93010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:14.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.994+0000 7ff954e00640 1 -- 172.21.15.1:0/2227572850 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2efd93a10 con 0x55b2efd8ae00 2024-09-12T01:27:14.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.994+0000 7ff954e00640 1 --2- 172.21.15.1:0/2227572850 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2efd8ae00 0x55b2efd92ad0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff93c0bae80 tx=0x7ff93c0f1010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cb273aa5cc776f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:14.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.995+0000 7ff942000640 1 -- 172.21.15.1:0/2227572850 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c049040 con 0x55b2efd8ae00 2024-09-12T01:27:14.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.995+0000 7ff942000640 1 -- 172.21.15.1:0/2227572850 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff93c0bb880 con 0x55b2efd8ae00 2024-09-12T01:27:14.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.995+0000 7ff942000640 1 -- 172.21.15.1:0/2227572850 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c0ec530 con 0x55b2efd8ae00 2024-09-12T01:27:14.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.995+0000 7ff95ac50740 1 -- 172.21.15.1:0/2227572850 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2efd8ae00 msgr2=0x55b2efd92ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:14.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.995+0000 7ff95ac50740 1 --2- 172.21.15.1:0/2227572850 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2efd8ae00 0x55b2efd92ad0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff93c0bae80 tx=0x7ff93c0f1010 comp rx=0 tx=0).stop 2024-09-12T01:27:14.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.995+0000 7ff95ac50740 1 -- 172.21.15.1:0/2227572850 shutdown_connections 2024-09-12T01:27:14.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.995+0000 7ff95ac50740 1 --2- 172.21.15.1:0/2227572850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2efd956c0 0x55b2efd93010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:14.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.995+0000 7ff95ac50740 1 --2- 172.21.15.1:0/2227572850 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2efd8ae00 0x55b2efd92ad0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:14.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.995+0000 7ff95ac50740 1 --2- 172.21.15.1:0/2227572850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2efd8a4e0 0x55b2efd8cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:14.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.995+0000 7ff95ac50740 1 -- 172.21.15.1:0/2227572850 >> 172.21.15.1:0/2227572850 conn(0x55b2efd86a70 msgr2=0x55b2efd94600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:14.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.996+0000 7ff95ac50740 1 -- 172.21.15.1:0/2227572850 shutdown_connections 2024-09-12T01:27:14.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.996+0000 7ff95ac50740 1 -- 172.21.15.1:0/2227572850 wait complete. 2024-09-12T01:27:14.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.996+0000 7ff95ac50740 1 -- start start 2024-09-12T01:27:14.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.996+0000 7ff95ac50740 1 Processor -- start 2024-09-12T01:27:14.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.997+0000 7ff95ac50740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2efd8a4e0 0x55b2efdaf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:14.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.997+0000 7ff95ac50740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2efd8ae00 0x55b2efdaff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:14.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.997+0000 7ff95ac50740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2efd956c0 0x55b2efdaa970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:14.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.997+0000 7ff95ac50740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b2efda7660 con 0x55b2efd956c0 2024-09-12T01:27:14.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.997+0000 7ff95ac50740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b2efda6f90 con 0x55b2efd8ae00 2024-09-12T01:27:14.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.997+0000 7ff95ac50740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b2efda7d30 con 0x55b2efd8a4e0 2024-09-12T01:27:15.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.997+0000 7ff95ac50740 10 client.0 _ll_register_callbacks cb 0x7ff944009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:27:15.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.997+0000 7ff954e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2efd8ae00 0x55b2efdaff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:15.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.997+0000 7ff956200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2efd956c0 0x55b2efdaa970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:15.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.997+0000 7ff954e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2efd8ae00 0x55b2efdaff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:55646/0 (socket says 172.21.15.1:55646) 2024-09-12T01:27:15.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.997+0000 7ff954e00640 1 -- 172.21.15.1:0/1081314914 learned_addr learned my addr 172.21.15.1:0/1081314914 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:27:15.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.997+0000 7ff955800640 1 --2- 172.21.15.1:0/1081314914 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2efd8a4e0 0x55b2efdaf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:15.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.997+0000 7ff954e00640 1 -- 172.21.15.1:0/1081314914 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2efd8a4e0 msgr2=0x55b2efdaf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:27:15.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.997+0000 7ff954e00640 1 --2- 172.21.15.1:0/1081314914 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2efd8a4e0 0x55b2efdaf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:15.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.997+0000 7ff954e00640 1 -- 172.21.15.1:0/1081314914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2efd956c0 msgr2=0x55b2efdaa970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:27:15.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.997+0000 7ff954e00640 1 --2- 172.21.15.1:0/1081314914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2efd956c0 0x55b2efdaa970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:15.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.997+0000 7ff954e00640 1 -- 172.21.15.1:0/1081314914 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2efdaaeb0 con 0x55b2efd8ae00 2024-09-12T01:27:15.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.997+0000 7ff955800640 1 --2- 172.21.15.1:0/1081314914 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2efd8a4e0 0x55b2efdaf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:27:15.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.998+0000 7ff954e00640 1 --2- 172.21.15.1:0/1081314914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2efd8ae00 0x55b2efdaff00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff93c000c00 tx=0x7ff93c0eee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b87e15c9d9a2a30 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:15.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.998+0000 7ff937400640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-12T01:27:15.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.998+0000 7ff937400640 1 -- 172.21.15.1:0/1081314914 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c049030 con 0x55b2efd8ae00 2024-09-12T01:27:15.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.998+0000 7ff937400640 1 -- 172.21.15.1:0/1081314914 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8802+0+0 (secure 0 0 0) 0x7ff93c0ec530 con 0x55b2efd8ae00 2024-09-12T01:27:15.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.998+0000 7ff95ac50740 1 -- 172.21.15.1:0/1081314914 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff93c0bb880 con 0x55b2efd8ae00 2024-09-12T01:27:15.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.998+0000 7ff937400640 1 -- 172.21.15.1:0/1081314914 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7ff93c0bb180 con 0x55b2efd8ae00 2024-09-12T01:27:15.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.998+0000 7ff95ac50740 10 client.24644 fetch_fsmap learned FSMap version 17 2024-09-12T01:27:15.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.998+0000 7ff95ac50740 1 -- 172.21.15.1:0/1081314914 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55b2efe11110 con 0x55b2efd8ae00 2024-09-12T01:27:15.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.998+0000 7ff937400640 1 -- 172.21.15.1:0/1081314914 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff93c0bb880 con 0x55b2efd8ae00 2024-09-12T01:27:15.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.998+0000 7ff937400640 1 -- 172.21.15.1:0/1081314914 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7ff93c03a030 con 0x55b2efd8ae00 2024-09-12T01:27:15.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.999+0000 7ff95ac50740 10 client.24644 fetch_fsmap finished waiting for FSMap version 17 2024-09-12T01:27:15.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.999+0000 7ff95ac50740 10 client.24644 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T01:27:15.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.999+0000 7ff95ac50740 20 client.24644 populate_metadata read hostname 'smithi001' 2024-09-12T01:27:15.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.999+0000 7ff95ac50740 1 --2- 172.21.15.1:0/1081314914 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55b2efe159e0 0x55b2efe146f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:15.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.999+0000 7ff95ac50740 4 client.24644 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T01:27:15.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.999+0000 7ff95ac50740 1 -- 172.21.15.1:0/1081314914 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "status"}) -- 0x55b2efcf7b30 con 0x55b2efe159e0 2024-09-12T01:27:15.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.999+0000 7ff955800640 1 --2- 172.21.15.1:0/1081314914 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55b2efe159e0 0x55b2efe146f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:15.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.999+0000 7ff955800640 1 --2- 172.21.15.1:0/1081314914 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55b2efe159e0 0x55b2efe146f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14a0c60fbbd3c770 server_cookie=8134ffdaf84114bc in_seq=0 out_seq=0 2024-09-12T01:27:15.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:14.999+0000 7ff937400640 10 client.24644 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:27:15.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:15.000+0000 7ff937400640 1 -- 172.21.15.1:0/1081314914 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55b2efcf7b30 con 0x55b2efe159e0 2024-09-12T01:27:15.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:15.000+0000 7ff937400640 10 client.24644 handle_command_reply: tid=0 2024-09-12T01:27:15.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:15.000+0000 7ff95ac50740 1 client.24644 shutdown 2024-09-12T01:27:15.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:15.000+0000 7ff95ac50740 1 -- 172.21.15.1:0/1081314914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2efd8ae00 msgr2=0x55b2efdaff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:15.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:15.000+0000 7ff95ac50740 1 --2- 172.21.15.1:0/1081314914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2efd8ae00 0x55b2efdaff00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff93c000c00 tx=0x7ff93c0eee40 comp rx=0 tx=0).stop 2024-09-12T01:27:15.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:15.000+0000 7ff95ac50740 1 -- 172.21.15.1:0/1081314914 shutdown_connections 2024-09-12T01:27:15.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:15.000+0000 7ff95ac50740 1 --2- 172.21.15.1:0/1081314914 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55b2efe159e0 0x55b2efe146f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:15.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:15.000+0000 7ff95ac50740 1 --2- 172.21.15.1:0/1081314914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b2efd956c0 0x55b2efdaa970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:15.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:15.000+0000 7ff95ac50740 1 --2- 172.21.15.1:0/1081314914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2efd8ae00 0x55b2efdaff00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:15.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:15.000+0000 7ff95ac50740 1 --2- 172.21.15.1:0/1081314914 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2efd8a4e0 0x55b2efdaf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:15.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:15.000+0000 7ff95ac50740 1 -- 172.21.15.1:0/1081314914 >> 172.21.15.1:0/1081314914 conn(0x55b2efd86a70 msgr2=0x55b2efda0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:15.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:15.000+0000 7ff937400640 0 client.24644 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:27:15.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:15.000+0000 7ff95ac50740 1 -- 172.21.15.1:0/1081314914 shutdown_connections 2024-09-12T01:27:15.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:15.000+0000 7ff95ac50740 1 -- 172.21.15.1:0/1081314914 wait complete. 2024-09-12T01:27:15.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:15.000+0000 7ff95ac50740 20 client.24644 trim_cache size 0 max 16384 2024-09-12T01:27:15.007 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-12T01:27:15.007 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "c0e4f180-70a4-11ef-bcea-c7b262605968", 2024-09-12T01:27:15.007 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 0, 2024-09-12T01:27:15.007 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24545, 2024-09-12T01:27:15.007 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-09-12T01:27:15.007 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-09-12T01:27:15.007 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-09-12T01:27:15.007 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 25.528916885000001, 2024-09-12T01:27:15.007 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 17, 2024-09-12T01:27:15.007 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 94, 2024-09-12T01:27:15.008 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 93, 2024-09-12T01:27:15.008 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 40.133498271000001 2024-09-12T01:27:15.008 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-12T01:27:15.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:15.001+0000 7ff956c00640 1 -- 172.21.15.1:0/3825037703 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff9380a2c00 msgr2=0x7ff9380a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:15.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:15.001+0000 7ff956c00640 1 --2- 172.21.15.1:0/3825037703 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff9380a2c00 0x7ff9380a50c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff948005fd0 tx=0x7ff948009500 comp rx=0 tx=0).stop 2024-09-12T01:27:15.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:15.001+0000 7ff956c00640 1 -- 172.21.15.1:0/3825037703 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9501047e0 msgr2=0x7ff95019a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:15.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:15.001+0000 7ff956c00640 1 --2- 172.21.15.1:0/3825037703 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9501047e0 0x7ff95019a990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff93c035040 tx=0x7ff93c033d90 comp rx=0 tx=0).stop 2024-09-12T01:27:15.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:15.001+0000 7ff956c00640 1 -- 172.21.15.1:0/3825037703 shutdown_connections 2024-09-12T01:27:15.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:15.001+0000 7ff956c00640 1 --2- 172.21.15.1:0/3825037703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff950108f60 0x7ff95019ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:15.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:15.001+0000 7ff956c00640 1 --2- 172.21.15.1:0/3825037703 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9501047e0 0x7ff95019a990 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:15.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:15.001+0000 7ff956c00640 1 --2- 172.21.15.1:0/3825037703 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff9380a2c00 0x7ff9380a50c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:15.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:15.001+0000 7ff956c00640 1 --2- 172.21.15.1:0/3825037703 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff950103e30 0x7ff95019a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:15.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:15.001+0000 7ff956c00640 1 -- 172.21.15.1:0/3825037703 >> 172.21.15.1:0/3825037703 conn(0x7ff9500ff8d0 msgr2=0x7ff950100c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:15.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:15.001+0000 7ff956c00640 1 -- 172.21.15.1:0/3825037703 shutdown_connections 2024-09-12T01:27:15.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:15.001+0000 7ff956c00640 1 -- 172.21.15.1:0/3825037703 wait complete. 2024-09-12T01:27:15.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:14 smithi104 ceph-mon[35219]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:27:15.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:15 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-09-12T01:27:15.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:15 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-09-12T01:27:15.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:15 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-09-12T01:27:15.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:15 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-09-12T01:27:15.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:15 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-09-12T01:27:15.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:15 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-09-12T01:27:15.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:15 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-09-12T01:27:15.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:15 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-09-12T01:27:15.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:15 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-09-12T01:27:15.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:15 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-09-12T01:27:15.877 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-12T01:27:15.877 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-12T01:27:15.878 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-12T01:27:15.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:15 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-09-12T01:27:15.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:15 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-09-12T01:27:15.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:15 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-09-12T01:27:15.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:15 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-09-12T01:27:15.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:15 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-09-12T01:27:15.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:15 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-09-12T01:27:15.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:15 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-09-12T01:27:15.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:15 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-09-12T01:27:15.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:15 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-09-12T01:27:15.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:15 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-09-12T01:27:16.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:15 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-09-12T01:27:16.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:15 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-09-12T01:27:16.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:15 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-09-12T01:27:16.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:15 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-09-12T01:27:16.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:15 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-09-12T01:27:16.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:15 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-09-12T01:27:16.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:15 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-09-12T01:27:16.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:15 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-09-12T01:27:16.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:15 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-09-12T01:27:16.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:15 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-09-12T01:27:16.258 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:27:16.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:16 smithi001 ceph-mon[30387]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:27:16.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:16 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-09-12T01:27:16.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:16 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-09-12T01:27:16.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:16 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-12T01:27:16.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:16 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-09-12T01:27:16.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:16 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-09-12T01:27:16.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:16 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-12T01:27:16.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:16 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-09-12T01:27:16.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:16 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-09-12T01:27:16.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:16 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-09-12T01:27:16.679 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:16 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-09-12T01:27:16.679 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:16 smithi001 ceph-mon[30387]: osdmap e95: 12 total, 12 up, 12 in 2024-09-12T01:27:16.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:16.851+0000 7ff13ba00640 1 -- 172.21.15.1:0/123597183 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff134112a30 msgr2=0x7ff134114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:16.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:16.851+0000 7ff13ba00640 1 --2- 172.21.15.1:0/123597183 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff134112a30 0x7ff134114e20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff12c009ca0 tx=0x7ff12c02a8c0 comp rx=0 tx=0).stop 2024-09-12T01:27:16.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:16.852+0000 7ff13ba00640 1 -- 172.21.15.1:0/123597183 shutdown_connections 2024-09-12T01:27:16.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:16.852+0000 7ff13ba00640 1 --2- 172.21.15.1:0/123597183 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff134112a30 0x7ff134114e20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:16.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:16.852+0000 7ff13ba00640 1 --2- 172.21.15.1:0/123597183 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff134077040 0x7ff1341124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:16.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:16.852+0000 7ff13ba00640 1 --2- 172.21.15.1:0/123597183 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff134076720 0x7ff134076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:16.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:16.852+0000 7ff13ba00640 1 -- 172.21.15.1:0/123597183 >> 172.21.15.1:0/123597183 conn(0x7ff1340ffa90 msgr2=0x7ff134101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:16.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:16.852+0000 7ff13ba00640 1 -- 172.21.15.1:0/123597183 shutdown_connections 2024-09-12T01:27:16.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:16.852+0000 7ff13ba00640 1 -- 172.21.15.1:0/123597183 wait complete. 2024-09-12T01:27:16.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:16.853+0000 7ff13ba00640 1 Processor -- start 2024-09-12T01:27:16.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:16.853+0000 7ff13ba00640 1 -- start start 2024-09-12T01:27:16.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:16.853+0000 7ff13ba00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff134076720 0x7ff1341a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:16.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:16.853+0000 7ff13ba00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff134077040 0x7ff1341a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:16.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:16.853+0000 7ff13ba00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff134112a30 0x7ff13419e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:16.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:16.853+0000 7ff13ba00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff1341185c0 con 0x7ff134076720 2024-09-12T01:27:16.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:16.853+0000 7ff13ba00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff134118440 con 0x7ff134112a30 2024-09-12T01:27:16.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:16.853+0000 7ff13ba00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff134118740 con 0x7ff134077040 2024-09-12T01:27:16.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:16.853+0000 7ff139000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff134076720 0x7ff1341a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:16.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:16.853+0000 7ff139000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff134076720 0x7ff1341a46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57096/0 (socket says 172.21.15.1:57096) 2024-09-12T01:27:16.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:16.853+0000 7ff139000640 1 -- 172.21.15.1:0/1132815156 learned_addr learned my addr 172.21.15.1:0/1132815156 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:27:16.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:16.853+0000 7ff139a00640 1 --2- 172.21.15.1:0/1132815156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff134112a30 0x7ff13419e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:16.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:16.853+0000 7ff133e00640 1 --2- 172.21.15.1:0/1132815156 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff134077040 0x7ff1341a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:16.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:16.854+0000 7ff139000640 1 -- 172.21.15.1:0/1132815156 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff134077040 msgr2=0x7ff1341a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:16.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:16.854+0000 7ff139000640 1 --2- 172.21.15.1:0/1132815156 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff134077040 0x7ff1341a4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:16.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:16.854+0000 7ff139000640 1 -- 172.21.15.1:0/1132815156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff134112a30 msgr2=0x7ff13419e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:16.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:16.854+0000 7ff139000640 1 --2- 172.21.15.1:0/1132815156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff134112a30 0x7ff13419e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:16.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:16.854+0000 7ff139000640 1 -- 172.21.15.1:0/1132815156 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff13419efd0 con 0x7ff134076720 2024-09-12T01:27:16.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:16.854+0000 7ff139000640 1 --2- 172.21.15.1:0/1132815156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff134076720 0x7ff1341a46b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff12800e9f0 tx=0x7ff12800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:16.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:16.854+0000 7ff131600640 1 -- 172.21.15.1:0/1132815156 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12800cd80 con 0x7ff134076720 2024-09-12T01:27:16.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:16.854+0000 7ff131600640 1 -- 172.21.15.1:0/1132815156 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff128010910 con 0x7ff134076720 2024-09-12T01:27:16.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:16.854+0000 7ff13ba00640 1 -- 172.21.15.1:0/1132815156 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff13419f2c0 con 0x7ff134076720 2024-09-12T01:27:16.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:16.854+0000 7ff131600640 1 -- 172.21.15.1:0/1132815156 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff128018990 con 0x7ff134076720 2024-09-12T01:27:16.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:16.855+0000 7ff13ba00640 1 -- 172.21.15.1:0/1132815156 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1341ab510 con 0x7ff134076720 2024-09-12T01:27:16.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:16.856+0000 7ff13ba00640 1 -- 172.21.15.1:0/1132815156 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff134077e60 con 0x7ff134076720 2024-09-12T01:27:16.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:16.858+0000 7ff131600640 1 -- 172.21.15.1:0/1132815156 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7ff128010ac0 con 0x7ff134076720 2024-09-12T01:27:16.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:16.859+0000 7ff131600640 1 --2- 172.21.15.1:0/1132815156 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff11c0a2c00 0x7ff11c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:16.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:16.859+0000 7ff131600640 1 -- 172.21.15.1:0/1132815156 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9092+0+0 (secure 0 0 0) 0x7ff1280cf670 con 0x7ff134076720 2024-09-12T01:27:16.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:16.859+0000 7ff133e00640 1 --2- 172.21.15.1:0/1132815156 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff11c0a2c00 0x7ff11c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:16.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:16.860+0000 7ff133e00640 1 --2- 172.21.15.1:0/1132815156 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff11c0a2c00 0x7ff11c0a50c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff134079220 tx=0x7ff12000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:16.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:16.863+0000 7ff131600640 1 -- 172.21.15.1:0/1132815156 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff12809bbc0 con 0x7ff134076720 2024-09-12T01:27:16.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:16 smithi053 ceph-mon[35158]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:27:16.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:16 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-09-12T01:27:16.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:16 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-09-12T01:27:16.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:16 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-12T01:27:16.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:16 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-09-12T01:27:16.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:16 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-09-12T01:27:16.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:16 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-12T01:27:16.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:16 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-09-12T01:27:16.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:16 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-09-12T01:27:16.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:16 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-09-12T01:27:16.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:16 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-09-12T01:27:16.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:16 smithi053 ceph-mon[35158]: osdmap e95: 12 total, 12 up, 12 in 2024-09-12T01:27:17.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:16 smithi104 ceph-mon[35219]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:27:17.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:16 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-09-12T01:27:17.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:16 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-09-12T01:27:17.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:16 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-12T01:27:17.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:16 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-09-12T01:27:17.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:16 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-09-12T01:27:17.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:16 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-12T01:27:17.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:16 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-09-12T01:27:17.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:16 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-09-12T01:27:17.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:16 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-09-12T01:27:17.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:16 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-09-12T01:27:17.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:16 smithi104 ceph-mon[35219]: osdmap e95: 12 total, 12 up, 12 in 2024-09-12T01:27:17.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:17.084+0000 7ff13ba00640 1 -- 172.21.15.1:0/1132815156 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff1341ab810 con 0x7ff134076720 2024-09-12T01:27:17.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:17.085+0000 7ff131600640 1 -- 172.21.15.1:0/1132815156 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7ff12809fec0 con 0x7ff134076720 2024-09-12T01:27:17.085 INFO:teuthology.orchestra.run.smithi001.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-12T01:27:17.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:17.088+0000 7ff13ba00640 1 -- 172.21.15.1:0/1132815156 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff11c0a2c00 msgr2=0x7ff11c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:17.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:17.088+0000 7ff13ba00640 1 --2- 172.21.15.1:0/1132815156 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff11c0a2c00 0x7ff11c0a50c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff134079220 tx=0x7ff12000b410 comp rx=0 tx=0).stop 2024-09-12T01:27:17.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:17.088+0000 7ff13ba00640 1 -- 172.21.15.1:0/1132815156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff134076720 msgr2=0x7ff1341a46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:17.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:17.088+0000 7ff13ba00640 1 --2- 172.21.15.1:0/1132815156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff134076720 0x7ff1341a46b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff12800e9f0 tx=0x7ff12800eeb0 comp rx=0 tx=0).stop 2024-09-12T01:27:17.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:17.088+0000 7ff13ba00640 1 -- 172.21.15.1:0/1132815156 shutdown_connections 2024-09-12T01:27:17.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:17.088+0000 7ff13ba00640 1 --2- 172.21.15.1:0/1132815156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff134112a30 0x7ff13419e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:17.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:17.088+0000 7ff13ba00640 1 --2- 172.21.15.1:0/1132815156 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff11c0a2c00 0x7ff11c0a50c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:17.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:17.088+0000 7ff13ba00640 1 --2- 172.21.15.1:0/1132815156 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff134077040 0x7ff1341a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:17.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:17.088+0000 7ff13ba00640 1 --2- 172.21.15.1:0/1132815156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff134076720 0x7ff1341a46b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:17.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:17.088+0000 7ff13ba00640 1 -- 172.21.15.1:0/1132815156 >> 172.21.15.1:0/1132815156 conn(0x7ff1340ffa90 msgr2=0x7ff134101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:17.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:17.088+0000 7ff13ba00640 1 -- 172.21.15.1:0/1132815156 shutdown_connections 2024-09-12T01:27:17.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:17.088+0000 7ff13ba00640 1 -- 172.21.15.1:0/1132815156 wait complete. 2024-09-12T01:27:17.783 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:17 smithi001 ceph-mon[30387]: osdmap e96: 12 total, 12 up, 12 in 2024-09-12T01:27:17.784 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:17 smithi001 ceph-mon[30387]: pgmap v327: 129 pgs: 7 peering, 122 active+clean; 579 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:27:17.784 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:17 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/1132815156' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T01:27:17.819 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-12T01:27:17.820 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T01:27:17.820 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi001.front.sepia.ceph.com 2024-09-12T01:27:17.820 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-12T01:27:17.820 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-12T01:27:17.820 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-12T01:27:17.820 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T01:27:17.820 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T01:27:17.820 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-12T01:27:17.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:27:17.820 DEBUG:teuthology.orchestra.run.smithi001:> ip addr 2024-09-12T01:27:17.856 INFO:teuthology.orchestra.run.smithi001.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T01:27:17.856 INFO:teuthology.orchestra.run.smithi001.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T01:27:17.857 INFO:teuthology.orchestra.run.smithi001.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T01:27:17.857 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:27:17.857 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 ::1/128 scope host 2024-09-12T01:27:17.857 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:27:17.857 INFO:teuthology.orchestra.run.smithi001.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T01:27:17.857 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7a brd ff:ff:ff:ff:ff:ff 2024-09-12T01:27:17.857 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f0 2024-09-12T01:27:17.857 INFO:teuthology.orchestra.run.smithi001.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T01:27:17.857 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7b brd ff:ff:ff:ff:ff:ff 2024-09-12T01:27:17.857 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f1 2024-09-12T01:27:17.857 INFO:teuthology.orchestra.run.smithi001.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T01:27:17.857 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e8 brd ff:ff:ff:ff:ff:ff 2024-09-12T01:27:17.857 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f0 2024-09-12T01:27:17.857 INFO:teuthology.orchestra.run.smithi001.stdout: inet 172.21.15.1/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T01:27:17.858 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft 42142sec preferred_lft 42142sec 2024-09-12T01:27:17.858 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::ec4:7aff:febd:15e8/64 scope link 2024-09-12T01:27:17.858 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:27:17.858 INFO:teuthology.orchestra.run.smithi001.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T01:27:17.858 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e9 brd ff:ff:ff:ff:ff:ff 2024-09-12T01:27:17.858 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f1 2024-09-12T01:27:17.859 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-12T01:27:17.859 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T01:27:17.859 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-09-12T01:27:17.859 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link add name ceph-brx type bridge 2024-09-12T01:27:17.860 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip addr flush dev ceph-brx 2024-09-12T01:27:17.860 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set ceph-brx up 2024-09-12T01:27:17.860 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-12T01:27:17.860 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-09-12T01:27:17.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T01:27:17.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:17 smithi053 ceph-mon[35158]: osdmap e96: 12 total, 12 up, 12 in 2024-09-12T01:27:17.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:17 smithi053 ceph-mon[35158]: pgmap v327: 129 pgs: 7 peering, 122 active+clean; 579 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:27:17.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:17 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/1132815156' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T01:27:18.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:17 smithi104 ceph-mon[35219]: osdmap e96: 12 total, 12 up, 12 in 2024-09-12T01:27:18.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:17 smithi104 ceph-mon[35219]: pgmap v327: 129 pgs: 7 peering, 122 active+clean; 579 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:27:18.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:17 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/1132815156' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T01:27:18.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:27:18.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:27:18.277 DEBUG:teuthology.orchestra.run.smithi001:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T01:27:18.333 INFO:teuthology.orchestra.run.smithi001.stdout:1 2024-09-12T01:27:18.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:27:18.337 DEBUG:teuthology.orchestra.run.smithi001:> route 2024-09-12T01:27:18.386 INFO:teuthology.orchestra.run.smithi001.stdout:Kernel IP routing table 2024-09-12T01:27:18.386 INFO:teuthology.orchestra.run.smithi001.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T01:27:18.387 INFO:teuthology.orchestra.run.smithi001.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T01:27:18.387 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T01:27:18.387 INFO:teuthology.orchestra.run.smithi001.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T01:27:18.387 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T01:27:18.387 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-09-12T01:27:18.387 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T01:27:18.387 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T01:27:18.387 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T01:27:18.388 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-09-12T01:27:18.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T01:27:19.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:27:19.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:27:19.037 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-09-12T01:27:19.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:27:19.074 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list-id 2024-09-12T01:27:19.150 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T01:27:19.150 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-09-12T01:27:19.150 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-12T01:27:19.150 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-12T01:27:19.150 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-09-12T01:27:19.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T01:27:19.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:27:19.330 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-12T01:27:19.331 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T01:27:19.331 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-09-12T01:27:19.331 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-12T01:27:19.331 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-12T01:27:19.331 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-12T01:27:19.331 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-12T01:27:19.331 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-12T01:27:19.331 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-09-12T01:27:19.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T01:27:19.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:27:19.582 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T01:27:19.582 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-09-12T01:27:19.582 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set brx.0 up 2024-09-12T01:27:19.582 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set dev brx.0 master ceph-brx 2024-09-12T01:27:19.582 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-09-12T01:27:19.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T01:27:19.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:27:19.750 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-12T01:27:19.750 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T01:27:19.750 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-12T01:27:19.786 INFO:teuthology.orchestra.run.smithi001.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-12T01:27:19.786 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T01:27:19.786 DEBUG:teuthology.orchestra.run.smithi001:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-12T01:27:19.861 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-09-12T01:27:19.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:19 smithi001 ceph-mon[30387]: pgmap v328: 129 pgs: 7 peering, 122 active+clean; 579 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-09-12T01:27:19.959 DEBUG:teuthology.orchestra.run.smithi001:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T01:27:19.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:19 smithi053 ceph-mon[35158]: pgmap v328: 129 pgs: 7 peering, 122 active+clean; 579 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-09-12T01:27:20.036 INFO:teuthology.orchestra.run.smithi001.stdout:/proc 2024-09-12T01:27:20.036 INFO:teuthology.orchestra.run.smithi001.stdout:/sys 2024-09-12T01:27:20.036 INFO:teuthology.orchestra.run.smithi001.stdout:/dev 2024-09-12T01:27:20.036 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/security 2024-09-12T01:27:20.036 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/shm 2024-09-12T01:27:20.036 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/pts 2024-09-12T01:27:20.037 INFO:teuthology.orchestra.run.smithi001.stdout:/run 2024-09-12T01:27:20.037 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/cgroup 2024-09-12T01:27:20.037 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/pstore 2024-09-12T01:27:20.037 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/bpf 2024-09-12T01:27:20.037 INFO:teuthology.orchestra.run.smithi001.stdout:/ 2024-09-12T01:27:20.037 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/selinux 2024-09-12T01:27:20.037 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T01:27:20.037 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/mqueue 2024-09-12T01:27:20.037 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/hugepages 2024-09-12T01:27:20.037 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/debug 2024-09-12T01:27:20.037 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/tracing 2024-09-12T01:27:20.037 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/fuse/connections 2024-09-12T01:27:20.037 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/config 2024-09-12T01:27:20.037 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-sysctl.service 2024-09-12T01:27:20.038 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-12T01:27:20.038 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-12T01:27:20.038 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/nfs/rpc_pipefs 2024-09-12T01:27:20.038 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/1000 2024-09-12T01:27:20.038 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph 2024-09-12T01:27:20.038 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay 2024-09-12T01:27:20.038 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/2e7727155d5da690420ee8e510883fea96911216c4604e231769998d1b99c05c/merged 2024-09-12T01:27:20.038 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/0e1f475017ce100f0ffd13258b2100850c1fd9abc4cb152fa48a70e7ac9d02c4/merged 2024-09-12T01:27:20.038 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/0 2024-09-12T01:27:20.038 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T01:27:20.038 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/6b760cd0558d91a86517269c07517d65e7f0165a8164c4c86bdc138935ccb500/merged 2024-09-12T01:27:20.038 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/e47cc1f328c85580bd63fff37e56620a7ee7a56db2e40bdaac9c89544e04d822/merged 2024-09-12T01:27:20.038 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/3d157f8de24f906ad6e38ca08e1879b9533652b8f4884b81a675fa2a63718ee5/merged 2024-09-12T01:27:20.039 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/1a664b3cd04309b2e6aeea512a88e029c90200745758709be7800320af2e32f9/merged 2024-09-12T01:27:20.039 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/ce25d328bc0cf8c83ad266038774ccd835e7e42cb2a715d40f3b0040c8fde8d3/merged 2024-09-12T01:27:20.039 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/fc4208d9dfbda187c4e5a810a61e21cebbf923cc80070249a14b63876ae736d5/merged 2024-09-12T01:27:20.039 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/85a1df038549a2bcbcb7cc38a360b971c8daf45035218a8135c7eedb37725f1a/merged 2024-09-12T01:27:20.039 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/cd9cb360987d7fd5e8cd542f602d5734d957a105663acd643126fd838ca231d6/merged 2024-09-12T01:27:20.039 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns 2024-09-12T01:27:20.039 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-12T01:27:20.039 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-12T01:27:20.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:27:20.040 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-09-12T01:27:20.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:19 smithi104 ceph-mon[35219]: pgmap v328: 129 pgs: 7 peering, 122 active+clean; 579 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-09-12T01:27:20.115 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-12T01:27:20.116 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-12T01:27:20.160 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-09-12T01:27:20.219 DEBUG:teuthology.orchestra.run.smithi001:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T01:27:20.255 INFO:teuthology.orchestra.run.smithi001.stdout:/proc 2024-09-12T01:27:20.255 INFO:teuthology.orchestra.run.smithi001.stdout:/sys 2024-09-12T01:27:20.255 INFO:teuthology.orchestra.run.smithi001.stdout:/dev 2024-09-12T01:27:20.256 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/security 2024-09-12T01:27:20.256 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/shm 2024-09-12T01:27:20.256 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/pts 2024-09-12T01:27:20.256 INFO:teuthology.orchestra.run.smithi001.stdout:/run 2024-09-12T01:27:20.256 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/cgroup 2024-09-12T01:27:20.256 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/pstore 2024-09-12T01:27:20.256 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/bpf 2024-09-12T01:27:20.256 INFO:teuthology.orchestra.run.smithi001.stdout:/ 2024-09-12T01:27:20.256 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/selinux 2024-09-12T01:27:20.256 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T01:27:20.256 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/mqueue 2024-09-12T01:27:20.256 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/hugepages 2024-09-12T01:27:20.256 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/debug 2024-09-12T01:27:20.257 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/tracing 2024-09-12T01:27:20.257 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/fuse/connections 2024-09-12T01:27:20.257 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/config 2024-09-12T01:27:20.257 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-sysctl.service 2024-09-12T01:27:20.257 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-12T01:27:20.257 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-12T01:27:20.257 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/nfs/rpc_pipefs 2024-09-12T01:27:20.257 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/1000 2024-09-12T01:27:20.257 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph 2024-09-12T01:27:20.257 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay 2024-09-12T01:27:20.257 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/2e7727155d5da690420ee8e510883fea96911216c4604e231769998d1b99c05c/merged 2024-09-12T01:27:20.257 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/0e1f475017ce100f0ffd13258b2100850c1fd9abc4cb152fa48a70e7ac9d02c4/merged 2024-09-12T01:27:20.257 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/0 2024-09-12T01:27:20.257 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T01:27:20.257 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/6b760cd0558d91a86517269c07517d65e7f0165a8164c4c86bdc138935ccb500/merged 2024-09-12T01:27:20.258 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/e47cc1f328c85580bd63fff37e56620a7ee7a56db2e40bdaac9c89544e04d822/merged 2024-09-12T01:27:20.258 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/3d157f8de24f906ad6e38ca08e1879b9533652b8f4884b81a675fa2a63718ee5/merged 2024-09-12T01:27:20.258 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/1a664b3cd04309b2e6aeea512a88e029c90200745758709be7800320af2e32f9/merged 2024-09-12T01:27:20.258 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/ce25d328bc0cf8c83ad266038774ccd835e7e42cb2a715d40f3b0040c8fde8d3/merged 2024-09-12T01:27:20.258 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/fc4208d9dfbda187c4e5a810a61e21cebbf923cc80070249a14b63876ae736d5/merged 2024-09-12T01:27:20.258 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/85a1df038549a2bcbcb7cc38a360b971c8daf45035218a8135c7eedb37725f1a/merged 2024-09-12T01:27:20.258 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/cd9cb360987d7fd5e8cd542f602d5734d957a105663acd643126fd838ca231d6/merged 2024-09-12T01:27:20.258 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns 2024-09-12T01:27:20.258 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-12T01:27:20.258 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-12T01:27:20.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:27:20.259 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-09-12T01:27:20.335 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi001.stderr:2024-09-12T01:27:20.336+0000 7f1ca3caf480 -1 init, newargv = 0x5651b33a73c0 newargc=15 2024-09-12T01:27:20.336 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi001.stderr:ceph-fuse[74898]: starting ceph client 2024-09-12T01:27:20.346 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi001.stderr:ceph-fuse[74898]: starting fuse 2024-09-12T01:27:21.437 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-09-12T01:27:21.502 DEBUG:teuthology.orchestra.run.smithi001:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T01:27:21.543 INFO:teuthology.orchestra.run.smithi001.stdout:/proc 2024-09-12T01:27:21.543 INFO:teuthology.orchestra.run.smithi001.stdout:/sys 2024-09-12T01:27:21.543 INFO:teuthology.orchestra.run.smithi001.stdout:/dev 2024-09-12T01:27:21.543 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/security 2024-09-12T01:27:21.543 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/shm 2024-09-12T01:27:21.543 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/pts 2024-09-12T01:27:21.543 INFO:teuthology.orchestra.run.smithi001.stdout:/run 2024-09-12T01:27:21.543 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/cgroup 2024-09-12T01:27:21.543 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/pstore 2024-09-12T01:27:21.543 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/bpf 2024-09-12T01:27:21.544 INFO:teuthology.orchestra.run.smithi001.stdout:/ 2024-09-12T01:27:21.544 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/selinux 2024-09-12T01:27:21.544 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T01:27:21.544 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/mqueue 2024-09-12T01:27:21.544 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/hugepages 2024-09-12T01:27:21.544 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/debug 2024-09-12T01:27:21.544 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/tracing 2024-09-12T01:27:21.544 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/fuse/connections 2024-09-12T01:27:21.544 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/config 2024-09-12T01:27:21.544 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-sysctl.service 2024-09-12T01:27:21.544 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-12T01:27:21.544 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-12T01:27:21.544 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/nfs/rpc_pipefs 2024-09-12T01:27:21.545 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/1000 2024-09-12T01:27:21.545 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph 2024-09-12T01:27:21.545 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay 2024-09-12T01:27:21.545 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/2e7727155d5da690420ee8e510883fea96911216c4604e231769998d1b99c05c/merged 2024-09-12T01:27:21.545 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/0e1f475017ce100f0ffd13258b2100850c1fd9abc4cb152fa48a70e7ac9d02c4/merged 2024-09-12T01:27:21.545 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/0 2024-09-12T01:27:21.545 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T01:27:21.545 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/6b760cd0558d91a86517269c07517d65e7f0165a8164c4c86bdc138935ccb500/merged 2024-09-12T01:27:21.545 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/e47cc1f328c85580bd63fff37e56620a7ee7a56db2e40bdaac9c89544e04d822/merged 2024-09-12T01:27:21.545 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/3d157f8de24f906ad6e38ca08e1879b9533652b8f4884b81a675fa2a63718ee5/merged 2024-09-12T01:27:21.545 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/1a664b3cd04309b2e6aeea512a88e029c90200745758709be7800320af2e32f9/merged 2024-09-12T01:27:21.545 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/ce25d328bc0cf8c83ad266038774ccd835e7e42cb2a715d40f3b0040c8fde8d3/merged 2024-09-12T01:27:21.545 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/fc4208d9dfbda187c4e5a810a61e21cebbf923cc80070249a14b63876ae736d5/merged 2024-09-12T01:27:21.546 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/85a1df038549a2bcbcb7cc38a360b971c8daf45035218a8135c7eedb37725f1a/merged 2024-09-12T01:27:21.546 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/containers/storage/overlay/cd9cb360987d7fd5e8cd542f602d5734d957a105663acd643126fd838ca231d6/merged 2024-09-12T01:27:21.546 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns 2024-09-12T01:27:21.546 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-12T01:27:21.546 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-12T01:27:21.546 INFO:teuthology.orchestra.run.smithi001.stdout:/home/ubuntu/cephtest/mnt.admin 2024-09-12T01:27:21.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:27:21.547 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-09-12T01:27:21.624 INFO:teuthology.orchestra.run.smithi001.stdout:120 2024-09-12T01:27:21.624 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-12T01:27:21.624 DEBUG:teuthology.orchestra.run.smithi001:> sudo stdin-killer -- python3 -c ' 2024-09-12T01:27:21.624 DEBUG:teuthology.orchestra.run.smithi001:> import glob 2024-09-12T01:27:21.624 DEBUG:teuthology.orchestra.run.smithi001:> import re 2024-09-12T01:27:21.624 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-09-12T01:27:21.625 DEBUG:teuthology.orchestra.run.smithi001:> import subprocess 2024-09-12T01:27:21.625 DEBUG:teuthology.orchestra.run.smithi001:> 2024-09-12T01:27:21.625 DEBUG:teuthology.orchestra.run.smithi001:> def _find_admin_socket(client_name): 2024-09-12T01:27:21.625 DEBUG:teuthology.orchestra.run.smithi001:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-12T01:27:21.625 DEBUG:teuthology.orchestra.run.smithi001:> files = glob.glob(asok_path) 2024-09-12T01:27:21.625 DEBUG:teuthology.orchestra.run.smithi001:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-12T01:27:21.625 DEBUG:teuthology.orchestra.run.smithi001:> 2024-09-12T01:27:21.625 DEBUG:teuthology.orchestra.run.smithi001:> # Given a non-glob path, it better be there 2024-09-12T01:27:21.625 DEBUG:teuthology.orchestra.run.smithi001:> if "*" not in asok_path: 2024-09-12T01:27:21.625 DEBUG:teuthology.orchestra.run.smithi001:> assert(len(files) == 1) 2024-09-12T01:27:21.625 DEBUG:teuthology.orchestra.run.smithi001:> return files[0] 2024-09-12T01:27:21.629 DEBUG:teuthology.orchestra.run.smithi001:> 2024-09-12T01:27:21.629 DEBUG:teuthology.orchestra.run.smithi001:> for f in files: 2024-09-12T01:27:21.629 DEBUG:teuthology.orchestra.run.smithi001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-12T01:27:21.629 DEBUG:teuthology.orchestra.run.smithi001:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-12T01:27:21.629 DEBUG:teuthology.orchestra.run.smithi001:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-12T01:27:21.629 DEBUG:teuthology.orchestra.run.smithi001:> contents = proc_f.read() 2024-09-12T01:27:21.630 DEBUG:teuthology.orchestra.run.smithi001:> if mountpoint in contents: 2024-09-12T01:27:21.630 DEBUG:teuthology.orchestra.run.smithi001:> return f 2024-09-12T01:27:21.630 DEBUG:teuthology.orchestra.run.smithi001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-12T01:27:21.630 DEBUG:teuthology.orchestra.run.smithi001:> 2024-09-12T01:27:21.630 DEBUG:teuthology.orchestra.run.smithi001:> print(_find_admin_socket("client.admin")) 2024-09-12T01:27:21.630 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-09-12T01:27:21.790 INFO:teuthology.orchestra.run.smithi001.stdout:/var/run/ceph/ceph-client.admin.74898.asok 2024-09-12T01:27:21.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:27:21.802 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:21 smithi001 ceph-mon[30387]: pgmap v329: 129 pgs: 7 peering, 122 active+clean; 579 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-09-12T01:27:21.804 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.74898.asok 2024-09-12T01:27:21.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:27:21.805 DEBUG:teuthology.orchestra.run.smithi001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.74898.asok status 2024-09-12T01:27:21.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:21 smithi053 ceph-mon[35158]: pgmap v329: 129 pgs: 7 peering, 122 active+clean; 579 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-09-12T01:27:21.990 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-12T01:27:21.990 INFO:teuthology.orchestra.run.smithi001.stdout: "metadata": { 2024-09-12T01:27:21.990 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", 2024-09-12T01:27:21.990 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T01:27:21.990 INFO:teuthology.orchestra.run.smithi001.stdout: "entity_id": "admin", 2024-09-12T01:27:21.990 INFO:teuthology.orchestra.run.smithi001.stdout: "hostname": "smithi001", 2024-09-12T01:27:21.991 INFO:teuthology.orchestra.run.smithi001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-12T01:27:21.991 INFO:teuthology.orchestra.run.smithi001.stdout: "pid": "74898", 2024-09-12T01:27:21.991 INFO:teuthology.orchestra.run.smithi001.stdout: "root": "/" 2024-09-12T01:27:21.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:27:21.991 INFO:teuthology.orchestra.run.smithi001.stdout: "dentry_count": 0, 2024-09-12T01:27:21.991 INFO:teuthology.orchestra.run.smithi001.stdout: "dentry_pinned_count": 0, 2024-09-12T01:27:21.991 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 14616, 2024-09-12T01:27:21.991 INFO:teuthology.orchestra.run.smithi001.stdout: "inst": { 2024-09-12T01:27:21.991 INFO:teuthology.orchestra.run.smithi001.stdout: "name": { 2024-09-12T01:27:21.991 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "client", 2024-09-12T01:27:21.991 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 14616 2024-09-12T01:27:21.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:27:21.991 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": { 2024-09-12T01:27:21.991 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-12T01:27:21.991 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "192.168.0.1:0", 2024-09-12T01:27:21.992 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 3878758584 2024-09-12T01:27:21.992 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:27:21.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:27:21.992 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": { 2024-09-12T01:27:21.992 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-12T01:27:21.992 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "192.168.0.1:0", 2024-09-12T01:27:21.992 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 3878758584 2024-09-12T01:27:21.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:27:21.992 INFO:teuthology.orchestra.run.smithi001.stdout: "inst_str": "client.14616 192.168.0.1:0/3878758584", 2024-09-12T01:27:21.992 INFO:teuthology.orchestra.run.smithi001.stdout: "addr_str": "192.168.0.1:0/3878758584", 2024-09-12T01:27:21.992 INFO:teuthology.orchestra.run.smithi001.stdout: "inode_count": 1, 2024-09-12T01:27:21.992 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_epoch": 17, 2024-09-12T01:27:21.992 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_epoch": 96, 2024-09-12T01:27:21.992 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_epoch_barrier": 0, 2024-09-12T01:27:21.993 INFO:teuthology.orchestra.run.smithi001.stdout: "blocklisted": false, 2024-09-12T01:27:21.993 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs" 2024-09-12T01:27:21.993 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-12T01:27:22.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:27:22.003 DEBUG:teuthology.orchestra.run.smithi001:> stat --file-system '--printf=%T 2024-09-12T01:27:22.003 DEBUG:teuthology.orchestra.run.smithi001:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-12T01:27:22.041 INFO:teuthology.orchestra.run.smithi001.stdout:fuseblk 2024-09-12T01:27:22.041 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-12T01:27:22.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:27:22.042 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-12T01:27:22.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:21 smithi104 ceph-mon[35219]: pgmap v329: 129 pgs: 7 peering, 122 active+clean; 579 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-09-12T01:27:22.139 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-12T01:27:22.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:22 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T01:27:22.261 INFO:teuthology.orchestra.run.smithi001.stdout:# file: . 2024-09-12T01:27:22.261 INFO:teuthology.orchestra.run.smithi001.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-12T01:27:22.261 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:27:22.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:27:22.270 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-12T01:27:22.270 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T01:27:22.306 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-12T01:27:22.725 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:27:23.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.303+0000 7fb263200640 1 -- 172.21.15.1:0/3378645513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb264100980 msgr2=0x7fb264100d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:23.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.303+0000 7fb263200640 1 --2- 172.21.15.1:0/3378645513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb264100980 0x7fb264100d60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb254009900 tx=0x7fb25402a470 comp rx=0 tx=0).stop 2024-09-12T01:27:23.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.303+0000 7fb263200640 1 -- 172.21.15.1:0/3378645513 shutdown_connections 2024-09-12T01:27:23.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.303+0000 7fb263200640 1 --2- 172.21.15.1:0/3378645513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb264105d90 0x7fb264106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:23.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.303+0000 7fb263200640 1 --2- 172.21.15.1:0/3378645513 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2641012a0 0x7fb264105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:23.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.303+0000 7fb263200640 1 --2- 172.21.15.1:0/3378645513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb264100980 0x7fb264100d60 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:23.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.303+0000 7fb263200640 1 -- 172.21.15.1:0/3378645513 >> 172.21.15.1:0/3378645513 conn(0x7fb2640fc610 msgr2=0x7fb2640fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:23.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.303+0000 7fb263200640 1 -- 172.21.15.1:0/3378645513 shutdown_connections 2024-09-12T01:27:23.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.303+0000 7fb263200640 1 -- 172.21.15.1:0/3378645513 wait complete. 2024-09-12T01:27:23.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.304+0000 7fb263200640 1 Processor -- start 2024-09-12T01:27:23.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.304+0000 7fb263200640 1 -- start start 2024-09-12T01:27:23.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.305+0000 7fb263200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb264100980 0x7fb2641a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:23.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.305+0000 7fb263200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2641012a0 0x7fb2641a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:23.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.305+0000 7fb263200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb264105d90 0x7fb26419e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:23.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.305+0000 7fb263200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb2641183c0 con 0x7fb264105d90 2024-09-12T01:27:23.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.305+0000 7fb263200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb264118240 con 0x7fb2641012a0 2024-09-12T01:27:23.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.305+0000 7fb263200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb264118540 con 0x7fb264100980 2024-09-12T01:27:23.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.305+0000 7fb262800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb264105d90 0x7fb26419e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:23.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.305+0000 7fb261e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb264100980 0x7fb2641a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:23.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.305+0000 7fb262800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb264105d90 0x7fb26419e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34522/0 (socket says 172.21.15.1:34522) 2024-09-12T01:27:23.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.305+0000 7fb262800640 1 -- 172.21.15.1:0/4079223344 learned_addr learned my addr 172.21.15.1:0/4079223344 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:27:23.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.305+0000 7fb261400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2641012a0 0x7fb2641a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:23.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.306+0000 7fb261e00640 1 -- 172.21.15.1:0/4079223344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2641012a0 msgr2=0x7fb2641a4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:23.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.306+0000 7fb261e00640 1 --2- 172.21.15.1:0/4079223344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2641012a0 0x7fb2641a4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:23.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.306+0000 7fb261e00640 1 -- 172.21.15.1:0/4079223344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb264105d90 msgr2=0x7fb26419e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:23.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.306+0000 7fb261e00640 1 --2- 172.21.15.1:0/4079223344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb264105d90 0x7fb26419e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:23.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.306+0000 7fb261e00640 1 -- 172.21.15.1:0/4079223344 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb26419efc0 con 0x7fb264100980 2024-09-12T01:27:23.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.306+0000 7fb261400640 1 --2- 172.21.15.1:0/4079223344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2641012a0 0x7fb2641a4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:27:23.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.306+0000 7fb261e00640 1 --2- 172.21.15.1:0/4079223344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb264100980 0x7fb2641a4680 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb254005940 tx=0x7fb2540040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:23.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.306+0000 7fb25aa00640 1 -- 172.21.15.1:0/4079223344 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25403f030 con 0x7fb264100980 2024-09-12T01:27:23.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.306+0000 7fb263200640 1 -- 172.21.15.1:0/4079223344 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb26419f250 con 0x7fb264100980 2024-09-12T01:27:23.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.307+0000 7fb263200640 1 -- 172.21.15.1:0/4079223344 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2641ab430 con 0x7fb264100980 2024-09-12T01:27:23.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.307+0000 7fb25aa00640 1 -- 172.21.15.1:0/4079223344 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2540044c0 con 0x7fb264100980 2024-09-12T01:27:23.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.307+0000 7fb25aa00640 1 -- 172.21.15.1:0/4079223344 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb254049060 con 0x7fb264100980 2024-09-12T01:27:23.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.308+0000 7fb263200640 1 -- 172.21.15.1:0/4079223344 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb26406a850 con 0x7fb264100980 2024-09-12T01:27:23.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.310+0000 7fb25aa00640 1 -- 172.21.15.1:0/4079223344 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fb254042030 con 0x7fb264100980 2024-09-12T01:27:23.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.311+0000 7fb25aa00640 1 --2- 172.21.15.1:0/4079223344 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fb2480a2c00 0x7fb2480a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:23.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.311+0000 7fb25aa00640 1 -- 172.21.15.1:0/4079223344 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9092+0+0 (secure 0 0 0) 0x7fb2540ef4c0 con 0x7fb264100980 2024-09-12T01:27:23.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.311+0000 7fb261400640 1 --2- 172.21.15.1:0/4079223344 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fb2480a2c00 0x7fb2480a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:23.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.312+0000 7fb261400640 1 --2- 172.21.15.1:0/4079223344 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fb2480a2c00 0x7fb2480a50c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb26419ffa0 tx=0x7fb250005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:23.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.313+0000 7fb25aa00640 1 -- 172.21.15.1:0/4079223344 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb2540bba10 con 0x7fb264100980 2024-09-12T01:27:23.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.513+0000 7fb263200640 1 -- 172.21.15.1:0/4079223344 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb264105770 con 0x7fb264100980 2024-09-12T01:27:23.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.514+0000 7fb25aa00640 1 -- 172.21.15.1:0/4079223344 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v96) ==== 81+0+580 (secure 0 0 0) 0x7fb2540bfd10 con 0x7fb264100980 2024-09-12T01:27:23.514 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/1373984726 2024-09-13T01:19:14.580637+0000 2024-09-12T01:27:23.514 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/1429963718 2024-09-13T01:19:14.580637+0000 2024-09-12T01:27:23.514 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6801/4101281615 2024-09-13T01:19:14.580637+0000 2024-09-12T01:27:23.514 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6800/4101281615 2024-09-13T01:19:14.580637+0000 2024-09-12T01:27:23.514 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/2564759586 2024-09-13T01:19:14.580637+0000 2024-09-12T01:27:23.514 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6801/2935968453 2024-09-13T01:18:45.882682+0000 2024-09-12T01:27:23.515 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6800/2935968453 2024-09-13T01:18:45.882682+0000 2024-09-12T01:27:23.515 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/1459848813 2024-09-13T01:18:45.882682+0000 2024-09-12T01:27:23.515 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/578605922 2024-09-13T01:18:45.882682+0000 2024-09-12T01:27:23.515 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/738035035 2024-09-13T01:18:45.882682+0000 2024-09-12T01:27:23.515 INFO:teuthology.orchestra.run.smithi001.stderr:listed 10 entries 2024-09-12T01:27:23.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.517+0000 7fb263200640 1 -- 172.21.15.1:0/4079223344 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fb2480a2c00 msgr2=0x7fb2480a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:23.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.517+0000 7fb263200640 1 --2- 172.21.15.1:0/4079223344 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fb2480a2c00 0x7fb2480a50c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb26419ffa0 tx=0x7fb250005f90 comp rx=0 tx=0).stop 2024-09-12T01:27:23.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.517+0000 7fb263200640 1 -- 172.21.15.1:0/4079223344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb264100980 msgr2=0x7fb2641a4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:23.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.517+0000 7fb263200640 1 --2- 172.21.15.1:0/4079223344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb264100980 0x7fb2641a4680 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb254005940 tx=0x7fb2540040c0 comp rx=0 tx=0).stop 2024-09-12T01:27:23.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.517+0000 7fb263200640 1 -- 172.21.15.1:0/4079223344 shutdown_connections 2024-09-12T01:27:23.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.517+0000 7fb263200640 1 --2- 172.21.15.1:0/4079223344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb264105d90 0x7fb26419e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:23.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.517+0000 7fb263200640 1 --2- 172.21.15.1:0/4079223344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2641012a0 0x7fb2641a4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:23.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.517+0000 7fb263200640 1 --2- 172.21.15.1:0/4079223344 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fb2480a2c00 0x7fb2480a50c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:23.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.517+0000 7fb263200640 1 --2- 172.21.15.1:0/4079223344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb264100980 0x7fb2641a4680 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:23.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.517+0000 7fb263200640 1 -- 172.21.15.1:0/4079223344 >> 172.21.15.1:0/4079223344 conn(0x7fb2640fc610 msgr2=0x7fb26410f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:23.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.517+0000 7fb263200640 1 -- 172.21.15.1:0/4079223344 shutdown_connections 2024-09-12T01:27:23.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:23.517+0000 7fb263200640 1 -- 172.21.15.1:0/4079223344 wait complete. 2024-09-12T01:27:23.912 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:23 smithi001 ceph-mon[30387]: pgmap v330: 129 pgs: 129 active+clean; 579 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 201 B/s, 0 objects/s recovering 2024-09-12T01:27:23.913 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:23 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/4079223344' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T01:27:23.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:23 smithi053 ceph-mon[35158]: pgmap v330: 129 pgs: 129 active+clean; 579 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 201 B/s, 0 objects/s recovering 2024-09-12T01:27:23.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:23 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/4079223344' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T01:27:24.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:23 smithi104 ceph-mon[35219]: pgmap v330: 129 pgs: 129 active+clean; 579 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 201 B/s, 0 objects/s recovering 2024-09-12T01:27:24.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:23 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/4079223344' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T01:27:24.302 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-12T01:27:24.302 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T01:27:24.340 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-12T01:27:24.769 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:27:25.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.460+0000 7f5934800640 1 -- 172.21.15.1:0/2119100231 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f592c101df0 msgr2=0x7f592c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:25.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.460+0000 7f5934800640 1 --2- 172.21.15.1:0/2119100231 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f592c101df0 0x7f592c1021d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5920009900 tx=0x7f592002a410 comp rx=0 tx=0).stop 2024-09-12T01:27:25.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.460+0000 7f5934800640 1 -- 172.21.15.1:0/2119100231 shutdown_connections 2024-09-12T01:27:25.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.460+0000 7f5934800640 1 --2- 172.21.15.1:0/2119100231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f592c106f20 0x7f592c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:25.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.460+0000 7f5934800640 1 --2- 172.21.15.1:0/2119100231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f592c1027a0 0x7f592c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:25.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.460+0000 7f5934800640 1 --2- 172.21.15.1:0/2119100231 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f592c101df0 0x7f592c1021d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:25.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.460+0000 7f5934800640 1 -- 172.21.15.1:0/2119100231 >> 172.21.15.1:0/2119100231 conn(0x7f592c0fd8b0 msgr2=0x7f592c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:25.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.460+0000 7f5934800640 1 -- 172.21.15.1:0/2119100231 shutdown_connections 2024-09-12T01:27:25.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.461+0000 7f5934800640 1 -- 172.21.15.1:0/2119100231 wait complete. 2024-09-12T01:27:25.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.461+0000 7f5934800640 1 Processor -- start 2024-09-12T01:27:25.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.462+0000 7f5934800640 1 -- start start 2024-09-12T01:27:25.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.462+0000 7f5934800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f592c101df0 0x7f592c19a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:25.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.462+0000 7f5934800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f592c1027a0 0x7f592c19abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:25.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.462+0000 7f5934800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f592c106f20 0x7f592c19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:25.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.462+0000 7f5934800640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f592c111f80 con 0x7f592c101df0 2024-09-12T01:27:25.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.462+0000 7f5934800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f592c111e00 con 0x7f592c1027a0 2024-09-12T01:27:25.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.462+0000 7f5934800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f592c112100 con 0x7f592c106f20 2024-09-12T01:27:25.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.462+0000 7f5931e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f592c101df0 0x7f592c19a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:25.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.462+0000 7f5931e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f592c101df0 0x7f592c19a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34542/0 (socket says 172.21.15.1:34542) 2024-09-12T01:27:25.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.462+0000 7f5931e00640 1 -- 172.21.15.1:0/589897456 learned_addr learned my addr 172.21.15.1:0/589897456 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:27:25.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.462+0000 7f5931400640 1 --2- 172.21.15.1:0/589897456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f592c1027a0 0x7f592c19abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:25.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.462+0000 7f5932800640 1 --2- 172.21.15.1:0/589897456 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f592c106f20 0x7f592c19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:25.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.463+0000 7f5931e00640 1 -- 172.21.15.1:0/589897456 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f592c106f20 msgr2=0x7f592c19ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:25.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.463+0000 7f5931e00640 1 --2- 172.21.15.1:0/589897456 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f592c106f20 0x7f592c19ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:25.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.463+0000 7f5931e00640 1 -- 172.21.15.1:0/589897456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f592c1027a0 msgr2=0x7f592c19abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:25.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.463+0000 7f5931e00640 1 --2- 172.21.15.1:0/589897456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f592c1027a0 0x7f592c19abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:25.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.463+0000 7f5931e00640 1 -- 172.21.15.1:0/589897456 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f592c19f630 con 0x7f592c101df0 2024-09-12T01:27:25.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.463+0000 7f5931400640 1 --2- 172.21.15.1:0/589897456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f592c1027a0 0x7f592c19abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:27:25.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.463+0000 7f5931e00640 1 --2- 172.21.15.1:0/589897456 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f592c101df0 0x7f592c19a680 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5920006010 tx=0x7f5920004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:25.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.463+0000 7f5926a00640 1 -- 172.21.15.1:0/589897456 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592003f030 con 0x7f592c101df0 2024-09-12T01:27:25.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.463+0000 7f5926a00640 1 -- 172.21.15.1:0/589897456 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5920004440 con 0x7f592c101df0 2024-09-12T01:27:25.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.463+0000 7f5934800640 1 -- 172.21.15.1:0/589897456 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f592c19f8c0 con 0x7f592c101df0 2024-09-12T01:27:25.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.464+0000 7f5926a00640 1 -- 172.21.15.1:0/589897456 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5920035070 con 0x7f592c101df0 2024-09-12T01:27:25.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.464+0000 7f5934800640 1 -- 172.21.15.1:0/589897456 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f592c1a7200 con 0x7f592c101df0 2024-09-12T01:27:25.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.465+0000 7f5934800640 1 -- 172.21.15.1:0/589897456 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f592c1032d0 con 0x7f592c101df0 2024-09-12T01:27:25.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.467+0000 7f5926a00640 1 -- 172.21.15.1:0/589897456 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f5920042030 con 0x7f592c101df0 2024-09-12T01:27:25.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.468+0000 7f5926a00640 1 --2- 172.21.15.1:0/589897456 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f59140a2b80 0x7f59140a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:25.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.468+0000 7f5926a00640 1 -- 172.21.15.1:0/589897456 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9092+0+0 (secure 0 0 0) 0x7f59200ef980 con 0x7f592c101df0 2024-09-12T01:27:25.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.469+0000 7f5931400640 1 --2- 172.21.15.1:0/589897456 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f59140a2b80 0x7f59140a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:25.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.469+0000 7f5931400640 1 --2- 172.21.15.1:0/589897456 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f59140a2b80 0x7f59140a5040 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f592c0ffe50 tx=0x7f591c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:25.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.471+0000 7f5926a00640 1 -- 172.21.15.1:0/589897456 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f59200bbed0 con 0x7f592c101df0 2024-09-12T01:27:25.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.669+0000 7f5934800640 1 -- 172.21.15.1:0/589897456 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f592c1021d0 con 0x7f592c101df0 2024-09-12T01:27:25.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.670+0000 7f5926a00640 1 -- 172.21.15.1:0/589897456 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v96) ==== 81+0+580 (secure 0 0 0) 0x7f59200c01d0 con 0x7f592c101df0 2024-09-12T01:27:25.670 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/1373984726 2024-09-13T01:19:14.580637+0000 2024-09-12T01:27:25.670 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/1429963718 2024-09-13T01:19:14.580637+0000 2024-09-12T01:27:25.670 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6801/4101281615 2024-09-13T01:19:14.580637+0000 2024-09-12T01:27:25.670 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6800/4101281615 2024-09-13T01:19:14.580637+0000 2024-09-12T01:27:25.670 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/2564759586 2024-09-13T01:19:14.580637+0000 2024-09-12T01:27:25.670 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6801/2935968453 2024-09-13T01:18:45.882682+0000 2024-09-12T01:27:25.670 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6800/2935968453 2024-09-13T01:18:45.882682+0000 2024-09-12T01:27:25.670 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/1459848813 2024-09-13T01:18:45.882682+0000 2024-09-12T01:27:25.670 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/578605922 2024-09-13T01:18:45.882682+0000 2024-09-12T01:27:25.670 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/738035035 2024-09-13T01:18:45.882682+0000 2024-09-12T01:27:25.671 INFO:teuthology.orchestra.run.smithi001.stderr:listed 10 entries 2024-09-12T01:27:25.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.672+0000 7f5934800640 1 -- 172.21.15.1:0/589897456 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f59140a2b80 msgr2=0x7f59140a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:25.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.672+0000 7f5934800640 1 --2- 172.21.15.1:0/589897456 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f59140a2b80 0x7f59140a5040 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f592c0ffe50 tx=0x7f591c005f90 comp rx=0 tx=0).stop 2024-09-12T01:27:25.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.672+0000 7f5934800640 1 -- 172.21.15.1:0/589897456 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f592c101df0 msgr2=0x7f592c19a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:25.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.672+0000 7f5934800640 1 --2- 172.21.15.1:0/589897456 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f592c101df0 0x7f592c19a680 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5920006010 tx=0x7f5920004040 comp rx=0 tx=0).stop 2024-09-12T01:27:25.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.673+0000 7f5934800640 1 -- 172.21.15.1:0/589897456 shutdown_connections 2024-09-12T01:27:25.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.673+0000 7f5934800640 1 --2- 172.21.15.1:0/589897456 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f59140a2b80 0x7f59140a5040 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:25.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.673+0000 7f5934800640 1 --2- 172.21.15.1:0/589897456 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f592c106f20 0x7f592c19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:25.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.673+0000 7f5934800640 1 --2- 172.21.15.1:0/589897456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f592c1027a0 0x7f592c19abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:25.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.673+0000 7f5934800640 1 --2- 172.21.15.1:0/589897456 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f592c101df0 0x7f592c19a680 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:25.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.673+0000 7f5934800640 1 -- 172.21.15.1:0/589897456 >> 172.21.15.1:0/589897456 conn(0x7f592c0fd8b0 msgr2=0x7f592c0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:25.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.673+0000 7f5934800640 1 -- 172.21.15.1:0/589897456 shutdown_connections 2024-09-12T01:27:25.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:25.673+0000 7f5934800640 1 -- 172.21.15.1:0/589897456 wait complete. 2024-09-12T01:27:25.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:25 smithi001 ceph-mon[30387]: pgmap v331: 129 pgs: 129 active+clean; 579 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 225 B/s wr, 0 op/s; 177 B/s, 0 objects/s recovering 2024-09-12T01:27:25.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:25 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/589897456' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T01:27:26.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:25 smithi104 ceph-mon[35219]: pgmap v331: 129 pgs: 129 active+clean; 579 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 225 B/s wr, 0 op/s; 177 B/s, 0 objects/s recovering 2024-09-12T01:27:26.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:25 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/589897456' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T01:27:26.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:25 smithi053 ceph-mon[35158]: pgmap v331: 129 pgs: 129 active+clean; 579 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 225 B/s wr, 0 op/s; 177 B/s, 0 objects/s recovering 2024-09-12T01:27:26.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:25 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/589897456' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T01:27:26.468 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi001.front.sepia.ceph.com... 2024-09-12T01:27:26.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:27:26.468 DEBUG:teuthology.orchestra.run.smithi001:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-12T01:27:26.525 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi001.stderr:ceph-fuse[74898]: fuse finished with error 0 and tester_r 0 2024-09-12T01:27:26.528 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T01:27:28.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:27 smithi104 ceph-mon[35219]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 508 B/s wr, 0 op/s; 160 B/s, 0 objects/s recovering 2024-09-12T01:27:28.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:27 smithi001 ceph-mon[30387]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 508 B/s wr, 0 op/s; 160 B/s, 0 objects/s recovering 2024-09-12T01:27:28.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:27 smithi053 ceph-mon[35158]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 508 B/s wr, 0 op/s; 160 B/s, 0 objects/s recovering 2024-09-12T01:27:30.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:29 smithi104 ceph-mon[35219]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s; 134 B/s, 0 objects/s recovering 2024-09-12T01:27:30.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:29 smithi001 ceph-mon[30387]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s; 134 B/s, 0 objects/s recovering 2024-09-12T01:27:30.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:29 smithi053 ceph-mon[35158]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s; 134 B/s, 0 objects/s recovering 2024-09-12T01:27:32.042 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:31 smithi053 ceph-mon[35158]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s; 134 B/s, 0 objects/s recovering 2024-09-12T01:27:32.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:31 smithi104 ceph-mon[35219]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s; 134 B/s, 0 objects/s recovering 2024-09-12T01:27:32.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:31 smithi001 ceph-mon[30387]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s; 134 B/s, 0 objects/s recovering 2024-09-12T01:27:32.630 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi001.front.sepia.ceph.com 2024-09-12T01:27:32.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:27:32.631 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-12T01:27:32.681 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T01:27:33.068 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:27:33.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.669+0000 7f1ec7200640 1 -- 172.21.15.1:0/2949424963 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ec01047e0 msgr2=0x7f1ec0108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:33.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.669+0000 7f1ec7200640 1 --2- 172.21.15.1:0/2949424963 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ec01047e0 0x7f1ec0108830 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0009900 tx=0x7f1eb002a410 comp rx=0 tx=0).stop 2024-09-12T01:27:33.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.670+0000 7f1ec7200640 1 -- 172.21.15.1:0/2949424963 shutdown_connections 2024-09-12T01:27:33.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.670+0000 7f1ec7200640 1 --2- 172.21.15.1:0/2949424963 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ec0108f60 0x7f1ec0110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:33.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.670+0000 7f1ec7200640 1 --2- 172.21.15.1:0/2949424963 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ec01047e0 0x7f1ec0108830 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:33.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.670+0000 7f1ec7200640 1 --2- 172.21.15.1:0/2949424963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ec0103e30 0x7f1ec0104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:33.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.670+0000 7f1ec7200640 1 -- 172.21.15.1:0/2949424963 >> 172.21.15.1:0/2949424963 conn(0x7f1ec00ff910 msgr2=0x7f1ec0101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:33.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.670+0000 7f1ec7200640 1 -- 172.21.15.1:0/2949424963 shutdown_connections 2024-09-12T01:27:33.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.670+0000 7f1ec7200640 1 -- 172.21.15.1:0/2949424963 wait complete. 2024-09-12T01:27:33.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.671+0000 7f1ec7200640 1 Processor -- start 2024-09-12T01:27:33.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.671+0000 7f1ec7200640 1 -- start start 2024-09-12T01:27:33.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.671+0000 7f1ec7200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ec0103e30 0x7f1ec019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:33.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.671+0000 7f1ec7200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ec01047e0 0x7f1ec019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:33.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.671+0000 7f1ec7200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ec0108f60 0x7f1ec019eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:33.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.671+0000 7f1ec7200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec0114010 con 0x7f1ec0103e30 2024-09-12T01:27:33.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.671+0000 7f1ec7200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec0113e90 con 0x7f1ec0108f60 2024-09-12T01:27:33.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.671+0000 7f1ec7200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec0114190 con 0x7f1ec01047e0 2024-09-12T01:27:33.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.671+0000 7f1ec5e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ec0103e30 0x7f1ec019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:33.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.671+0000 7f1ec5400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ec01047e0 0x7f1ec019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:33.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.672+0000 7f1ec5e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ec0103e30 0x7f1ec019a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44900/0 (socket says 172.21.15.1:44900) 2024-09-12T01:27:33.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.672+0000 7f1ec5e00640 1 -- 172.21.15.1:0/4131880998 learned_addr learned my addr 172.21.15.1:0/4131880998 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:27:33.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.672+0000 7f1ec5400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ec01047e0 0x7f1ec019aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:35342/0 (socket says 172.21.15.1:35342) 2024-09-12T01:27:33.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.672+0000 7f1ec6800640 1 --2- 172.21.15.1:0/4131880998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ec0108f60 0x7f1ec019eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:33.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.672+0000 7f1ec5400640 1 -- 172.21.15.1:0/4131880998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ec0108f60 msgr2=0x7f1ec019eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:33.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.672+0000 7f1ec5400640 1 --2- 172.21.15.1:0/4131880998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ec0108f60 0x7f1ec019eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:33.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.672+0000 7f1ec5400640 1 -- 172.21.15.1:0/4131880998 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ec0103e30 msgr2=0x7f1ec019a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:33.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.672+0000 7f1ec5400640 1 --2- 172.21.15.1:0/4131880998 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ec0103e30 0x7f1ec019a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:33.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.672+0000 7f1ec5400640 1 -- 172.21.15.1:0/4131880998 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec019f480 con 0x7f1ec01047e0 2024-09-12T01:27:33.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.672+0000 7f1ec5e00640 1 --2- 172.21.15.1:0/4131880998 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ec0103e30 0x7f1ec019a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:27:33.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.672+0000 7f1ec5400640 1 --2- 172.21.15.1:0/4131880998 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ec01047e0 0x7f1ec019aa10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1eb002a930 tx=0x7f1eb0003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:33.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.673+0000 7f1ebaa00640 1 -- 172.21.15.1:0/4131880998 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb003f070 con 0x7f1ec01047e0 2024-09-12T01:27:33.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.673+0000 7f1ec7200640 1 -- 172.21.15.1:0/4131880998 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ec019f710 con 0x7f1ec01047e0 2024-09-12T01:27:33.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.673+0000 7f1ec7200640 1 -- 172.21.15.1:0/4131880998 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ec01a7050 con 0x7f1ec01047e0 2024-09-12T01:27:33.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.673+0000 7f1ebaa00640 1 -- 172.21.15.1:0/4131880998 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1eb003b940 con 0x7f1ec01047e0 2024-09-12T01:27:33.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.673+0000 7f1ebaa00640 1 -- 172.21.15.1:0/4131880998 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0049060 con 0x7f1ec01047e0 2024-09-12T01:27:33.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.674+0000 7f1ec7200640 1 -- 172.21.15.1:0/4131880998 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ea4005290 con 0x7f1ec01047e0 2024-09-12T01:27:33.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.676+0000 7f1ebaa00640 1 -- 172.21.15.1:0/4131880998 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f1eb0042070 con 0x7f1ec01047e0 2024-09-12T01:27:33.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.677+0000 7f1ebaa00640 1 --2- 172.21.15.1:0/4131880998 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1ea80a2c00 0x7f1ea80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:33.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.678+0000 7f1ebaa00640 1 -- 172.21.15.1:0/4131880998 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9092+0+0 (secure 0 0 0) 0x7f1eb00efa20 con 0x7f1ec01047e0 2024-09-12T01:27:33.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.678+0000 7f1ec5e00640 1 --2- 172.21.15.1:0/4131880998 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1ea80a2c00 0x7f1ea80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:33.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.678+0000 7f1ec5e00640 1 --2- 172.21.15.1:0/4131880998 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1ea80a2c00 0x7f1ea80a50c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4011500 tx=0x7f1eb4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:33.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.680+0000 7f1ebaa00640 1 -- 172.21.15.1:0/4131880998 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1eb00bc5c0 con 0x7f1ec01047e0 2024-09-12T01:27:33.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.870+0000 7f1ec7200640 1 -- 172.21.15.1:0/4131880998 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ea4005850 con 0x7f1ec01047e0 2024-09-12T01:27:33.870 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:33 smithi001 ceph-mon[30387]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s; 134 B/s, 0 objects/s recovering 2024-09-12T01:27:33.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.872+0000 7f1ebaa00640 1 -- 172.21.15.1:0/4131880998 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24657 (secure 0 0 0) 0x7f1eb00bbec0 con 0x7f1ec01047e0 2024-09-12T01:27:33.872 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:27:33.872 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":96,"fsid":"c0e4f180-70a4-11ef-bcea-c7b262605968","created":"2024-09-12T01:18:19.337406+0000","modified":"2024-09-12T01:27:16.608027+0000","last_up_change":"2024-09-12T01:25:50.601477+0000","last_in_change":"2024-09-12T01:25:27.674600+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:21:41.204242+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:26:43.066475+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:26:46.112771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T01:26:56.040130+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b9a396ae-bf97-462f-9ef9-b10ef6b9d337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6803","nonce":1125335136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6805","nonce":1125335136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6809","nonce":1125335136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6807","nonce":1125335136}]},"public_addr":"172.21.15.1:6803/1125335136","cluster_addr":"172.21.15.1:6805/1125335136","heartbeat_back_addr":"172.21.15.1:6809/1125335136","heartbeat_front_addr":"172.21.15.1:6807/1125335136","state":["exists","up"]},{"osd":1,"uuid":"dfe7a8a3-3700-4789-ac64-88660de9e0d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6801","nonce":3131049165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6803","nonce":3131049165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6807","nonce":3131049165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6805","nonce":3131049165}]},"public_addr":"172.21.15.53:6801/3131049165","cluster_addr":"172.21.15.53:6803/3131049165","heartbeat_back_addr":"172.21.15.53:6807/3131049165","heartbeat_front_addr":"172.21.15.53:6805/3131049165","state":["exists","up"]},{"osd":2,"uuid":"5d9d3603-ba7c-4115-9267-12abcb20d55b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6801","nonce":1133285636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6803","nonce":1133285636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6807","nonce":1133285636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6805","nonce":1133285636}]},"public_addr":"172.21.15.104:6801/1133285636","cluster_addr":"172.21.15.104:6803/1133285636","heartbeat_back_addr":"172.21.15.104:6807/1133285636","heartbeat_front_addr":"172.21.15.104:6805/1133285636","state":["exists","up"]},{"osd":3,"uuid":"1e063c66-41b4-4db1-a964-28e122ce628e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6811","nonce":1247468803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6813","nonce":1247468803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6817","nonce":1247468803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6815","nonce":1247468803}]},"public_addr":"172.21.15.1:6811/1247468803","cluster_addr":"172.21.15.1:6813/1247468803","heartbeat_back_addr":"172.21.15.1:6817/1247468803","heartbeat_front_addr":"172.21.15.1:6815/1247468803","state":["exists","up"]},{"osd":4,"uuid":"fe89ece0-2171-45d5-a06d-e400f77233e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6809","nonce":2673413639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6811","nonce":2673413639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6815","nonce":2673413639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6813","nonce":2673413639}]},"public_addr":"172.21.15.53:6809/2673413639","cluster_addr":"172.21.15.53:6811/2673413639","heartbeat_back_addr":"172.21.15.53:6815/2673413639","heartbeat_front_addr":"172.21.15.53:6813/2673413639","state":["exists","up"]},{"osd":5,"uuid":"daab795c-9942-4d16-aa34-8bba30a4b99b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6809","nonce":3131498790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6811","nonce":3131498790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6815","nonce":3131498790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6813","nonce":3131498790}]},"public_addr":"172.21.15.104:6809/3131498790","cluster_addr":"172.21.15.104:6811/3131498790","heartbeat_back_addr":"172.21.15.104:6815/3131498790","heartbeat_front_addr":"172.21.15.104:6813/3131498790","state":["exists","up"]},{"osd":6,"uuid":"f31d8633-e435-499b-9d46-0aa11a2481cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6819","nonce":3413067918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6821","nonce":3413067918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6825","nonce":3413067918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6823","nonce":3413067918}]},"public_addr":"172.21.15.1:6819/3413067918","cluster_addr":"172.21.15.1:6821/3413067918","heartbeat_back_addr":"172.21.15.1:6825/3413067918","heartbeat_front_addr":"172.21.15.1:6823/3413067918","state":["exists","up"]},{"osd":7,"uuid":"e64da49e-d31e-4801-b6cc-634a233b4761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6817","nonce":2841484136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6819","nonce":2841484136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6823","nonce":2841484136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6821","nonce":2841484136}]},"public_addr":"172.21.15.53:6817/2841484136","cluster_addr":"172.21.15.53:6819/2841484136","heartbeat_back_addr":"172.21.15.53:6823/2841484136","heartbeat_front_addr":"172.21.15.53:6821/2841484136","state":["exists","up"]},{"osd":8,"uuid":"fdfb5a42-dca7-4874-b479-176b4a191e21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6817","nonce":941453547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6819","nonce":941453547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6823","nonce":941453547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6821","nonce":941453547}]},"public_addr":"172.21.15.104:6817/941453547","cluster_addr":"172.21.15.104:6819/941453547","heartbeat_back_addr":"172.21.15.104:6823/941453547","heartbeat_front_addr":"172.21.15.104:6821/941453547","state":["exists","up"]},{"osd":9,"uuid":"7f81537a-cae5-4bb8-aec2-4eb07deb7958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6827","nonce":958287586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6829","nonce":958287586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6833","nonce":958287586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6831","nonce":958287586}]},"public_addr":"172.21.15.1:6827/958287586","cluster_addr":"172.21.15.1:6829/958287586","heartbeat_back_addr":"172.21.15.1:6833/958287586","heartbeat_front_addr":"172.21.15.1:6831/958287586","state":["exists","up"]},{"osd":10,"uuid":"e3c9c37d-46b9-4585-990d-69c447453bd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6825","nonce":3357909498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6827","nonce":3357909498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6831","nonce":3357909498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6829","nonce":3357909498}]},"public_addr":"172.21.15.53:6825/3357909498","cluster_addr":"172.21.15.53:6827/3357909498","heartbeat_back_addr":"172.21.15.53:6831/3357909498","heartbeat_front_addr":"172.21.15.53:6829/3357909498","state":["exists","up"]},{"osd":11,"uuid":"474b5345-f2ae-4e0e-a197-53ab11f58c04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6825","nonce":3381427987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6827","nonce":3381427987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6831","nonce":3381427987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6829","nonce":3381427987}]},"public_addr":"172.21.15.104:6825/3381427987","cluster_addr":"172.21.15.104:6827/3381427987","heartbeat_back_addr":"172.21.15.104:6831/3381427987","heartbeat_front_addr":"172.21.15.104:6829/3381427987","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:20:46.903181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:21:12.735556+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:21:37.868739+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:22:09.141228+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:22:36.438563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:23:02.220948+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:23:33.505881+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:24:00.982176+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:24:26.010444+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:24:56.711121+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:25:23.846495+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:25:48.815427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1373984726":"2024-09-13T01:19:14.580637+0000","172.21.15.1:0/1429963718":"2024-09-13T01:19:14.580637+0000","172.21.15.1:6801/4101281615":"2024-09-13T01:19:14.580637+0000","172.21.15.1:6800/4101281615":"2024-09-13T01:19:14.580637+0000","172.21.15.1:0/2564759586":"2024-09-13T01:19:14.580637+0000","172.21.15.1:6801/2935968453":"2024-09-13T01:18:45.882682+0000","172.21.15.1:6800/2935968453":"2024-09-13T01:18:45.882682+0000","172.21.15.1:0/1459848813":"2024-09-13T01:18:45.882682+0000","172.21.15.1:0/578605922":"2024-09-13T01:18:45.882682+0000","172.21.15.1:0/738035035":"2024-09-13T01:18:45.882682+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:27:33.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.874+0000 7f1ec7200640 1 -- 172.21.15.1:0/4131880998 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1ea80a2c00 msgr2=0x7f1ea80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:33.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.874+0000 7f1ec7200640 1 --2- 172.21.15.1:0/4131880998 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1ea80a2c00 0x7f1ea80a50c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4011500 tx=0x7f1eb4005f90 comp rx=0 tx=0).stop 2024-09-12T01:27:33.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.874+0000 7f1ec7200640 1 -- 172.21.15.1:0/4131880998 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ec01047e0 msgr2=0x7f1ec019aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:33.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.874+0000 7f1ec7200640 1 --2- 172.21.15.1:0/4131880998 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ec01047e0 0x7f1ec019aa10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1eb002a930 tx=0x7f1eb0003b20 comp rx=0 tx=0).stop 2024-09-12T01:27:33.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.874+0000 7f1ec7200640 1 -- 172.21.15.1:0/4131880998 shutdown_connections 2024-09-12T01:27:33.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.874+0000 7f1ec7200640 1 --2- 172.21.15.1:0/4131880998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ec0108f60 0x7f1ec019eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:33.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.874+0000 7f1ec7200640 1 --2- 172.21.15.1:0/4131880998 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1ea80a2c00 0x7f1ea80a50c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:33.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.874+0000 7f1ec7200640 1 --2- 172.21.15.1:0/4131880998 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ec01047e0 0x7f1ec019aa10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:33.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.874+0000 7f1ec7200640 1 --2- 172.21.15.1:0/4131880998 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ec0103e30 0x7f1ec019a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:33.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.874+0000 7f1ec7200640 1 -- 172.21.15.1:0/4131880998 >> 172.21.15.1:0/4131880998 conn(0x7f1ec00ff910 msgr2=0x7f1ec0100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:33.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.874+0000 7f1ec7200640 1 -- 172.21.15.1:0/4131880998 shutdown_connections 2024-09-12T01:27:33.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:33.875+0000 7f1ec7200640 1 -- 172.21.15.1:0/4131880998 wait complete. 2024-09-12T01:27:34.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:33 smithi104 ceph-mon[35219]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s; 134 B/s, 0 objects/s recovering 2024-09-12T01:27:34.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:33 smithi053 ceph-mon[35158]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s; 134 B/s, 0 objects/s recovering 2024-09-12T01:27:34.834 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T01:27:35.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:34 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/4131880998' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:27:35.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:34 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/4131880998' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:27:35.220 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:27:35.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:34 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/4131880998' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:27:35.985 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:35 smithi001 ceph-mon[30387]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-12T01:27:35.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:35.984+0000 7f9591600640 1 -- 172.21.15.1:0/4260003494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f958c1049a0 msgr2=0x7f958c1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:35.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:35.984+0000 7f9591600640 1 --2- 172.21.15.1:0/4260003494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f958c1049a0 0x7f958c1089f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9574009900 tx=0x7f957402a410 comp rx=0 tx=0).stop 2024-09-12T01:27:35.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:35.985+0000 7f9591600640 1 -- 172.21.15.1:0/4260003494 shutdown_connections 2024-09-12T01:27:35.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:35.985+0000 7f9591600640 1 --2- 172.21.15.1:0/4260003494 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f958c109120 0x7f958c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:35.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:35.985+0000 7f9591600640 1 --2- 172.21.15.1:0/4260003494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f958c1049a0 0x7f958c1089f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:35.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:35.985+0000 7f9591600640 1 --2- 172.21.15.1:0/4260003494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f958c103ff0 0x7f958c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:35.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:35.985+0000 7f9591600640 1 -- 172.21.15.1:0/4260003494 >> 172.21.15.1:0/4260003494 conn(0x7f958c0ffa90 msgr2=0x7f958c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:35.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:35.985+0000 7f9591600640 1 -- 172.21.15.1:0/4260003494 shutdown_connections 2024-09-12T01:27:35.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:35.985+0000 7f9591600640 1 -- 172.21.15.1:0/4260003494 wait complete. 2024-09-12T01:27:35.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:35.986+0000 7f9591600640 1 Processor -- start 2024-09-12T01:27:35.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:35.986+0000 7f9591600640 1 -- start start 2024-09-12T01:27:35.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:35.986+0000 7f9591600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f958c103ff0 0x7f958c19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:35.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:35.986+0000 7f9591600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f958c1049a0 0x7f958c19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:35.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:35.986+0000 7f9591600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f958c109120 0x7f958c19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:35.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:35.986+0000 7f9591600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f958c114190 con 0x7f958c1049a0 2024-09-12T01:27:35.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:35.986+0000 7f9591600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f958c114010 con 0x7f958c103ff0 2024-09-12T01:27:35.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:35.986+0000 7f9591600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f958c114310 con 0x7f958c109120 2024-09-12T01:27:35.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:35.987+0000 7f9583e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f958c1049a0 0x7f958c19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:35.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:35.987+0000 7f958aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f958c103ff0 0x7f958c19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:35.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:35.987+0000 7f958aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f958c103ff0 0x7f958c19a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:41726/0 (socket says 172.21.15.1:41726) 2024-09-12T01:27:35.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:35.987+0000 7f958aa00640 1 -- 172.21.15.1:0/792467279 learned_addr learned my addr 172.21.15.1:0/792467279 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:27:35.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:35.987+0000 7f958b400640 1 --2- 172.21.15.1:0/792467279 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f958c109120 0x7f958c19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:35.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:35.987+0000 7f9583e00640 1 -- 172.21.15.1:0/792467279 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f958c109120 msgr2=0x7f958c19eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:35.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:35.987+0000 7f9583e00640 1 --2- 172.21.15.1:0/792467279 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f958c109120 0x7f958c19eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:35.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:35.987+0000 7f9583e00640 1 -- 172.21.15.1:0/792467279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f958c103ff0 msgr2=0x7f958c19a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:35.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:35.987+0000 7f9583e00640 1 --2- 172.21.15.1:0/792467279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f958c103ff0 0x7f958c19a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:35.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:35.987+0000 7f9583e00640 1 -- 172.21.15.1:0/792467279 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f958c19f5a0 con 0x7f958c1049a0 2024-09-12T01:27:35.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:35.987+0000 7f9583e00640 1 --2- 172.21.15.1:0/792467279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f958c1049a0 0x7f958c19ab30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f957402a930 tx=0x7f9574003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:35.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:35.988+0000 7f9583400640 1 -- 172.21.15.1:0/792467279 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957403f070 con 0x7f958c1049a0 2024-09-12T01:27:35.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:35.988+0000 7f9583400640 1 -- 172.21.15.1:0/792467279 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9574031c90 con 0x7f958c1049a0 2024-09-12T01:27:35.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:35.988+0000 7f9591600640 1 -- 172.21.15.1:0/792467279 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f958c19f830 con 0x7f958c1049a0 2024-09-12T01:27:35.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:35.988+0000 7f9583400640 1 -- 172.21.15.1:0/792467279 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9574035030 con 0x7f958c1049a0 2024-09-12T01:27:35.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:35.988+0000 7f9591600640 1 -- 172.21.15.1:0/792467279 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f958c1a7170 con 0x7f958c1049a0 2024-09-12T01:27:35.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:35.989+0000 7f9591600640 1 -- 172.21.15.1:0/792467279 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f958c1054d0 con 0x7f958c1049a0 2024-09-12T01:27:35.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:35.991+0000 7f9583400640 1 -- 172.21.15.1:0/792467279 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f9574042070 con 0x7f958c1049a0 2024-09-12T01:27:35.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:35.992+0000 7f9583400640 1 --2- 172.21.15.1:0/792467279 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f95700a2b80 0x7f95700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:35.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:35.992+0000 7f9583400640 1 -- 172.21.15.1:0/792467279 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9092+0+0 (secure 0 0 0) 0x7f95740efcf0 con 0x7f958c1049a0 2024-09-12T01:27:35.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:35.992+0000 7f958aa00640 1 --2- 172.21.15.1:0/792467279 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f95700a2b80 0x7f95700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:35.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:35.993+0000 7f958aa00640 1 --2- 172.21.15.1:0/792467279 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f95700a2b80 0x7f95700a5040 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f957c0095f0 tx=0x7f957c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:35.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:35.996+0000 7f9583400640 1 -- 172.21.15.1:0/792467279 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f95740bc240 con 0x7f958c1049a0 2024-09-12T01:27:36.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:35 smithi104 ceph-mon[35219]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-12T01:27:36.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:36.185+0000 7f9591600640 1 -- 172.21.15.1:0/792467279 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f958c1a7340 con 0x7f958c1049a0 2024-09-12T01:27:36.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:36.186+0000 7f9583400640 1 -- 172.21.15.1:0/792467279 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24657 (secure 0 0 0) 0x7f95740c0540 con 0x7f958c1049a0 2024-09-12T01:27:36.186 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:27:36.187 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":96,"fsid":"c0e4f180-70a4-11ef-bcea-c7b262605968","created":"2024-09-12T01:18:19.337406+0000","modified":"2024-09-12T01:27:16.608027+0000","last_up_change":"2024-09-12T01:25:50.601477+0000","last_in_change":"2024-09-12T01:25:27.674600+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:21:41.204242+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:26:43.066475+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:26:46.112771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T01:26:56.040130+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b9a396ae-bf97-462f-9ef9-b10ef6b9d337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6803","nonce":1125335136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6805","nonce":1125335136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6809","nonce":1125335136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6807","nonce":1125335136}]},"public_addr":"172.21.15.1:6803/1125335136","cluster_addr":"172.21.15.1:6805/1125335136","heartbeat_back_addr":"172.21.15.1:6809/1125335136","heartbeat_front_addr":"172.21.15.1:6807/1125335136","state":["exists","up"]},{"osd":1,"uuid":"dfe7a8a3-3700-4789-ac64-88660de9e0d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6801","nonce":3131049165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6803","nonce":3131049165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6807","nonce":3131049165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6805","nonce":3131049165}]},"public_addr":"172.21.15.53:6801/3131049165","cluster_addr":"172.21.15.53:6803/3131049165","heartbeat_back_addr":"172.21.15.53:6807/3131049165","heartbeat_front_addr":"172.21.15.53:6805/3131049165","state":["exists","up"]},{"osd":2,"uuid":"5d9d3603-ba7c-4115-9267-12abcb20d55b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6801","nonce":1133285636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6803","nonce":1133285636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6807","nonce":1133285636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6805","nonce":1133285636}]},"public_addr":"172.21.15.104:6801/1133285636","cluster_addr":"172.21.15.104:6803/1133285636","heartbeat_back_addr":"172.21.15.104:6807/1133285636","heartbeat_front_addr":"172.21.15.104:6805/1133285636","state":["exists","up"]},{"osd":3,"uuid":"1e063c66-41b4-4db1-a964-28e122ce628e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6811","nonce":1247468803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6813","nonce":1247468803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6817","nonce":1247468803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6815","nonce":1247468803}]},"public_addr":"172.21.15.1:6811/1247468803","cluster_addr":"172.21.15.1:6813/1247468803","heartbeat_back_addr":"172.21.15.1:6817/1247468803","heartbeat_front_addr":"172.21.15.1:6815/1247468803","state":["exists","up"]},{"osd":4,"uuid":"fe89ece0-2171-45d5-a06d-e400f77233e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6809","nonce":2673413639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6811","nonce":2673413639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6815","nonce":2673413639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6813","nonce":2673413639}]},"public_addr":"172.21.15.53:6809/2673413639","cluster_addr":"172.21.15.53:6811/2673413639","heartbeat_back_addr":"172.21.15.53:6815/2673413639","heartbeat_front_addr":"172.21.15.53:6813/2673413639","state":["exists","up"]},{"osd":5,"uuid":"daab795c-9942-4d16-aa34-8bba30a4b99b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6809","nonce":3131498790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6811","nonce":3131498790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6815","nonce":3131498790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6813","nonce":3131498790}]},"public_addr":"172.21.15.104:6809/3131498790","cluster_addr":"172.21.15.104:6811/3131498790","heartbeat_back_addr":"172.21.15.104:6815/3131498790","heartbeat_front_addr":"172.21.15.104:6813/3131498790","state":["exists","up"]},{"osd":6,"uuid":"f31d8633-e435-499b-9d46-0aa11a2481cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6819","nonce":3413067918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6821","nonce":3413067918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6825","nonce":3413067918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6823","nonce":3413067918}]},"public_addr":"172.21.15.1:6819/3413067918","cluster_addr":"172.21.15.1:6821/3413067918","heartbeat_back_addr":"172.21.15.1:6825/3413067918","heartbeat_front_addr":"172.21.15.1:6823/3413067918","state":["exists","up"]},{"osd":7,"uuid":"e64da49e-d31e-4801-b6cc-634a233b4761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6817","nonce":2841484136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6819","nonce":2841484136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6823","nonce":2841484136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6821","nonce":2841484136}]},"public_addr":"172.21.15.53:6817/2841484136","cluster_addr":"172.21.15.53:6819/2841484136","heartbeat_back_addr":"172.21.15.53:6823/2841484136","heartbeat_front_addr":"172.21.15.53:6821/2841484136","state":["exists","up"]},{"osd":8,"uuid":"fdfb5a42-dca7-4874-b479-176b4a191e21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6817","nonce":941453547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6819","nonce":941453547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6823","nonce":941453547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6821","nonce":941453547}]},"public_addr":"172.21.15.104:6817/941453547","cluster_addr":"172.21.15.104:6819/941453547","heartbeat_back_addr":"172.21.15.104:6823/941453547","heartbeat_front_addr":"172.21.15.104:6821/941453547","state":["exists","up"]},{"osd":9,"uuid":"7f81537a-cae5-4bb8-aec2-4eb07deb7958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6827","nonce":958287586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6829","nonce":958287586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6833","nonce":958287586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6831","nonce":958287586}]},"public_addr":"172.21.15.1:6827/958287586","cluster_addr":"172.21.15.1:6829/958287586","heartbeat_back_addr":"172.21.15.1:6833/958287586","heartbeat_front_addr":"172.21.15.1:6831/958287586","state":["exists","up"]},{"osd":10,"uuid":"e3c9c37d-46b9-4585-990d-69c447453bd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6825","nonce":3357909498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6827","nonce":3357909498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6831","nonce":3357909498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6829","nonce":3357909498}]},"public_addr":"172.21.15.53:6825/3357909498","cluster_addr":"172.21.15.53:6827/3357909498","heartbeat_back_addr":"172.21.15.53:6831/3357909498","heartbeat_front_addr":"172.21.15.53:6829/3357909498","state":["exists","up"]},{"osd":11,"uuid":"474b5345-f2ae-4e0e-a197-53ab11f58c04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6825","nonce":3381427987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6827","nonce":3381427987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6831","nonce":3381427987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6829","nonce":3381427987}]},"public_addr":"172.21.15.104:6825/3381427987","cluster_addr":"172.21.15.104:6827/3381427987","heartbeat_back_addr":"172.21.15.104:6831/3381427987","heartbeat_front_addr":"172.21.15.104:6829/3381427987","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:20:46.903181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:21:12.735556+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:21:37.868739+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:22:09.141228+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:22:36.438563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:23:02.220948+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:23:33.505881+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:24:00.982176+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:24:26.010444+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:24:56.711121+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:25:23.846495+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:25:48.815427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1373984726":"2024-09-13T01:19:14.580637+0000","172.21.15.1:0/1429963718":"2024-09-13T01:19:14.580637+0000","172.21.15.1:6801/4101281615":"2024-09-13T01:19:14.580637+0000","172.21.15.1:6800/4101281615":"2024-09-13T01:19:14.580637+0000","172.21.15.1:0/2564759586":"2024-09-13T01:19:14.580637+0000","172.21.15.1:6801/2935968453":"2024-09-13T01:18:45.882682+0000","172.21.15.1:6800/2935968453":"2024-09-13T01:18:45.882682+0000","172.21.15.1:0/1459848813":"2024-09-13T01:18:45.882682+0000","172.21.15.1:0/578605922":"2024-09-13T01:18:45.882682+0000","172.21.15.1:0/738035035":"2024-09-13T01:18:45.882682+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:27:36.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:36.189+0000 7f9591600640 1 -- 172.21.15.1:0/792467279 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f95700a2b80 msgr2=0x7f95700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:36.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:36.189+0000 7f9591600640 1 --2- 172.21.15.1:0/792467279 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f95700a2b80 0x7f95700a5040 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f957c0095f0 tx=0x7f957c005f90 comp rx=0 tx=0).stop 2024-09-12T01:27:36.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:36.189+0000 7f9591600640 1 -- 172.21.15.1:0/792467279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f958c1049a0 msgr2=0x7f958c19ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:36.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:36.189+0000 7f9591600640 1 --2- 172.21.15.1:0/792467279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f958c1049a0 0x7f958c19ab30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f957402a930 tx=0x7f9574003b20 comp rx=0 tx=0).stop 2024-09-12T01:27:36.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:36.189+0000 7f9591600640 1 -- 172.21.15.1:0/792467279 shutdown_connections 2024-09-12T01:27:36.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:36.189+0000 7f9591600640 1 --2- 172.21.15.1:0/792467279 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f95700a2b80 0x7f95700a5040 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:36.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:36.189+0000 7f9591600640 1 --2- 172.21.15.1:0/792467279 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f958c109120 0x7f958c19eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:36.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:36.189+0000 7f9591600640 1 --2- 172.21.15.1:0/792467279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f958c1049a0 0x7f958c19ab30 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:36.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:36.189+0000 7f9591600640 1 --2- 172.21.15.1:0/792467279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f958c103ff0 0x7f958c19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:36.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:36.189+0000 7f9591600640 1 -- 172.21.15.1:0/792467279 >> 172.21.15.1:0/792467279 conn(0x7f958c0ffa90 msgr2=0x7f958c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:36.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:36.189+0000 7f9591600640 1 -- 172.21.15.1:0/792467279 shutdown_connections 2024-09-12T01:27:36.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:36.189+0000 7f9591600640 1 -- 172.21.15.1:0/792467279 wait complete. 2024-09-12T01:27:36.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:35 smithi053 ceph-mon[35158]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-12T01:27:37.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:36 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/792467279' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:27:37.085 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:36 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/792467279' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:27:37.126 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-12T01:27:37.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:36 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/792467279' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:27:37.501 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:27:37.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:37 smithi001 ceph-mon[30387]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-12T01:27:38.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:37 smithi104 ceph-mon[35219]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-12T01:27:38.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.112+0000 7ffacf600640 1 -- 172.21.15.1:0/2727375355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffac8109140 msgr2=0x7ffac8110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:38.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.112+0000 7ffacf600640 1 --2- 172.21.15.1:0/2727375355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffac8109140 0x7ffac8110cc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009ca0 tx=0x7ffac002a8c0 comp rx=0 tx=0).stop 2024-09-12T01:27:38.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.112+0000 7ffacf600640 1 -- 172.21.15.1:0/2727375355 shutdown_connections 2024-09-12T01:27:38.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.112+0000 7ffacf600640 1 --2- 172.21.15.1:0/2727375355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffac8109140 0x7ffac8110cc0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:38.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.112+0000 7ffacf600640 1 --2- 172.21.15.1:0/2727375355 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffac81049c0 0x7ffac8108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:38.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.112+0000 7ffacf600640 1 --2- 172.21.15.1:0/2727375355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffac8104010 0x7ffac81043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:38.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.112+0000 7ffacf600640 1 -- 172.21.15.1:0/2727375355 >> 172.21.15.1:0/2727375355 conn(0x7ffac80ffa50 msgr2=0x7ffac8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:38.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.113+0000 7ffacf600640 1 -- 172.21.15.1:0/2727375355 shutdown_connections 2024-09-12T01:27:38.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.113+0000 7ffacf600640 1 -- 172.21.15.1:0/2727375355 wait complete. 2024-09-12T01:27:38.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.113+0000 7ffacf600640 1 Processor -- start 2024-09-12T01:27:38.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.114+0000 7ffacf600640 1 -- start start 2024-09-12T01:27:38.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.114+0000 7ffacf600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffac8104010 0x7ffac819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:38.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.114+0000 7ffacf600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffac81049c0 0x7ffac819ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:38.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.114+0000 7ffacf600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffac8109140 0x7ffac819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:38.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.114+0000 7ffacf600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ffac81141c0 con 0x7ffac81049c0 2024-09-12T01:27:38.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.114+0000 7ffacf600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ffac8114040 con 0x7ffac8104010 2024-09-12T01:27:38.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.114+0000 7ffacf600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ffac8114340 con 0x7ffac8109140 2024-09-12T01:27:38.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.114+0000 7ffac7e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffac81049c0 0x7ffac819ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:38.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.114+0000 7ffaccc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffac8104010 0x7ffac819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:38.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.114+0000 7ffac7e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffac81049c0 0x7ffac819ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44946/0 (socket says 172.21.15.1:44946) 2024-09-12T01:27:38.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.114+0000 7ffac7e00640 1 -- 172.21.15.1:0/3307154134 learned_addr learned my addr 172.21.15.1:0/3307154134 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:27:38.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.115+0000 7ffacd600640 1 --2- 172.21.15.1:0/3307154134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffac8109140 0x7ffac819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:38.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.115+0000 7ffac7e00640 1 -- 172.21.15.1:0/3307154134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffac8109140 msgr2=0x7ffac819ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:38.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.115+0000 7ffac7e00640 1 --2- 172.21.15.1:0/3307154134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffac8109140 0x7ffac819ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:38.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.115+0000 7ffac7e00640 1 -- 172.21.15.1:0/3307154134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffac8104010 msgr2=0x7ffac819a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:38.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.115+0000 7ffac7e00640 1 --2- 172.21.15.1:0/3307154134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffac8104010 0x7ffac819a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:38.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.115+0000 7ffac7e00640 1 -- 172.21.15.1:0/3307154134 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffac819f670 con 0x7ffac81049c0 2024-09-12T01:27:38.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.115+0000 7ffaccc00640 1 --2- 172.21.15.1:0/3307154134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffac8104010 0x7ffac819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:27:38.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.115+0000 7ffac7e00640 1 --2- 172.21.15.1:0/3307154134 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffac81049c0 0x7ffac819ac00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ffab400e9f0 tx=0x7ffab400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:38.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.115+0000 7ffac5600640 1 -- 172.21.15.1:0/3307154134 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab400cd80 con 0x7ffac81049c0 2024-09-12T01:27:38.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.115+0000 7ffac5600640 1 -- 172.21.15.1:0/3307154134 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffab4005500 con 0x7ffac81049c0 2024-09-12T01:27:38.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.115+0000 7ffacf600640 1 -- 172.21.15.1:0/3307154134 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffac819f960 con 0x7ffac81049c0 2024-09-12T01:27:38.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.115+0000 7ffac5600640 1 -- 172.21.15.1:0/3307154134 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab4010690 con 0x7ffac81049c0 2024-09-12T01:27:38.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.115+0000 7ffacf600640 1 -- 172.21.15.1:0/3307154134 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffac81a7240 con 0x7ffac81049c0 2024-09-12T01:27:38.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.116+0000 7ffacf600640 1 -- 172.21.15.1:0/3307154134 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffac81043f0 con 0x7ffac81049c0 2024-09-12T01:27:38.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.118+0000 7ffac5600640 1 -- 172.21.15.1:0/3307154134 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7ffab40046c0 con 0x7ffac81049c0 2024-09-12T01:27:38.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.120+0000 7ffac5600640 1 --2- 172.21.15.1:0/3307154134 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ffab00a2b80 0x7ffab00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:38.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.120+0000 7ffac5600640 1 -- 172.21.15.1:0/3307154134 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9092+0+0 (secure 0 0 0) 0x7ffab40cfa30 con 0x7ffac81049c0 2024-09-12T01:27:38.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.120+0000 7ffaccc00640 1 --2- 172.21.15.1:0/3307154134 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ffab00a2b80 0x7ffab00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:38.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.121+0000 7ffaccc00640 1 --2- 172.21.15.1:0/3307154134 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ffab00a2b80 0x7ffab00a5040 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ffabc009fc0 tx=0x7ffabc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:38.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.122+0000 7ffac5600640 1 -- 172.21.15.1:0/3307154134 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffab40541b0 con 0x7ffac81049c0 2024-09-12T01:27:38.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:37 smithi053 ceph-mon[35158]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-12T01:27:38.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.346+0000 7ffacf600640 1 -- 172.21.15.1:0/3307154134 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ffac819ba20 con 0x7ffac81049c0 2024-09-12T01:27:38.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.775+0000 7ffac5600640 1 -- 172.21.15.1:0/3307154134 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7ffab409c5d0 con 0x7ffac81049c0 2024-09-12T01:27:38.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.778+0000 7ffacf600640 1 -- 172.21.15.1:0/3307154134 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ffab00a2b80 msgr2=0x7ffab00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:38.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.778+0000 7ffacf600640 1 --2- 172.21.15.1:0/3307154134 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ffab00a2b80 0x7ffab00a5040 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ffabc009fc0 tx=0x7ffabc005f90 comp rx=0 tx=0).stop 2024-09-12T01:27:38.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.778+0000 7ffacf600640 1 -- 172.21.15.1:0/3307154134 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffac81049c0 msgr2=0x7ffac819ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:38.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.778+0000 7ffacf600640 1 --2- 172.21.15.1:0/3307154134 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffac81049c0 0x7ffac819ac00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ffab400e9f0 tx=0x7ffab400edf0 comp rx=0 tx=0).stop 2024-09-12T01:27:38.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.778+0000 7ffacf600640 1 -- 172.21.15.1:0/3307154134 shutdown_connections 2024-09-12T01:27:38.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.778+0000 7ffacf600640 1 --2- 172.21.15.1:0/3307154134 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ffab00a2b80 0x7ffab00a5040 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:38.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.778+0000 7ffacf600640 1 --2- 172.21.15.1:0/3307154134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffac8109140 0x7ffac819ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:38.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.778+0000 7ffacf600640 1 --2- 172.21.15.1:0/3307154134 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffac81049c0 0x7ffac819ac00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:38.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.778+0000 7ffacf600640 1 --2- 172.21.15.1:0/3307154134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffac8104010 0x7ffac819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:38.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.778+0000 7ffacf600640 1 -- 172.21.15.1:0/3307154134 >> 172.21.15.1:0/3307154134 conn(0x7ffac80ffa50 msgr2=0x7ffac8100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:38.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.778+0000 7ffacf600640 1 -- 172.21.15.1:0/3307154134 shutdown_connections 2024-09-12T01:27:38.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:38.779+0000 7ffacf600640 1 -- 172.21.15.1:0/3307154134 wait complete. 2024-09-12T01:27:39.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:38 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/3307154134' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T01:27:39.120 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:38 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/3307154134' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T01:27:39.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:38 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/3307154134' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T01:27:39.517 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-12T01:27:39.518 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-09-12T01:27:39.897 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:27:40.035 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:39 smithi001 ceph-mon[30387]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:27:40.035 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:39 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/3307154134' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-12T01:27:40.035 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:39 smithi001 ceph-mon[30387]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T01:27:40.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:39 smithi104 ceph-mon[35219]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:27:40.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:39 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/3307154134' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-12T01:27:40.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:39 smithi104 ceph-mon[35219]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T01:27:40.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:39 smithi053 ceph-mon[35158]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:27:40.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:39 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/3307154134' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-12T01:27:40.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:39 smithi053 ceph-mon[35158]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T01:27:40.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:40.658+0000 7fe603e00640 1 -- 172.21.15.1:0/1257926693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6041019c0 msgr2=0x7fe604101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:40.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:40.658+0000 7fe603e00640 1 --2- 172.21.15.1:0/1257926693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6041019c0 0x7fe604101e20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec009930 tx=0x7fe5ec02a420 comp rx=0 tx=0).stop 2024-09-12T01:27:40.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:40.658+0000 7fe603e00640 1 -- 172.21.15.1:0/1257926693 shutdown_connections 2024-09-12T01:27:40.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:40.658+0000 7fe603e00640 1 --2- 172.21.15.1:0/1257926693 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe604102360 0x7fe60410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:40.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:40.658+0000 7fe603e00640 1 --2- 172.21.15.1:0/1257926693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6041019c0 0x7fe604101e20 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:40.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:40.658+0000 7fe603e00640 1 --2- 172.21.15.1:0/1257926693 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6041079c0 0x7fe604107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:40.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:40.658+0000 7fe603e00640 1 -- 172.21.15.1:0/1257926693 >> 172.21.15.1:0/1257926693 conn(0x7fe6040fd740 msgr2=0x7fe6040ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:40.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:40.658+0000 7fe603e00640 1 -- 172.21.15.1:0/1257926693 shutdown_connections 2024-09-12T01:27:40.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:40.659+0000 7fe603e00640 1 -- 172.21.15.1:0/1257926693 wait complete. 2024-09-12T01:27:40.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:40.659+0000 7fe603e00640 1 Processor -- start 2024-09-12T01:27:40.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:40.660+0000 7fe603e00640 1 -- start start 2024-09-12T01:27:40.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:40.660+0000 7fe603e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6041019c0 0x7fe60419a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:40.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:40.660+0000 7fe603e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe604102360 0x7fe60419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:40.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:40.660+0000 7fe603e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6041079c0 0x7fe60419ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:40.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:40.660+0000 7fe603e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe604111e10 con 0x7fe6041019c0 2024-09-12T01:27:40.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:40.660+0000 7fe603e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe604111c90 con 0x7fe604102360 2024-09-12T01:27:40.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:40.660+0000 7fe603e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe604111f90 con 0x7fe6041079c0 2024-09-12T01:27:40.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:40.660+0000 7fe602a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6041019c0 0x7fe60419a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:40.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:40.660+0000 7fe602a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6041019c0 0x7fe60419a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44368/0 (socket says 172.21.15.1:44368) 2024-09-12T01:27:40.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:40.660+0000 7fe602a00640 1 -- 172.21.15.1:0/1349405105 learned_addr learned my addr 172.21.15.1:0/1349405105 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:27:40.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:40.660+0000 7fe603400640 1 --2- 172.21.15.1:0/1349405105 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6041079c0 0x7fe60419ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:40.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:40.660+0000 7fe5fa000640 1 --2- 172.21.15.1:0/1349405105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe604102360 0x7fe60419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:40.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:40.661+0000 7fe602a00640 1 -- 172.21.15.1:0/1349405105 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6041079c0 msgr2=0x7fe60419ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:40.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:40.661+0000 7fe602a00640 1 --2- 172.21.15.1:0/1349405105 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6041079c0 0x7fe60419ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:40.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:40.661+0000 7fe602a00640 1 -- 172.21.15.1:0/1349405105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe604102360 msgr2=0x7fe60419aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:40.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:40.661+0000 7fe602a00640 1 --2- 172.21.15.1:0/1349405105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe604102360 0x7fe60419aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:40.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:40.661+0000 7fe602a00640 1 -- 172.21.15.1:0/1349405105 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe60419f520 con 0x7fe6041019c0 2024-09-12T01:27:40.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:40.661+0000 7fe5fa000640 1 --2- 172.21.15.1:0/1349405105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe604102360 0x7fe60419aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:27:40.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:40.661+0000 7fe603400640 1 --2- 172.21.15.1:0/1349405105 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6041079c0 0x7fe60419ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:27:40.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:40.661+0000 7fe602a00640 1 --2- 172.21.15.1:0/1349405105 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6041019c0 0x7fe60419a570 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe5f400e9f0 tx=0x7fe5f400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:40.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:40.662+0000 7fe608a00640 1 -- 172.21.15.1:0/1349405105 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f400cd80 con 0x7fe6041019c0 2024-09-12T01:27:40.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:40.662+0000 7fe608a00640 1 -- 172.21.15.1:0/1349405105 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5f4010910 con 0x7fe6041019c0 2024-09-12T01:27:40.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:40.662+0000 7fe603e00640 1 -- 172.21.15.1:0/1349405105 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe60419f810 con 0x7fe6041019c0 2024-09-12T01:27:40.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:40.662+0000 7fe608a00640 1 -- 172.21.15.1:0/1349405105 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f4018990 con 0x7fe6041019c0 2024-09-12T01:27:40.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:40.662+0000 7fe603e00640 1 -- 172.21.15.1:0/1349405105 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6041a70f0 con 0x7fe6041019c0 2024-09-12T01:27:40.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:40.663+0000 7fe603e00640 1 -- 172.21.15.1:0/1349405105 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe604101e20 con 0x7fe6041019c0 2024-09-12T01:27:40.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:40.665+0000 7fe608a00640 1 -- 172.21.15.1:0/1349405105 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fe5f4010ac0 con 0x7fe6041019c0 2024-09-12T01:27:40.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:40.666+0000 7fe608a00640 1 --2- 172.21.15.1:0/1349405105 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe5e80a2b80 0x7fe5e80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:40.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:40.666+0000 7fe608a00640 1 -- 172.21.15.1:0/1349405105 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9092+0+0 (secure 0 0 0) 0x7fe5f40cf740 con 0x7fe6041019c0 2024-09-12T01:27:40.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:40.666+0000 7fe5fa000640 1 --2- 172.21.15.1:0/1349405105 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe5e80a2b80 0x7fe5e80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:40.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:40.667+0000 7fe5fa000640 1 --2- 172.21.15.1:0/1349405105 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe5e80a2b80 0x7fe5e80a5040 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec006010 tx=0x7fe5ec005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:40.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:40.670+0000 7fe608a00640 1 -- 172.21.15.1:0/1349405105 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe5f409c2e0 con 0x7fe6041019c0 2024-09-12T01:27:40.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:40.893+0000 7fe603e00640 1 -- 172.21.15.1:0/1349405105 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7fe6041a7320 con 0x7fe6041019c0 2024-09-12T01:27:41.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:41.289+0000 7fe608a00640 1 -- 172.21.15.1:0/1349405105 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7fe5f409bbe0 con 0x7fe6041019c0 2024-09-12T01:27:41.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:41.292+0000 7fe603e00640 1 -- 172.21.15.1:0/1349405105 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe5e80a2b80 msgr2=0x7fe5e80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:41.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:41.292+0000 7fe603e00640 1 --2- 172.21.15.1:0/1349405105 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe5e80a2b80 0x7fe5e80a5040 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec006010 tx=0x7fe5ec005a20 comp rx=0 tx=0).stop 2024-09-12T01:27:41.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:41.292+0000 7fe603e00640 1 -- 172.21.15.1:0/1349405105 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6041019c0 msgr2=0x7fe60419a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:41.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:41.292+0000 7fe603e00640 1 --2- 172.21.15.1:0/1349405105 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6041019c0 0x7fe60419a570 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe5f400e9f0 tx=0x7fe5f400eeb0 comp rx=0 tx=0).stop 2024-09-12T01:27:41.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:41.292+0000 7fe603e00640 1 -- 172.21.15.1:0/1349405105 shutdown_connections 2024-09-12T01:27:41.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:41.292+0000 7fe603e00640 1 --2- 172.21.15.1:0/1349405105 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe5e80a2b80 0x7fe5e80a5040 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:41.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:41.292+0000 7fe603e00640 1 --2- 172.21.15.1:0/1349405105 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6041079c0 0x7fe60419ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:41.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:41.292+0000 7fe603e00640 1 --2- 172.21.15.1:0/1349405105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe604102360 0x7fe60419aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:41.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:41.292+0000 7fe603e00640 1 --2- 172.21.15.1:0/1349405105 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6041019c0 0x7fe60419a570 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:41.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:41.292+0000 7fe603e00640 1 -- 172.21.15.1:0/1349405105 >> 172.21.15.1:0/1349405105 conn(0x7fe6040fd740 msgr2=0x7fe60410d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:41.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:41.292+0000 7fe603e00640 1 -- 172.21.15.1:0/1349405105 shutdown_connections 2024-09-12T01:27:41.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:41.293+0000 7fe603e00640 1 -- 172.21.15.1:0/1349405105 wait complete. 2024-09-12T01:27:42.029 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:41 smithi001 ceph-mon[30387]: pgmap v339: 129 pgs: 129 active+clean; 581 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:27:42.030 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:41 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/1349405105' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-12T01:27:42.030 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:41 smithi001 ceph-mon[30387]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-12T01:27:42.030 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:41 smithi001 ceph-mon[30387]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-12T01:27:42.030 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:41 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/1349405105' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-12T01:27:42.030 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:41 smithi001 ceph-mon[30387]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-12T01:27:42.030 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:41 smithi001 ceph-mon[30387]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-12T01:27:42.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:41 smithi104 ceph-mon[35219]: pgmap v339: 129 pgs: 129 active+clean; 581 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:27:42.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:41 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/1349405105' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-12T01:27:42.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:41 smithi104 ceph-mon[35219]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-12T01:27:42.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:41 smithi104 ceph-mon[35219]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-12T01:27:42.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:41 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/1349405105' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-12T01:27:42.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:41 smithi104 ceph-mon[35219]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-12T01:27:42.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:41 smithi104 ceph-mon[35219]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-12T01:27:42.076 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-12T01:27:42.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:41 smithi053 ceph-mon[35158]: pgmap v339: 129 pgs: 129 active+clean; 581 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:27:42.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:41 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/1349405105' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-12T01:27:42.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:41 smithi053 ceph-mon[35158]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-12T01:27:42.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:41 smithi053 ceph-mon[35158]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-12T01:27:42.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:41 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/1349405105' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-12T01:27:42.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:41 smithi053 ceph-mon[35158]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-12T01:27:42.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:41 smithi053 ceph-mon[35158]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-12T01:27:42.467 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:27:43.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.033+0000 7fa540600640 1 -- 172.21.15.1:0/3037681304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa538077040 msgr2=0x7fa5381124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:43.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.033+0000 7fa540600640 1 --2- 172.21.15.1:0/3037681304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa538077040 0x7fa5381124f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa524009900 tx=0x7fa52402a3f0 comp rx=0 tx=0).stop 2024-09-12T01:27:43.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.033+0000 7fa540600640 1 -- 172.21.15.1:0/3037681304 shutdown_connections 2024-09-12T01:27:43.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.033+0000 7fa540600640 1 --2- 172.21.15.1:0/3037681304 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa538112a30 0x7fa538114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:43.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.033+0000 7fa540600640 1 --2- 172.21.15.1:0/3037681304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa538077040 0x7fa5381124f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:43.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.033+0000 7fa540600640 1 --2- 172.21.15.1:0/3037681304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa538076720 0x7fa538076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:43.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.033+0000 7fa540600640 1 -- 172.21.15.1:0/3037681304 >> 172.21.15.1:0/3037681304 conn(0x7fa5380ffa90 msgr2=0x7fa538101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:43.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.033+0000 7fa540600640 1 -- 172.21.15.1:0/3037681304 shutdown_connections 2024-09-12T01:27:43.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.033+0000 7fa540600640 1 -- 172.21.15.1:0/3037681304 wait complete. 2024-09-12T01:27:43.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.034+0000 7fa540600640 1 Processor -- start 2024-09-12T01:27:43.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.034+0000 7fa540600640 1 -- start start 2024-09-12T01:27:43.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.035+0000 7fa540600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa538076720 0x7fa5381a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:43.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.035+0000 7fa540600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa538077040 0x7fa5381a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:43.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.035+0000 7fa540600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa538112a30 0x7fa53819e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:43.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.035+0000 7fa540600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa538118600 con 0x7fa538077040 2024-09-12T01:27:43.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.035+0000 7fa540600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa538118480 con 0x7fa538076720 2024-09-12T01:27:43.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.035+0000 7fa540600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa538118780 con 0x7fa538112a30 2024-09-12T01:27:43.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.035+0000 7fa53d200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa538077040 0x7fa5381a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:43.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.035+0000 7fa53dc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa538076720 0x7fa5381a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:43.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.035+0000 7fa53dc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa538076720 0x7fa5381a4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:42220/0 (socket says 172.21.15.1:42220) 2024-09-12T01:27:43.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.035+0000 7fa53dc00640 1 -- 172.21.15.1:0/312483409 learned_addr learned my addr 172.21.15.1:0/312483409 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:27:43.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.035+0000 7fa53e600640 1 --2- 172.21.15.1:0/312483409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa538112a30 0x7fa53819e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:43.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.035+0000 7fa53dc00640 1 -- 172.21.15.1:0/312483409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa538112a30 msgr2=0x7fa53819e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:43.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.035+0000 7fa53dc00640 1 --2- 172.21.15.1:0/312483409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa538112a30 0x7fa53819e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:43.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.035+0000 7fa53dc00640 1 -- 172.21.15.1:0/312483409 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa538077040 msgr2=0x7fa5381a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:43.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.035+0000 7fa53dc00640 1 --2- 172.21.15.1:0/312483409 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa538077040 0x7fa5381a4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:43.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.035+0000 7fa53dc00640 1 -- 172.21.15.1:0/312483409 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa53819eff0 con 0x7fa538076720 2024-09-12T01:27:43.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.036+0000 7fa53dc00640 1 --2- 172.21.15.1:0/312483409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa538076720 0x7fa5381a4700 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa52c00ea60 tx=0x7fa52c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:43.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.036+0000 7fa536000640 1 -- 172.21.15.1:0/312483409 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52c00cde0 con 0x7fa538076720 2024-09-12T01:27:43.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.036+0000 7fa536000640 1 -- 172.21.15.1:0/312483409 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa52c0035f0 con 0x7fa538076720 2024-09-12T01:27:43.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.036+0000 7fa540600640 1 -- 172.21.15.1:0/312483409 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa53819f2e0 con 0x7fa538076720 2024-09-12T01:27:43.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.036+0000 7fa536000640 1 -- 172.21.15.1:0/312483409 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52c010670 con 0x7fa538076720 2024-09-12T01:27:43.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.036+0000 7fa540600640 1 -- 172.21.15.1:0/312483409 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5381ab5d0 con 0x7fa538076720 2024-09-12T01:27:43.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.037+0000 7fa540600640 1 -- 172.21.15.1:0/312483409 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa538076b00 con 0x7fa538076720 2024-09-12T01:27:43.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.039+0000 7fa536000640 1 -- 172.21.15.1:0/312483409 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fa52c0037a0 con 0x7fa538076720 2024-09-12T01:27:43.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.041+0000 7fa536000640 1 --2- 172.21.15.1:0/312483409 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fa5200a2b80 0x7fa5200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:43.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.041+0000 7fa536000640 1 -- 172.21.15.1:0/312483409 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9092+0+0 (secure 0 0 0) 0x7fa52c0cf770 con 0x7fa538076720 2024-09-12T01:27:43.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.041+0000 7fa53d200640 1 --2- 172.21.15.1:0/312483409 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fa5200a2b80 0x7fa5200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:43.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.041+0000 7fa53d200640 1 --2- 172.21.15.1:0/312483409 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fa5200a2b80 0x7fa5200a5040 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa524000c00 tx=0x7fa524005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:43.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.043+0000 7fa536000640 1 -- 172.21.15.1:0/312483409 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa52c09c3a0 con 0x7fa538076720 2024-09-12T01:27:43.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.269+0000 7fa540600640 1 -- 172.21.15.1:0/312483409 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fa53819fdf0 con 0x7fa538076720 2024-09-12T01:27:43.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.305+0000 7fa536000640 1 -- 172.21.15.1:0/312483409 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7fa53819fdf0 con 0x7fa538076720 2024-09-12T01:27:43.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.308+0000 7fa540600640 1 -- 172.21.15.1:0/312483409 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fa5200a2b80 msgr2=0x7fa5200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:43.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.308+0000 7fa540600640 1 --2- 172.21.15.1:0/312483409 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fa5200a2b80 0x7fa5200a5040 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa524000c00 tx=0x7fa524005b50 comp rx=0 tx=0).stop 2024-09-12T01:27:43.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.308+0000 7fa540600640 1 -- 172.21.15.1:0/312483409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa538076720 msgr2=0x7fa5381a4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:43.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.308+0000 7fa540600640 1 --2- 172.21.15.1:0/312483409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa538076720 0x7fa5381a4700 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa52c00ea60 tx=0x7fa52c00ef20 comp rx=0 tx=0).stop 2024-09-12T01:27:43.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.308+0000 7fa540600640 1 -- 172.21.15.1:0/312483409 shutdown_connections 2024-09-12T01:27:43.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.308+0000 7fa540600640 1 --2- 172.21.15.1:0/312483409 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fa5200a2b80 0x7fa5200a5040 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:43.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.308+0000 7fa540600640 1 --2- 172.21.15.1:0/312483409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa538112a30 0x7fa53819e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:43.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.308+0000 7fa540600640 1 --2- 172.21.15.1:0/312483409 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa538077040 0x7fa5381a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:43.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.308+0000 7fa540600640 1 --2- 172.21.15.1:0/312483409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa538076720 0x7fa5381a4700 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:43.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.308+0000 7fa540600640 1 -- 172.21.15.1:0/312483409 >> 172.21.15.1:0/312483409 conn(0x7fa5380ffa90 msgr2=0x7fa538101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:43.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.309+0000 7fa540600640 1 -- 172.21.15.1:0/312483409 shutdown_connections 2024-09-12T01:27:43.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:43.309+0000 7fa540600640 1 -- 172.21.15.1:0/312483409 wait complete. 2024-09-12T01:27:43.542 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:43 smithi001 ceph-mon[30387]: mds.? [v2:172.21.15.104:6838/1459976041,v1:172.21.15.104:6839/1459976041] up:active 2024-09-12T01:27:43.542 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:43 smithi001 ceph-mon[30387]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-12T01:27:43.542 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:43 smithi001 ceph-mon[30387]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-12T01:27:43.542 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:43 smithi001 ceph-mon[30387]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-12T01:27:43.543 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:43 smithi001 ceph-mon[30387]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-12T01:27:43.543 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:43 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:27:43.543 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:43 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/312483409' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T01:27:43.543 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:43 smithi001 ceph-mon[30387]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T01:27:43.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:43 smithi104 ceph-mon[35219]: mds.? [v2:172.21.15.104:6838/1459976041,v1:172.21.15.104:6839/1459976041] up:active 2024-09-12T01:27:43.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:43 smithi104 ceph-mon[35219]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-12T01:27:43.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:43 smithi104 ceph-mon[35219]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-12T01:27:43.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:43 smithi104 ceph-mon[35219]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-12T01:27:43.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:43 smithi104 ceph-mon[35219]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-12T01:27:43.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:43 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:27:43.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:43 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/312483409' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T01:27:43.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:43 smithi104 ceph-mon[35219]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T01:27:43.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:43 smithi053 ceph-mon[35158]: mds.? [v2:172.21.15.104:6838/1459976041,v1:172.21.15.104:6839/1459976041] up:active 2024-09-12T01:27:43.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:43 smithi053 ceph-mon[35158]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-12T01:27:43.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:43 smithi053 ceph-mon[35158]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-12T01:27:43.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:43 smithi053 ceph-mon[35158]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-12T01:27:43.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:43 smithi053 ceph-mon[35158]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-12T01:27:43.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:43 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:27:43.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:43 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/312483409' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T01:27:43.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:43 smithi053 ceph-mon[35158]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T01:27:44.208 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-12T01:27:44.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:44 smithi001 ceph-mon[30387]: pgmap v340: 129 pgs: 129 active+clean; 581 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-12T01:27:44.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:44 smithi001 ceph-mon[30387]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-12T01:27:44.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:44 smithi001 ceph-mon[30387]: mds.? [v2:172.21.15.53:6836/2848743754,v1:172.21.15.53:6837/2848743754] up:active 2024-09-12T01:27:44.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:44 smithi001 ceph-mon[30387]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-12T01:27:44.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:44 smithi001 ceph-mon[30387]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-12T01:27:44.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:44 smithi001 ceph-mon[30387]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-12T01:27:44.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:44 smithi001 ceph-mon[30387]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-12T01:27:44.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:44 smithi053 ceph-mon[35158]: pgmap v340: 129 pgs: 129 active+clean; 581 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-12T01:27:44.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:44 smithi053 ceph-mon[35158]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-12T01:27:44.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:44 smithi053 ceph-mon[35158]: mds.? [v2:172.21.15.53:6836/2848743754,v1:172.21.15.53:6837/2848743754] up:active 2024-09-12T01:27:44.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:44 smithi053 ceph-mon[35158]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-12T01:27:44.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:44 smithi053 ceph-mon[35158]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-12T01:27:44.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:44 smithi053 ceph-mon[35158]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-12T01:27:44.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:44 smithi053 ceph-mon[35158]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-12T01:27:44.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:44 smithi104 ceph-mon[35219]: pgmap v340: 129 pgs: 129 active+clean; 581 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-12T01:27:44.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:44 smithi104 ceph-mon[35219]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-12T01:27:44.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:44 smithi104 ceph-mon[35219]: mds.? [v2:172.21.15.53:6836/2848743754,v1:172.21.15.53:6837/2848743754] up:active 2024-09-12T01:27:44.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:44 smithi104 ceph-mon[35219]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-12T01:27:44.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:44 smithi104 ceph-mon[35219]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-12T01:27:44.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:44 smithi104 ceph-mon[35219]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-12T01:27:44.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:44 smithi104 ceph-mon[35219]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-12T01:27:45.219 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:27:45.563 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:45 smithi001 ceph-mon[30387]: mds.? [v2:172.21.15.104:6836/464720247,v1:172.21.15.104:6837/464720247] up:active 2024-09-12T01:27:45.563 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:45 smithi001 ceph-mon[30387]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-12T01:27:45.563 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:45 smithi001 ceph-mon[30387]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-12T01:27:45.563 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:45 smithi001 ceph-mon[30387]: Cluster is now healthy 2024-09-12T01:27:45.564 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:45 smithi001 ceph-mon[30387]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-09-12T01:27:45.564 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:45 smithi001 ceph-mon[30387]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-12T01:27:45.564 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:45 smithi001 ceph-mon[30387]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-12T01:27:45.564 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:45 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:27:45.564 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:45 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:27:45.564 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:45 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:27:45.564 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:45 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:27:45.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:45 smithi104 ceph-mon[35219]: mds.? [v2:172.21.15.104:6836/464720247,v1:172.21.15.104:6837/464720247] up:active 2024-09-12T01:27:45.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:45 smithi104 ceph-mon[35219]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-12T01:27:45.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:45 smithi104 ceph-mon[35219]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-12T01:27:45.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:45 smithi104 ceph-mon[35219]: Cluster is now healthy 2024-09-12T01:27:45.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:45 smithi104 ceph-mon[35219]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-09-12T01:27:45.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:45 smithi104 ceph-mon[35219]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-12T01:27:45.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:45 smithi104 ceph-mon[35219]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-12T01:27:45.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:45 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:27:45.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:45 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:27:45.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:45 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:27:45.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:45 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:27:45.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:45 smithi053 ceph-mon[35158]: mds.? [v2:172.21.15.104:6836/464720247,v1:172.21.15.104:6837/464720247] up:active 2024-09-12T01:27:45.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:45 smithi053 ceph-mon[35158]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-12T01:27:45.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:45 smithi053 ceph-mon[35158]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-12T01:27:45.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:45 smithi053 ceph-mon[35158]: Cluster is now healthy 2024-09-12T01:27:45.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:45 smithi053 ceph-mon[35158]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-09-12T01:27:45.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:45 smithi053 ceph-mon[35158]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-12T01:27:45.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:45 smithi053 ceph-mon[35158]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-12T01:27:45.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:45 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:27:45.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:45 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:27:45.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:45 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:27:45.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:45 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:27:45.741 INFO:journalctl@ceph.mds.b.smithi053.stdout:Sep 12 01:27:45 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-b[65497]: 2024-09-12T01:27:45.430+0000 7fcfc64e1640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-12T01:27:45.741 INFO:journalctl@ceph.mds.b.smithi053.stdout:Sep 12 01:27:45 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-b[65497]: 2024-09-12T01:27:45.430+0000 7fcfc64e1640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-12T01:27:45.741 INFO:journalctl@ceph.mds.b.smithi053.stdout:Sep 12 01:27:45 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-b[65497]: 2024-09-12T01:27:45.430+0000 7fcfc64e1640 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-09-12T01:27:45.741 INFO:journalctl@ceph.mds.b.smithi053.stdout:Sep 12 01:27:45 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-b[65497]: 2024-09-12T01:27:45.430+0000 7fcfc64e1640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-09-12T01:27:45.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:45.942+0000 7f6d99e00640 1 -- 172.21.15.1:0/2692081461 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d94112a30 msgr2=0x7f6d94114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:45.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:45.942+0000 7f6d99e00640 1 --2- 172.21.15.1:0/2692081461 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d94112a30 0x7f6d94114e20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c009ca0 tx=0x7f6d8c02a8c0 comp rx=0 tx=0).stop 2024-09-12T01:27:45.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:45.942+0000 7f6d99e00640 1 -- 172.21.15.1:0/2692081461 shutdown_connections 2024-09-12T01:27:45.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:45.942+0000 7f6d99e00640 1 --2- 172.21.15.1:0/2692081461 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d94112a30 0x7f6d94114e20 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:45.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:45.942+0000 7f6d99e00640 1 --2- 172.21.15.1:0/2692081461 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d94077040 0x7f6d941124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:45.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:45.942+0000 7f6d99e00640 1 --2- 172.21.15.1:0/2692081461 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d94076720 0x7f6d94076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:45.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:45.942+0000 7f6d99e00640 1 -- 172.21.15.1:0/2692081461 >> 172.21.15.1:0/2692081461 conn(0x7f6d940ffa90 msgr2=0x7f6d94101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:45.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:45.942+0000 7f6d99e00640 1 -- 172.21.15.1:0/2692081461 shutdown_connections 2024-09-12T01:27:45.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:45.942+0000 7f6d99e00640 1 -- 172.21.15.1:0/2692081461 wait complete. 2024-09-12T01:27:45.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:45.943+0000 7f6d99e00640 1 Processor -- start 2024-09-12T01:27:45.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:45.943+0000 7f6d99e00640 1 -- start start 2024-09-12T01:27:45.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:45.944+0000 7f6d99e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d94076720 0x7f6d941a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:45.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:45.944+0000 7f6d99e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d94077040 0x7f6d941a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:45.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:45.944+0000 7f6d99e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d94112a30 0x7f6d9419e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:45.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:45.944+0000 7f6d99e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6d941185e0 con 0x7f6d94077040 2024-09-12T01:27:45.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:45.944+0000 7f6d99e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6d94118460 con 0x7f6d94076720 2024-09-12T01:27:45.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:45.944+0000 7f6d99e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6d94118760 con 0x7f6d94112a30 2024-09-12T01:27:45.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:45.944+0000 7f6d8be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d94077040 0x7f6d941a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:45.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:45.944+0000 7f6d92a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d94076720 0x7f6d941a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:45.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:45.944+0000 7f6d8be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d94077040 0x7f6d941a4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44410/0 (socket says 172.21.15.1:44410) 2024-09-12T01:27:45.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:45.944+0000 7f6d8be00640 1 -- 172.21.15.1:0/2439789916 learned_addr learned my addr 172.21.15.1:0/2439789916 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:27:45.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:45.945+0000 7f6d93400640 1 --2- 172.21.15.1:0/2439789916 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d94112a30 0x7f6d9419e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:45.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:45.945+0000 7f6d8be00640 1 -- 172.21.15.1:0/2439789916 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d94112a30 msgr2=0x7f6d9419e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:45.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:45.945+0000 7f6d8be00640 1 --2- 172.21.15.1:0/2439789916 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d94112a30 0x7f6d9419e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:45.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:45.945+0000 7f6d8be00640 1 -- 172.21.15.1:0/2439789916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d94076720 msgr2=0x7f6d941a4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:45.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:45.945+0000 7f6d8be00640 1 --2- 172.21.15.1:0/2439789916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d94076720 0x7f6d941a4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:45.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:45.945+0000 7f6d8be00640 1 -- 172.21.15.1:0/2439789916 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d9419f040 con 0x7f6d94077040 2024-09-12T01:27:45.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:45.945+0000 7f6d92a00640 1 --2- 172.21.15.1:0/2439789916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d94076720 0x7f6d941a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:27:45.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:45.945+0000 7f6d8be00640 1 --2- 172.21.15.1:0/2439789916 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d94077040 0x7f6d941a4c90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c00e9f0 tx=0x7f6d7c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:45.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:45.946+0000 7f6d98a00640 1 -- 172.21.15.1:0/2439789916 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c00cd80 con 0x7f6d94077040 2024-09-12T01:27:45.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:45.946+0000 7f6d98a00640 1 -- 172.21.15.1:0/2439789916 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d7c005500 con 0x7f6d94077040 2024-09-12T01:27:45.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:45.946+0000 7f6d99e00640 1 -- 172.21.15.1:0/2439789916 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d9419f330 con 0x7f6d94077040 2024-09-12T01:27:45.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:45.946+0000 7f6d98a00640 1 -- 172.21.15.1:0/2439789916 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c010690 con 0x7f6d94077040 2024-09-12T01:27:45.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:45.946+0000 7f6d99e00640 1 -- 172.21.15.1:0/2439789916 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d941ab620 con 0x7f6d94077040 2024-09-12T01:27:45.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:45.947+0000 7f6d99e00640 1 -- 172.21.15.1:0/2439789916 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d94076b00 con 0x7f6d94077040 2024-09-12T01:27:45.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:45.949+0000 7f6d98a00640 1 -- 172.21.15.1:0/2439789916 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f6d7c0046c0 con 0x7f6d94077040 2024-09-12T01:27:45.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:45.950+0000 7f6d98a00640 1 --2- 172.21.15.1:0/2439789916 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f6d780a2b80 0x7f6d780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:45.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:45.951+0000 7f6d98a00640 1 -- 172.21.15.1:0/2439789916 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9092+0+0 (secure 0 0 0) 0x7f6d7c0cf850 con 0x7f6d94077040 2024-09-12T01:27:45.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:45.951+0000 7f6d92a00640 1 --2- 172.21.15.1:0/2439789916 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f6d780a2b80 0x7f6d780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:45.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:45.952+0000 7f6d92a00640 1 --2- 172.21.15.1:0/2439789916 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f6d780a2b80 0x7f6d780a5040 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6d9419fa50 tx=0x7f6d84005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:45.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:45.955+0000 7f6d98a00640 1 -- 172.21.15.1:0/2439789916 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6d7c09c3f0 con 0x7f6d94077040 2024-09-12T01:27:46.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:46.204+0000 7f6d99e00640 1 -- 172.21.15.1:0/2439789916 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f6d9419fe40 con 0x7f6d94077040 2024-09-12T01:27:46.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:46.332+0000 7f6d98a00640 1 -- 172.21.15.1:0/2439789916 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v28) ==== 115+0+0 (secure 0 0 0) 0x7f6d9419fe40 con 0x7f6d94077040 2024-09-12T01:27:46.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:46.335+0000 7f6d99e00640 1 -- 172.21.15.1:0/2439789916 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f6d780a2b80 msgr2=0x7f6d780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:46.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:46.335+0000 7f6d99e00640 1 --2- 172.21.15.1:0/2439789916 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f6d780a2b80 0x7f6d780a5040 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6d9419fa50 tx=0x7f6d84005f90 comp rx=0 tx=0).stop 2024-09-12T01:27:46.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:46.335+0000 7f6d99e00640 1 -- 172.21.15.1:0/2439789916 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d94077040 msgr2=0x7f6d941a4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:46.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:46.335+0000 7f6d99e00640 1 --2- 172.21.15.1:0/2439789916 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d94077040 0x7f6d941a4c90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c00e9f0 tx=0x7f6d7c00edf0 comp rx=0 tx=0).stop 2024-09-12T01:27:46.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:46.336+0000 7f6d99e00640 1 -- 172.21.15.1:0/2439789916 shutdown_connections 2024-09-12T01:27:46.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:46.336+0000 7f6d99e00640 1 --2- 172.21.15.1:0/2439789916 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f6d780a2b80 0x7f6d780a5040 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:46.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:46.336+0000 7f6d99e00640 1 --2- 172.21.15.1:0/2439789916 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d94112a30 0x7f6d9419e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:46.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:46.336+0000 7f6d99e00640 1 --2- 172.21.15.1:0/2439789916 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d94077040 0x7f6d941a4c90 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:46.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:46.336+0000 7f6d99e00640 1 --2- 172.21.15.1:0/2439789916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d94076720 0x7f6d941a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:46.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:46.336+0000 7f6d99e00640 1 -- 172.21.15.1:0/2439789916 >> 172.21.15.1:0/2439789916 conn(0x7f6d940ffa90 msgr2=0x7f6d94113630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:46.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:46.336+0000 7f6d99e00640 1 -- 172.21.15.1:0/2439789916 shutdown_connections 2024-09-12T01:27:46.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:46.336+0000 7f6d99e00640 1 -- 172.21.15.1:0/2439789916 wait complete. 2024-09-12T01:27:46.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:46 smithi104 ceph-mon[35219]: pgmap v341: 129 pgs: 129 active+clean; 582 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 4 op/s 2024-09-12T01:27:46.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:46 smithi104 ceph-mon[35219]: mds.? [v2:172.21.15.104:6834/1547038540,v1:172.21.15.104:6835/1547038540] up:active 2024-09-12T01:27:46.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:46 smithi104 ceph-mon[35219]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-12T01:27:46.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:46 smithi104 ceph-mon[35219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T01:27:46.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:46 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/2439789916' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-12T01:27:46.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:46 smithi001 ceph-mon[30387]: pgmap v341: 129 pgs: 129 active+clean; 582 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 4 op/s 2024-09-12T01:27:46.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:46 smithi001 ceph-mon[30387]: mds.? [v2:172.21.15.104:6834/1547038540,v1:172.21.15.104:6835/1547038540] up:active 2024-09-12T01:27:46.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:46 smithi001 ceph-mon[30387]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-12T01:27:46.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:46 smithi001 ceph-mon[30387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T01:27:46.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:46 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/2439789916' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-12T01:27:46.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:46 smithi053 ceph-mon[35158]: pgmap v341: 129 pgs: 129 active+clean; 582 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 4 op/s 2024-09-12T01:27:46.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:46 smithi053 ceph-mon[35158]: mds.? [v2:172.21.15.104:6834/1547038540,v1:172.21.15.104:6835/1547038540] up:active 2024-09-12T01:27:46.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:46 smithi053 ceph-mon[35158]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-12T01:27:46.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:46 smithi053 ceph-mon[35158]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T01:27:46.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:46 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/2439789916' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-12T01:27:47.170 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-09-12T01:27:47.170 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-12T01:27:47.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:47 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/2439789916' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-12T01:27:47.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:47 smithi001 ceph-mon[30387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T01:27:47.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:47 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:27:47.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:47 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:27:47.590 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:27:47.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:47 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/2439789916' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-12T01:27:47.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:47 smithi053 ceph-mon[35158]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T01:27:47.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:47 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:27:47.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:47 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:27:47.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:47 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/2439789916' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-12T01:27:47.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:47 smithi104 ceph-mon[35219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T01:27:47.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:47 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:27:47.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:47 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:27:48.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.249+0000 7f9057c00640 1 -- 172.21.15.1:0/993124348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9050101de0 msgr2=0x7f90501021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:48.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.249+0000 7f9057c00640 1 --2- 172.21.15.1:0/993124348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9050101de0 0x7f90501021c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9040009900 tx=0x7f904002a3f0 comp rx=0 tx=0).stop 2024-09-12T01:27:48.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.249+0000 7f9057c00640 1 -- 172.21.15.1:0/993124348 shutdown_connections 2024-09-12T01:27:48.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.249+0000 7f9057c00640 1 --2- 172.21.15.1:0/993124348 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9050106f10 0x7f905010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:48.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.249+0000 7f9057c00640 1 --2- 172.21.15.1:0/993124348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9050102790 0x7f90501067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:48.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.249+0000 7f9057c00640 1 --2- 172.21.15.1:0/993124348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9050101de0 0x7f90501021c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:48.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.249+0000 7f9057c00640 1 -- 172.21.15.1:0/993124348 >> 172.21.15.1:0/993124348 conn(0x7f90500fd8c0 msgr2=0x7f90500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:48.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.249+0000 7f9057c00640 1 -- 172.21.15.1:0/993124348 shutdown_connections 2024-09-12T01:27:48.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.249+0000 7f9057c00640 1 -- 172.21.15.1:0/993124348 wait complete. 2024-09-12T01:27:48.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.250+0000 7f9057c00640 1 Processor -- start 2024-09-12T01:27:48.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.250+0000 7f9057c00640 1 -- start start 2024-09-12T01:27:48.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.250+0000 7f9057c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9050101de0 0x7f905019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:48.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.251+0000 7f9057c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9050102790 0x7f905019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:48.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.251+0000 7f9057c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9050106f10 0x7f905019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:48.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.251+0000 7f9057c00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9050111fc0 con 0x7f9050101de0 2024-09-12T01:27:48.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.251+0000 7f9057c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9050111e40 con 0x7f9050106f10 2024-09-12T01:27:48.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.251+0000 7f9057c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9050112140 con 0x7f9050102790 2024-09-12T01:27:48.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.251+0000 7f9055200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9050101de0 0x7f905019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:48.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.251+0000 7f9055200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9050101de0 0x7f905019a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44428/0 (socket says 172.21.15.1:44428) 2024-09-12T01:27:48.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.251+0000 7f9055200640 1 -- 172.21.15.1:0/942877737 learned_addr learned my addr 172.21.15.1:0/942877737 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:27:48.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.251+0000 7f9055c00640 1 --2- 172.21.15.1:0/942877737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9050106f10 0x7f905019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:48.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.251+0000 7f9047e00640 1 --2- 172.21.15.1:0/942877737 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9050102790 0x7f905019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:48.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.251+0000 7f9055200640 1 -- 172.21.15.1:0/942877737 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9050102790 msgr2=0x7f905019ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:48.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.251+0000 7f9055200640 1 --2- 172.21.15.1:0/942877737 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9050102790 0x7f905019ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:48.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.251+0000 7f9055200640 1 -- 172.21.15.1:0/942877737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9050106f10 msgr2=0x7f905019eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:48.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.251+0000 7f9055200640 1 --2- 172.21.15.1:0/942877737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9050106f10 0x7f905019eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:48.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.251+0000 7f9055200640 1 -- 172.21.15.1:0/942877737 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f905019f5d0 con 0x7f9050101de0 2024-09-12T01:27:48.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.252+0000 7f9055200640 1 --2- 172.21.15.1:0/942877737 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9050101de0 0x7f905019a620 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9040004770 tx=0x7f9040003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:48.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.252+0000 7f904e000640 1 -- 172.21.15.1:0/942877737 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904003f070 con 0x7f9050101de0 2024-09-12T01:27:48.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.252+0000 7f904e000640 1 -- 172.21.15.1:0/942877737 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9040031ca0 con 0x7f9050101de0 2024-09-12T01:27:48.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.252+0000 7f904e000640 1 -- 172.21.15.1:0/942877737 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9040035030 con 0x7f9050101de0 2024-09-12T01:27:48.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.252+0000 7f9057c00640 1 -- 172.21.15.1:0/942877737 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f905019f860 con 0x7f9050101de0 2024-09-12T01:27:48.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.252+0000 7f9057c00640 1 -- 172.21.15.1:0/942877737 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f905019fcf0 con 0x7f9050101de0 2024-09-12T01:27:48.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.253+0000 7f9057c00640 1 -- 172.21.15.1:0/942877737 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90501021c0 con 0x7f9050101de0 2024-09-12T01:27:48.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.255+0000 7f904e000640 1 -- 172.21.15.1:0/942877737 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f9040042070 con 0x7f9050101de0 2024-09-12T01:27:48.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.256+0000 7f904e000640 1 --2- 172.21.15.1:0/942877737 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f90340a2b80 0x7f90340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:48.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.256+0000 7f904e000640 1 -- 172.21.15.1:0/942877737 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9092+0+0 (secure 0 0 0) 0x7f90400efa60 con 0x7f9050101de0 2024-09-12T01:27:48.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.256+0000 7f9047e00640 1 --2- 172.21.15.1:0/942877737 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f90340a2b80 0x7f90340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:48.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.257+0000 7f9047e00640 1 --2- 172.21.15.1:0/942877737 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f90340a2b80 0x7f90340a5040 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9050076480 tx=0x7f9038005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:48.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.260+0000 7f904e000640 1 -- 172.21.15.1:0/942877737 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f90400bbfb0 con 0x7f9050101de0 2024-09-12T01:27:48.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:48 smithi001 ceph-mon[30387]: pgmap v342: 129 pgs: 129 active+clean; 585 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2024-09-12T01:27:48.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:48 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:27:48.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:48 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:27:48.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:48 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:27:48.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.474+0000 7f9057c00640 1 -- 172.21.15.1:0/942877737 --> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f905010de00 con 0x7f90340a2b80 2024-09-12T01:27:48.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.508+0000 7f904e000640 1 -- 172.21.15.1:0/942877737 <== mgr.14150 v2:172.21.15.1:6800/1928662456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f905010de00 con 0x7f90340a2b80 2024-09-12T01:27:48.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.510+0000 7f9057c00640 1 -- 172.21.15.1:0/942877737 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f90340a2b80 msgr2=0x7f90340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:48.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.510+0000 7f9057c00640 1 --2- 172.21.15.1:0/942877737 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f90340a2b80 0x7f90340a5040 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9050076480 tx=0x7f9038005f90 comp rx=0 tx=0).stop 2024-09-12T01:27:48.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.510+0000 7f9057c00640 1 -- 172.21.15.1:0/942877737 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9050101de0 msgr2=0x7f905019a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:48.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.510+0000 7f9057c00640 1 --2- 172.21.15.1:0/942877737 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9050101de0 0x7f905019a620 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9040004770 tx=0x7f9040003c10 comp rx=0 tx=0).stop 2024-09-12T01:27:48.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.511+0000 7f9057c00640 1 -- 172.21.15.1:0/942877737 shutdown_connections 2024-09-12T01:27:48.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.511+0000 7f9057c00640 1 --2- 172.21.15.1:0/942877737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9050106f10 0x7f905019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:48.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.511+0000 7f9057c00640 1 --2- 172.21.15.1:0/942877737 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f90340a2b80 0x7f90340a5040 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:48.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.511+0000 7f9057c00640 1 --2- 172.21.15.1:0/942877737 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9050102790 0x7f905019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:48.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.511+0000 7f9057c00640 1 --2- 172.21.15.1:0/942877737 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9050101de0 0x7f905019a620 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:48.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.511+0000 7f9057c00640 1 -- 172.21.15.1:0/942877737 >> 172.21.15.1:0/942877737 conn(0x7f90500fd8c0 msgr2=0x7f90500feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:48.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.511+0000 7f9057c00640 1 -- 172.21.15.1:0/942877737 shutdown_connections 2024-09-12T01:27:48.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:48.511+0000 7f9057c00640 1 -- 172.21.15.1:0/942877737 wait complete. 2024-09-12T01:27:48.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:48 smithi053 ceph-mon[35158]: pgmap v342: 129 pgs: 129 active+clean; 585 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2024-09-12T01:27:48.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:48 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:27:48.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:48 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:27:48.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:48 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:27:48.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:48 smithi104 ceph-mon[35219]: pgmap v342: 129 pgs: 129 active+clean; 585 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2024-09-12T01:27:48.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:48 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:27:48.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:48 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:27:48.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:48 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:27:48.928 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:27:48 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:27:48.480+0000 7feafdc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T01:27:48.928 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:27:48 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:27:48.480+0000 7feafdc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T01:27:48.928 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:27:48 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:27:48.480+0000 7feafdc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T01:27:48.928 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:27:48 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:27:48.480+0000 7feafdc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T01:27:48.928 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:27:48 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:27:48.480+0000 7feafdc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T01:27:49.357 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-12T01:27:49.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:49 smithi001 ceph-mon[30387]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:27:49.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:49 smithi053 ceph-mon[35158]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:27:49.742 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:27:49.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:49 smithi104 ceph-mon[35219]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:27:50.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.303+0000 7ff7d4000640 1 -- 172.21.15.1:0/3972384885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7cc102790 msgr2=0x7ff7cc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:50.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.303+0000 7ff7d4000640 1 --2- 172.21.15.1:0/3972384885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7cc102790 0x7ff7cc1067e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8009900 tx=0x7ff7b802a3f0 comp rx=0 tx=0).stop 2024-09-12T01:27:50.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.303+0000 7ff7d4000640 1 -- 172.21.15.1:0/3972384885 shutdown_connections 2024-09-12T01:27:50.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.303+0000 7ff7d4000640 1 --2- 172.21.15.1:0/3972384885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7cc106f10 0x7ff7cc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:50.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.303+0000 7ff7d4000640 1 --2- 172.21.15.1:0/3972384885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7cc102790 0x7ff7cc1067e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:50.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.303+0000 7ff7d4000640 1 --2- 172.21.15.1:0/3972384885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7cc101de0 0x7ff7cc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:50.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.303+0000 7ff7d4000640 1 -- 172.21.15.1:0/3972384885 >> 172.21.15.1:0/3972384885 conn(0x7ff7cc0fd8c0 msgr2=0x7ff7cc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:50.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.303+0000 7ff7d4000640 1 -- 172.21.15.1:0/3972384885 shutdown_connections 2024-09-12T01:27:50.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.303+0000 7ff7d4000640 1 -- 172.21.15.1:0/3972384885 wait complete. 2024-09-12T01:27:50.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.304+0000 7ff7d4000640 1 Processor -- start 2024-09-12T01:27:50.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.304+0000 7ff7d4000640 1 -- start start 2024-09-12T01:27:50.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.305+0000 7ff7d4000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7cc101de0 0x7ff7cc19a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:50.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.305+0000 7ff7d4000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7cc102790 0x7ff7cc19abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:50.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.305+0000 7ff7d4000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7cc106f10 0x7ff7cc19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:50.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.305+0000 7ff7d4000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff7cc111fc0 con 0x7ff7cc106f10 2024-09-12T01:27:50.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.305+0000 7ff7d4000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff7cc111e40 con 0x7ff7cc102790 2024-09-12T01:27:50.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.305+0000 7ff7d4000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff7cc112140 con 0x7ff7cc101de0 2024-09-12T01:27:50.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.305+0000 7ff7d2000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7cc106f10 0x7ff7cc19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:50.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.305+0000 7ff7d2000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7cc106f10 0x7ff7cc19ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59030/0 (socket says 172.21.15.1:59030) 2024-09-12T01:27:50.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.305+0000 7ff7d1600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7cc101de0 0x7ff7cc19a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:50.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.305+0000 7ff7d2000640 1 -- 172.21.15.1:0/1768280696 learned_addr learned my addr 172.21.15.1:0/1768280696 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:27:50.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.305+0000 7ff7d0c00640 1 --2- 172.21.15.1:0/1768280696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7cc102790 0x7ff7cc19abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:50.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.305+0000 7ff7d1600640 1 -- 172.21.15.1:0/1768280696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7cc102790 msgr2=0x7ff7cc19abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:50.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.305+0000 7ff7d1600640 1 --2- 172.21.15.1:0/1768280696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7cc102790 0x7ff7cc19abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:50.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.305+0000 7ff7d1600640 1 -- 172.21.15.1:0/1768280696 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7cc106f10 msgr2=0x7ff7cc19ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:50.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.305+0000 7ff7d1600640 1 --2- 172.21.15.1:0/1768280696 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7cc106f10 0x7ff7cc19ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:50.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.305+0000 7ff7d1600640 1 -- 172.21.15.1:0/1768280696 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7cc19f630 con 0x7ff7cc101de0 2024-09-12T01:27:50.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.306+0000 7ff7d2000640 1 --2- 172.21.15.1:0/1768280696 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7cc106f10 0x7ff7cc19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:27:50.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.306+0000 7ff7d1600640 1 --2- 172.21.15.1:0/1768280696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7cc101de0 0x7ff7cc19a680 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff7c000e9f0 tx=0x7ff7c000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:50.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.306+0000 7ff7ca000640 1 -- 172.21.15.1:0/1768280696 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c000ccc0 con 0x7ff7cc101de0 2024-09-12T01:27:50.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.306+0000 7ff7ca000640 1 -- 172.21.15.1:0/1768280696 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7c0010910 con 0x7ff7cc101de0 2024-09-12T01:27:50.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.306+0000 7ff7d4000640 1 -- 172.21.15.1:0/1768280696 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7cc19f920 con 0x7ff7cc101de0 2024-09-12T01:27:50.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.306+0000 7ff7ca000640 1 -- 172.21.15.1:0/1768280696 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c0018960 con 0x7ff7cc101de0 2024-09-12T01:27:50.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.306+0000 7ff7d4000640 1 -- 172.21.15.1:0/1768280696 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7cc1a7200 con 0x7ff7cc101de0 2024-09-12T01:27:50.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.307+0000 7ff7d4000640 1 -- 172.21.15.1:0/1768280696 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7cc1021c0 con 0x7ff7cc101de0 2024-09-12T01:27:50.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.309+0000 7ff7ca000640 1 -- 172.21.15.1:0/1768280696 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7ff7c0010ac0 con 0x7ff7cc101de0 2024-09-12T01:27:50.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.311+0000 7ff7ca000640 1 --2- 172.21.15.1:0/1768280696 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff7b40a2c10 0x7ff7b40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:50.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.311+0000 7ff7ca000640 1 -- 172.21.15.1:0/1768280696 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9092+0+0 (secure 0 0 0) 0x7ff7c00cfde0 con 0x7ff7cc101de0 2024-09-12T01:27:50.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.311+0000 7ff7d0c00640 1 --2- 172.21.15.1:0/1768280696 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff7b40a2c10 0x7ff7b40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:50.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.312+0000 7ff7d0c00640 1 --2- 172.21.15.1:0/1768280696 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff7b40a2c10 0x7ff7b40a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8000c00 tx=0x7ff7b8005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:50.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.313+0000 7ff7ca000640 1 -- 172.21.15.1:0/1768280696 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff7c009c1d0 con 0x7ff7cc101de0 2024-09-12T01:27:50.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.527+0000 7ff7d4000640 1 -- 172.21.15.1:0/1768280696 --> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7ff7cc0fec40 con 0x7ff7b40a2c10 2024-09-12T01:27:50.528 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:50 smithi001 ceph-mon[30387]: pgmap v343: 129 pgs: 129 active+clean; 587 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 3.9 KiB/s wr, 15 op/s 2024-09-12T01:27:50.528 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:50 smithi001 ceph-mon[30387]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-09-12T01:27:50.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.552+0000 7ff7ca000640 1 -- 172.21.15.1:0/1768280696 <== mgr.14150 v2:172.21.15.1:6800/1928662456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff7cc0fec40 con 0x7ff7b40a2c10 2024-09-12T01:27:50.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.554+0000 7ff7d4000640 1 -- 172.21.15.1:0/1768280696 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff7b40a2c10 msgr2=0x7ff7b40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:50.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.554+0000 7ff7d4000640 1 --2- 172.21.15.1:0/1768280696 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff7b40a2c10 0x7ff7b40a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8000c00 tx=0x7ff7b8005b50 comp rx=0 tx=0).stop 2024-09-12T01:27:50.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.554+0000 7ff7d4000640 1 -- 172.21.15.1:0/1768280696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7cc101de0 msgr2=0x7ff7cc19a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:50.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.554+0000 7ff7d4000640 1 --2- 172.21.15.1:0/1768280696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7cc101de0 0x7ff7cc19a680 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff7c000e9f0 tx=0x7ff7c000eeb0 comp rx=0 tx=0).stop 2024-09-12T01:27:50.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.554+0000 7ff7d4000640 1 -- 172.21.15.1:0/1768280696 shutdown_connections 2024-09-12T01:27:50.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.554+0000 7ff7d4000640 1 --2- 172.21.15.1:0/1768280696 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7cc106f10 0x7ff7cc19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:50.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.554+0000 7ff7d4000640 1 --2- 172.21.15.1:0/1768280696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7cc102790 0x7ff7cc19abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:50.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.554+0000 7ff7d4000640 1 --2- 172.21.15.1:0/1768280696 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff7b40a2c10 0x7ff7b40a50d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:50.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.554+0000 7ff7d4000640 1 --2- 172.21.15.1:0/1768280696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7cc101de0 0x7ff7cc19a680 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:50.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.554+0000 7ff7d4000640 1 -- 172.21.15.1:0/1768280696 >> 172.21.15.1:0/1768280696 conn(0x7ff7cc0fd8c0 msgr2=0x7ff7cc0ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:50.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.554+0000 7ff7d4000640 1 -- 172.21.15.1:0/1768280696 shutdown_connections 2024-09-12T01:27:50.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:50.555+0000 7ff7d4000640 1 -- 172.21.15.1:0/1768280696 wait complete. 2024-09-12T01:27:50.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:50 smithi053 ceph-mon[35158]: pgmap v343: 129 pgs: 129 active+clean; 587 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 3.9 KiB/s wr, 15 op/s 2024-09-12T01:27:50.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:50 smithi053 ceph-mon[35158]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-09-12T01:27:50.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:50 smithi104 ceph-mon[35219]: pgmap v343: 129 pgs: 129 active+clean; 587 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 3.9 KiB/s wr, 15 op/s 2024-09-12T01:27:50.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:50 smithi104 ceph-mon[35219]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-09-12T01:27:51.484 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-12T01:27:51.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:51 smithi001 ceph-mon[30387]: from='client.24712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:27:51.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:51 smithi053 ceph-mon[35158]: from='client.24712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:27:51.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:51 smithi104 ceph-mon[35219]: from='client.24712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:27:51.861 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:27:52.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.450+0000 7f40d9200640 1 -- 172.21.15.1:0/3374250882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40d4102790 msgr2=0x7f40d41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:52.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.450+0000 7f40d9200640 1 --2- 172.21.15.1:0/3374250882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40d4102790 0x7f40d41067e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f40bc009900 tx=0x7f40bc02a3f0 comp rx=0 tx=0).stop 2024-09-12T01:27:52.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.450+0000 7f40d9200640 1 -- 172.21.15.1:0/3374250882 shutdown_connections 2024-09-12T01:27:52.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.450+0000 7f40d9200640 1 --2- 172.21.15.1:0/3374250882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40d4106f10 0x7f40d410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:52.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.450+0000 7f40d9200640 1 --2- 172.21.15.1:0/3374250882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40d4102790 0x7f40d41067e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:52.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.450+0000 7f40d9200640 1 --2- 172.21.15.1:0/3374250882 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40d4101de0 0x7f40d41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:52.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.450+0000 7f40d9200640 1 -- 172.21.15.1:0/3374250882 >> 172.21.15.1:0/3374250882 conn(0x7f40d40fd8c0 msgr2=0x7f40d40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:52.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.450+0000 7f40d9200640 1 -- 172.21.15.1:0/3374250882 shutdown_connections 2024-09-12T01:27:52.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.450+0000 7f40d9200640 1 -- 172.21.15.1:0/3374250882 wait complete. 2024-09-12T01:27:52.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.451+0000 7f40d9200640 1 Processor -- start 2024-09-12T01:27:52.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.451+0000 7f40d9200640 1 -- start start 2024-09-12T01:27:52.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.451+0000 7f40d9200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40d4101de0 0x7f40d4198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:52.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.451+0000 7f40d9200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40d4102790 0x7f40d41989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:52.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.451+0000 7f40d9200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40d4106f10 0x7f40d419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:52.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.451+0000 7f40d9200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f40d4111fc0 con 0x7f40d4106f10 2024-09-12T01:27:52.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.451+0000 7f40d9200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f40d4111e40 con 0x7f40d4102790 2024-09-12T01:27:52.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.451+0000 7f40d9200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f40d4112140 con 0x7f40d4101de0 2024-09-12T01:27:52.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.452+0000 7f40c9400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40d4102790 0x7f40d41989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:52.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.452+0000 7f40d2800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40d4106f10 0x7f40d419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:52.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.452+0000 7f40c9400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40d4102790 0x7f40d41989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:48944/0 (socket says 172.21.15.1:48944) 2024-09-12T01:27:52.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.452+0000 7f40c9400640 1 -- 172.21.15.1:0/1178595945 learned_addr learned my addr 172.21.15.1:0/1178595945 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:27:52.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.452+0000 7f40d1e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40d4101de0 0x7f40d4198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:52.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.452+0000 7f40c9400640 1 -- 172.21.15.1:0/1178595945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40d4101de0 msgr2=0x7f40d4198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:52.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.452+0000 7f40c9400640 1 --2- 172.21.15.1:0/1178595945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40d4101de0 0x7f40d4198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:52.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.452+0000 7f40c9400640 1 -- 172.21.15.1:0/1178595945 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40d4106f10 msgr2=0x7f40d419cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:52.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.452+0000 7f40c9400640 1 --2- 172.21.15.1:0/1178595945 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40d4106f10 0x7f40d419cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:52.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.452+0000 7f40c9400640 1 -- 172.21.15.1:0/1178595945 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40d419d410 con 0x7f40d4102790 2024-09-12T01:27:52.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.452+0000 7f40d2800640 1 --2- 172.21.15.1:0/1178595945 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40d4106f10 0x7f40d419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:27:52.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.452+0000 7f40c9400640 1 --2- 172.21.15.1:0/1178595945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40d4102790 0x7f40d41989a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f40bc004770 tx=0x7f40bc003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:52.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.453+0000 7f40cb400640 1 -- 172.21.15.1:0/1178595945 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40bc03f070 con 0x7f40d4102790 2024-09-12T01:27:52.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.453+0000 7f40cb400640 1 -- 172.21.15.1:0/1178595945 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40bc031ca0 con 0x7f40d4102790 2024-09-12T01:27:52.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.453+0000 7f40cb400640 1 -- 172.21.15.1:0/1178595945 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40bc035030 con 0x7f40d4102790 2024-09-12T01:27:52.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.453+0000 7f40d9200640 1 -- 172.21.15.1:0/1178595945 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40d419d6a0 con 0x7f40d4102790 2024-09-12T01:27:52.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.453+0000 7f40d9200640 1 -- 172.21.15.1:0/1178595945 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40d41a4fe0 con 0x7f40d4102790 2024-09-12T01:27:52.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.454+0000 7f40d9200640 1 -- 172.21.15.1:0/1178595945 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40d41021c0 con 0x7f40d4102790 2024-09-12T01:27:52.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.456+0000 7f40cb400640 1 -- 172.21.15.1:0/1178595945 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f40bc042070 con 0x7f40d4102790 2024-09-12T01:27:52.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.457+0000 7f40cb400640 1 --2- 172.21.15.1:0/1178595945 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f40b80a2c90 0x7f40b80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:52.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.457+0000 7f40cb400640 1 -- 172.21.15.1:0/1178595945 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9092+0+0 (secure 0 0 0) 0x7f40bc0efaa0 con 0x7f40d4102790 2024-09-12T01:27:52.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.457+0000 7f40d1e00640 1 --2- 172.21.15.1:0/1178595945 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f40b80a2c90 0x7f40b80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:52.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.458+0000 7f40d1e00640 1 --2- 172.21.15.1:0/1178595945 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f40b80a2c90 0x7f40b80a5150 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f40c4005fd0 tx=0x7f40c4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:52.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.461+0000 7f40cb400640 1 -- 172.21.15.1:0/1178595945 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f40bc0bbf50 con 0x7f40d4102790 2024-09-12T01:27:52.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.675+0000 7f40d9200640 1 -- 172.21.15.1:0/1178595945 --> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f40d410a120 con 0x7f40b80a2c90 2024-09-12T01:27:52.675 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:52 smithi001 ceph-mon[30387]: pgmap v344: 129 pgs: 129 active+clean; 587 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.2 KiB/s wr, 19 op/s 2024-09-12T01:27:52.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.681+0000 7f40cb400640 1 -- 172.21.15.1:0/1178595945 <== mgr.14150 v2:172.21.15.1:6800/1928662456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f40d410a120 con 0x7f40b80a2c90 2024-09-12T01:27:52.680 INFO:teuthology.orchestra.run.smithi001.stdout:/volumes/qa/sv_0/b54a3529-b6e2-4300-9c59-2893c93f842b 2024-09-12T01:27:52.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.682+0000 7f40d9200640 1 -- 172.21.15.1:0/1178595945 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f40b80a2c90 msgr2=0x7f40b80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:52.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.682+0000 7f40d9200640 1 --2- 172.21.15.1:0/1178595945 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f40b80a2c90 0x7f40b80a5150 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f40c4005fd0 tx=0x7f40c4009500 comp rx=0 tx=0).stop 2024-09-12T01:27:52.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.682+0000 7f40d9200640 1 -- 172.21.15.1:0/1178595945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40d4102790 msgr2=0x7f40d41989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:52.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.682+0000 7f40d9200640 1 --2- 172.21.15.1:0/1178595945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40d4102790 0x7f40d41989a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f40bc004770 tx=0x7f40bc003c10 comp rx=0 tx=0).stop 2024-09-12T01:27:52.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.682+0000 7f40d9200640 1 -- 172.21.15.1:0/1178595945 shutdown_connections 2024-09-12T01:27:52.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.682+0000 7f40d9200640 1 --2- 172.21.15.1:0/1178595945 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40d4106f10 0x7f40d419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:52.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.682+0000 7f40d9200640 1 --2- 172.21.15.1:0/1178595945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40d4102790 0x7f40d41989a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:52.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.682+0000 7f40d9200640 1 --2- 172.21.15.1:0/1178595945 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f40b80a2c90 0x7f40b80a5150 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:52.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.682+0000 7f40d9200640 1 --2- 172.21.15.1:0/1178595945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40d4101de0 0x7f40d4198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:52.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.682+0000 7f40d9200640 1 -- 172.21.15.1:0/1178595945 >> 172.21.15.1:0/1178595945 conn(0x7f40d40fd8c0 msgr2=0x7f40d40fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:52.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.682+0000 7f40d9200640 1 -- 172.21.15.1:0/1178595945 shutdown_connections 2024-09-12T01:27:52.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:52.682+0000 7f40d9200640 1 -- 172.21.15.1:0/1178595945 wait complete. 2024-09-12T01:27:52.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:52 smithi053 ceph-mon[35158]: pgmap v344: 129 pgs: 129 active+clean; 587 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.2 KiB/s wr, 19 op/s 2024-09-12T01:27:52.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:52 smithi104 ceph-mon[35219]: pgmap v344: 129 pgs: 129 active+clean; 587 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.2 KiB/s wr, 19 op/s 2024-09-12T01:27:53.456 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-12T01:27:53.837 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:27:54.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.423+0000 7f855a800640 1 -- 172.21.15.1:0/966608080 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8554102790 msgr2=0x7f85541067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:54.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.423+0000 7f855a800640 1 --2- 172.21.15.1:0/966608080 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8554102790 0x7f85541067e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f853c009900 tx=0x7f853c02a410 comp rx=0 tx=0).stop 2024-09-12T01:27:54.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.423+0000 7f855a800640 1 -- 172.21.15.1:0/966608080 shutdown_connections 2024-09-12T01:27:54.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.423+0000 7f855a800640 1 --2- 172.21.15.1:0/966608080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8554106f10 0x7f855410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:54.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.423+0000 7f855a800640 1 --2- 172.21.15.1:0/966608080 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8554102790 0x7f85541067e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:54.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.423+0000 7f855a800640 1 --2- 172.21.15.1:0/966608080 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8554101de0 0x7f85541021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:54.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.423+0000 7f855a800640 1 -- 172.21.15.1:0/966608080 >> 172.21.15.1:0/966608080 conn(0x7f85540fd8c0 msgr2=0x7f85540ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:54.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.423+0000 7f855a800640 1 -- 172.21.15.1:0/966608080 shutdown_connections 2024-09-12T01:27:54.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.423+0000 7f855a800640 1 -- 172.21.15.1:0/966608080 wait complete. 2024-09-12T01:27:54.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.424+0000 7f855a800640 1 Processor -- start 2024-09-12T01:27:54.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.424+0000 7f855a800640 1 -- start start 2024-09-12T01:27:54.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.425+0000 7f855a800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8554101de0 0x7f85540744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:54.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.425+0000 7f855a800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8554102790 0x7f8554074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:54.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.425+0000 7f855a800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8554106f10 0x7f8554074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:54.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.425+0000 7f855a800640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8554111f70 con 0x7f8554102790 2024-09-12T01:27:54.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.425+0000 7f855a800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8554111df0 con 0x7f8554106f10 2024-09-12T01:27:54.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.425+0000 7f855a800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f85541120f0 con 0x7f8554101de0 2024-09-12T01:27:54.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.425+0000 7f854be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8554102790 0x7f8554074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:54.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.425+0000 7f8553400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8554101de0 0x7f85540744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:54.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.425+0000 7f8553400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8554101de0 0x7f85540744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:38606/0 (socket says 172.21.15.1:38606) 2024-09-12T01:27:54.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.425+0000 7f8553400640 1 -- 172.21.15.1:0/2674213367 learned_addr learned my addr 172.21.15.1:0/2674213367 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:27:54.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.425+0000 7f8553e00640 1 --2- 172.21.15.1:0/2674213367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8554106f10 0x7f8554074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:54.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.425+0000 7f8553400640 1 -- 172.21.15.1:0/2674213367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8554106f10 msgr2=0x7f8554074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:54.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.425+0000 7f8553400640 1 --2- 172.21.15.1:0/2674213367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8554106f10 0x7f8554074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:54.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.425+0000 7f8553400640 1 -- 172.21.15.1:0/2674213367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8554102790 msgr2=0x7f8554074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:54.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.425+0000 7f8553400640 1 --2- 172.21.15.1:0/2674213367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8554102790 0x7f8554074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:54.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.425+0000 7f8553400640 1 -- 172.21.15.1:0/2674213367 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85541a6970 con 0x7f8554101de0 2024-09-12T01:27:54.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.425+0000 7f854be00640 1 --2- 172.21.15.1:0/2674213367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8554102790 0x7f8554074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:27:54.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.426+0000 7f8553400640 1 --2- 172.21.15.1:0/2674213367 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8554101de0 0x7f85540744f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f854400e9f0 tx=0x7f854400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:54.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.426+0000 7f8550c00640 1 -- 172.21.15.1:0/2674213367 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f854400cd80 con 0x7f8554101de0 2024-09-12T01:27:54.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.426+0000 7f855a800640 1 -- 172.21.15.1:0/2674213367 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85541a6c60 con 0x7f8554101de0 2024-09-12T01:27:54.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.426+0000 7f8550c00640 1 -- 172.21.15.1:0/2674213367 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8544005500 con 0x7f8554101de0 2024-09-12T01:27:54.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.426+0000 7f8550c00640 1 -- 172.21.15.1:0/2674213367 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8544010690 con 0x7f8554101de0 2024-09-12T01:27:54.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.426+0000 7f855a800640 1 -- 172.21.15.1:0/2674213367 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85541a7120 con 0x7f8554101de0 2024-09-12T01:27:54.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.427+0000 7f855a800640 1 -- 172.21.15.1:0/2674213367 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85541021c0 con 0x7f8554101de0 2024-09-12T01:27:54.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.429+0000 7f8550c00640 1 -- 172.21.15.1:0/2674213367 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f85440046c0 con 0x7f8554101de0 2024-09-12T01:27:54.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.430+0000 7f8550c00640 1 --2- 172.21.15.1:0/2674213367 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f85380a2c10 0x7f85380a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:54.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.431+0000 7f8550c00640 1 -- 172.21.15.1:0/2674213367 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9092+0+0 (secure 0 0 0) 0x7f85440cf9b0 con 0x7f8554101de0 2024-09-12T01:27:54.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.431+0000 7f854be00640 1 --2- 172.21.15.1:0/2674213367 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f85380a2c10 0x7f85380a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:54.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.431+0000 7f854be00640 1 --2- 172.21.15.1:0/2674213367 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f85380a2c10 0x7f85380a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f853c009230 tx=0x7f853c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:54.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.435+0000 7f8550c00640 1 -- 172.21.15.1:0/2674213367 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f854409c3f0 con 0x7f8554101de0 2024-09-12T01:27:54.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.647+0000 7f855a800640 1 -- 172.21.15.1:0/2674213367 --> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f855410a1c0 con 0x7f85380a2c10 2024-09-12T01:27:54.647 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:54 smithi001 ceph-mon[30387]: from='client.24725 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:27:54.647 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:54 smithi001 ceph-mon[30387]: pgmap v345: 129 pgs: 129 active+clean; 608 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.2 KiB/s wr, 21 op/s 2024-09-12T01:27:54.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.676+0000 7f8550c00640 1 -- 172.21.15.1:0/2674213367 <== mgr.14150 v2:172.21.15.1:6800/1928662456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f853c0316b0 con 0x7f85380a2c10 2024-09-12T01:27:54.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.679+0000 7f855a800640 1 -- 172.21.15.1:0/2674213367 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f85380a2c10 msgr2=0x7f85380a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:54.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.679+0000 7f855a800640 1 --2- 172.21.15.1:0/2674213367 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f85380a2c10 0x7f85380a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f853c009230 tx=0x7f853c0058d0 comp rx=0 tx=0).stop 2024-09-12T01:27:54.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.679+0000 7f855a800640 1 -- 172.21.15.1:0/2674213367 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8554101de0 msgr2=0x7f85540744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:54.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.679+0000 7f855a800640 1 --2- 172.21.15.1:0/2674213367 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8554101de0 0x7f85540744f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f854400e9f0 tx=0x7f854400edf0 comp rx=0 tx=0).stop 2024-09-12T01:27:54.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.679+0000 7f855a800640 1 -- 172.21.15.1:0/2674213367 shutdown_connections 2024-09-12T01:27:54.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.679+0000 7f855a800640 1 --2- 172.21.15.1:0/2674213367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8554106f10 0x7f8554074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:54.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.679+0000 7f855a800640 1 --2- 172.21.15.1:0/2674213367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8554102790 0x7f8554074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:54.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.679+0000 7f855a800640 1 --2- 172.21.15.1:0/2674213367 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f85380a2c10 0x7f85380a50d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:54.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.679+0000 7f855a800640 1 --2- 172.21.15.1:0/2674213367 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8554101de0 0x7f85540744f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:54.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.679+0000 7f855a800640 1 -- 172.21.15.1:0/2674213367 >> 172.21.15.1:0/2674213367 conn(0x7f85540fd8c0 msgr2=0x7f85540fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:54.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.679+0000 7f855a800640 1 -- 172.21.15.1:0/2674213367 shutdown_connections 2024-09-12T01:27:54.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:54.679+0000 7f855a800640 1 -- 172.21.15.1:0/2674213367 wait complete. 2024-09-12T01:27:54.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:54 smithi053 ceph-mon[35158]: from='client.24725 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:27:54.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:54 smithi053 ceph-mon[35158]: pgmap v345: 129 pgs: 129 active+clean; 608 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.2 KiB/s wr, 21 op/s 2024-09-12T01:27:54.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:54 smithi104 ceph-mon[35219]: from='client.24725 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:27:54.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:54 smithi104 ceph-mon[35219]: pgmap v345: 129 pgs: 129 active+clean; 608 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.2 KiB/s wr, 21 op/s 2024-09-12T01:27:55.515 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-12T01:27:55.892 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:27:56.535 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:56 smithi001 ceph-mon[30387]: from='client.24724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:27:56.536 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:56 smithi001 ceph-mon[30387]: pgmap v346: 129 pgs: 129 active+clean; 608 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.4 KiB/s wr, 23 op/s 2024-09-12T01:27:56.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.535+0000 7f67afc00640 1 -- 172.21.15.1:0/2095433419 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67a8101df0 msgr2=0x7f67a81021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:56.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.535+0000 7f67afc00640 1 --2- 172.21.15.1:0/2095433419 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67a8101df0 0x7f67a81021d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f679c009900 tx=0x7f679c02a410 comp rx=0 tx=0).stop 2024-09-12T01:27:56.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.535+0000 7f67afc00640 1 -- 172.21.15.1:0/2095433419 shutdown_connections 2024-09-12T01:27:56.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.535+0000 7f67afc00640 1 --2- 172.21.15.1:0/2095433419 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f67a8106f20 0x7f67a810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:56.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.535+0000 7f67afc00640 1 --2- 172.21.15.1:0/2095433419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67a81027a0 0x7f67a81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:56.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.535+0000 7f67afc00640 1 --2- 172.21.15.1:0/2095433419 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67a8101df0 0x7f67a81021d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:56.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.535+0000 7f67afc00640 1 -- 172.21.15.1:0/2095433419 >> 172.21.15.1:0/2095433419 conn(0x7f67a80fd8b0 msgr2=0x7f67a80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:56.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.535+0000 7f67afc00640 1 -- 172.21.15.1:0/2095433419 shutdown_connections 2024-09-12T01:27:56.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.535+0000 7f67afc00640 1 -- 172.21.15.1:0/2095433419 wait complete. 2024-09-12T01:27:56.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.536+0000 7f67afc00640 1 Processor -- start 2024-09-12T01:27:56.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.536+0000 7f67afc00640 1 -- start start 2024-09-12T01:27:56.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.537+0000 7f67afc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67a8101df0 0x7f67a819a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:56.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.537+0000 7f67afc00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f67a81027a0 0x7f67a819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:56.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.537+0000 7f67afc00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67a8106f20 0x7f67a819ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:56.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.537+0000 7f67afc00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f67a8111f80 con 0x7f67a8106f20 2024-09-12T01:27:56.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.537+0000 7f67afc00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f67a8111e00 con 0x7f67a8101df0 2024-09-12T01:27:56.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.537+0000 7f67afc00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f67a8112100 con 0x7f67a81027a0 2024-09-12T01:27:56.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.537+0000 7f67ad200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67a8101df0 0x7f67a819a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:56.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.537+0000 7f67adc00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67a8106f20 0x7f67a819ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:56.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.537+0000 7f67ad200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67a8101df0 0x7f67a819a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:48974/0 (socket says 172.21.15.1:48974) 2024-09-12T01:27:56.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.537+0000 7f67ad200640 1 -- 172.21.15.1:0/2412183663 learned_addr learned my addr 172.21.15.1:0/2412183663 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:27:56.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.537+0000 7f67adc00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67a8106f20 0x7f67a819ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59096/0 (socket says 172.21.15.1:59096) 2024-09-12T01:27:56.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.537+0000 7f67a7e00640 1 --2- 172.21.15.1:0/2412183663 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f67a81027a0 0x7f67a819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:56.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.538+0000 7f67ad200640 1 -- 172.21.15.1:0/2412183663 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f67a81027a0 msgr2=0x7f67a819ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:56.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.538+0000 7f67ad200640 1 --2- 172.21.15.1:0/2412183663 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f67a81027a0 0x7f67a819ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:56.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.538+0000 7f67ad200640 1 -- 172.21.15.1:0/2412183663 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67a8106f20 msgr2=0x7f67a819ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:56.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.538+0000 7f67ad200640 1 --2- 172.21.15.1:0/2412183663 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67a8106f20 0x7f67a819ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:56.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.538+0000 7f67ad200640 1 -- 172.21.15.1:0/2412183663 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67a819f570 con 0x7f67a8101df0 2024-09-12T01:27:56.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.538+0000 7f67adc00640 1 --2- 172.21.15.1:0/2412183663 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67a8106f20 0x7f67a819ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:27:56.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.538+0000 7f67ad200640 1 --2- 172.21.15.1:0/2412183663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67a8101df0 0x7f67a819a5c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f679c006010 tx=0x7f679c004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:56.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.538+0000 7f67a5600640 1 -- 172.21.15.1:0/2412183663 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679c03f030 con 0x7f67a8101df0 2024-09-12T01:27:56.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.539+0000 7f67afc00640 1 -- 172.21.15.1:0/2412183663 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67a819f800 con 0x7f67a8101df0 2024-09-12T01:27:56.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.539+0000 7f67a5600640 1 -- 172.21.15.1:0/2412183663 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f679c004440 con 0x7f67a8101df0 2024-09-12T01:27:56.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.539+0000 7f67a5600640 1 -- 172.21.15.1:0/2412183663 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679c035070 con 0x7f67a8101df0 2024-09-12T01:27:56.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.539+0000 7f67afc00640 1 -- 172.21.15.1:0/2412183663 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67a81a7140 con 0x7f67a8101df0 2024-09-12T01:27:56.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.540+0000 7f67afc00640 1 -- 172.21.15.1:0/2412183663 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67a81021d0 con 0x7f67a8101df0 2024-09-12T01:27:56.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.543+0000 7f67a5600640 1 -- 172.21.15.1:0/2412183663 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f679c042030 con 0x7f67a8101df0 2024-09-12T01:27:56.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.545+0000 7f67a5600640 1 --2- 172.21.15.1:0/2412183663 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f67900a2c10 0x7f67900a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:56.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.545+0000 7f67a5600640 1 -- 172.21.15.1:0/2412183663 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9092+0+0 (secure 0 0 0) 0x7f679c0ef8e0 con 0x7f67a8101df0 2024-09-12T01:27:56.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.545+0000 7f67a7e00640 1 --2- 172.21.15.1:0/2412183663 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f67900a2c10 0x7f67900a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:56.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.546+0000 7f67a7e00640 1 --2- 172.21.15.1:0/2412183663 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f67900a2c10 0x7f67900a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f67a80ff6e0 tx=0x7f6794005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:56.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.549+0000 7f67a5600640 1 -- 172.21.15.1:0/2412183663 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f679c0bbcd0 con 0x7f67a8101df0 2024-09-12T01:27:56.627 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:56 smithi104 ceph-mon[35219]: from='client.24724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:27:56.627 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:56 smithi104 ceph-mon[35219]: pgmap v346: 129 pgs: 129 active+clean; 608 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.4 KiB/s wr, 23 op/s 2024-09-12T01:27:56.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:56 smithi053 ceph-mon[35158]: from='client.24724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:27:56.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:56 smithi053 ceph-mon[35158]: pgmap v346: 129 pgs: 129 active+clean; 608 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.4 KiB/s wr, 23 op/s 2024-09-12T01:27:56.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.764+0000 7f67afc00640 1 -- 172.21.15.1:0/2412183663 --> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f67a8075430 con 0x7f67900a2c10 2024-09-12T01:27:56.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.769+0000 7f67a5600640 1 -- 172.21.15.1:0/2412183663 <== mgr.14150 v2:172.21.15.1:6800/1928662456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f67a8075430 con 0x7f67900a2c10 2024-09-12T01:27:56.769 INFO:teuthology.orchestra.run.smithi001.stdout:/volumes/qa/sv_1/68dfb7ec-f69e-4d16-ba25-71857c337627 2024-09-12T01:27:56.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.771+0000 7f67afc00640 1 -- 172.21.15.1:0/2412183663 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f67900a2c10 msgr2=0x7f67900a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:56.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.771+0000 7f67afc00640 1 --2- 172.21.15.1:0/2412183663 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f67900a2c10 0x7f67900a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f67a80ff6e0 tx=0x7f6794005f90 comp rx=0 tx=0).stop 2024-09-12T01:27:56.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.771+0000 7f67afc00640 1 -- 172.21.15.1:0/2412183663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67a8101df0 msgr2=0x7f67a819a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:56.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.771+0000 7f67afc00640 1 --2- 172.21.15.1:0/2412183663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67a8101df0 0x7f67a819a5c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f679c006010 tx=0x7f679c004040 comp rx=0 tx=0).stop 2024-09-12T01:27:56.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.771+0000 7f67afc00640 1 -- 172.21.15.1:0/2412183663 shutdown_connections 2024-09-12T01:27:56.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.771+0000 7f67afc00640 1 --2- 172.21.15.1:0/2412183663 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67a8106f20 0x7f67a819ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:56.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.771+0000 7f67afc00640 1 --2- 172.21.15.1:0/2412183663 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f67900a2c10 0x7f67900a50d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:56.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.771+0000 7f67afc00640 1 --2- 172.21.15.1:0/2412183663 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f67a81027a0 0x7f67a819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:56.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.771+0000 7f67afc00640 1 --2- 172.21.15.1:0/2412183663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67a8101df0 0x7f67a819a5c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:56.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.772+0000 7f67afc00640 1 -- 172.21.15.1:0/2412183663 >> 172.21.15.1:0/2412183663 conn(0x7f67a80fd8b0 msgr2=0x7f67a80feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:56.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.772+0000 7f67afc00640 1 -- 172.21.15.1:0/2412183663 shutdown_connections 2024-09-12T01:27:56.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:56.772+0000 7f67afc00640 1 -- 172.21.15.1:0/2412183663 wait complete. 2024-09-12T01:27:57.634 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:27:58.019 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:27:58.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:58 smithi001 ceph-mon[30387]: pgmap v347: 129 pgs: 129 active+clean; 632 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.1 KiB/s wr, 22 op/s 2024-09-12T01:27:58.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:58 smithi001 ceph-mon[30387]: from='client.24737 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:27:58.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.639+0000 7f8109400640 1 -- 172.21.15.1:0/579948298 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81041047e0 msgr2=0x7f8104108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:58.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.639+0000 7f8109400640 1 --2- 172.21.15.1:0/579948298 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81041047e0 0x7f8104108830 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f80ec009900 tx=0x7f80ec02a3f0 comp rx=0 tx=0).stop 2024-09-12T01:27:58.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.639+0000 7f8109400640 1 -- 172.21.15.1:0/579948298 shutdown_connections 2024-09-12T01:27:58.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.639+0000 7f8109400640 1 --2- 172.21.15.1:0/579948298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8104108f60 0x7f8104110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:58.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.639+0000 7f8109400640 1 --2- 172.21.15.1:0/579948298 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81041047e0 0x7f8104108830 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:58.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.639+0000 7f8109400640 1 --2- 172.21.15.1:0/579948298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8104103e30 0x7f8104104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:58.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.639+0000 7f8109400640 1 -- 172.21.15.1:0/579948298 >> 172.21.15.1:0/579948298 conn(0x7f81040ff8d0 msgr2=0x7f8104101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:58.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.639+0000 7f8109400640 1 -- 172.21.15.1:0/579948298 shutdown_connections 2024-09-12T01:27:58.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.639+0000 7f8109400640 1 -- 172.21.15.1:0/579948298 wait complete. 2024-09-12T01:27:58.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.640+0000 7f8109400640 1 Processor -- start 2024-09-12T01:27:58.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.640+0000 7f8109400640 1 -- start start 2024-09-12T01:27:58.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.641+0000 7f8109400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8104103e30 0x7f810419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:58.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.641+0000 7f8109400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81041047e0 0x7f810419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:58.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.641+0000 7f8109400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8104108f60 0x7f810419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:58.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.641+0000 7f8109400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8104114010 con 0x7f8104103e30 2024-09-12T01:27:58.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.641+0000 7f8109400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8104113e90 con 0x7f8104108f60 2024-09-12T01:27:58.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.641+0000 7f8109400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8104114190 con 0x7f81041047e0 2024-09-12T01:27:58.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.641+0000 7f8103e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8104103e30 0x7f810419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:58.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.641+0000 7f8103e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8104103e30 0x7f810419a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59114/0 (socket says 172.21.15.1:59114) 2024-09-12T01:27:58.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.641+0000 7f8103e00640 1 -- 172.21.15.1:0/4269791567 learned_addr learned my addr 172.21.15.1:0/4269791567 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:27:58.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.641+0000 7f80fb400640 1 --2- 172.21.15.1:0/4269791567 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81041047e0 0x7f810419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:58.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.641+0000 7f8108a00640 1 --2- 172.21.15.1:0/4269791567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8104108f60 0x7f810419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:58.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.642+0000 7f8103e00640 1 -- 172.21.15.1:0/4269791567 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81041047e0 msgr2=0x7f810419a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:58.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.642+0000 7f8103e00640 1 --2- 172.21.15.1:0/4269791567 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81041047e0 0x7f810419a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:58.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.642+0000 7f8103e00640 1 -- 172.21.15.1:0/4269791567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8104108f60 msgr2=0x7f810419ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:58.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.642+0000 7f8103e00640 1 --2- 172.21.15.1:0/4269791567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8104108f60 0x7f810419ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:58.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.642+0000 7f8103e00640 1 -- 172.21.15.1:0/4269791567 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f810419f400 con 0x7f8104103e30 2024-09-12T01:27:58.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.642+0000 7f8103e00640 1 --2- 172.21.15.1:0/4269791567 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8104103e30 0x7f810419a450 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f80f400e9f0 tx=0x7f80f400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:58.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.642+0000 7f8101600640 1 -- 172.21.15.1:0/4269791567 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f400cd80 con 0x7f8104103e30 2024-09-12T01:27:58.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.642+0000 7f8109400640 1 -- 172.21.15.1:0/4269791567 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f810419f6f0 con 0x7f8104103e30 2024-09-12T01:27:58.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.642+0000 7f8101600640 1 -- 172.21.15.1:0/4269791567 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80f4010910 con 0x7f8104103e30 2024-09-12T01:27:58.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.642+0000 7f8101600640 1 -- 172.21.15.1:0/4269791567 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f4018990 con 0x7f8104103e30 2024-09-12T01:27:58.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.642+0000 7f8109400640 1 -- 172.21.15.1:0/4269791567 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81041a6fd0 con 0x7f8104103e30 2024-09-12T01:27:58.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.644+0000 7f8109400640 1 -- 172.21.15.1:0/4269791567 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8104104210 con 0x7f8104103e30 2024-09-12T01:27:58.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.645+0000 7f8101600640 1 -- 172.21.15.1:0/4269791567 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f80f4010ac0 con 0x7f8104103e30 2024-09-12T01:27:58.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.647+0000 7f8101600640 1 --2- 172.21.15.1:0/4269791567 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f80e80a2c10 0x7f80e80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:27:58.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.647+0000 7f8101600640 1 -- 172.21.15.1:0/4269791567 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9092+0+0 (secure 0 0 0) 0x7f80f40cf890 con 0x7f8104103e30 2024-09-12T01:27:58.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.647+0000 7f80fb400640 1 --2- 172.21.15.1:0/4269791567 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f80e80a2c10 0x7f80e80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:27:58.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.648+0000 7f80fb400640 1 --2- 172.21.15.1:0/4269791567 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f80e80a2c10 0x7f80e80a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f80ec000c00 tx=0x7f80ec005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:27:58.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.651+0000 7f8101600640 1 -- 172.21.15.1:0/4269791567 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f80f409bc80 con 0x7f8104103e30 2024-09-12T01:27:58.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:58 smithi053 ceph-mon[35158]: pgmap v347: 129 pgs: 129 active+clean; 632 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.1 KiB/s wr, 22 op/s 2024-09-12T01:27:58.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:58 smithi053 ceph-mon[35158]: from='client.24737 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:27:58.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:58 smithi104 ceph-mon[35219]: pgmap v347: 129 pgs: 129 active+clean; 632 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.1 KiB/s wr, 22 op/s 2024-09-12T01:27:58.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:58 smithi104 ceph-mon[35219]: from='client.24737 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:27:58.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.872+0000 7f8109400640 1 -- 172.21.15.1:0/4269791567 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f810410c1a0 con 0x7f8104103e30 2024-09-12T01:27:58.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.873+0000 7f8101600640 1 -- 172.21.15.1:0/4269791567 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11962 (secure 0 0 0) 0x7f80f409ff80 con 0x7f8104103e30 2024-09-12T01:27:58.873 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:27:58.873 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":28,"btime":"2024-09-12T01:27:46:332106+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14430,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/572192306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":572192306},{"type":"v1","addr":"172.21.15.1:6835","nonce":572192306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14442,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/1050734277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":1050734277},{"type":"v1","addr":"172.21.15.1:6837","nonce":1050734277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:26:48.555447+0000","modified":"2024-09-12T01:27:46.206001+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24545,"mds_1":24511,"mds_2":24506,"mds_3":24493,"mds_4":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14481":{"gid":14481,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/3651849993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":3651849993},{"type":"v1","addr":"172.21.15.1:6839","nonce":3651849993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14493":{"gid":14493,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2464781973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2464781973},{"type":"v1","addr":"172.21.15.1:6841","nonce":2464781973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2071825633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2071825633},{"type":"v1","addr":"172.21.15.104:6833","nonce":2071825633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/2930310711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2930310711},{"type":"v1","addr":"172.21.15.53:6833","nonce":2930310711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2032381725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2032381725},{"type":"v1","addr":"172.21.15.53:6835","nonce":2032381725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.104:6835/1547038540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1547038540},{"type":"v1","addr":"172.21.15.104:6835","nonce":1547038540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":23,"addr":"172.21.15.104:6837/464720247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":464720247},{"type":"v1","addr":"172.21.15.104:6837","nonce":464720247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":22,"addr":"172.21.15.53:6837/2848743754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2848743754},{"type":"v1","addr":"172.21.15.53:6837","nonce":2848743754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":21,"addr":"172.21.15.104:6839/1459976041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1459976041},{"type":"v1","addr":"172.21.15.104:6839","nonce":1459976041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.53:6839/4019389301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":4019389301},{"type":"v1","addr":"172.21.15.53:6839","nonce":4019389301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24487,24493,24506,24545,24511]},"id":1}]} 2024-09-12T01:27:58.873 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 28 2024-09-12T01:27:58.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.875+0000 7f8109400640 1 -- 172.21.15.1:0/4269791567 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f80e80a2c10 msgr2=0x7f80e80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:58.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.875+0000 7f8109400640 1 --2- 172.21.15.1:0/4269791567 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f80e80a2c10 0x7f80e80a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f80ec000c00 tx=0x7f80ec005b50 comp rx=0 tx=0).stop 2024-09-12T01:27:58.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.875+0000 7f8109400640 1 -- 172.21.15.1:0/4269791567 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8104103e30 msgr2=0x7f810419a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:27:58.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.875+0000 7f8109400640 1 --2- 172.21.15.1:0/4269791567 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8104103e30 0x7f810419a450 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f80f400e9f0 tx=0x7f80f400eeb0 comp rx=0 tx=0).stop 2024-09-12T01:27:58.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.875+0000 7f8109400640 1 -- 172.21.15.1:0/4269791567 shutdown_connections 2024-09-12T01:27:58.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.875+0000 7f8109400640 1 --2- 172.21.15.1:0/4269791567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8104108f60 0x7f810419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:58.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.875+0000 7f8109400640 1 --2- 172.21.15.1:0/4269791567 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f80e80a2c10 0x7f80e80a50d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:58.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.875+0000 7f8109400640 1 --2- 172.21.15.1:0/4269791567 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81041047e0 0x7f810419a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:58.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.875+0000 7f8109400640 1 --2- 172.21.15.1:0/4269791567 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8104103e30 0x7f810419a450 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:27:58.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.875+0000 7f8109400640 1 -- 172.21.15.1:0/4269791567 >> 172.21.15.1:0/4269791567 conn(0x7f81040ff8d0 msgr2=0x7f8104100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:27:58.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.876+0000 7f8109400640 1 -- 172.21.15.1:0/4269791567 shutdown_connections 2024-09-12T01:27:58.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:27:58.876+0000 7f8109400640 1 -- 172.21.15.1:0/4269791567 wait complete. 2024-09-12T01:27:59.596 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:27:59 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/4269791567' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:27:59.634 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T01:27:59.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:27:59 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/4269791567' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:27:59.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:27:59 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/4269791567' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:28:00.014 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:28:00.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:00 smithi001 ceph-mon[30387]: pgmap v348: 129 pgs: 129 active+clean; 632 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.1 KiB/s wr, 15 op/s 2024-09-12T01:28:00.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.651+0000 7fde1b200640 1 -- 172.21.15.1:0/4290639706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde14103e30 msgr2=0x7fde14104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:00.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.651+0000 7fde1b200640 1 --2- 172.21.15.1:0/4290639706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde14103e30 0x7fde14104210 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fde08009900 tx=0x7fde0802a410 comp rx=0 tx=0).stop 2024-09-12T01:28:00.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.651+0000 7fde1b200640 1 -- 172.21.15.1:0/4290639706 shutdown_connections 2024-09-12T01:28:00.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.651+0000 7fde1b200640 1 --2- 172.21.15.1:0/4290639706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde14108f60 0x7fde14110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:00.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.651+0000 7fde1b200640 1 --2- 172.21.15.1:0/4290639706 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde141047e0 0x7fde14108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:00.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.651+0000 7fde1b200640 1 --2- 172.21.15.1:0/4290639706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde14103e30 0x7fde14104210 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:00.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.651+0000 7fde1b200640 1 -- 172.21.15.1:0/4290639706 >> 172.21.15.1:0/4290639706 conn(0x7fde140ff8d0 msgr2=0x7fde14101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:00.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.651+0000 7fde1b200640 1 -- 172.21.15.1:0/4290639706 shutdown_connections 2024-09-12T01:28:00.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.651+0000 7fde1b200640 1 -- 172.21.15.1:0/4290639706 wait complete. 2024-09-12T01:28:00.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.652+0000 7fde1b200640 1 Processor -- start 2024-09-12T01:28:00.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.652+0000 7fde1b200640 1 -- start start 2024-09-12T01:28:00.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.652+0000 7fde1b200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde14103e30 0x7fde1419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:00.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.652+0000 7fde1b200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde141047e0 0x7fde1419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:00.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.652+0000 7fde1b200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde14108f60 0x7fde1419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:00.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.652+0000 7fde1b200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fde14113fc0 con 0x7fde141047e0 2024-09-12T01:28:00.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.652+0000 7fde1b200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fde14113e40 con 0x7fde14103e30 2024-09-12T01:28:00.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.652+0000 7fde1b200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fde14114140 con 0x7fde14108f60 2024-09-12T01:28:00.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.653+0000 7fde19400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde141047e0 0x7fde1419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:00.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.653+0000 7fde19e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde14103e30 0x7fde1419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:00.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.653+0000 7fde19400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde141047e0 0x7fde1419a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40048/0 (socket says 172.21.15.1:40048) 2024-09-12T01:28:00.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.653+0000 7fde19400640 1 -- 172.21.15.1:0/2594410804 learned_addr learned my addr 172.21.15.1:0/2594410804 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:28:00.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.653+0000 7fde1a800640 1 --2- 172.21.15.1:0/2594410804 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde14108f60 0x7fde1419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:00.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.653+0000 7fde19e00640 1 -- 172.21.15.1:0/2594410804 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde14108f60 msgr2=0x7fde1419ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:00.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.653+0000 7fde19e00640 1 --2- 172.21.15.1:0/2594410804 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde14108f60 0x7fde1419ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:00.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.653+0000 7fde19e00640 1 -- 172.21.15.1:0/2594410804 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde141047e0 msgr2=0x7fde1419a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:00.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.653+0000 7fde19e00640 1 --2- 172.21.15.1:0/2594410804 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde141047e0 0x7fde1419a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:00.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.653+0000 7fde19e00640 1 -- 172.21.15.1:0/2594410804 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde1419f460 con 0x7fde14103e30 2024-09-12T01:28:00.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.654+0000 7fde19e00640 1 --2- 172.21.15.1:0/2594410804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde14103e30 0x7fde1419a4b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fde08005940 tx=0x7fde08004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:00.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.654+0000 7fde0ea00640 1 -- 172.21.15.1:0/2594410804 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0803f030 con 0x7fde14103e30 2024-09-12T01:28:00.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.654+0000 7fde0ea00640 1 -- 172.21.15.1:0/2594410804 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde08003ad0 con 0x7fde14103e30 2024-09-12T01:28:00.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.654+0000 7fde0ea00640 1 -- 172.21.15.1:0/2594410804 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde08035070 con 0x7fde14103e30 2024-09-12T01:28:00.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.654+0000 7fde1b200640 1 -- 172.21.15.1:0/2594410804 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde1419f6f0 con 0x7fde14103e30 2024-09-12T01:28:00.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.654+0000 7fde1b200640 1 -- 172.21.15.1:0/2594410804 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde1419fb80 con 0x7fde14103e30 2024-09-12T01:28:00.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.655+0000 7fde1b200640 1 -- 172.21.15.1:0/2594410804 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde14104210 con 0x7fde14103e30 2024-09-12T01:28:00.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.657+0000 7fde0ea00640 1 -- 172.21.15.1:0/2594410804 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fde08042030 con 0x7fde14103e30 2024-09-12T01:28:00.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.658+0000 7fde0ea00640 1 --2- 172.21.15.1:0/2594410804 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fddfc0a2c10 0x7fddfc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:00.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.659+0000 7fde0ea00640 1 -- 172.21.15.1:0/2594410804 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9092+0+0 (secure 0 0 0) 0x7fde080efd30 con 0x7fde14103e30 2024-09-12T01:28:00.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.659+0000 7fde19400640 1 --2- 172.21.15.1:0/2594410804 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fddfc0a2c10 0x7fddfc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:00.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.660+0000 7fde19400640 1 --2- 172.21.15.1:0/2594410804 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fddfc0a2c10 0x7fddfc0a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fde140ffd60 tx=0x7fde04005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:00.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.661+0000 7fde0ea00640 1 -- 172.21.15.1:0/2594410804 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fde080bc120 con 0x7fde14103e30 2024-09-12T01:28:00.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:00 smithi053 ceph-mon[35158]: pgmap v348: 129 pgs: 129 active+clean; 632 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.1 KiB/s wr, 15 op/s 2024-09-12T01:28:00.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:00 smithi104 ceph-mon[35219]: pgmap v348: 129 pgs: 129 active+clean; 632 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.1 KiB/s wr, 15 op/s 2024-09-12T01:28:00.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.850+0000 7fde1b200640 1 -- 172.21.15.1:0/2594410804 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fde1419b810 con 0x7fde14103e30 2024-09-12T01:28:00.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.852+0000 7fde0ea00640 1 -- 172.21.15.1:0/2594410804 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24657 (secure 0 0 0) 0x7fde1419b810 con 0x7fde14103e30 2024-09-12T01:28:00.852 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:28:00.852 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":96,"fsid":"c0e4f180-70a4-11ef-bcea-c7b262605968","created":"2024-09-12T01:18:19.337406+0000","modified":"2024-09-12T01:27:16.608027+0000","last_up_change":"2024-09-12T01:25:50.601477+0000","last_in_change":"2024-09-12T01:25:27.674600+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:21:41.204242+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:26:43.066475+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:26:46.112771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T01:26:56.040130+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b9a396ae-bf97-462f-9ef9-b10ef6b9d337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6803","nonce":1125335136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6805","nonce":1125335136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6809","nonce":1125335136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6807","nonce":1125335136}]},"public_addr":"172.21.15.1:6803/1125335136","cluster_addr":"172.21.15.1:6805/1125335136","heartbeat_back_addr":"172.21.15.1:6809/1125335136","heartbeat_front_addr":"172.21.15.1:6807/1125335136","state":["exists","up"]},{"osd":1,"uuid":"dfe7a8a3-3700-4789-ac64-88660de9e0d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6801","nonce":3131049165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6803","nonce":3131049165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6807","nonce":3131049165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6805","nonce":3131049165}]},"public_addr":"172.21.15.53:6801/3131049165","cluster_addr":"172.21.15.53:6803/3131049165","heartbeat_back_addr":"172.21.15.53:6807/3131049165","heartbeat_front_addr":"172.21.15.53:6805/3131049165","state":["exists","up"]},{"osd":2,"uuid":"5d9d3603-ba7c-4115-9267-12abcb20d55b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6801","nonce":1133285636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6803","nonce":1133285636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6807","nonce":1133285636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6805","nonce":1133285636}]},"public_addr":"172.21.15.104:6801/1133285636","cluster_addr":"172.21.15.104:6803/1133285636","heartbeat_back_addr":"172.21.15.104:6807/1133285636","heartbeat_front_addr":"172.21.15.104:6805/1133285636","state":["exists","up"]},{"osd":3,"uuid":"1e063c66-41b4-4db1-a964-28e122ce628e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6811","nonce":1247468803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6813","nonce":1247468803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6817","nonce":1247468803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6815","nonce":1247468803}]},"public_addr":"172.21.15.1:6811/1247468803","cluster_addr":"172.21.15.1:6813/1247468803","heartbeat_back_addr":"172.21.15.1:6817/1247468803","heartbeat_front_addr":"172.21.15.1:6815/1247468803","state":["exists","up"]},{"osd":4,"uuid":"fe89ece0-2171-45d5-a06d-e400f77233e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6809","nonce":2673413639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6811","nonce":2673413639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6815","nonce":2673413639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6813","nonce":2673413639}]},"public_addr":"172.21.15.53:6809/2673413639","cluster_addr":"172.21.15.53:6811/2673413639","heartbeat_back_addr":"172.21.15.53:6815/2673413639","heartbeat_front_addr":"172.21.15.53:6813/2673413639","state":["exists","up"]},{"osd":5,"uuid":"daab795c-9942-4d16-aa34-8bba30a4b99b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6809","nonce":3131498790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6811","nonce":3131498790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6815","nonce":3131498790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6813","nonce":3131498790}]},"public_addr":"172.21.15.104:6809/3131498790","cluster_addr":"172.21.15.104:6811/3131498790","heartbeat_back_addr":"172.21.15.104:6815/3131498790","heartbeat_front_addr":"172.21.15.104:6813/3131498790","state":["exists","up"]},{"osd":6,"uuid":"f31d8633-e435-499b-9d46-0aa11a2481cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6819","nonce":3413067918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6821","nonce":3413067918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6825","nonce":3413067918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6823","nonce":3413067918}]},"public_addr":"172.21.15.1:6819/3413067918","cluster_addr":"172.21.15.1:6821/3413067918","heartbeat_back_addr":"172.21.15.1:6825/3413067918","heartbeat_front_addr":"172.21.15.1:6823/3413067918","state":["exists","up"]},{"osd":7,"uuid":"e64da49e-d31e-4801-b6cc-634a233b4761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6817","nonce":2841484136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6819","nonce":2841484136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6823","nonce":2841484136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6821","nonce":2841484136}]},"public_addr":"172.21.15.53:6817/2841484136","cluster_addr":"172.21.15.53:6819/2841484136","heartbeat_back_addr":"172.21.15.53:6823/2841484136","heartbeat_front_addr":"172.21.15.53:6821/2841484136","state":["exists","up"]},{"osd":8,"uuid":"fdfb5a42-dca7-4874-b479-176b4a191e21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6817","nonce":941453547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6819","nonce":941453547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6823","nonce":941453547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6821","nonce":941453547}]},"public_addr":"172.21.15.104:6817/941453547","cluster_addr":"172.21.15.104:6819/941453547","heartbeat_back_addr":"172.21.15.104:6823/941453547","heartbeat_front_addr":"172.21.15.104:6821/941453547","state":["exists","up"]},{"osd":9,"uuid":"7f81537a-cae5-4bb8-aec2-4eb07deb7958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6827","nonce":958287586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6829","nonce":958287586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6833","nonce":958287586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6831","nonce":958287586}]},"public_addr":"172.21.15.1:6827/958287586","cluster_addr":"172.21.15.1:6829/958287586","heartbeat_back_addr":"172.21.15.1:6833/958287586","heartbeat_front_addr":"172.21.15.1:6831/958287586","state":["exists","up"]},{"osd":10,"uuid":"e3c9c37d-46b9-4585-990d-69c447453bd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6825","nonce":3357909498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6827","nonce":3357909498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6831","nonce":3357909498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6829","nonce":3357909498}]},"public_addr":"172.21.15.53:6825/3357909498","cluster_addr":"172.21.15.53:6827/3357909498","heartbeat_back_addr":"172.21.15.53:6831/3357909498","heartbeat_front_addr":"172.21.15.53:6829/3357909498","state":["exists","up"]},{"osd":11,"uuid":"474b5345-f2ae-4e0e-a197-53ab11f58c04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6825","nonce":3381427987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6827","nonce":3381427987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6831","nonce":3381427987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6829","nonce":3381427987}]},"public_addr":"172.21.15.104:6825/3381427987","cluster_addr":"172.21.15.104:6827/3381427987","heartbeat_back_addr":"172.21.15.104:6831/3381427987","heartbeat_front_addr":"172.21.15.104:6829/3381427987","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:20:46.903181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:21:12.735556+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:21:37.868739+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:22:09.141228+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:22:36.438563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:23:02.220948+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:23:33.505881+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:24:00.982176+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:24:26.010444+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:24:56.711121+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:25:23.846495+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:25:48.815427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1373984726":"2024-09-13T01:19:14.580637+0000","172.21.15.1:0/1429963718":"2024-09-13T01:19:14.580637+0000","172.21.15.1:6801/4101281615":"2024-09-13T01:19:14.580637+0000","172.21.15.1:6800/4101281615":"2024-09-13T01:19:14.580637+0000","172.21.15.1:0/2564759586":"2024-09-13T01:19:14.580637+0000","172.21.15.1:6801/2935968453":"2024-09-13T01:18:45.882682+0000","172.21.15.1:6800/2935968453":"2024-09-13T01:18:45.882682+0000","172.21.15.1:0/1459848813":"2024-09-13T01:18:45.882682+0000","172.21.15.1:0/578605922":"2024-09-13T01:18:45.882682+0000","172.21.15.1:0/738035035":"2024-09-13T01:18:45.882682+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:28:00.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.854+0000 7fde1b200640 1 -- 172.21.15.1:0/2594410804 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fddfc0a2c10 msgr2=0x7fddfc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:00.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.854+0000 7fde1b200640 1 --2- 172.21.15.1:0/2594410804 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fddfc0a2c10 0x7fddfc0a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fde140ffd60 tx=0x7fde04005f90 comp rx=0 tx=0).stop 2024-09-12T01:28:00.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.854+0000 7fde1b200640 1 -- 172.21.15.1:0/2594410804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde14103e30 msgr2=0x7fde1419a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:00.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.854+0000 7fde1b200640 1 --2- 172.21.15.1:0/2594410804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde14103e30 0x7fde1419a4b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fde08005940 tx=0x7fde08004270 comp rx=0 tx=0).stop 2024-09-12T01:28:00.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.855+0000 7fde1b200640 1 -- 172.21.15.1:0/2594410804 shutdown_connections 2024-09-12T01:28:00.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.855+0000 7fde1b200640 1 --2- 172.21.15.1:0/2594410804 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fddfc0a2c10 0x7fddfc0a50d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:00.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.855+0000 7fde1b200640 1 --2- 172.21.15.1:0/2594410804 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde14108f60 0x7fde1419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:00.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.855+0000 7fde1b200640 1 --2- 172.21.15.1:0/2594410804 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde141047e0 0x7fde1419a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:00.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.855+0000 7fde1b200640 1 --2- 172.21.15.1:0/2594410804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde14103e30 0x7fde1419a4b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:00.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.855+0000 7fde1b200640 1 -- 172.21.15.1:0/2594410804 >> 172.21.15.1:0/2594410804 conn(0x7fde140ff8d0 msgr2=0x7fde14100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:00.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.855+0000 7fde1b200640 1 -- 172.21.15.1:0/2594410804 shutdown_connections 2024-09-12T01:28:00.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:00.855+0000 7fde1b200640 1 -- 172.21.15.1:0/2594410804 wait complete. 2024-09-12T01:28:01.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:01 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/2594410804' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:28:01.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:01 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/2594410804' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:28:01.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:01 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/2594410804' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:28:01.932 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-12T01:28:01.932 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T01:28:02.315 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:28:02.641 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:02 smithi053 ceph-mon[35158]: pgmap v349: 129 pgs: 129 active+clean; 632 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 5.0 KiB/s wr, 14 op/s 2024-09-12T01:28:02.657 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:02 smithi001 ceph-mon[30387]: pgmap v349: 129 pgs: 129 active+clean; 632 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 5.0 KiB/s wr, 14 op/s 2024-09-12T01:28:02.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:02 smithi104 ceph-mon[35219]: pgmap v349: 129 pgs: 129 active+clean; 632 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 5.0 KiB/s wr, 14 op/s 2024-09-12T01:28:02.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:02.948+0000 7f9075800640 1 -- 172.21.15.1:0/3360718588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9070106f10 msgr2=0x7f907010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:02.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:02.948+0000 7f9075800640 1 --2- 172.21.15.1:0/3360718588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9070106f10 0x7f907010ea90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9068009ca0 tx=0x7f906802a8c0 comp rx=0 tx=0).stop 2024-09-12T01:28:02.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:02.948+0000 7f9075800640 1 -- 172.21.15.1:0/3360718588 shutdown_connections 2024-09-12T01:28:02.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:02.948+0000 7f9075800640 1 --2- 172.21.15.1:0/3360718588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9070106f10 0x7f907010ea90 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:02.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:02.948+0000 7f9075800640 1 --2- 172.21.15.1:0/3360718588 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9070102790 0x7f90701067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:02.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:02.948+0000 7f9075800640 1 --2- 172.21.15.1:0/3360718588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9070101de0 0x7f90701021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:02.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:02.948+0000 7f9075800640 1 -- 172.21.15.1:0/3360718588 >> 172.21.15.1:0/3360718588 conn(0x7f90700fd8c0 msgr2=0x7f90700ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:02.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:02.948+0000 7f9075800640 1 -- 172.21.15.1:0/3360718588 shutdown_connections 2024-09-12T01:28:02.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:02.948+0000 7f9075800640 1 -- 172.21.15.1:0/3360718588 wait complete. 2024-09-12T01:28:02.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:02.949+0000 7f9075800640 1 Processor -- start 2024-09-12T01:28:02.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:02.949+0000 7f9075800640 1 -- start start 2024-09-12T01:28:02.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:02.949+0000 7f9075800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9070101de0 0x7f907019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:02.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:02.949+0000 7f9075800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9070102790 0x7f907019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:02.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:02.949+0000 7f9075800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9070106f10 0x7f907019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:02.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:02.949+0000 7f9075800640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9070111fc0 con 0x7f9070102790 2024-09-12T01:28:02.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:02.949+0000 7f9075800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9070111e40 con 0x7f9070106f10 2024-09-12T01:28:02.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:02.949+0000 7f9075800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9070112140 con 0x7f9070101de0 2024-09-12T01:28:02.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:02.950+0000 7f9067e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9070102790 0x7f907019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:02.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:02.950+0000 7f906f400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9070106f10 0x7f907019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:02.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:02.950+0000 7f9067e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9070102790 0x7f907019aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40058/0 (socket says 172.21.15.1:40058) 2024-09-12T01:28:02.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:02.950+0000 7f9067e00640 1 -- 172.21.15.1:0/836970577 learned_addr learned my addr 172.21.15.1:0/836970577 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:28:02.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:02.950+0000 7f906ea00640 1 --2- 172.21.15.1:0/836970577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9070101de0 0x7f907019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:02.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:02.950+0000 7f906f400640 1 -- 172.21.15.1:0/836970577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9070101de0 msgr2=0x7f907019a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:02.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:02.950+0000 7f906f400640 1 --2- 172.21.15.1:0/836970577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9070101de0 0x7f907019a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:02.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:02.950+0000 7f906f400640 1 -- 172.21.15.1:0/836970577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9070102790 msgr2=0x7f907019aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:02.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:02.950+0000 7f906f400640 1 --2- 172.21.15.1:0/836970577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9070102790 0x7f907019aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:02.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:02.950+0000 7f906f400640 1 -- 172.21.15.1:0/836970577 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f907019f610 con 0x7f9070106f10 2024-09-12T01:28:02.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:02.950+0000 7f906ea00640 1 --2- 172.21.15.1:0/836970577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9070101de0 0x7f907019a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:28:02.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:02.950+0000 7f906f400640 1 --2- 172.21.15.1:0/836970577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9070106f10 0x7f907019ef30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9068005e30 tx=0x7f9068003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:02.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:02.951+0000 7f9067400640 1 -- 172.21.15.1:0/836970577 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9068031650 con 0x7f9070106f10 2024-09-12T01:28:02.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:02.951+0000 7f9075800640 1 -- 172.21.15.1:0/836970577 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f907019f8a0 con 0x7f9070106f10 2024-09-12T01:28:02.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:02.951+0000 7f9067400640 1 -- 172.21.15.1:0/836970577 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90680317f0 con 0x7f9070106f10 2024-09-12T01:28:02.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:02.951+0000 7f9067400640 1 -- 172.21.15.1:0/836970577 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906802f990 con 0x7f9070106f10 2024-09-12T01:28:02.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:02.951+0000 7f9075800640 1 -- 172.21.15.1:0/836970577 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90701a71e0 con 0x7f9070106f10 2024-09-12T01:28:02.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:02.952+0000 7f9075800640 1 -- 172.21.15.1:0/836970577 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90701021c0 con 0x7f9070106f10 2024-09-12T01:28:02.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:02.954+0000 7f9067400640 1 -- 172.21.15.1:0/836970577 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f906802fb30 con 0x7f9070106f10 2024-09-12T01:28:02.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:02.955+0000 7f9067400640 1 --2- 172.21.15.1:0/836970577 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f90540a2c10 0x7f90540a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:02.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:02.955+0000 7f9067400640 1 -- 172.21.15.1:0/836970577 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9092+0+0 (secure 0 0 0) 0x7f90680f07d0 con 0x7f9070106f10 2024-09-12T01:28:02.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:02.955+0000 7f906ea00640 1 --2- 172.21.15.1:0/836970577 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f90540a2c10 0x7f90540a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:02.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:02.956+0000 7f906ea00640 1 --2- 172.21.15.1:0/836970577 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f90540a2c10 0x7f90540a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f90600095f0 tx=0x7f9060005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:02.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:02.958+0000 7f9067400640 1 -- 172.21.15.1:0/836970577 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f90680319a0 con 0x7f9070106f10 2024-09-12T01:28:03.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:03.139+0000 7f9075800640 1 -- 172.21.15.1:0/836970577 --> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9070075430 con 0x7f90540a2c10 2024-09-12T01:28:03.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:03.150+0000 7f9067400640 1 -- 172.21.15.1:0/836970577 <== mgr.14150 v2:172.21.15.1:6800/1928662456 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340422 (secure 0 0 0) 0x7f9070075430 con 0x7f90540a2c10 2024-09-12T01:28:03.156 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:28:03.158 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-12T01:28:03.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:03.155+0000 7f9075800640 1 -- 172.21.15.1:0/836970577 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f90540a2c10 msgr2=0x7f90540a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:03.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:03.155+0000 7f9075800640 1 --2- 172.21.15.1:0/836970577 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f90540a2c10 0x7f90540a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f90600095f0 tx=0x7f9060005f90 comp rx=0 tx=0).stop 2024-09-12T01:28:03.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:03.155+0000 7f9075800640 1 -- 172.21.15.1:0/836970577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9070106f10 msgr2=0x7f907019ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:03.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:03.155+0000 7f9075800640 1 --2- 172.21.15.1:0/836970577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9070106f10 0x7f907019ef30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9068005e30 tx=0x7f9068003f10 comp rx=0 tx=0).stop 2024-09-12T01:28:03.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:03.155+0000 7f9075800640 1 -- 172.21.15.1:0/836970577 shutdown_connections 2024-09-12T01:28:03.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:03.155+0000 7f9075800640 1 --2- 172.21.15.1:0/836970577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9070106f10 0x7f907019ef30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:03.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:03.155+0000 7f9075800640 1 --2- 172.21.15.1:0/836970577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9070102790 0x7f907019aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:03.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:03.155+0000 7f9075800640 1 --2- 172.21.15.1:0/836970577 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f90540a2c10 0x7f90540a50d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:03.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:03.155+0000 7f9075800640 1 --2- 172.21.15.1:0/836970577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9070101de0 0x7f907019a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:03.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:03.155+0000 7f9075800640 1 -- 172.21.15.1:0/836970577 >> 172.21.15.1:0/836970577 conn(0x7f90700fd8c0 msgr2=0x7f90700fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:03.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:03.155+0000 7f9075800640 1 -- 172.21.15.1:0/836970577 shutdown_connections 2024-09-12T01:28:03.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:03.155+0000 7f9075800640 1 -- 172.21.15.1:0/836970577 wait complete. 2024-09-12T01:28:04.044 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":350,"stamp":"2024-09-12T01:28:02.692238+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654952,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":322,"num_read_kb":282,"num_write":422,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1182348,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":347,"ondisk_log_size":347,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":401704,"kb_used_data":15016,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124540120,"statfs":{"total":1151940427776,"available":1151529082880,"internally_reserved":0,"allocated":15376384,"data_stored":8459136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":395752762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":53532,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":131,"num_read_kb":125,"num_write":28,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003181"},"pg_stats":[{"pgid":"2.1f","version":"96'13","reported_seq":68,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:58.359997+0000","last_change":"2024-09-12T01:26:44.440383+0000","last_active":"2024-09-12T01:27:58.359997+0000","last_peered":"2024-09-12T01:27:58.359997+0000","last_clean":"2024-09-12T01:27:58.359997+0000","last_became_active":"2024-09-12T01:26:44.440048+0000","last_became_peered":"2024-09-12T01:26:44.440048+0000","last_unstale":"2024-09-12T01:27:58.359997+0000","last_undegraded":"2024-09-12T01:27:58.359997+0000","last_fullsized":"2024-09-12T01:27:58.359997+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:05:26.956556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.727618+0000","last_change":"2024-09-12T01:26:47.483952+0000","last_active":"2024-09-12T01:27:16.727618+0000","last_peered":"2024-09-12T01:27:16.727618+0000","last_clean":"2024-09-12T01:27:16.727618+0000","last_became_active":"2024-09-12T01:26:47.483679+0000","last_became_peered":"2024-09-12T01:26:47.483679+0000","last_unstale":"2024-09-12T01:27:16.727618+0000","last_undegraded":"2024-09-12T01:27:16.727618+0000","last_fullsized":"2024-09-12T01:27:16.727618+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:46:53.997935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.614152+0000","last_change":"2024-09-12T01:26:57.529904+0000","last_active":"2024-09-12T01:27:16.614152+0000","last_peered":"2024-09-12T01:27:16.614152+0000","last_clean":"2024-09-12T01:27:16.614152+0000","last_became_active":"2024-09-12T01:26:57.529332+0000","last_became_peered":"2024-09-12T01:26:57.529332+0000","last_unstale":"2024-09-12T01:27:16.614152+0000","last_undegraded":"2024-09-12T01:27:16.614152+0000","last_fullsized":"2024-09-12T01:27:16.614152+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:54.733150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"96'2","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.727867+0000","last_change":"2024-09-12T01:26:44.448605+0000","last_active":"2024-09-12T01:27:16.727867+0000","last_peered":"2024-09-12T01:27:16.727867+0000","last_clean":"2024-09-12T01:27:16.727867+0000","last_became_active":"2024-09-12T01:26:44.447035+0000","last_became_peered":"2024-09-12T01:26:44.447035+0000","last_unstale":"2024-09-12T01:27:16.727867+0000","last_undegraded":"2024-09-12T01:27:16.727867+0000","last_fullsized":"2024-09-12T01:27:16.727867+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:29:44.222912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.612376+0000","last_change":"2024-09-12T01:26:47.475058+0000","last_active":"2024-09-12T01:27:16.612376+0000","last_peered":"2024-09-12T01:27:16.612376+0000","last_clean":"2024-09-12T01:27:16.612376+0000","last_became_active":"2024-09-12T01:26:47.474057+0000","last_became_peered":"2024-09-12T01:26:47.474057+0000","last_unstale":"2024-09-12T01:27:16.612376+0000","last_undegraded":"2024-09-12T01:27:16.612376+0000","last_fullsized":"2024-09-12T01:27:16.612376+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:58:20.039831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.617128+0000","last_change":"2024-09-12T01:26:57.532139+0000","last_active":"2024-09-12T01:27:16.617128+0000","last_peered":"2024-09-12T01:27:16.617128+0000","last_clean":"2024-09-12T01:27:16.617128+0000","last_became_active":"2024-09-12T01:26:57.531718+0000","last_became_peered":"2024-09-12T01:26:57.531718+0000","last_unstale":"2024-09-12T01:27:16.617128+0000","last_undegraded":"2024-09-12T01:27:16.617128+0000","last_fullsized":"2024-09-12T01:27:16.617128+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:10.414782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"96'12","reported_seq":62,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:45.338125+0000","last_change":"2024-09-12T01:27:16.631523+0000","last_active":"2024-09-12T01:27:45.338125+0000","last_peered":"2024-09-12T01:27:45.338125+0000","last_clean":"2024-09-12T01:27:45.338125+0000","last_became_active":"2024-09-12T01:27:16.622591+0000","last_became_peered":"2024-09-12T01:27:16.622591+0000","last_unstale":"2024-09-12T01:27:45.338125+0000","last_undegraded":"2024-09-12T01:27:45.338125+0000","last_fullsized":"2024-09-12T01:27:45.338125+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:14:12.793775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.616175+0000","last_change":"2024-09-12T01:26:47.477891+0000","last_active":"2024-09-12T01:27:16.616175+0000","last_peered":"2024-09-12T01:27:16.616175+0000","last_clean":"2024-09-12T01:27:16.616175+0000","last_became_active":"2024-09-12T01:26:47.477107+0000","last_became_peered":"2024-09-12T01:26:47.477107+0000","last_unstale":"2024-09-12T01:27:16.616175+0000","last_undegraded":"2024-09-12T01:27:16.616175+0000","last_fullsized":"2024-09-12T01:27:16.616175+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:35:17.262896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":31,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.832962+0000","last_change":"2024-09-12T01:27:16.832962+0000","last_active":"2024-09-12T01:27:16.832962+0000","last_peered":"2024-09-12T01:27:16.832962+0000","last_clean":"2024-09-12T01:27:16.832962+0000","last_became_active":"2024-09-12T01:27:16.832463+0000","last_became_peered":"2024-09-12T01:27:16.832463+0000","last_unstale":"2024-09-12T01:27:16.832962+0000","last_undegraded":"2024-09-12T01:27:16.832962+0000","last_fullsized":"2024-09-12T01:27:16.832962+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:10:24.321133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"96'5","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:44.326346+0000","last_change":"2024-09-12T01:26:44.445375+0000","last_active":"2024-09-12T01:27:44.326346+0000","last_peered":"2024-09-12T01:27:44.326346+0000","last_clean":"2024-09-12T01:27:44.326346+0000","last_became_active":"2024-09-12T01:26:44.444820+0000","last_became_peered":"2024-09-12T01:26:44.444820+0000","last_unstale":"2024-09-12T01:27:44.326346+0000","last_undegraded":"2024-09-12T01:27:44.326346+0000","last_fullsized":"2024-09-12T01:27:44.326346+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:05:40.790222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.831260+0000","last_change":"2024-09-12T01:26:47.480650+0000","last_active":"2024-09-12T01:27:16.831260+0000","last_peered":"2024-09-12T01:27:16.831260+0000","last_clean":"2024-09-12T01:27:16.831260+0000","last_became_active":"2024-09-12T01:26:47.480266+0000","last_became_peered":"2024-09-12T01:26:47.480266+0000","last_unstale":"2024-09-12T01:27:16.831260+0000","last_undegraded":"2024-09-12T01:27:16.831260+0000","last_fullsized":"2024-09-12T01:27:16.831260+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:00.218397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.617076+0000","last_change":"2024-09-12T01:26:57.532582+0000","last_active":"2024-09-12T01:27:16.617076+0000","last_peered":"2024-09-12T01:27:16.617076+0000","last_clean":"2024-09-12T01:27:16.617076+0000","last_became_active":"2024-09-12T01:26:57.531966+0000","last_became_peered":"2024-09-12T01:26:57.531966+0000","last_unstale":"2024-09-12T01:27:16.617076+0000","last_undegraded":"2024-09-12T01:27:16.617076+0000","last_fullsized":"2024-09-12T01:27:16.617076+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:01:13.886678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.618598+0000","last_change":"2024-09-12T01:26:47.486567+0000","last_active":"2024-09-12T01:27:16.618598+0000","last_peered":"2024-09-12T01:27:16.618598+0000","last_clean":"2024-09-12T01:27:16.618598+0000","last_became_active":"2024-09-12T01:26:47.486219+0000","last_became_peered":"2024-09-12T01:26:47.486219+0000","last_unstale":"2024-09-12T01:27:16.618598+0000","last_undegraded":"2024-09-12T01:27:16.618598+0000","last_fullsized":"2024-09-12T01:27:16.618598+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:48:53.889111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"96'4","reported_seq":18,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:41.303051+0000","last_change":"2024-09-12T01:27:16.624095+0000","last_active":"2024-09-12T01:27:41.303051+0000","last_peered":"2024-09-12T01:27:41.303051+0000","last_clean":"2024-09-12T01:27:41.303051+0000","last_became_active":"2024-09-12T01:27:16.620392+0000","last_became_peered":"2024-09-12T01:27:16.620392+0000","last_unstale":"2024-09-12T01:27:41.303051+0000","last_undegraded":"2024-09-12T01:27:41.303051+0000","last_fullsized":"2024-09-12T01:27:41.303051+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:56:05.516283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.618083+0000","last_change":"2024-09-12T01:26:57.729865+0000","last_active":"2024-09-12T01:27:16.618083+0000","last_peered":"2024-09-12T01:27:16.618083+0000","last_clean":"2024-09-12T01:27:16.618083+0000","last_became_active":"2024-09-12T01:26:57.728819+0000","last_became_peered":"2024-09-12T01:26:57.728819+0000","last_unstale":"2024-09-12T01:27:16.618083+0000","last_undegraded":"2024-09-12T01:27:16.618083+0000","last_fullsized":"2024-09-12T01:27:16.618083+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:11.337683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.727426+0000","last_change":"2024-09-12T01:26:47.485503+0000","last_active":"2024-09-12T01:27:16.727426+0000","last_peered":"2024-09-12T01:27:16.727426+0000","last_clean":"2024-09-12T01:27:16.727426+0000","last_became_active":"2024-09-12T01:26:47.485187+0000","last_became_peered":"2024-09-12T01:26:47.485187+0000","last_unstale":"2024-09-12T01:27:16.727426+0000","last_undegraded":"2024-09-12T01:27:16.727426+0000","last_fullsized":"2024-09-12T01:27:16.727426+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:24:52.544761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"96'3","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:43.321325+0000","last_change":"2024-09-12T01:26:44.447010+0000","last_active":"2024-09-12T01:27:43.321325+0000","last_peered":"2024-09-12T01:27:43.321325+0000","last_clean":"2024-09-12T01:27:43.321325+0000","last_became_active":"2024-09-12T01:26:44.446643+0000","last_became_peered":"2024-09-12T01:26:44.446643+0000","last_unstale":"2024-09-12T01:27:43.321325+0000","last_undegraded":"2024-09-12T01:27:43.321325+0000","last_fullsized":"2024-09-12T01:27:43.321325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:56:33.298732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"96'1","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.618128+0000","last_change":"2024-09-12T01:26:57.524687+0000","last_active":"2024-09-12T01:27:16.618128+0000","last_peered":"2024-09-12T01:27:16.618128+0000","last_clean":"2024-09-12T01:27:16.618128+0000","last_became_active":"2024-09-12T01:26:57.524201+0000","last_became_peered":"2024-09-12T01:26:57.524201+0000","last_unstale":"2024-09-12T01:27:16.618128+0000","last_undegraded":"2024-09-12T01:27:16.618128+0000","last_fullsized":"2024-09-12T01:27:16.618128+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:51:49.147460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.617325+0000","last_change":"2024-09-12T01:26:47.478432+0000","last_active":"2024-09-12T01:27:16.617325+0000","last_peered":"2024-09-12T01:27:16.617325+0000","last_clean":"2024-09-12T01:27:16.617325+0000","last_became_active":"2024-09-12T01:26:47.478109+0000","last_became_peered":"2024-09-12T01:26:47.478109+0000","last_unstale":"2024-09-12T01:27:16.617325+0000","last_undegraded":"2024-09-12T01:27:16.617325+0000","last_fullsized":"2024-09-12T01:27:16.617325+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:30:19.917875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"96'7","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:44.327496+0000","last_change":"2024-09-12T01:26:44.441961+0000","last_active":"2024-09-12T01:27:44.327496+0000","last_peered":"2024-09-12T01:27:44.327496+0000","last_clean":"2024-09-12T01:27:44.327496+0000","last_became_active":"2024-09-12T01:26:44.441644+0000","last_became_peered":"2024-09-12T01:26:44.441644+0000","last_unstale":"2024-09-12T01:27:44.327496+0000","last_undegraded":"2024-09-12T01:27:44.327496+0000","last_fullsized":"2024-09-12T01:27:44.327496+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:40:48.561119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.831832+0000","last_change":"2024-09-12T01:26:57.531164+0000","last_active":"2024-09-12T01:27:16.831832+0000","last_peered":"2024-09-12T01:27:16.831832+0000","last_clean":"2024-09-12T01:27:16.831832+0000","last_became_active":"2024-09-12T01:26:57.530513+0000","last_became_peered":"2024-09-12T01:26:57.530513+0000","last_unstale":"2024-09-12T01:27:16.831832+0000","last_undegraded":"2024-09-12T01:27:16.831832+0000","last_fullsized":"2024-09-12T01:27:16.831832+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:08:55.385620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.617323+0000","last_change":"2024-09-12T01:26:47.475562+0000","last_active":"2024-09-12T01:27:16.617323+0000","last_peered":"2024-09-12T01:27:16.617323+0000","last_clean":"2024-09-12T01:27:16.617323+0000","last_became_active":"2024-09-12T01:26:47.475167+0000","last_became_peered":"2024-09-12T01:26:47.475167+0000","last_unstale":"2024-09-12T01:27:16.617323+0000","last_undegraded":"2024-09-12T01:27:16.617323+0000","last_fullsized":"2024-09-12T01:27:16.617323+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:39:34.706811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"96'2","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:41.301425+0000","last_change":"2024-09-12T01:26:44.447264+0000","last_active":"2024-09-12T01:27:41.301425+0000","last_peered":"2024-09-12T01:27:41.301425+0000","last_clean":"2024-09-12T01:27:41.301425+0000","last_became_active":"2024-09-12T01:26:44.446885+0000","last_became_peered":"2024-09-12T01:26:44.446885+0000","last_unstale":"2024-09-12T01:27:41.301425+0000","last_undegraded":"2024-09-12T01:27:41.301425+0000","last_fullsized":"2024-09-12T01:27:41.301425+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:56:27.229949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.618060+0000","last_change":"2024-09-12T01:26:57.729792+0000","last_active":"2024-09-12T01:27:16.618060+0000","last_peered":"2024-09-12T01:27:16.618060+0000","last_clean":"2024-09-12T01:27:16.618060+0000","last_became_active":"2024-09-12T01:26:57.729252+0000","last_became_peered":"2024-09-12T01:26:57.729252+0000","last_unstale":"2024-09-12T01:27:16.618060+0000","last_undegraded":"2024-09-12T01:27:16.618060+0000","last_fullsized":"2024-09-12T01:27:16.618060+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:34:57.106270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.612181+0000","last_change":"2024-09-12T01:26:47.479179+0000","last_active":"2024-09-12T01:27:16.612181+0000","last_peered":"2024-09-12T01:27:16.612181+0000","last_clean":"2024-09-12T01:27:16.612181+0000","last_became_active":"2024-09-12T01:26:47.478783+0000","last_became_peered":"2024-09-12T01:26:47.478783+0000","last_unstale":"2024-09-12T01:27:16.612181+0000","last_undegraded":"2024-09-12T01:27:16.612181+0000","last_fullsized":"2024-09-12T01:27:16.612181+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:23:22.548509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"96'7","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:44.328377+0000","last_change":"2024-09-12T01:26:44.471854+0000","last_active":"2024-09-12T01:27:44.328377+0000","last_peered":"2024-09-12T01:27:44.328377+0000","last_clean":"2024-09-12T01:27:44.328377+0000","last_became_active":"2024-09-12T01:26:44.471511+0000","last_became_peered":"2024-09-12T01:26:44.471511+0000","last_unstale":"2024-09-12T01:27:44.328377+0000","last_undegraded":"2024-09-12T01:27:44.328377+0000","last_fullsized":"2024-09-12T01:27:44.328377+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:00:26.639984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.614289+0000","last_change":"2024-09-12T01:26:57.520467+0000","last_active":"2024-09-12T01:27:16.614289+0000","last_peered":"2024-09-12T01:27:16.614289+0000","last_clean":"2024-09-12T01:27:16.614289+0000","last_became_active":"2024-09-12T01:26:57.520048+0000","last_became_peered":"2024-09-12T01:26:57.520048+0000","last_unstale":"2024-09-12T01:27:16.614289+0000","last_undegraded":"2024-09-12T01:27:16.614289+0000","last_fullsized":"2024-09-12T01:27:16.614289+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:42:05.303804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.727805+0000","last_change":"2024-09-12T01:26:47.476484+0000","last_active":"2024-09-12T01:27:16.727805+0000","last_peered":"2024-09-12T01:27:16.727805+0000","last_clean":"2024-09-12T01:27:16.727805+0000","last_became_active":"2024-09-12T01:26:47.475610+0000","last_became_peered":"2024-09-12T01:26:47.475610+0000","last_unstale":"2024-09-12T01:27:16.727805+0000","last_undegraded":"2024-09-12T01:27:16.727805+0000","last_fullsized":"2024-09-12T01:27:16.727805+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:44.637986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"96'12","reported_seq":51,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:57.487267+0000","last_change":"2024-09-12T01:26:44.442988+0000","last_active":"2024-09-12T01:27:57.487267+0000","last_peered":"2024-09-12T01:27:57.487267+0000","last_clean":"2024-09-12T01:27:57.487267+0000","last_became_active":"2024-09-12T01:26:44.442612+0000","last_became_peered":"2024-09-12T01:26:44.442612+0000","last_unstale":"2024-09-12T01:27:57.487267+0000","last_undegraded":"2024-09-12T01:27:57.487267+0000","last_fullsized":"2024-09-12T01:27:57.487267+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:38:25.486470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.727142+0000","last_change":"2024-09-12T01:26:57.729762+0000","last_active":"2024-09-12T01:27:16.727142+0000","last_peered":"2024-09-12T01:27:16.727142+0000","last_clean":"2024-09-12T01:27:16.727142+0000","last_became_active":"2024-09-12T01:26:57.728944+0000","last_became_peered":"2024-09-12T01:26:57.728944+0000","last_unstale":"2024-09-12T01:27:16.727142+0000","last_undegraded":"2024-09-12T01:27:16.727142+0000","last_fullsized":"2024-09-12T01:27:16.727142+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:33:58.030332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.615545+0000","last_change":"2024-09-12T01:26:47.482512+0000","last_active":"2024-09-12T01:27:16.615545+0000","last_peered":"2024-09-12T01:27:16.615545+0000","last_clean":"2024-09-12T01:27:16.615545+0000","last_became_active":"2024-09-12T01:26:47.481344+0000","last_became_peered":"2024-09-12T01:26:47.481344+0000","last_unstale":"2024-09-12T01:27:16.615545+0000","last_undegraded":"2024-09-12T01:27:16.615545+0000","last_fullsized":"2024-09-12T01:27:16.615545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:52:22.920248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"96'4","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:44.326689+0000","last_change":"2024-09-12T01:26:44.447654+0000","last_active":"2024-09-12T01:27:44.326689+0000","last_peered":"2024-09-12T01:27:44.326689+0000","last_clean":"2024-09-12T01:27:44.326689+0000","last_became_active":"2024-09-12T01:26:44.447148+0000","last_became_peered":"2024-09-12T01:26:44.447148+0000","last_unstale":"2024-09-12T01:27:44.326689+0000","last_undegraded":"2024-09-12T01:27:44.326689+0000","last_fullsized":"2024-09-12T01:27:44.326689+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:40:02.928329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.616781+0000","last_change":"2024-09-12T01:26:57.530191+0000","last_active":"2024-09-12T01:27:16.616781+0000","last_peered":"2024-09-12T01:27:16.616781+0000","last_clean":"2024-09-12T01:27:16.616781+0000","last_became_active":"2024-09-12T01:26:57.529752+0000","last_became_peered":"2024-09-12T01:26:57.529752+0000","last_unstale":"2024-09-12T01:27:16.616781+0000","last_undegraded":"2024-09-12T01:27:16.616781+0000","last_fullsized":"2024-09-12T01:27:16.616781+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:00:16.639297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.612993+0000","last_change":"2024-09-12T01:26:47.478991+0000","last_active":"2024-09-12T01:27:16.612993+0000","last_peered":"2024-09-12T01:27:16.612993+0000","last_clean":"2024-09-12T01:27:16.612993+0000","last_became_active":"2024-09-12T01:26:47.478228+0000","last_became_peered":"2024-09-12T01:26:47.478228+0000","last_unstale":"2024-09-12T01:27:16.612993+0000","last_undegraded":"2024-09-12T01:27:16.612993+0000","last_fullsized":"2024-09-12T01:27:16.612993+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:45.236546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"96'1","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.831760+0000","last_change":"2024-09-12T01:26:44.441143+0000","last_active":"2024-09-12T01:27:16.831760+0000","last_peered":"2024-09-12T01:27:16.831760+0000","last_clean":"2024-09-12T01:27:16.831760+0000","last_became_active":"2024-09-12T01:26:44.439361+0000","last_became_peered":"2024-09-12T01:26:44.439361+0000","last_unstale":"2024-09-12T01:27:16.831760+0000","last_undegraded":"2024-09-12T01:27:16.831760+0000","last_fullsized":"2024-09-12T01:27:16.831760+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:22:34.257977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.617430+0000","last_change":"2024-09-12T01:26:57.729010+0000","last_active":"2024-09-12T01:27:16.617430+0000","last_peered":"2024-09-12T01:27:16.617430+0000","last_clean":"2024-09-12T01:27:16.617430+0000","last_became_active":"2024-09-12T01:26:57.728576+0000","last_became_peered":"2024-09-12T01:26:57.728576+0000","last_unstale":"2024-09-12T01:27:16.617430+0000","last_undegraded":"2024-09-12T01:27:16.617430+0000","last_fullsized":"2024-09-12T01:27:16.617430+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:48:48.798028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.832723+0000","last_change":"2024-09-12T01:27:16.832723+0000","last_active":"2024-09-12T01:27:16.832723+0000","last_peered":"2024-09-12T01:27:16.832723+0000","last_clean":"2024-09-12T01:27:16.832723+0000","last_became_active":"2024-09-12T01:27:16.832241+0000","last_became_peered":"2024-09-12T01:27:16.832241+0000","last_unstale":"2024-09-12T01:27:16.832723+0000","last_undegraded":"2024-09-12T01:27:16.832723+0000","last_fullsized":"2024-09-12T01:27:16.832723+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:26:09.610468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"96'6","reported_seq":61,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:56.358349+0000","last_change":"2024-09-12T01:26:44.440802+0000","last_active":"2024-09-12T01:27:56.358349+0000","last_peered":"2024-09-12T01:27:56.358349+0000","last_clean":"2024-09-12T01:27:56.358349+0000","last_became_active":"2024-09-12T01:26:44.439686+0000","last_became_peered":"2024-09-12T01:26:44.439686+0000","last_unstale":"2024-09-12T01:27:56.358349+0000","last_undegraded":"2024-09-12T01:27:56.358349+0000","last_fullsized":"2024-09-12T01:27:56.358349+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:56.608975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.615436+0000","last_change":"2024-09-12T01:26:57.537046+0000","last_active":"2024-09-12T01:27:16.615436+0000","last_peered":"2024-09-12T01:27:16.615436+0000","last_clean":"2024-09-12T01:27:16.615436+0000","last_became_active":"2024-09-12T01:26:57.536589+0000","last_became_peered":"2024-09-12T01:26:57.536589+0000","last_unstale":"2024-09-12T01:27:16.615436+0000","last_undegraded":"2024-09-12T01:27:16.615436+0000","last_fullsized":"2024-09-12T01:27:16.615436+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:24:54.232963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.831136+0000","last_change":"2024-09-12T01:26:47.476288+0000","last_active":"2024-09-12T01:27:16.831136+0000","last_peered":"2024-09-12T01:27:16.831136+0000","last_clean":"2024-09-12T01:27:16.831136+0000","last_became_active":"2024-09-12T01:26:47.475836+0000","last_became_peered":"2024-09-12T01:26:47.475836+0000","last_unstale":"2024-09-12T01:27:16.831136+0000","last_undegraded":"2024-09-12T01:27:16.831136+0000","last_fullsized":"2024-09-12T01:27:16.831136+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:39:43.017253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"96'5","reported_seq":61,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:59.358978+0000","last_change":"2024-09-12T01:26:44.446539+0000","last_active":"2024-09-12T01:27:59.358978+0000","last_peered":"2024-09-12T01:27:59.358978+0000","last_clean":"2024-09-12T01:27:59.358978+0000","last_became_active":"2024-09-12T01:26:44.446219+0000","last_became_peered":"2024-09-12T01:26:44.446219+0000","last_unstale":"2024-09-12T01:27:59.358978+0000","last_undegraded":"2024-09-12T01:27:59.358978+0000","last_fullsized":"2024-09-12T01:27:59.358978+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:50:32.584078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.615779+0000","last_change":"2024-09-12T01:26:57.729445+0000","last_active":"2024-09-12T01:27:16.615779+0000","last_peered":"2024-09-12T01:27:16.615779+0000","last_clean":"2024-09-12T01:27:16.615779+0000","last_became_active":"2024-09-12T01:26:57.728948+0000","last_became_peered":"2024-09-12T01:26:57.728948+0000","last_unstale":"2024-09-12T01:27:16.615779+0000","last_undegraded":"2024-09-12T01:27:16.615779+0000","last_fullsized":"2024-09-12T01:27:16.615779+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:33:27.017565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.614620+0000","last_change":"2024-09-12T01:26:47.481965+0000","last_active":"2024-09-12T01:27:16.614620+0000","last_peered":"2024-09-12T01:27:16.614620+0000","last_clean":"2024-09-12T01:27:16.614620+0000","last_became_active":"2024-09-12T01:26:47.481600+0000","last_became_peered":"2024-09-12T01:26:47.481600+0000","last_unstale":"2024-09-12T01:27:16.614620+0000","last_undegraded":"2024-09-12T01:27:16.614620+0000","last_fullsized":"2024-09-12T01:27:16.614620+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:20.078494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"96'4","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:42.307242+0000","last_change":"2024-09-12T01:27:16.832522+0000","last_active":"2024-09-12T01:27:42.307242+0000","last_peered":"2024-09-12T01:27:42.307242+0000","last_clean":"2024-09-12T01:27:42.307242+0000","last_became_active":"2024-09-12T01:27:16.832090+0000","last_became_peered":"2024-09-12T01:27:16.832090+0000","last_unstale":"2024-09-12T01:27:42.307242+0000","last_undegraded":"2024-09-12T01:27:42.307242+0000","last_fullsized":"2024-09-12T01:27:42.307242+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:15:48.061559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.831364+0000","last_change":"2024-09-12T01:26:57.729622+0000","last_active":"2024-09-12T01:27:16.831364+0000","last_peered":"2024-09-12T01:27:16.831364+0000","last_clean":"2024-09-12T01:27:16.831364+0000","last_became_active":"2024-09-12T01:26:57.729175+0000","last_became_peered":"2024-09-12T01:26:57.729175+0000","last_unstale":"2024-09-12T01:27:16.831364+0000","last_undegraded":"2024-09-12T01:27:16.831364+0000","last_fullsized":"2024-09-12T01:27:16.831364+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:41.691685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.727458+0000","last_change":"2024-09-12T01:26:47.479155+0000","last_active":"2024-09-12T01:27:16.727458+0000","last_peered":"2024-09-12T01:27:16.727458+0000","last_clean":"2024-09-12T01:27:16.727458+0000","last_became_active":"2024-09-12T01:26:47.478781+0000","last_became_peered":"2024-09-12T01:26:47.478781+0000","last_unstale":"2024-09-12T01:27:16.727458+0000","last_undegraded":"2024-09-12T01:27:16.727458+0000","last_fullsized":"2024-09-12T01:27:16.727458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:40:49.730016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"96'6","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:45.336494+0000","last_change":"2024-09-12T01:26:44.437048+0000","last_active":"2024-09-12T01:27:45.336494+0000","last_peered":"2024-09-12T01:27:45.336494+0000","last_clean":"2024-09-12T01:27:45.336494+0000","last_became_active":"2024-09-12T01:26:44.436702+0000","last_became_peered":"2024-09-12T01:26:44.436702+0000","last_unstale":"2024-09-12T01:27:45.336494+0000","last_undegraded":"2024-09-12T01:27:45.336494+0000","last_fullsized":"2024-09-12T01:27:45.336494+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:15:18.857093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":358,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.618344+0000","last_change":"2024-09-12T01:25:28.683724+0000","last_active":"2024-09-12T01:27:16.618344+0000","last_peered":"2024-09-12T01:27:16.618344+0000","last_clean":"2024-09-12T01:27:16.618344+0000","last_became_active":"2024-09-12T01:25:28.683155+0000","last_became_peered":"2024-09-12T01:25:28.683155+0000","last_unstale":"2024-09-12T01:27:16.618344+0000","last_undegraded":"2024-09-12T01:27:16.618344+0000","last_fullsized":"2024-09-12T01:27:16.618344+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:21:41.529611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:21:41.529611+0000","last_clean_scrub_stamp":"2024-09-12T01:21:41.529611+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:13:02.724915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"96'1","reported_seq":32,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.833617+0000","last_change":"2024-09-12T01:27:16.833028+0000","last_active":"2024-09-12T01:27:16.833617+0000","last_peered":"2024-09-12T01:27:16.833617+0000","last_clean":"2024-09-12T01:27:16.833617+0000","last_became_active":"2024-09-12T01:27:16.832416+0000","last_became_peered":"2024-09-12T01:27:16.832416+0000","last_unstale":"2024-09-12T01:27:16.833617+0000","last_undegraded":"2024-09-12T01:27:16.833617+0000","last_fullsized":"2024-09-12T01:27:16.833617+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:49:43.008107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.619425+0000","last_change":"2024-09-12T01:26:47.475467+0000","last_active":"2024-09-12T01:27:16.619425+0000","last_peered":"2024-09-12T01:27:16.619425+0000","last_clean":"2024-09-12T01:27:16.619425+0000","last_became_active":"2024-09-12T01:26:47.474724+0000","last_became_peered":"2024-09-12T01:26:47.474724+0000","last_unstale":"2024-09-12T01:27:16.619425+0000","last_undegraded":"2024-09-12T01:27:16.619425+0000","last_fullsized":"2024-09-12T01:27:16.619425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:56:54.651180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"96'6","reported_seq":65,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:59.362236+0000","last_change":"2024-09-12T01:26:44.472048+0000","last_active":"2024-09-12T01:27:59.362236+0000","last_peered":"2024-09-12T01:27:59.362236+0000","last_clean":"2024-09-12T01:27:59.362236+0000","last_became_active":"2024-09-12T01:26:44.471697+0000","last_became_peered":"2024-09-12T01:26:44.471697+0000","last_unstale":"2024-09-12T01:27:59.362236+0000","last_undegraded":"2024-09-12T01:27:59.362236+0000","last_fullsized":"2024-09-12T01:27:59.362236+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:44:13.071983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":19,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.616587+0000","last_change":"2024-09-12T01:26:57.528837+0000","last_active":"2024-09-12T01:27:16.616587+0000","last_peered":"2024-09-12T01:27:16.616587+0000","last_clean":"2024-09-12T01:27:16.616587+0000","last_became_active":"2024-09-12T01:26:57.528285+0000","last_became_peered":"2024-09-12T01:26:57.528285+0000","last_unstale":"2024-09-12T01:27:16.616587+0000","last_undegraded":"2024-09-12T01:27:16.616587+0000","last_fullsized":"2024-09-12T01:27:16.616587+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:12:13.534002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.618818+0000","last_change":"2024-09-12T01:26:47.483456+0000","last_active":"2024-09-12T01:27:16.618818+0000","last_peered":"2024-09-12T01:27:16.618818+0000","last_clean":"2024-09-12T01:27:16.618818+0000","last_became_active":"2024-09-12T01:26:47.482189+0000","last_became_peered":"2024-09-12T01:26:47.482189+0000","last_unstale":"2024-09-12T01:27:16.618818+0000","last_undegraded":"2024-09-12T01:27:16.618818+0000","last_fullsized":"2024-09-12T01:27:16.618818+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:18:55.466266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"96'3","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:44.328146+0000","last_change":"2024-09-12T01:26:44.443507+0000","last_active":"2024-09-12T01:27:44.328146+0000","last_peered":"2024-09-12T01:27:44.328146+0000","last_clean":"2024-09-12T01:27:44.328146+0000","last_became_active":"2024-09-12T01:26:44.443147+0000","last_became_peered":"2024-09-12T01:26:44.443147+0000","last_unstale":"2024-09-12T01:27:44.328146+0000","last_undegraded":"2024-09-12T01:27:44.328146+0000","last_fullsized":"2024-09-12T01:27:44.328146+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:38:40.433581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.612289+0000","last_change":"2024-09-12T01:26:57.523029+0000","last_active":"2024-09-12T01:27:16.612289+0000","last_peered":"2024-09-12T01:27:16.612289+0000","last_clean":"2024-09-12T01:27:16.612289+0000","last_became_active":"2024-09-12T01:26:57.522565+0000","last_became_peered":"2024-09-12T01:26:57.522565+0000","last_unstale":"2024-09-12T01:27:16.612289+0000","last_undegraded":"2024-09-12T01:27:16.612289+0000","last_fullsized":"2024-09-12T01:27:16.612289+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:36:27.432126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.619446+0000","last_change":"2024-09-12T01:26:47.471958+0000","last_active":"2024-09-12T01:27:16.619446+0000","last_peered":"2024-09-12T01:27:16.619446+0000","last_clean":"2024-09-12T01:27:16.619446+0000","last_became_active":"2024-09-12T01:26:47.471621+0000","last_became_peered":"2024-09-12T01:26:47.471621+0000","last_unstale":"2024-09-12T01:27:16.619446+0000","last_undegraded":"2024-09-12T01:27:16.619446+0000","last_fullsized":"2024-09-12T01:27:16.619446+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:46:38.004188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"96'5","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:44.328360+0000","last_change":"2024-09-12T01:26:44.442852+0000","last_active":"2024-09-12T01:27:44.328360+0000","last_peered":"2024-09-12T01:27:44.328360+0000","last_clean":"2024-09-12T01:27:44.328360+0000","last_became_active":"2024-09-12T01:26:44.442487+0000","last_became_peered":"2024-09-12T01:26:44.442487+0000","last_unstale":"2024-09-12T01:27:44.328360+0000","last_undegraded":"2024-09-12T01:27:44.328360+0000","last_fullsized":"2024-09-12T01:27:44.328360+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:04:32.688925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.830495+0000","last_change":"2024-09-12T01:26:57.521155+0000","last_active":"2024-09-12T01:27:16.830495+0000","last_peered":"2024-09-12T01:27:16.830495+0000","last_clean":"2024-09-12T01:27:16.830495+0000","last_became_active":"2024-09-12T01:26:57.520721+0000","last_became_peered":"2024-09-12T01:26:57.520721+0000","last_unstale":"2024-09-12T01:27:16.830495+0000","last_undegraded":"2024-09-12T01:27:16.830495+0000","last_fullsized":"2024-09-12T01:27:16.830495+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:58:52.004697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.616687+0000","last_change":"2024-09-12T01:26:47.482564+0000","last_active":"2024-09-12T01:27:16.616687+0000","last_peered":"2024-09-12T01:27:16.616687+0000","last_clean":"2024-09-12T01:27:16.616687+0000","last_became_active":"2024-09-12T01:26:47.482215+0000","last_became_peered":"2024-09-12T01:26:47.482215+0000","last_unstale":"2024-09-12T01:27:16.616687+0000","last_undegraded":"2024-09-12T01:27:16.616687+0000","last_fullsized":"2024-09-12T01:27:16.616687+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:27:40.882473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"96'2","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:44.302545+0000","last_change":"2024-09-12T01:26:44.442774+0000","last_active":"2024-09-12T01:27:44.302545+0000","last_peered":"2024-09-12T01:27:44.302545+0000","last_clean":"2024-09-12T01:27:44.302545+0000","last_became_active":"2024-09-12T01:26:44.442389+0000","last_became_peered":"2024-09-12T01:26:44.442389+0000","last_unstale":"2024-09-12T01:27:44.302545+0000","last_undegraded":"2024-09-12T01:27:44.302545+0000","last_fullsized":"2024-09-12T01:27:44.302545+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:22.628392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.830638+0000","last_change":"2024-09-12T01:26:57.526182+0000","last_active":"2024-09-12T01:27:16.830638+0000","last_peered":"2024-09-12T01:27:16.830638+0000","last_clean":"2024-09-12T01:27:16.830638+0000","last_became_active":"2024-09-12T01:26:57.525869+0000","last_became_peered":"2024-09-12T01:26:57.525869+0000","last_unstale":"2024-09-12T01:27:16.830638+0000","last_undegraded":"2024-09-12T01:27:16.830638+0000","last_fullsized":"2024-09-12T01:27:16.830638+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:40:09.994361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.830839+0000","last_change":"2024-09-12T01:26:47.475872+0000","last_active":"2024-09-12T01:27:16.830839+0000","last_peered":"2024-09-12T01:27:16.830839+0000","last_clean":"2024-09-12T01:27:16.830839+0000","last_became_active":"2024-09-12T01:26:47.475536+0000","last_became_peered":"2024-09-12T01:26:47.475536+0000","last_unstale":"2024-09-12T01:27:16.830839+0000","last_undegraded":"2024-09-12T01:27:16.830839+0000","last_fullsized":"2024-09-12T01:27:16.830839+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:03:37.519530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"96'17","reported_seq":57,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:44.324198+0000","last_change":"2024-09-12T01:26:44.448843+0000","last_active":"2024-09-12T01:27:44.324198+0000","last_peered":"2024-09-12T01:27:44.324198+0000","last_clean":"2024-09-12T01:27:44.324198+0000","last_became_active":"2024-09-12T01:26:44.448451+0000","last_became_peered":"2024-09-12T01:26:44.448451+0000","last_unstale":"2024-09-12T01:27:44.324198+0000","last_undegraded":"2024-09-12T01:27:44.324198+0000","last_fullsized":"2024-09-12T01:27:44.324198+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:59:21.422174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.615765+0000","last_change":"2024-09-12T01:26:57.541164+0000","last_active":"2024-09-12T01:27:16.615765+0000","last_peered":"2024-09-12T01:27:16.615765+0000","last_clean":"2024-09-12T01:27:16.615765+0000","last_became_active":"2024-09-12T01:26:57.540746+0000","last_became_peered":"2024-09-12T01:26:57.540746+0000","last_unstale":"2024-09-12T01:27:16.615765+0000","last_undegraded":"2024-09-12T01:27:16.615765+0000","last_fullsized":"2024-09-12T01:27:16.615765+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:58:26.870141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.618651+0000","last_change":"2024-09-12T01:26:47.473261+0000","last_active":"2024-09-12T01:27:16.618651+0000","last_peered":"2024-09-12T01:27:16.618651+0000","last_clean":"2024-09-12T01:27:16.618651+0000","last_became_active":"2024-09-12T01:26:47.472902+0000","last_became_peered":"2024-09-12T01:26:47.472902+0000","last_unstale":"2024-09-12T01:27:16.618651+0000","last_undegraded":"2024-09-12T01:27:16.618651+0000","last_fullsized":"2024-09-12T01:27:16.618651+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:58:41.080240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"96'6","reported_seq":62,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:57.356815+0000","last_change":"2024-09-12T01:26:44.440503+0000","last_active":"2024-09-12T01:27:57.356815+0000","last_peered":"2024-09-12T01:27:57.356815+0000","last_clean":"2024-09-12T01:27:57.356815+0000","last_became_active":"2024-09-12T01:26:44.440128+0000","last_became_peered":"2024-09-12T01:26:44.440128+0000","last_unstale":"2024-09-12T01:27:57.356815+0000","last_undegraded":"2024-09-12T01:27:57.356815+0000","last_fullsized":"2024-09-12T01:27:57.356815+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:39:35.373682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.831878+0000","last_change":"2024-09-12T01:26:57.533883+0000","last_active":"2024-09-12T01:27:16.831878+0000","last_peered":"2024-09-12T01:27:16.831878+0000","last_clean":"2024-09-12T01:27:16.831878+0000","last_became_active":"2024-09-12T01:26:57.533329+0000","last_became_peered":"2024-09-12T01:26:57.533329+0000","last_unstale":"2024-09-12T01:27:16.831878+0000","last_undegraded":"2024-09-12T01:27:16.831878+0000","last_fullsized":"2024-09-12T01:27:16.831878+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:17.655219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.618886+0000","last_change":"2024-09-12T01:26:47.480276+0000","last_active":"2024-09-12T01:27:16.618886+0000","last_peered":"2024-09-12T01:27:16.618886+0000","last_clean":"2024-09-12T01:27:16.618886+0000","last_became_active":"2024-09-12T01:26:47.479965+0000","last_became_peered":"2024-09-12T01:26:47.479965+0000","last_unstale":"2024-09-12T01:27:16.618886+0000","last_undegraded":"2024-09-12T01:27:16.618886+0000","last_fullsized":"2024-09-12T01:27:16.618886+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:24:10.339000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"96'5","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:44.328224+0000","last_change":"2024-09-12T01:26:44.442346+0000","last_active":"2024-09-12T01:27:44.328224+0000","last_peered":"2024-09-12T01:27:44.328224+0000","last_clean":"2024-09-12T01:27:44.328224+0000","last_became_active":"2024-09-12T01:26:44.442044+0000","last_became_peered":"2024-09-12T01:26:44.442044+0000","last_unstale":"2024-09-12T01:27:44.328224+0000","last_undegraded":"2024-09-12T01:27:44.328224+0000","last_fullsized":"2024-09-12T01:27:44.328224+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:47:00.193554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.616223+0000","last_change":"2024-09-12T01:26:57.531650+0000","last_active":"2024-09-12T01:27:16.616223+0000","last_peered":"2024-09-12T01:27:16.616223+0000","last_clean":"2024-09-12T01:27:16.616223+0000","last_became_active":"2024-09-12T01:26:57.531184+0000","last_became_peered":"2024-09-12T01:26:57.531184+0000","last_unstale":"2024-09-12T01:27:16.616223+0000","last_undegraded":"2024-09-12T01:27:16.616223+0000","last_fullsized":"2024-09-12T01:27:16.616223+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:42:56.191490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.615182+0000","last_change":"2024-09-12T01:26:47.480344+0000","last_active":"2024-09-12T01:27:16.615182+0000","last_peered":"2024-09-12T01:27:16.615182+0000","last_clean":"2024-09-12T01:27:16.615182+0000","last_became_active":"2024-09-12T01:26:47.479966+0000","last_became_peered":"2024-09-12T01:26:47.479966+0000","last_unstale":"2024-09-12T01:27:16.615182+0000","last_undegraded":"2024-09-12T01:27:16.615182+0000","last_fullsized":"2024-09-12T01:27:16.615182+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:44:31.968977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"96'9","reported_seq":51,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:45.336523+0000","last_change":"2024-09-12T01:26:44.447708+0000","last_active":"2024-09-12T01:27:45.336523+0000","last_peered":"2024-09-12T01:27:45.336523+0000","last_clean":"2024-09-12T01:27:45.336523+0000","last_became_active":"2024-09-12T01:26:44.446974+0000","last_became_peered":"2024-09-12T01:26:44.446974+0000","last_unstale":"2024-09-12T01:27:45.336523+0000","last_undegraded":"2024-09-12T01:27:45.336523+0000","last_fullsized":"2024-09-12T01:27:45.336523+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:01:10.440044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.831215+0000","last_change":"2024-09-12T01:26:57.533128+0000","last_active":"2024-09-12T01:27:16.831215+0000","last_peered":"2024-09-12T01:27:16.831215+0000","last_clean":"2024-09-12T01:27:16.831215+0000","last_became_active":"2024-09-12T01:26:57.532731+0000","last_became_peered":"2024-09-12T01:26:57.532731+0000","last_unstale":"2024-09-12T01:27:16.831215+0000","last_undegraded":"2024-09-12T01:27:16.831215+0000","last_fullsized":"2024-09-12T01:27:16.831215+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:09:06.118313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.618798+0000","last_change":"2024-09-12T01:26:47.471545+0000","last_active":"2024-09-12T01:27:16.618798+0000","last_peered":"2024-09-12T01:27:16.618798+0000","last_clean":"2024-09-12T01:27:16.618798+0000","last_became_active":"2024-09-12T01:26:47.471234+0000","last_became_peered":"2024-09-12T01:26:47.471234+0000","last_unstale":"2024-09-12T01:27:16.618798+0000","last_undegraded":"2024-09-12T01:27:16.618798+0000","last_fullsized":"2024-09-12T01:27:16.618798+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:24:00.844203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"96'31","reported_seq":126,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:28:01.361940+0000","last_change":"2024-09-12T01:27:16.629235+0000","last_active":"2024-09-12T01:28:01.361940+0000","last_peered":"2024-09-12T01:28:01.361940+0000","last_clean":"2024-09-12T01:28:01.361940+0000","last_became_active":"2024-09-12T01:27:16.621719+0000","last_became_peered":"2024-09-12T01:27:16.621719+0000","last_unstale":"2024-09-12T01:28:01.361940+0000","last_undegraded":"2024-09-12T01:28:01.361940+0000","last_fullsized":"2024-09-12T01:28:01.361940+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:45:43.011452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56749,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":40,"num_read_kb":75,"num_write":33,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.831355+0000","last_change":"2024-09-12T01:26:57.521950+0000","last_active":"2024-09-12T01:27:16.831355+0000","last_peered":"2024-09-12T01:27:16.831355+0000","last_clean":"2024-09-12T01:27:16.831355+0000","last_became_active":"2024-09-12T01:26:57.521574+0000","last_became_peered":"2024-09-12T01:26:57.521574+0000","last_unstale":"2024-09-12T01:27:16.831355+0000","last_undegraded":"2024-09-12T01:27:16.831355+0000","last_fullsized":"2024-09-12T01:27:16.831355+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:23.086233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.616172+0000","last_change":"2024-09-12T01:26:47.478212+0000","last_active":"2024-09-12T01:27:16.616172+0000","last_peered":"2024-09-12T01:27:16.616172+0000","last_clean":"2024-09-12T01:27:16.616172+0000","last_became_active":"2024-09-12T01:26:47.477553+0000","last_became_peered":"2024-09-12T01:26:47.477553+0000","last_unstale":"2024-09-12T01:27:16.616172+0000","last_undegraded":"2024-09-12T01:27:16.616172+0000","last_fullsized":"2024-09-12T01:27:16.616172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:48:22.390040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"96'5","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:43.320016+0000","last_change":"2024-09-12T01:26:44.441558+0000","last_active":"2024-09-12T01:27:43.320016+0000","last_peered":"2024-09-12T01:27:43.320016+0000","last_clean":"2024-09-12T01:27:43.320016+0000","last_became_active":"2024-09-12T01:26:44.441099+0000","last_became_peered":"2024-09-12T01:26:44.441099+0000","last_unstale":"2024-09-12T01:27:43.320016+0000","last_undegraded":"2024-09-12T01:27:43.320016+0000","last_fullsized":"2024-09-12T01:27:43.320016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:07:40.566150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.617323+0000","last_change":"2024-09-12T01:26:57.524838+0000","last_active":"2024-09-12T01:27:16.617323+0000","last_peered":"2024-09-12T01:27:16.617323+0000","last_clean":"2024-09-12T01:27:16.617323+0000","last_became_active":"2024-09-12T01:26:57.524182+0000","last_became_peered":"2024-09-12T01:26:57.524182+0000","last_unstale":"2024-09-12T01:27:16.617323+0000","last_undegraded":"2024-09-12T01:27:16.617323+0000","last_fullsized":"2024-09-12T01:27:16.617323+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:18:55.493116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.612416+0000","last_change":"2024-09-12T01:26:47.478562+0000","last_active":"2024-09-12T01:27:16.612416+0000","last_peered":"2024-09-12T01:27:16.612416+0000","last_clean":"2024-09-12T01:27:16.612416+0000","last_became_active":"2024-09-12T01:26:47.478223+0000","last_became_peered":"2024-09-12T01:26:47.478223+0000","last_unstale":"2024-09-12T01:27:16.612416+0000","last_undegraded":"2024-09-12T01:27:16.612416+0000","last_fullsized":"2024-09-12T01:27:16.612416+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:32:20.472870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"96'4","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:44.325427+0000","last_change":"2024-09-12T01:26:44.443434+0000","last_active":"2024-09-12T01:27:44.325427+0000","last_peered":"2024-09-12T01:27:44.325427+0000","last_clean":"2024-09-12T01:27:44.325427+0000","last_became_active":"2024-09-12T01:26:44.443097+0000","last_became_peered":"2024-09-12T01:26:44.443097+0000","last_unstale":"2024-09-12T01:27:44.325427+0000","last_undegraded":"2024-09-12T01:27:44.325427+0000","last_fullsized":"2024-09-12T01:27:44.325427+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:48:56.626378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.831047+0000","last_change":"2024-09-12T01:26:57.532507+0000","last_active":"2024-09-12T01:27:16.831047+0000","last_peered":"2024-09-12T01:27:16.831047+0000","last_clean":"2024-09-12T01:27:16.831047+0000","last_became_active":"2024-09-12T01:26:57.529757+0000","last_became_peered":"2024-09-12T01:26:57.529757+0000","last_unstale":"2024-09-12T01:27:16.831047+0000","last_undegraded":"2024-09-12T01:27:16.831047+0000","last_fullsized":"2024-09-12T01:27:16.831047+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:35.589141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.616354+0000","last_change":"2024-09-12T01:26:47.474632+0000","last_active":"2024-09-12T01:27:16.616354+0000","last_peered":"2024-09-12T01:27:16.616354+0000","last_clean":"2024-09-12T01:27:16.616354+0000","last_became_active":"2024-09-12T01:26:47.473791+0000","last_became_peered":"2024-09-12T01:26:47.473791+0000","last_unstale":"2024-09-12T01:27:16.616354+0000","last_undegraded":"2024-09-12T01:27:16.616354+0000","last_fullsized":"2024-09-12T01:27:16.616354+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:58:45.241657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"96'4","reported_seq":77,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:28:01.360991+0000","last_change":"2024-09-12T01:26:44.445045+0000","last_active":"2024-09-12T01:28:01.360991+0000","last_peered":"2024-09-12T01:28:01.360991+0000","last_clean":"2024-09-12T01:28:01.360991+0000","last_became_active":"2024-09-12T01:26:44.444668+0000","last_became_peered":"2024-09-12T01:26:44.444668+0000","last_unstale":"2024-09-12T01:28:01.360991+0000","last_undegraded":"2024-09-12T01:28:01.360991+0000","last_fullsized":"2024-09-12T01:28:01.360991+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:05:14.112438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.727294+0000","last_change":"2024-09-12T01:26:57.523792+0000","last_active":"2024-09-12T01:27:16.727294+0000","last_peered":"2024-09-12T01:27:16.727294+0000","last_clean":"2024-09-12T01:27:16.727294+0000","last_became_active":"2024-09-12T01:26:57.523361+0000","last_became_peered":"2024-09-12T01:26:57.523361+0000","last_unstale":"2024-09-12T01:27:16.727294+0000","last_undegraded":"2024-09-12T01:27:16.727294+0000","last_fullsized":"2024-09-12T01:27:16.727294+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:42:46.177077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.611989+0000","last_change":"2024-09-12T01:26:57.534204+0000","last_active":"2024-09-12T01:27:16.611989+0000","last_peered":"2024-09-12T01:27:16.611989+0000","last_clean":"2024-09-12T01:27:16.611989+0000","last_became_active":"2024-09-12T01:26:57.532094+0000","last_became_peered":"2024-09-12T01:26:57.532094+0000","last_unstale":"2024-09-12T01:27:16.611989+0000","last_undegraded":"2024-09-12T01:27:16.611989+0000","last_fullsized":"2024-09-12T01:27:16.611989+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:43:17.714896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.831451+0000","last_change":"2024-09-12T01:26:47.480169+0000","last_active":"2024-09-12T01:27:16.831451+0000","last_peered":"2024-09-12T01:27:16.831451+0000","last_clean":"2024-09-12T01:27:16.831451+0000","last_became_active":"2024-09-12T01:26:47.479853+0000","last_became_peered":"2024-09-12T01:26:47.479853+0000","last_unstale":"2024-09-12T01:27:16.831451+0000","last_undegraded":"2024-09-12T01:27:16.831451+0000","last_fullsized":"2024-09-12T01:27:16.831451+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:58:10.578637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"96'3","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:44.326245+0000","last_change":"2024-09-12T01:26:44.440641+0000","last_active":"2024-09-12T01:27:44.326245+0000","last_peered":"2024-09-12T01:27:44.326245+0000","last_clean":"2024-09-12T01:27:44.326245+0000","last_became_active":"2024-09-12T01:26:44.440315+0000","last_became_peered":"2024-09-12T01:26:44.440315+0000","last_unstale":"2024-09-12T01:27:44.326245+0000","last_undegraded":"2024-09-12T01:27:44.326245+0000","last_fullsized":"2024-09-12T01:27:44.326245+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:21:52.919820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.616589+0000","last_change":"2024-09-12T01:26:57.532046+0000","last_active":"2024-09-12T01:27:16.616589+0000","last_peered":"2024-09-12T01:27:16.616589+0000","last_clean":"2024-09-12T01:27:16.616589+0000","last_became_active":"2024-09-12T01:26:57.531660+0000","last_became_peered":"2024-09-12T01:26:57.531660+0000","last_unstale":"2024-09-12T01:27:16.616589+0000","last_undegraded":"2024-09-12T01:27:16.616589+0000","last_fullsized":"2024-09-12T01:27:16.616589+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:26:12.650687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.727258+0000","last_change":"2024-09-12T01:26:47.476383+0000","last_active":"2024-09-12T01:27:16.727258+0000","last_peered":"2024-09-12T01:27:16.727258+0000","last_clean":"2024-09-12T01:27:16.727258+0000","last_became_active":"2024-09-12T01:26:47.475759+0000","last_became_peered":"2024-09-12T01:26:47.475759+0000","last_unstale":"2024-09-12T01:27:16.727258+0000","last_undegraded":"2024-09-12T01:27:16.727258+0000","last_fullsized":"2024-09-12T01:27:16.727258+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:14:38.898332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"96'3","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:42.309751+0000","last_change":"2024-09-12T01:26:44.445418+0000","last_active":"2024-09-12T01:27:42.309751+0000","last_peered":"2024-09-12T01:27:42.309751+0000","last_clean":"2024-09-12T01:27:42.309751+0000","last_became_active":"2024-09-12T01:26:44.444396+0000","last_became_peered":"2024-09-12T01:26:44.444396+0000","last_unstale":"2024-09-12T01:27:42.309751+0000","last_undegraded":"2024-09-12T01:27:42.309751+0000","last_fullsized":"2024-09-12T01:27:42.309751+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:16:16.595947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"96'7","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:43.319874+0000","last_change":"2024-09-12T01:27:16.620062+0000","last_active":"2024-09-12T01:27:43.319874+0000","last_peered":"2024-09-12T01:27:43.319874+0000","last_clean":"2024-09-12T01:27:43.319874+0000","last_became_active":"2024-09-12T01:27:16.619031+0000","last_became_peered":"2024-09-12T01:27:16.619031+0000","last_unstale":"2024-09-12T01:27:43.319874+0000","last_undegraded":"2024-09-12T01:27:43.319874+0000","last_fullsized":"2024-09-12T01:27:43.319874+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:04:04.329211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.831159+0000","last_change":"2024-09-12T01:26:47.478706+0000","last_active":"2024-09-12T01:27:16.831159+0000","last_peered":"2024-09-12T01:27:16.831159+0000","last_clean":"2024-09-12T01:27:16.831159+0000","last_became_active":"2024-09-12T01:26:47.478339+0000","last_became_peered":"2024-09-12T01:26:47.478339+0000","last_unstale":"2024-09-12T01:27:16.831159+0000","last_undegraded":"2024-09-12T01:27:16.831159+0000","last_fullsized":"2024-09-12T01:27:16.831159+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:20:37.180485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.614983+0000","last_change":"2024-09-12T01:26:57.533268+0000","last_active":"2024-09-12T01:27:16.614983+0000","last_peered":"2024-09-12T01:27:16.614983+0000","last_clean":"2024-09-12T01:27:16.614983+0000","last_became_active":"2024-09-12T01:26:57.532862+0000","last_became_peered":"2024-09-12T01:26:57.532862+0000","last_unstale":"2024-09-12T01:27:16.614983+0000","last_undegraded":"2024-09-12T01:27:16.614983+0000","last_fullsized":"2024-09-12T01:27:16.614983+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:53.659734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"96'5","reported_seq":61,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:58.357220+0000","last_change":"2024-09-12T01:26:44.443113+0000","last_active":"2024-09-12T01:27:58.357220+0000","last_peered":"2024-09-12T01:27:58.357220+0000","last_clean":"2024-09-12T01:27:58.357220+0000","last_became_active":"2024-09-12T01:26:44.442642+0000","last_became_peered":"2024-09-12T01:26:44.442642+0000","last_unstale":"2024-09-12T01:27:58.357220+0000","last_undegraded":"2024-09-12T01:27:58.357220+0000","last_fullsized":"2024-09-12T01:27:58.357220+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:16:20.143475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.618053+0000","last_change":"2024-09-12T01:26:47.476115+0000","last_active":"2024-09-12T01:27:16.618053+0000","last_peered":"2024-09-12T01:27:16.618053+0000","last_clean":"2024-09-12T01:27:16.618053+0000","last_became_active":"2024-09-12T01:26:47.475815+0000","last_became_peered":"2024-09-12T01:26:47.475815+0000","last_unstale":"2024-09-12T01:27:16.618053+0000","last_undegraded":"2024-09-12T01:27:16.618053+0000","last_fullsized":"2024-09-12T01:27:16.618053+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:50.775693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.615201+0000","last_change":"2024-09-12T01:26:57.518831+0000","last_active":"2024-09-12T01:27:16.615201+0000","last_peered":"2024-09-12T01:27:16.615201+0000","last_clean":"2024-09-12T01:27:16.615201+0000","last_became_active":"2024-09-12T01:26:57.518439+0000","last_became_peered":"2024-09-12T01:26:57.518439+0000","last_unstale":"2024-09-12T01:27:16.615201+0000","last_undegraded":"2024-09-12T01:27:16.615201+0000","last_fullsized":"2024-09-12T01:27:16.615201+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:49:14.845248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.832809+0000","last_change":"2024-09-12T01:27:16.832809+0000","last_active":"2024-09-12T01:27:16.832809+0000","last_peered":"2024-09-12T01:27:16.832809+0000","last_clean":"2024-09-12T01:27:16.832809+0000","last_became_active":"2024-09-12T01:27:16.832243+0000","last_became_peered":"2024-09-12T01:27:16.832243+0000","last_unstale":"2024-09-12T01:27:16.832809+0000","last_undegraded":"2024-09-12T01:27:16.832809+0000","last_fullsized":"2024-09-12T01:27:16.832809+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:55:00.002884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.614548+0000","last_change":"2024-09-12T01:26:47.475959+0000","last_active":"2024-09-12T01:27:16.614548+0000","last_peered":"2024-09-12T01:27:16.614548+0000","last_clean":"2024-09-12T01:27:16.614548+0000","last_became_active":"2024-09-12T01:26:47.475660+0000","last_became_peered":"2024-09-12T01:26:47.475660+0000","last_unstale":"2024-09-12T01:27:16.614548+0000","last_undegraded":"2024-09-12T01:27:16.614548+0000","last_fullsized":"2024-09-12T01:27:16.614548+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:42.556069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.614300+0000","last_change":"2024-09-12T01:26:47.474380+0000","last_active":"2024-09-12T01:27:16.614300+0000","last_peered":"2024-09-12T01:27:16.614300+0000","last_clean":"2024-09-12T01:27:16.614300+0000","last_became_active":"2024-09-12T01:26:47.474050+0000","last_became_peered":"2024-09-12T01:26:47.474050+0000","last_unstale":"2024-09-12T01:27:16.614300+0000","last_undegraded":"2024-09-12T01:27:16.614300+0000","last_fullsized":"2024-09-12T01:27:16.614300+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:19:05.032866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.612222+0000","last_change":"2024-09-12T01:26:47.477056+0000","last_active":"2024-09-12T01:27:16.612222+0000","last_peered":"2024-09-12T01:27:16.612222+0000","last_clean":"2024-09-12T01:27:16.612222+0000","last_became_active":"2024-09-12T01:26:47.476641+0000","last_became_peered":"2024-09-12T01:26:47.476641+0000","last_unstale":"2024-09-12T01:27:16.612222+0000","last_undegraded":"2024-09-12T01:27:16.612222+0000","last_fullsized":"2024-09-12T01:27:16.612222+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:52.924471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.832763+0000","last_change":"2024-09-12T01:27:16.832763+0000","last_active":"2024-09-12T01:27:16.832763+0000","last_peered":"2024-09-12T01:27:16.832763+0000","last_clean":"2024-09-12T01:27:16.832763+0000","last_became_active":"2024-09-12T01:27:16.832396+0000","last_became_peered":"2024-09-12T01:27:16.832396+0000","last_unstale":"2024-09-12T01:27:16.832763+0000","last_undegraded":"2024-09-12T01:27:16.832763+0000","last_fullsized":"2024-09-12T01:27:16.832763+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:59.444289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.616683+0000","last_change":"2024-09-12T01:26:47.472125+0000","last_active":"2024-09-12T01:27:16.616683+0000","last_peered":"2024-09-12T01:27:16.616683+0000","last_clean":"2024-09-12T01:27:16.616683+0000","last_became_active":"2024-09-12T01:26:47.471758+0000","last_became_peered":"2024-09-12T01:26:47.471758+0000","last_unstale":"2024-09-12T01:27:16.616683+0000","last_undegraded":"2024-09-12T01:27:16.616683+0000","last_fullsized":"2024-09-12T01:27:16.616683+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:38.949566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.615278+0000","last_change":"2024-09-12T01:26:47.473584+0000","last_active":"2024-09-12T01:27:16.615278+0000","last_peered":"2024-09-12T01:27:16.615278+0000","last_clean":"2024-09-12T01:27:16.615278+0000","last_became_active":"2024-09-12T01:26:47.470089+0000","last_became_peered":"2024-09-12T01:26:47.470089+0000","last_unstale":"2024-09-12T01:27:16.615278+0000","last_undegraded":"2024-09-12T01:27:16.615278+0000","last_fullsized":"2024-09-12T01:27:16.615278+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:44:58.878810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.727704+0000","last_change":"2024-09-12T01:26:47.486547+0000","last_active":"2024-09-12T01:27:16.727704+0000","last_peered":"2024-09-12T01:27:16.727704+0000","last_clean":"2024-09-12T01:27:16.727704+0000","last_became_active":"2024-09-12T01:26:47.486105+0000","last_became_peered":"2024-09-12T01:26:47.486105+0000","last_unstale":"2024-09-12T01:27:16.727704+0000","last_undegraded":"2024-09-12T01:27:16.727704+0000","last_fullsized":"2024-09-12T01:27:16.727704+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:21:18.820930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.830807+0000","last_change":"2024-09-12T01:26:47.480459+0000","last_active":"2024-09-12T01:27:16.830807+0000","last_peered":"2024-09-12T01:27:16.830807+0000","last_clean":"2024-09-12T01:27:16.830807+0000","last_became_active":"2024-09-12T01:26:47.478917+0000","last_became_peered":"2024-09-12T01:26:47.478917+0000","last_unstale":"2024-09-12T01:27:16.830807+0000","last_undegraded":"2024-09-12T01:27:16.830807+0000","last_fullsized":"2024-09-12T01:27:16.830807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:20:54.470888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.727398+0000","last_change":"2024-09-12T01:26:47.480744+0000","last_active":"2024-09-12T01:27:16.727398+0000","last_peered":"2024-09-12T01:27:16.727398+0000","last_clean":"2024-09-12T01:27:16.727398+0000","last_became_active":"2024-09-12T01:26:47.480354+0000","last_became_peered":"2024-09-12T01:26:47.480354+0000","last_unstale":"2024-09-12T01:27:16.727398+0000","last_undegraded":"2024-09-12T01:27:16.727398+0000","last_fullsized":"2024-09-12T01:27:16.727398+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:02:04.670945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.616718+0000","last_change":"2024-09-12T01:26:47.476431+0000","last_active":"2024-09-12T01:27:16.616718+0000","last_peered":"2024-09-12T01:27:16.616718+0000","last_clean":"2024-09-12T01:27:16.616718+0000","last_became_active":"2024-09-12T01:26:47.476110+0000","last_became_peered":"2024-09-12T01:26:47.476110+0000","last_unstale":"2024-09-12T01:27:16.616718+0000","last_undegraded":"2024-09-12T01:27:16.616718+0000","last_fullsized":"2024-09-12T01:27:16.616718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:34:24.086677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.615591+0000","last_change":"2024-09-12T01:26:47.484568+0000","last_active":"2024-09-12T01:27:16.615591+0000","last_peered":"2024-09-12T01:27:16.615591+0000","last_clean":"2024-09-12T01:27:16.615591+0000","last_became_active":"2024-09-12T01:26:47.484165+0000","last_became_peered":"2024-09-12T01:26:47.484165+0000","last_unstale":"2024-09-12T01:27:16.615591+0000","last_undegraded":"2024-09-12T01:27:16.615591+0000","last_fullsized":"2024-09-12T01:27:16.615591+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:26:51.324754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.616471+0000","last_change":"2024-09-12T01:26:47.482608+0000","last_active":"2024-09-12T01:27:16.616471+0000","last_peered":"2024-09-12T01:27:16.616471+0000","last_clean":"2024-09-12T01:27:16.616471+0000","last_became_active":"2024-09-12T01:26:47.482299+0000","last_became_peered":"2024-09-12T01:26:47.482299+0000","last_unstale":"2024-09-12T01:27:16.616471+0000","last_undegraded":"2024-09-12T01:27:16.616471+0000","last_fullsized":"2024-09-12T01:27:16.616471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:48:39.516539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.615626+0000","last_change":"2024-09-12T01:26:47.472976+0000","last_active":"2024-09-12T01:27:16.615626+0000","last_peered":"2024-09-12T01:27:16.615626+0000","last_clean":"2024-09-12T01:27:16.615626+0000","last_became_active":"2024-09-12T01:26:47.472495+0000","last_became_peered":"2024-09-12T01:26:47.472495+0000","last_unstale":"2024-09-12T01:27:16.615626+0000","last_undegraded":"2024-09-12T01:27:16.615626+0000","last_fullsized":"2024-09-12T01:27:16.615626+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:57.556675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.831973+0000","last_change":"2024-09-12T01:26:47.476004+0000","last_active":"2024-09-12T01:27:16.831973+0000","last_peered":"2024-09-12T01:27:16.831973+0000","last_clean":"2024-09-12T01:27:16.831973+0000","last_became_active":"2024-09-12T01:26:47.475440+0000","last_became_peered":"2024-09-12T01:26:47.475440+0000","last_unstale":"2024-09-12T01:27:16.831973+0000","last_undegraded":"2024-09-12T01:27:16.831973+0000","last_fullsized":"2024-09-12T01:27:16.831973+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:36:47.650577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.617378+0000","last_change":"2024-09-12T01:26:47.483848+0000","last_active":"2024-09-12T01:27:16.617378+0000","last_peered":"2024-09-12T01:27:16.617378+0000","last_clean":"2024-09-12T01:27:16.617378+0000","last_became_active":"2024-09-12T01:26:47.483446+0000","last_became_peered":"2024-09-12T01:26:47.483446+0000","last_unstale":"2024-09-12T01:27:16.617378+0000","last_undegraded":"2024-09-12T01:27:16.617378+0000","last_fullsized":"2024-09-12T01:27:16.617378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:47.844458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.615704+0000","last_change":"2024-09-12T01:26:47.473801+0000","last_active":"2024-09-12T01:27:16.615704+0000","last_peered":"2024-09-12T01:27:16.615704+0000","last_clean":"2024-09-12T01:27:16.615704+0000","last_became_active":"2024-09-12T01:26:47.473268+0000","last_became_peered":"2024-09-12T01:26:47.473268+0000","last_unstale":"2024-09-12T01:27:16.615704+0000","last_undegraded":"2024-09-12T01:27:16.615704+0000","last_fullsized":"2024-09-12T01:27:16.615704+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:13:30.595643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.611890+0000","last_change":"2024-09-12T01:26:47.475778+0000","last_active":"2024-09-12T01:27:16.611890+0000","last_peered":"2024-09-12T01:27:16.611890+0000","last_clean":"2024-09-12T01:27:16.611890+0000","last_became_active":"2024-09-12T01:26:47.475477+0000","last_became_peered":"2024-09-12T01:26:47.475477+0000","last_unstale":"2024-09-12T01:27:16.611890+0000","last_undegraded":"2024-09-12T01:27:16.611890+0000","last_fullsized":"2024-09-12T01:27:16.611890+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:56.555424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.611897+0000","last_change":"2024-09-12T01:26:47.474736+0000","last_active":"2024-09-12T01:27:16.611897+0000","last_peered":"2024-09-12T01:27:16.611897+0000","last_clean":"2024-09-12T01:27:16.611897+0000","last_became_active":"2024-09-12T01:26:47.474378+0000","last_became_peered":"2024-09-12T01:26:47.474378+0000","last_unstale":"2024-09-12T01:27:16.611897+0000","last_undegraded":"2024-09-12T01:27:16.611897+0000","last_fullsized":"2024-09-12T01:27:16.611897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:02:37.818552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.615491+0000","last_change":"2024-09-12T01:26:47.479586+0000","last_active":"2024-09-12T01:27:16.615491+0000","last_peered":"2024-09-12T01:27:16.615491+0000","last_clean":"2024-09-12T01:27:16.615491+0000","last_became_active":"2024-09-12T01:26:47.473216+0000","last_became_peered":"2024-09-12T01:26:47.473216+0000","last_unstale":"2024-09-12T01:27:16.615491+0000","last_undegraded":"2024-09-12T01:27:16.615491+0000","last_fullsized":"2024-09-12T01:27:16.615491+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:05:28.994081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.618661+0000","last_change":"2024-09-12T01:26:47.475634+0000","last_active":"2024-09-12T01:27:16.618661+0000","last_peered":"2024-09-12T01:27:16.618661+0000","last_clean":"2024-09-12T01:27:16.618661+0000","last_became_active":"2024-09-12T01:26:47.475319+0000","last_became_peered":"2024-09-12T01:26:47.475319+0000","last_unstale":"2024-09-12T01:27:16.618661+0000","last_undegraded":"2024-09-12T01:27:16.618661+0000","last_fullsized":"2024-09-12T01:27:16.618661+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:40.095276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.619362+0000","last_change":"2024-09-12T01:26:47.475721+0000","last_active":"2024-09-12T01:27:16.619362+0000","last_peered":"2024-09-12T01:27:16.619362+0000","last_clean":"2024-09-12T01:27:16.619362+0000","last_became_active":"2024-09-12T01:26:47.475149+0000","last_became_peered":"2024-09-12T01:26:47.475149+0000","last_unstale":"2024-09-12T01:27:16.619362+0000","last_undegraded":"2024-09-12T01:27:16.619362+0000","last_fullsized":"2024-09-12T01:27:16.619362+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:27:19.946756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.613490+0000","last_change":"2024-09-12T01:26:47.482667+0000","last_active":"2024-09-12T01:27:16.613490+0000","last_peered":"2024-09-12T01:27:16.613490+0000","last_clean":"2024-09-12T01:27:16.613490+0000","last_became_active":"2024-09-12T01:26:47.482258+0000","last_became_peered":"2024-09-12T01:26:47.482258+0000","last_unstale":"2024-09-12T01:27:16.613490+0000","last_undegraded":"2024-09-12T01:27:16.613490+0000","last_fullsized":"2024-09-12T01:27:16.613490+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:10.449000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.616427+0000","last_change":"2024-09-12T01:26:47.475406+0000","last_active":"2024-09-12T01:27:16.616427+0000","last_peered":"2024-09-12T01:27:16.616427+0000","last_clean":"2024-09-12T01:27:16.616427+0000","last_became_active":"2024-09-12T01:26:47.474846+0000","last_became_peered":"2024-09-12T01:26:47.474846+0000","last_unstale":"2024-09-12T01:27:16.616427+0000","last_undegraded":"2024-09-12T01:27:16.616427+0000","last_fullsized":"2024-09-12T01:27:16.616427+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:54:02.527691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.727210+0000","last_change":"2024-09-12T01:26:47.477383+0000","last_active":"2024-09-12T01:27:16.727210+0000","last_peered":"2024-09-12T01:27:16.727210+0000","last_clean":"2024-09-12T01:27:16.727210+0000","last_became_active":"2024-09-12T01:26:47.476977+0000","last_became_peered":"2024-09-12T01:26:47.476977+0000","last_unstale":"2024-09-12T01:27:16.727210+0000","last_undegraded":"2024-09-12T01:27:16.727210+0000","last_fullsized":"2024-09-12T01:27:16.727210+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:51:45.310556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.617664+0000","last_change":"2024-09-12T01:26:47.473686+0000","last_active":"2024-09-12T01:27:16.617664+0000","last_peered":"2024-09-12T01:27:16.617664+0000","last_clean":"2024-09-12T01:27:16.617664+0000","last_became_active":"2024-09-12T01:26:47.473363+0000","last_became_peered":"2024-09-12T01:26:47.473363+0000","last_unstale":"2024-09-12T01:27:16.617664+0000","last_undegraded":"2024-09-12T01:27:16.617664+0000","last_fullsized":"2024-09-12T01:27:16.617664+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:08:23.238434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.612749+0000","last_change":"2024-09-12T01:26:47.479310+0000","last_active":"2024-09-12T01:27:16.612749+0000","last_peered":"2024-09-12T01:27:16.612749+0000","last_clean":"2024-09-12T01:27:16.612749+0000","last_became_active":"2024-09-12T01:26:47.477416+0000","last_became_peered":"2024-09-12T01:26:47.477416+0000","last_unstale":"2024-09-12T01:27:16.612749+0000","last_undegraded":"2024-09-12T01:27:16.612749+0000","last_fullsized":"2024-09-12T01:27:16.612749+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:01:55.023455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.617943+0000","last_change":"2024-09-12T01:26:47.478455+0000","last_active":"2024-09-12T01:27:16.617943+0000","last_peered":"2024-09-12T01:27:16.617943+0000","last_clean":"2024-09-12T01:27:16.617943+0000","last_became_active":"2024-09-12T01:26:47.478093+0000","last_became_peered":"2024-09-12T01:26:47.478093+0000","last_unstale":"2024-09-12T01:27:16.617943+0000","last_undegraded":"2024-09-12T01:27:16.617943+0000","last_fullsized":"2024-09-12T01:27:16.617943+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:04:41.626677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.618563+0000","last_change":"2024-09-12T01:26:47.479385+0000","last_active":"2024-09-12T01:27:16.618563+0000","last_peered":"2024-09-12T01:27:16.618563+0000","last_clean":"2024-09-12T01:27:16.618563+0000","last_became_active":"2024-09-12T01:26:47.479025+0000","last_became_peered":"2024-09-12T01:26:47.479025+0000","last_unstale":"2024-09-12T01:27:16.618563+0000","last_undegraded":"2024-09-12T01:27:16.618563+0000","last_fullsized":"2024-09-12T01:27:16.618563+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:11.927111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.612094+0000","last_change":"2024-09-12T01:26:47.461567+0000","last_active":"2024-09-12T01:27:16.612094+0000","last_peered":"2024-09-12T01:27:16.612094+0000","last_clean":"2024-09-12T01:27:16.612094+0000","last_became_active":"2024-09-12T01:26:47.461230+0000","last_became_peered":"2024-09-12T01:26:47.461230+0000","last_unstale":"2024-09-12T01:27:16.612094+0000","last_undegraded":"2024-09-12T01:27:16.612094+0000","last_fullsized":"2024-09-12T01:27:16.612094+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:09:24.519448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.612091+0000","last_change":"2024-09-12T01:26:47.472323+0000","last_active":"2024-09-12T01:27:16.612091+0000","last_peered":"2024-09-12T01:27:16.612091+0000","last_clean":"2024-09-12T01:27:16.612091+0000","last_became_active":"2024-09-12T01:26:47.469837+0000","last_became_peered":"2024-09-12T01:26:47.469837+0000","last_unstale":"2024-09-12T01:27:16.612091+0000","last_undegraded":"2024-09-12T01:27:16.612091+0000","last_fullsized":"2024-09-12T01:27:16.612091+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:46.202656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.617266+0000","last_change":"2024-09-12T01:26:47.479707+0000","last_active":"2024-09-12T01:27:16.617266+0000","last_peered":"2024-09-12T01:27:16.617266+0000","last_clean":"2024-09-12T01:27:16.617266+0000","last_became_active":"2024-09-12T01:26:47.479368+0000","last_became_peered":"2024-09-12T01:26:47.479368+0000","last_unstale":"2024-09-12T01:27:16.617266+0000","last_undegraded":"2024-09-12T01:27:16.617266+0000","last_fullsized":"2024-09-12T01:27:16.617266+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:04:30.129381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64350,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":196,"num_read_kb":173,"num_write":239,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":185399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416412,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1101824,"data_stored":545474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645345,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37096,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708056,"statfs":{"total":95995035648,"available":95957049344,"internally_reserved":0,"allocated":1794048,"data_stored":1193135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808870,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1126400,"data_stored":545011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070507,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":545389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332144,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37024,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708128,"statfs":{"total":95995035648,"available":95957123072,"internally_reserved":0,"allocated":1720320,"data_stored":1142281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495672,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":540908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757309,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37032,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708120,"statfs":{"total":95995035648,"available":95957114880,"internally_reserved":0,"allocated":1728512,"data_stored":1137077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920835,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1167360,"data_stored":594080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182472,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1114112,"data_stored":540223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36436,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708716,"statfs":{"total":95995035648,"available":95957725184,"internally_reserved":0,"allocated":1118208,"data_stored":542158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36472,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708680,"statfs":{"total":95995035648,"available":95957688320,"internally_reserved":0,"allocated":1155072,"data_stored":591036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36444,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708708,"statfs":{"total":95995035648,"available":95957716992,"internally_reserved":0,"allocated":1126400,"data_stored":542364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":58894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":50191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":58878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T01:28:04.050 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T01:28:04.432 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:28:04.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:04 smithi001 ceph-mon[30387]: pgmap v350: 129 pgs: 129 active+clean; 640 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.4 KiB/s wr, 13 op/s 2024-09-12T01:28:04.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:04 smithi001 ceph-mon[30387]: from='client.24755 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:28:04.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:04 smithi053 ceph-mon[35158]: pgmap v350: 129 pgs: 129 active+clean; 640 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.4 KiB/s wr, 13 op/s 2024-09-12T01:28:04.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:04 smithi053 ceph-mon[35158]: from='client.24755 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:28:04.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:04 smithi104 ceph-mon[35219]: pgmap v350: 129 pgs: 129 active+clean; 640 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.4 KiB/s wr, 13 op/s 2024-09-12T01:28:04.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:04 smithi104 ceph-mon[35219]: from='client.24755 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:28:04.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:04.966+0000 7ffbf5e00640 1 -- 172.21.15.1:0/4290640402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffbf01049c0 msgr2=0x7ffbf0108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:04.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:04.966+0000 7ffbf5e00640 1 --2- 172.21.15.1:0/4290640402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffbf01049c0 0x7ffbf0108a10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8009900 tx=0x7ffbd802a3f0 comp rx=0 tx=0).stop 2024-09-12T01:28:04.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:04.966+0000 7ffbf5e00640 1 -- 172.21.15.1:0/4290640402 shutdown_connections 2024-09-12T01:28:04.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:04.966+0000 7ffbf5e00640 1 --2- 172.21.15.1:0/4290640402 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbf0109140 0x7ffbf0110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:04.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:04.966+0000 7ffbf5e00640 1 --2- 172.21.15.1:0/4290640402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffbf01049c0 0x7ffbf0108a10 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:04.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:04.966+0000 7ffbf5e00640 1 --2- 172.21.15.1:0/4290640402 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffbf0104010 0x7ffbf01043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:04.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:04.966+0000 7ffbf5e00640 1 -- 172.21.15.1:0/4290640402 >> 172.21.15.1:0/4290640402 conn(0x7ffbf00ffa50 msgr2=0x7ffbf0101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:04.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:04.966+0000 7ffbf5e00640 1 -- 172.21.15.1:0/4290640402 shutdown_connections 2024-09-12T01:28:04.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:04.966+0000 7ffbf5e00640 1 -- 172.21.15.1:0/4290640402 wait complete. 2024-09-12T01:28:04.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:04.967+0000 7ffbf5e00640 1 Processor -- start 2024-09-12T01:28:04.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:04.967+0000 7ffbf5e00640 1 -- start start 2024-09-12T01:28:04.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:04.967+0000 7ffbf5e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffbf0104010 0x7ffbf019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:04.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:04.967+0000 7ffbf5e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbf01049c0 0x7ffbf019abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:04.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:04.968+0000 7ffbf5e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffbf0109140 0x7ffbf019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:04.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:04.968+0000 7ffbf5e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf01141b0 con 0x7ffbf0109140 2024-09-12T01:28:04.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:04.968+0000 7ffbf5e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf0114030 con 0x7ffbf01049c0 2024-09-12T01:28:04.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:04.968+0000 7ffbf5e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf0114330 con 0x7ffbf0104010 2024-09-12T01:28:04.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:04.968+0000 7ffbef400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffbf0109140 0x7ffbf019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:04.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:04.968+0000 7ffbeea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffbf0104010 0x7ffbf019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:04.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:04.968+0000 7ffbeea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffbf0104010 0x7ffbf019a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:42454/0 (socket says 172.21.15.1:42454) 2024-09-12T01:28:04.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:04.968+0000 7ffbe6000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbf01049c0 0x7ffbf019abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:04.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:04.968+0000 7ffbeea00640 1 -- 172.21.15.1:0/711663877 learned_addr learned my addr 172.21.15.1:0/711663877 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:28:04.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:04.968+0000 7ffbeea00640 1 -- 172.21.15.1:0/711663877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbf01049c0 msgr2=0x7ffbf019abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:04.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:04.968+0000 7ffbeea00640 1 --2- 172.21.15.1:0/711663877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbf01049c0 0x7ffbf019abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:04.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:04.968+0000 7ffbeea00640 1 -- 172.21.15.1:0/711663877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffbf0109140 msgr2=0x7ffbf019ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:04.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:04.968+0000 7ffbeea00640 1 --2- 172.21.15.1:0/711663877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffbf0109140 0x7ffbf019ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:04.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:04.968+0000 7ffbeea00640 1 -- 172.21.15.1:0/711663877 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbf019f660 con 0x7ffbf0104010 2024-09-12T01:28:04.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:04.968+0000 7ffbef400640 1 --2- 172.21.15.1:0/711663877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffbf0109140 0x7ffbf019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:28:04.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:04.969+0000 7ffbeea00640 1 --2- 172.21.15.1:0/711663877 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffbf0104010 0x7ffbf019a6b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffbe000eb90 tx=0x7ffbe000ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:04.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:04.969+0000 7ffbf4a00640 1 -- 172.21.15.1:0/711663877 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe0004b20 con 0x7ffbf0104010 2024-09-12T01:28:04.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:04.969+0000 7ffbf4a00640 1 -- 172.21.15.1:0/711663877 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbe0004cc0 con 0x7ffbf0104010 2024-09-12T01:28:04.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:04.969+0000 7ffbf4a00640 1 -- 172.21.15.1:0/711663877 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe0010720 con 0x7ffbf0104010 2024-09-12T01:28:04.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:04.969+0000 7ffbf5e00640 1 -- 172.21.15.1:0/711663877 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbf019f950 con 0x7ffbf0104010 2024-09-12T01:28:04.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:04.969+0000 7ffbf5e00640 1 -- 172.21.15.1:0/711663877 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbf01a7230 con 0x7ffbf0104010 2024-09-12T01:28:04.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:04.970+0000 7ffbf5e00640 1 -- 172.21.15.1:0/711663877 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbf01043f0 con 0x7ffbf0104010 2024-09-12T01:28:04.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:04.972+0000 7ffbf4a00640 1 -- 172.21.15.1:0/711663877 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7ffbe0044b60 con 0x7ffbf0104010 2024-09-12T01:28:04.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:04.973+0000 7ffbf4a00640 1 --2- 172.21.15.1:0/711663877 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ffbd40a2c10 0x7ffbd40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:04.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:04.974+0000 7ffbf4a00640 1 -- 172.21.15.1:0/711663877 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9092+0+0 (secure 0 0 0) 0x7ffbe00cfba0 con 0x7ffbf0104010 2024-09-12T01:28:04.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:04.974+0000 7ffbe6000640 1 --2- 172.21.15.1:0/711663877 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ffbd40a2c10 0x7ffbd40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:04.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:04.974+0000 7ffbe6000640 1 --2- 172.21.15.1:0/711663877 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ffbd40a2c10 0x7ffbd40a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8000c00 tx=0x7ffbd8005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:04.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:04.976+0000 7ffbf4a00640 1 -- 172.21.15.1:0/711663877 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffbe00cfe20 con 0x7ffbf0104010 2024-09-12T01:28:05.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:05.158+0000 7ffbf5e00640 1 -- 172.21.15.1:0/711663877 --> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffbf010c3f0 con 0x7ffbd40a2c10 2024-09-12T01:28:05.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:05.167+0000 7ffbf4a00640 1 -- 172.21.15.1:0/711663877 <== mgr.14150 v2:172.21.15.1:6800/1928662456 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340421 (secure 0 0 0) 0x7ffbd8004190 con 0x7ffbd40a2c10 2024-09-12T01:28:05.170 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:28:05.172 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-12T01:28:05.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:05.171+0000 7ffbf5e00640 1 -- 172.21.15.1:0/711663877 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ffbd40a2c10 msgr2=0x7ffbd40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:05.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:05.171+0000 7ffbf5e00640 1 --2- 172.21.15.1:0/711663877 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ffbd40a2c10 0x7ffbd40a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8000c00 tx=0x7ffbd8005b50 comp rx=0 tx=0).stop 2024-09-12T01:28:05.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:05.171+0000 7ffbf5e00640 1 -- 172.21.15.1:0/711663877 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffbf0104010 msgr2=0x7ffbf019a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:05.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:05.171+0000 7ffbf5e00640 1 --2- 172.21.15.1:0/711663877 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffbf0104010 0x7ffbf019a6b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffbe000eb90 tx=0x7ffbe000ef90 comp rx=0 tx=0).stop 2024-09-12T01:28:05.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:05.171+0000 7ffbf5e00640 1 -- 172.21.15.1:0/711663877 shutdown_connections 2024-09-12T01:28:05.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:05.171+0000 7ffbf5e00640 1 --2- 172.21.15.1:0/711663877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffbf0109140 0x7ffbf019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:05.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:05.171+0000 7ffbf5e00640 1 --2- 172.21.15.1:0/711663877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbf01049c0 0x7ffbf019abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:05.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:05.171+0000 7ffbf5e00640 1 --2- 172.21.15.1:0/711663877 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ffbd40a2c10 0x7ffbd40a50d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:05.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:05.171+0000 7ffbf5e00640 1 --2- 172.21.15.1:0/711663877 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffbf0104010 0x7ffbf019a6b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:05.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:05.171+0000 7ffbf5e00640 1 -- 172.21.15.1:0/711663877 >> 172.21.15.1:0/711663877 conn(0x7ffbf00ffa50 msgr2=0x7ffbf0100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:05.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:05.171+0000 7ffbf5e00640 1 -- 172.21.15.1:0/711663877 shutdown_connections 2024-09-12T01:28:05.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:05.171+0000 7ffbf5e00640 1 -- 172.21.15.1:0/711663877 wait complete. 2024-09-12T01:28:06.133 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":351,"stamp":"2024-09-12T01:28:04.692670+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654952,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":337,"num_read_kb":287,"num_write":426,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1182348,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":349,"ondisk_log_size":349,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":401712,"kb_used_data":15024,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124540112,"statfs":{"total":1151940427776,"available":1151529074688,"internally_reserved":0,"allocated":15384576,"data_stored":8466787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":395752762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32796,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":89,"num_write":24,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003000"},"pg_stats":[{"pgid":"2.1f","version":"96'13","reported_seq":73,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:28:03.366185+0000","last_change":"2024-09-12T01:26:44.440383+0000","last_active":"2024-09-12T01:28:03.366185+0000","last_peered":"2024-09-12T01:28:03.366185+0000","last_clean":"2024-09-12T01:28:03.366185+0000","last_became_active":"2024-09-12T01:26:44.440048+0000","last_became_peered":"2024-09-12T01:26:44.440048+0000","last_unstale":"2024-09-12T01:28:03.366185+0000","last_undegraded":"2024-09-12T01:28:03.366185+0000","last_fullsized":"2024-09-12T01:28:03.366185+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:05:26.956556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.727618+0000","last_change":"2024-09-12T01:26:47.483952+0000","last_active":"2024-09-12T01:27:16.727618+0000","last_peered":"2024-09-12T01:27:16.727618+0000","last_clean":"2024-09-12T01:27:16.727618+0000","last_became_active":"2024-09-12T01:26:47.483679+0000","last_became_peered":"2024-09-12T01:26:47.483679+0000","last_unstale":"2024-09-12T01:27:16.727618+0000","last_undegraded":"2024-09-12T01:27:16.727618+0000","last_fullsized":"2024-09-12T01:27:16.727618+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:46:53.997935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.614152+0000","last_change":"2024-09-12T01:26:57.529904+0000","last_active":"2024-09-12T01:27:16.614152+0000","last_peered":"2024-09-12T01:27:16.614152+0000","last_clean":"2024-09-12T01:27:16.614152+0000","last_became_active":"2024-09-12T01:26:57.529332+0000","last_became_peered":"2024-09-12T01:26:57.529332+0000","last_unstale":"2024-09-12T01:27:16.614152+0000","last_undegraded":"2024-09-12T01:27:16.614152+0000","last_fullsized":"2024-09-12T01:27:16.614152+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:54.733150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"96'2","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.727867+0000","last_change":"2024-09-12T01:26:44.448605+0000","last_active":"2024-09-12T01:27:16.727867+0000","last_peered":"2024-09-12T01:27:16.727867+0000","last_clean":"2024-09-12T01:27:16.727867+0000","last_became_active":"2024-09-12T01:26:44.447035+0000","last_became_peered":"2024-09-12T01:26:44.447035+0000","last_unstale":"2024-09-12T01:27:16.727867+0000","last_undegraded":"2024-09-12T01:27:16.727867+0000","last_fullsized":"2024-09-12T01:27:16.727867+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:29:44.222912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.612376+0000","last_change":"2024-09-12T01:26:47.475058+0000","last_active":"2024-09-12T01:27:16.612376+0000","last_peered":"2024-09-12T01:27:16.612376+0000","last_clean":"2024-09-12T01:27:16.612376+0000","last_became_active":"2024-09-12T01:26:47.474057+0000","last_became_peered":"2024-09-12T01:26:47.474057+0000","last_unstale":"2024-09-12T01:27:16.612376+0000","last_undegraded":"2024-09-12T01:27:16.612376+0000","last_fullsized":"2024-09-12T01:27:16.612376+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:58:20.039831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.617128+0000","last_change":"2024-09-12T01:26:57.532139+0000","last_active":"2024-09-12T01:27:16.617128+0000","last_peered":"2024-09-12T01:27:16.617128+0000","last_clean":"2024-09-12T01:27:16.617128+0000","last_became_active":"2024-09-12T01:26:57.531718+0000","last_became_peered":"2024-09-12T01:26:57.531718+0000","last_unstale":"2024-09-12T01:27:16.617128+0000","last_undegraded":"2024-09-12T01:27:16.617128+0000","last_fullsized":"2024-09-12T01:27:16.617128+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:10.414782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"96'12","reported_seq":62,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:45.338125+0000","last_change":"2024-09-12T01:27:16.631523+0000","last_active":"2024-09-12T01:27:45.338125+0000","last_peered":"2024-09-12T01:27:45.338125+0000","last_clean":"2024-09-12T01:27:45.338125+0000","last_became_active":"2024-09-12T01:27:16.622591+0000","last_became_peered":"2024-09-12T01:27:16.622591+0000","last_unstale":"2024-09-12T01:27:45.338125+0000","last_undegraded":"2024-09-12T01:27:45.338125+0000","last_fullsized":"2024-09-12T01:27:45.338125+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:14:12.793775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.616175+0000","last_change":"2024-09-12T01:26:47.477891+0000","last_active":"2024-09-12T01:27:16.616175+0000","last_peered":"2024-09-12T01:27:16.616175+0000","last_clean":"2024-09-12T01:27:16.616175+0000","last_became_active":"2024-09-12T01:26:47.477107+0000","last_became_peered":"2024-09-12T01:26:47.477107+0000","last_unstale":"2024-09-12T01:27:16.616175+0000","last_undegraded":"2024-09-12T01:27:16.616175+0000","last_fullsized":"2024-09-12T01:27:16.616175+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:35:17.262896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":31,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.832962+0000","last_change":"2024-09-12T01:27:16.832962+0000","last_active":"2024-09-12T01:27:16.832962+0000","last_peered":"2024-09-12T01:27:16.832962+0000","last_clean":"2024-09-12T01:27:16.832962+0000","last_became_active":"2024-09-12T01:27:16.832463+0000","last_became_peered":"2024-09-12T01:27:16.832463+0000","last_unstale":"2024-09-12T01:27:16.832962+0000","last_undegraded":"2024-09-12T01:27:16.832962+0000","last_fullsized":"2024-09-12T01:27:16.832962+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:10:24.321133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"96'5","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:44.326346+0000","last_change":"2024-09-12T01:26:44.445375+0000","last_active":"2024-09-12T01:27:44.326346+0000","last_peered":"2024-09-12T01:27:44.326346+0000","last_clean":"2024-09-12T01:27:44.326346+0000","last_became_active":"2024-09-12T01:26:44.444820+0000","last_became_peered":"2024-09-12T01:26:44.444820+0000","last_unstale":"2024-09-12T01:27:44.326346+0000","last_undegraded":"2024-09-12T01:27:44.326346+0000","last_fullsized":"2024-09-12T01:27:44.326346+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:05:40.790222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.831260+0000","last_change":"2024-09-12T01:26:47.480650+0000","last_active":"2024-09-12T01:27:16.831260+0000","last_peered":"2024-09-12T01:27:16.831260+0000","last_clean":"2024-09-12T01:27:16.831260+0000","last_became_active":"2024-09-12T01:26:47.480266+0000","last_became_peered":"2024-09-12T01:26:47.480266+0000","last_unstale":"2024-09-12T01:27:16.831260+0000","last_undegraded":"2024-09-12T01:27:16.831260+0000","last_fullsized":"2024-09-12T01:27:16.831260+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:00.218397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.617076+0000","last_change":"2024-09-12T01:26:57.532582+0000","last_active":"2024-09-12T01:27:16.617076+0000","last_peered":"2024-09-12T01:27:16.617076+0000","last_clean":"2024-09-12T01:27:16.617076+0000","last_became_active":"2024-09-12T01:26:57.531966+0000","last_became_peered":"2024-09-12T01:26:57.531966+0000","last_unstale":"2024-09-12T01:27:16.617076+0000","last_undegraded":"2024-09-12T01:27:16.617076+0000","last_fullsized":"2024-09-12T01:27:16.617076+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:01:13.886678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.618598+0000","last_change":"2024-09-12T01:26:47.486567+0000","last_active":"2024-09-12T01:27:16.618598+0000","last_peered":"2024-09-12T01:27:16.618598+0000","last_clean":"2024-09-12T01:27:16.618598+0000","last_became_active":"2024-09-12T01:26:47.486219+0000","last_became_peered":"2024-09-12T01:26:47.486219+0000","last_unstale":"2024-09-12T01:27:16.618598+0000","last_undegraded":"2024-09-12T01:27:16.618598+0000","last_fullsized":"2024-09-12T01:27:16.618598+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:48:53.889111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"96'4","reported_seq":18,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:41.303051+0000","last_change":"2024-09-12T01:27:16.624095+0000","last_active":"2024-09-12T01:27:41.303051+0000","last_peered":"2024-09-12T01:27:41.303051+0000","last_clean":"2024-09-12T01:27:41.303051+0000","last_became_active":"2024-09-12T01:27:16.620392+0000","last_became_peered":"2024-09-12T01:27:16.620392+0000","last_unstale":"2024-09-12T01:27:41.303051+0000","last_undegraded":"2024-09-12T01:27:41.303051+0000","last_fullsized":"2024-09-12T01:27:41.303051+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:56:05.516283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.618083+0000","last_change":"2024-09-12T01:26:57.729865+0000","last_active":"2024-09-12T01:27:16.618083+0000","last_peered":"2024-09-12T01:27:16.618083+0000","last_clean":"2024-09-12T01:27:16.618083+0000","last_became_active":"2024-09-12T01:26:57.728819+0000","last_became_peered":"2024-09-12T01:26:57.728819+0000","last_unstale":"2024-09-12T01:27:16.618083+0000","last_undegraded":"2024-09-12T01:27:16.618083+0000","last_fullsized":"2024-09-12T01:27:16.618083+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:11.337683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.727426+0000","last_change":"2024-09-12T01:26:47.485503+0000","last_active":"2024-09-12T01:27:16.727426+0000","last_peered":"2024-09-12T01:27:16.727426+0000","last_clean":"2024-09-12T01:27:16.727426+0000","last_became_active":"2024-09-12T01:26:47.485187+0000","last_became_peered":"2024-09-12T01:26:47.485187+0000","last_unstale":"2024-09-12T01:27:16.727426+0000","last_undegraded":"2024-09-12T01:27:16.727426+0000","last_fullsized":"2024-09-12T01:27:16.727426+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:24:52.544761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"96'3","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:43.321325+0000","last_change":"2024-09-12T01:26:44.447010+0000","last_active":"2024-09-12T01:27:43.321325+0000","last_peered":"2024-09-12T01:27:43.321325+0000","last_clean":"2024-09-12T01:27:43.321325+0000","last_became_active":"2024-09-12T01:26:44.446643+0000","last_became_peered":"2024-09-12T01:26:44.446643+0000","last_unstale":"2024-09-12T01:27:43.321325+0000","last_undegraded":"2024-09-12T01:27:43.321325+0000","last_fullsized":"2024-09-12T01:27:43.321325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:56:33.298732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"96'1","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.618128+0000","last_change":"2024-09-12T01:26:57.524687+0000","last_active":"2024-09-12T01:27:16.618128+0000","last_peered":"2024-09-12T01:27:16.618128+0000","last_clean":"2024-09-12T01:27:16.618128+0000","last_became_active":"2024-09-12T01:26:57.524201+0000","last_became_peered":"2024-09-12T01:26:57.524201+0000","last_unstale":"2024-09-12T01:27:16.618128+0000","last_undegraded":"2024-09-12T01:27:16.618128+0000","last_fullsized":"2024-09-12T01:27:16.618128+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:51:49.147460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.617325+0000","last_change":"2024-09-12T01:26:47.478432+0000","last_active":"2024-09-12T01:27:16.617325+0000","last_peered":"2024-09-12T01:27:16.617325+0000","last_clean":"2024-09-12T01:27:16.617325+0000","last_became_active":"2024-09-12T01:26:47.478109+0000","last_became_peered":"2024-09-12T01:26:47.478109+0000","last_unstale":"2024-09-12T01:27:16.617325+0000","last_undegraded":"2024-09-12T01:27:16.617325+0000","last_fullsized":"2024-09-12T01:27:16.617325+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:30:19.917875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"96'7","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:44.327496+0000","last_change":"2024-09-12T01:26:44.441961+0000","last_active":"2024-09-12T01:27:44.327496+0000","last_peered":"2024-09-12T01:27:44.327496+0000","last_clean":"2024-09-12T01:27:44.327496+0000","last_became_active":"2024-09-12T01:26:44.441644+0000","last_became_peered":"2024-09-12T01:26:44.441644+0000","last_unstale":"2024-09-12T01:27:44.327496+0000","last_undegraded":"2024-09-12T01:27:44.327496+0000","last_fullsized":"2024-09-12T01:27:44.327496+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:40:48.561119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.831832+0000","last_change":"2024-09-12T01:26:57.531164+0000","last_active":"2024-09-12T01:27:16.831832+0000","last_peered":"2024-09-12T01:27:16.831832+0000","last_clean":"2024-09-12T01:27:16.831832+0000","last_became_active":"2024-09-12T01:26:57.530513+0000","last_became_peered":"2024-09-12T01:26:57.530513+0000","last_unstale":"2024-09-12T01:27:16.831832+0000","last_undegraded":"2024-09-12T01:27:16.831832+0000","last_fullsized":"2024-09-12T01:27:16.831832+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:08:55.385620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.617323+0000","last_change":"2024-09-12T01:26:47.475562+0000","last_active":"2024-09-12T01:27:16.617323+0000","last_peered":"2024-09-12T01:27:16.617323+0000","last_clean":"2024-09-12T01:27:16.617323+0000","last_became_active":"2024-09-12T01:26:47.475167+0000","last_became_peered":"2024-09-12T01:26:47.475167+0000","last_unstale":"2024-09-12T01:27:16.617323+0000","last_undegraded":"2024-09-12T01:27:16.617323+0000","last_fullsized":"2024-09-12T01:27:16.617323+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:39:34.706811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"96'2","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:41.301425+0000","last_change":"2024-09-12T01:26:44.447264+0000","last_active":"2024-09-12T01:27:41.301425+0000","last_peered":"2024-09-12T01:27:41.301425+0000","last_clean":"2024-09-12T01:27:41.301425+0000","last_became_active":"2024-09-12T01:26:44.446885+0000","last_became_peered":"2024-09-12T01:26:44.446885+0000","last_unstale":"2024-09-12T01:27:41.301425+0000","last_undegraded":"2024-09-12T01:27:41.301425+0000","last_fullsized":"2024-09-12T01:27:41.301425+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:56:27.229949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.618060+0000","last_change":"2024-09-12T01:26:57.729792+0000","last_active":"2024-09-12T01:27:16.618060+0000","last_peered":"2024-09-12T01:27:16.618060+0000","last_clean":"2024-09-12T01:27:16.618060+0000","last_became_active":"2024-09-12T01:26:57.729252+0000","last_became_peered":"2024-09-12T01:26:57.729252+0000","last_unstale":"2024-09-12T01:27:16.618060+0000","last_undegraded":"2024-09-12T01:27:16.618060+0000","last_fullsized":"2024-09-12T01:27:16.618060+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:34:57.106270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.612181+0000","last_change":"2024-09-12T01:26:47.479179+0000","last_active":"2024-09-12T01:27:16.612181+0000","last_peered":"2024-09-12T01:27:16.612181+0000","last_clean":"2024-09-12T01:27:16.612181+0000","last_became_active":"2024-09-12T01:26:47.478783+0000","last_became_peered":"2024-09-12T01:26:47.478783+0000","last_unstale":"2024-09-12T01:27:16.612181+0000","last_undegraded":"2024-09-12T01:27:16.612181+0000","last_fullsized":"2024-09-12T01:27:16.612181+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:23:22.548509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"96'7","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:44.328377+0000","last_change":"2024-09-12T01:26:44.471854+0000","last_active":"2024-09-12T01:27:44.328377+0000","last_peered":"2024-09-12T01:27:44.328377+0000","last_clean":"2024-09-12T01:27:44.328377+0000","last_became_active":"2024-09-12T01:26:44.471511+0000","last_became_peered":"2024-09-12T01:26:44.471511+0000","last_unstale":"2024-09-12T01:27:44.328377+0000","last_undegraded":"2024-09-12T01:27:44.328377+0000","last_fullsized":"2024-09-12T01:27:44.328377+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:00:26.639984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.614289+0000","last_change":"2024-09-12T01:26:57.520467+0000","last_active":"2024-09-12T01:27:16.614289+0000","last_peered":"2024-09-12T01:27:16.614289+0000","last_clean":"2024-09-12T01:27:16.614289+0000","last_became_active":"2024-09-12T01:26:57.520048+0000","last_became_peered":"2024-09-12T01:26:57.520048+0000","last_unstale":"2024-09-12T01:27:16.614289+0000","last_undegraded":"2024-09-12T01:27:16.614289+0000","last_fullsized":"2024-09-12T01:27:16.614289+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:42:05.303804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.727805+0000","last_change":"2024-09-12T01:26:47.476484+0000","last_active":"2024-09-12T01:27:16.727805+0000","last_peered":"2024-09-12T01:27:16.727805+0000","last_clean":"2024-09-12T01:27:16.727805+0000","last_became_active":"2024-09-12T01:26:47.475610+0000","last_became_peered":"2024-09-12T01:26:47.475610+0000","last_unstale":"2024-09-12T01:27:16.727805+0000","last_undegraded":"2024-09-12T01:27:16.727805+0000","last_fullsized":"2024-09-12T01:27:16.727805+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:44.637986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"96'14","reported_seq":53,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:28:01.488121+0000","last_change":"2024-09-12T01:26:44.442988+0000","last_active":"2024-09-12T01:28:01.488121+0000","last_peered":"2024-09-12T01:28:01.488121+0000","last_clean":"2024-09-12T01:28:01.488121+0000","last_became_active":"2024-09-12T01:26:44.442612+0000","last_became_peered":"2024-09-12T01:26:44.442612+0000","last_unstale":"2024-09-12T01:28:01.488121+0000","last_undegraded":"2024-09-12T01:28:01.488121+0000","last_fullsized":"2024-09-12T01:28:01.488121+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:38:25.486470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.727142+0000","last_change":"2024-09-12T01:26:57.729762+0000","last_active":"2024-09-12T01:27:16.727142+0000","last_peered":"2024-09-12T01:27:16.727142+0000","last_clean":"2024-09-12T01:27:16.727142+0000","last_became_active":"2024-09-12T01:26:57.728944+0000","last_became_peered":"2024-09-12T01:26:57.728944+0000","last_unstale":"2024-09-12T01:27:16.727142+0000","last_undegraded":"2024-09-12T01:27:16.727142+0000","last_fullsized":"2024-09-12T01:27:16.727142+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:33:58.030332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.615545+0000","last_change":"2024-09-12T01:26:47.482512+0000","last_active":"2024-09-12T01:27:16.615545+0000","last_peered":"2024-09-12T01:27:16.615545+0000","last_clean":"2024-09-12T01:27:16.615545+0000","last_became_active":"2024-09-12T01:26:47.481344+0000","last_became_peered":"2024-09-12T01:26:47.481344+0000","last_unstale":"2024-09-12T01:27:16.615545+0000","last_undegraded":"2024-09-12T01:27:16.615545+0000","last_fullsized":"2024-09-12T01:27:16.615545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:52:22.920248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"96'4","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:44.326689+0000","last_change":"2024-09-12T01:26:44.447654+0000","last_active":"2024-09-12T01:27:44.326689+0000","last_peered":"2024-09-12T01:27:44.326689+0000","last_clean":"2024-09-12T01:27:44.326689+0000","last_became_active":"2024-09-12T01:26:44.447148+0000","last_became_peered":"2024-09-12T01:26:44.447148+0000","last_unstale":"2024-09-12T01:27:44.326689+0000","last_undegraded":"2024-09-12T01:27:44.326689+0000","last_fullsized":"2024-09-12T01:27:44.326689+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:40:02.928329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.616781+0000","last_change":"2024-09-12T01:26:57.530191+0000","last_active":"2024-09-12T01:27:16.616781+0000","last_peered":"2024-09-12T01:27:16.616781+0000","last_clean":"2024-09-12T01:27:16.616781+0000","last_became_active":"2024-09-12T01:26:57.529752+0000","last_became_peered":"2024-09-12T01:26:57.529752+0000","last_unstale":"2024-09-12T01:27:16.616781+0000","last_undegraded":"2024-09-12T01:27:16.616781+0000","last_fullsized":"2024-09-12T01:27:16.616781+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:00:16.639297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.612993+0000","last_change":"2024-09-12T01:26:47.478991+0000","last_active":"2024-09-12T01:27:16.612993+0000","last_peered":"2024-09-12T01:27:16.612993+0000","last_clean":"2024-09-12T01:27:16.612993+0000","last_became_active":"2024-09-12T01:26:47.478228+0000","last_became_peered":"2024-09-12T01:26:47.478228+0000","last_unstale":"2024-09-12T01:27:16.612993+0000","last_undegraded":"2024-09-12T01:27:16.612993+0000","last_fullsized":"2024-09-12T01:27:16.612993+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:45.236546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"96'1","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.831760+0000","last_change":"2024-09-12T01:26:44.441143+0000","last_active":"2024-09-12T01:27:16.831760+0000","last_peered":"2024-09-12T01:27:16.831760+0000","last_clean":"2024-09-12T01:27:16.831760+0000","last_became_active":"2024-09-12T01:26:44.439361+0000","last_became_peered":"2024-09-12T01:26:44.439361+0000","last_unstale":"2024-09-12T01:27:16.831760+0000","last_undegraded":"2024-09-12T01:27:16.831760+0000","last_fullsized":"2024-09-12T01:27:16.831760+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:22:34.257977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.617430+0000","last_change":"2024-09-12T01:26:57.729010+0000","last_active":"2024-09-12T01:27:16.617430+0000","last_peered":"2024-09-12T01:27:16.617430+0000","last_clean":"2024-09-12T01:27:16.617430+0000","last_became_active":"2024-09-12T01:26:57.728576+0000","last_became_peered":"2024-09-12T01:26:57.728576+0000","last_unstale":"2024-09-12T01:27:16.617430+0000","last_undegraded":"2024-09-12T01:27:16.617430+0000","last_fullsized":"2024-09-12T01:27:16.617430+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:48:48.798028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.832723+0000","last_change":"2024-09-12T01:27:16.832723+0000","last_active":"2024-09-12T01:27:16.832723+0000","last_peered":"2024-09-12T01:27:16.832723+0000","last_clean":"2024-09-12T01:27:16.832723+0000","last_became_active":"2024-09-12T01:27:16.832241+0000","last_became_peered":"2024-09-12T01:27:16.832241+0000","last_unstale":"2024-09-12T01:27:16.832723+0000","last_undegraded":"2024-09-12T01:27:16.832723+0000","last_fullsized":"2024-09-12T01:27:16.832723+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:26:09.610468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"96'6","reported_seq":66,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:28:01.364117+0000","last_change":"2024-09-12T01:26:44.440802+0000","last_active":"2024-09-12T01:28:01.364117+0000","last_peered":"2024-09-12T01:28:01.364117+0000","last_clean":"2024-09-12T01:28:01.364117+0000","last_became_active":"2024-09-12T01:26:44.439686+0000","last_became_peered":"2024-09-12T01:26:44.439686+0000","last_unstale":"2024-09-12T01:28:01.364117+0000","last_undegraded":"2024-09-12T01:28:01.364117+0000","last_fullsized":"2024-09-12T01:28:01.364117+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:56.608975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.615436+0000","last_change":"2024-09-12T01:26:57.537046+0000","last_active":"2024-09-12T01:27:16.615436+0000","last_peered":"2024-09-12T01:27:16.615436+0000","last_clean":"2024-09-12T01:27:16.615436+0000","last_became_active":"2024-09-12T01:26:57.536589+0000","last_became_peered":"2024-09-12T01:26:57.536589+0000","last_unstale":"2024-09-12T01:27:16.615436+0000","last_undegraded":"2024-09-12T01:27:16.615436+0000","last_fullsized":"2024-09-12T01:27:16.615436+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:24:54.232963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.831136+0000","last_change":"2024-09-12T01:26:47.476288+0000","last_active":"2024-09-12T01:27:16.831136+0000","last_peered":"2024-09-12T01:27:16.831136+0000","last_clean":"2024-09-12T01:27:16.831136+0000","last_became_active":"2024-09-12T01:26:47.475836+0000","last_became_peered":"2024-09-12T01:26:47.475836+0000","last_unstale":"2024-09-12T01:27:16.831136+0000","last_undegraded":"2024-09-12T01:27:16.831136+0000","last_fullsized":"2024-09-12T01:27:16.831136+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:39:43.017253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"96'5","reported_seq":61,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:59.358978+0000","last_change":"2024-09-12T01:26:44.446539+0000","last_active":"2024-09-12T01:27:59.358978+0000","last_peered":"2024-09-12T01:27:59.358978+0000","last_clean":"2024-09-12T01:27:59.358978+0000","last_became_active":"2024-09-12T01:26:44.446219+0000","last_became_peered":"2024-09-12T01:26:44.446219+0000","last_unstale":"2024-09-12T01:27:59.358978+0000","last_undegraded":"2024-09-12T01:27:59.358978+0000","last_fullsized":"2024-09-12T01:27:59.358978+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:50:32.584078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.615779+0000","last_change":"2024-09-12T01:26:57.729445+0000","last_active":"2024-09-12T01:27:16.615779+0000","last_peered":"2024-09-12T01:27:16.615779+0000","last_clean":"2024-09-12T01:27:16.615779+0000","last_became_active":"2024-09-12T01:26:57.728948+0000","last_became_peered":"2024-09-12T01:26:57.728948+0000","last_unstale":"2024-09-12T01:27:16.615779+0000","last_undegraded":"2024-09-12T01:27:16.615779+0000","last_fullsized":"2024-09-12T01:27:16.615779+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:33:27.017565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.614620+0000","last_change":"2024-09-12T01:26:47.481965+0000","last_active":"2024-09-12T01:27:16.614620+0000","last_peered":"2024-09-12T01:27:16.614620+0000","last_clean":"2024-09-12T01:27:16.614620+0000","last_became_active":"2024-09-12T01:26:47.481600+0000","last_became_peered":"2024-09-12T01:26:47.481600+0000","last_unstale":"2024-09-12T01:27:16.614620+0000","last_undegraded":"2024-09-12T01:27:16.614620+0000","last_fullsized":"2024-09-12T01:27:16.614620+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:20.078494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"96'4","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:42.307242+0000","last_change":"2024-09-12T01:27:16.832522+0000","last_active":"2024-09-12T01:27:42.307242+0000","last_peered":"2024-09-12T01:27:42.307242+0000","last_clean":"2024-09-12T01:27:42.307242+0000","last_became_active":"2024-09-12T01:27:16.832090+0000","last_became_peered":"2024-09-12T01:27:16.832090+0000","last_unstale":"2024-09-12T01:27:42.307242+0000","last_undegraded":"2024-09-12T01:27:42.307242+0000","last_fullsized":"2024-09-12T01:27:42.307242+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:15:48.061559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.831364+0000","last_change":"2024-09-12T01:26:57.729622+0000","last_active":"2024-09-12T01:27:16.831364+0000","last_peered":"2024-09-12T01:27:16.831364+0000","last_clean":"2024-09-12T01:27:16.831364+0000","last_became_active":"2024-09-12T01:26:57.729175+0000","last_became_peered":"2024-09-12T01:26:57.729175+0000","last_unstale":"2024-09-12T01:27:16.831364+0000","last_undegraded":"2024-09-12T01:27:16.831364+0000","last_fullsized":"2024-09-12T01:27:16.831364+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:41.691685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.727458+0000","last_change":"2024-09-12T01:26:47.479155+0000","last_active":"2024-09-12T01:27:16.727458+0000","last_peered":"2024-09-12T01:27:16.727458+0000","last_clean":"2024-09-12T01:27:16.727458+0000","last_became_active":"2024-09-12T01:26:47.478781+0000","last_became_peered":"2024-09-12T01:26:47.478781+0000","last_unstale":"2024-09-12T01:27:16.727458+0000","last_undegraded":"2024-09-12T01:27:16.727458+0000","last_fullsized":"2024-09-12T01:27:16.727458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:40:49.730016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"96'6","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:45.336494+0000","last_change":"2024-09-12T01:26:44.437048+0000","last_active":"2024-09-12T01:27:45.336494+0000","last_peered":"2024-09-12T01:27:45.336494+0000","last_clean":"2024-09-12T01:27:45.336494+0000","last_became_active":"2024-09-12T01:26:44.436702+0000","last_became_peered":"2024-09-12T01:26:44.436702+0000","last_unstale":"2024-09-12T01:27:45.336494+0000","last_undegraded":"2024-09-12T01:27:45.336494+0000","last_fullsized":"2024-09-12T01:27:45.336494+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:15:18.857093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":358,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.618344+0000","last_change":"2024-09-12T01:25:28.683724+0000","last_active":"2024-09-12T01:27:16.618344+0000","last_peered":"2024-09-12T01:27:16.618344+0000","last_clean":"2024-09-12T01:27:16.618344+0000","last_became_active":"2024-09-12T01:25:28.683155+0000","last_became_peered":"2024-09-12T01:25:28.683155+0000","last_unstale":"2024-09-12T01:27:16.618344+0000","last_undegraded":"2024-09-12T01:27:16.618344+0000","last_fullsized":"2024-09-12T01:27:16.618344+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:21:41.529611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:21:41.529611+0000","last_clean_scrub_stamp":"2024-09-12T01:21:41.529611+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:13:02.724915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"96'1","reported_seq":32,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.833617+0000","last_change":"2024-09-12T01:27:16.833028+0000","last_active":"2024-09-12T01:27:16.833617+0000","last_peered":"2024-09-12T01:27:16.833617+0000","last_clean":"2024-09-12T01:27:16.833617+0000","last_became_active":"2024-09-12T01:27:16.832416+0000","last_became_peered":"2024-09-12T01:27:16.832416+0000","last_unstale":"2024-09-12T01:27:16.833617+0000","last_undegraded":"2024-09-12T01:27:16.833617+0000","last_fullsized":"2024-09-12T01:27:16.833617+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:49:43.008107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.619425+0000","last_change":"2024-09-12T01:26:47.475467+0000","last_active":"2024-09-12T01:27:16.619425+0000","last_peered":"2024-09-12T01:27:16.619425+0000","last_clean":"2024-09-12T01:27:16.619425+0000","last_became_active":"2024-09-12T01:26:47.474724+0000","last_became_peered":"2024-09-12T01:26:47.474724+0000","last_unstale":"2024-09-12T01:27:16.619425+0000","last_undegraded":"2024-09-12T01:27:16.619425+0000","last_fullsized":"2024-09-12T01:27:16.619425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:56:54.651180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"96'6","reported_seq":65,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:59.362236+0000","last_change":"2024-09-12T01:26:44.472048+0000","last_active":"2024-09-12T01:27:59.362236+0000","last_peered":"2024-09-12T01:27:59.362236+0000","last_clean":"2024-09-12T01:27:59.362236+0000","last_became_active":"2024-09-12T01:26:44.471697+0000","last_became_peered":"2024-09-12T01:26:44.471697+0000","last_unstale":"2024-09-12T01:27:59.362236+0000","last_undegraded":"2024-09-12T01:27:59.362236+0000","last_fullsized":"2024-09-12T01:27:59.362236+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:44:13.071983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":19,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.616587+0000","last_change":"2024-09-12T01:26:57.528837+0000","last_active":"2024-09-12T01:27:16.616587+0000","last_peered":"2024-09-12T01:27:16.616587+0000","last_clean":"2024-09-12T01:27:16.616587+0000","last_became_active":"2024-09-12T01:26:57.528285+0000","last_became_peered":"2024-09-12T01:26:57.528285+0000","last_unstale":"2024-09-12T01:27:16.616587+0000","last_undegraded":"2024-09-12T01:27:16.616587+0000","last_fullsized":"2024-09-12T01:27:16.616587+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:12:13.534002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.618818+0000","last_change":"2024-09-12T01:26:47.483456+0000","last_active":"2024-09-12T01:27:16.618818+0000","last_peered":"2024-09-12T01:27:16.618818+0000","last_clean":"2024-09-12T01:27:16.618818+0000","last_became_active":"2024-09-12T01:26:47.482189+0000","last_became_peered":"2024-09-12T01:26:47.482189+0000","last_unstale":"2024-09-12T01:27:16.618818+0000","last_undegraded":"2024-09-12T01:27:16.618818+0000","last_fullsized":"2024-09-12T01:27:16.618818+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:18:55.466266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"96'3","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:44.328146+0000","last_change":"2024-09-12T01:26:44.443507+0000","last_active":"2024-09-12T01:27:44.328146+0000","last_peered":"2024-09-12T01:27:44.328146+0000","last_clean":"2024-09-12T01:27:44.328146+0000","last_became_active":"2024-09-12T01:26:44.443147+0000","last_became_peered":"2024-09-12T01:26:44.443147+0000","last_unstale":"2024-09-12T01:27:44.328146+0000","last_undegraded":"2024-09-12T01:27:44.328146+0000","last_fullsized":"2024-09-12T01:27:44.328146+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:38:40.433581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.612289+0000","last_change":"2024-09-12T01:26:57.523029+0000","last_active":"2024-09-12T01:27:16.612289+0000","last_peered":"2024-09-12T01:27:16.612289+0000","last_clean":"2024-09-12T01:27:16.612289+0000","last_became_active":"2024-09-12T01:26:57.522565+0000","last_became_peered":"2024-09-12T01:26:57.522565+0000","last_unstale":"2024-09-12T01:27:16.612289+0000","last_undegraded":"2024-09-12T01:27:16.612289+0000","last_fullsized":"2024-09-12T01:27:16.612289+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:36:27.432126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.619446+0000","last_change":"2024-09-12T01:26:47.471958+0000","last_active":"2024-09-12T01:27:16.619446+0000","last_peered":"2024-09-12T01:27:16.619446+0000","last_clean":"2024-09-12T01:27:16.619446+0000","last_became_active":"2024-09-12T01:26:47.471621+0000","last_became_peered":"2024-09-12T01:26:47.471621+0000","last_unstale":"2024-09-12T01:27:16.619446+0000","last_undegraded":"2024-09-12T01:27:16.619446+0000","last_fullsized":"2024-09-12T01:27:16.619446+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:46:38.004188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"96'5","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:44.328360+0000","last_change":"2024-09-12T01:26:44.442852+0000","last_active":"2024-09-12T01:27:44.328360+0000","last_peered":"2024-09-12T01:27:44.328360+0000","last_clean":"2024-09-12T01:27:44.328360+0000","last_became_active":"2024-09-12T01:26:44.442487+0000","last_became_peered":"2024-09-12T01:26:44.442487+0000","last_unstale":"2024-09-12T01:27:44.328360+0000","last_undegraded":"2024-09-12T01:27:44.328360+0000","last_fullsized":"2024-09-12T01:27:44.328360+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:04:32.688925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.830495+0000","last_change":"2024-09-12T01:26:57.521155+0000","last_active":"2024-09-12T01:27:16.830495+0000","last_peered":"2024-09-12T01:27:16.830495+0000","last_clean":"2024-09-12T01:27:16.830495+0000","last_became_active":"2024-09-12T01:26:57.520721+0000","last_became_peered":"2024-09-12T01:26:57.520721+0000","last_unstale":"2024-09-12T01:27:16.830495+0000","last_undegraded":"2024-09-12T01:27:16.830495+0000","last_fullsized":"2024-09-12T01:27:16.830495+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:58:52.004697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.616687+0000","last_change":"2024-09-12T01:26:47.482564+0000","last_active":"2024-09-12T01:27:16.616687+0000","last_peered":"2024-09-12T01:27:16.616687+0000","last_clean":"2024-09-12T01:27:16.616687+0000","last_became_active":"2024-09-12T01:26:47.482215+0000","last_became_peered":"2024-09-12T01:26:47.482215+0000","last_unstale":"2024-09-12T01:27:16.616687+0000","last_undegraded":"2024-09-12T01:27:16.616687+0000","last_fullsized":"2024-09-12T01:27:16.616687+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:27:40.882473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"96'2","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:44.302545+0000","last_change":"2024-09-12T01:26:44.442774+0000","last_active":"2024-09-12T01:27:44.302545+0000","last_peered":"2024-09-12T01:27:44.302545+0000","last_clean":"2024-09-12T01:27:44.302545+0000","last_became_active":"2024-09-12T01:26:44.442389+0000","last_became_peered":"2024-09-12T01:26:44.442389+0000","last_unstale":"2024-09-12T01:27:44.302545+0000","last_undegraded":"2024-09-12T01:27:44.302545+0000","last_fullsized":"2024-09-12T01:27:44.302545+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:22.628392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.830638+0000","last_change":"2024-09-12T01:26:57.526182+0000","last_active":"2024-09-12T01:27:16.830638+0000","last_peered":"2024-09-12T01:27:16.830638+0000","last_clean":"2024-09-12T01:27:16.830638+0000","last_became_active":"2024-09-12T01:26:57.525869+0000","last_became_peered":"2024-09-12T01:26:57.525869+0000","last_unstale":"2024-09-12T01:27:16.830638+0000","last_undegraded":"2024-09-12T01:27:16.830638+0000","last_fullsized":"2024-09-12T01:27:16.830638+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:40:09.994361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.830839+0000","last_change":"2024-09-12T01:26:47.475872+0000","last_active":"2024-09-12T01:27:16.830839+0000","last_peered":"2024-09-12T01:27:16.830839+0000","last_clean":"2024-09-12T01:27:16.830839+0000","last_became_active":"2024-09-12T01:26:47.475536+0000","last_became_peered":"2024-09-12T01:26:47.475536+0000","last_unstale":"2024-09-12T01:27:16.830839+0000","last_undegraded":"2024-09-12T01:27:16.830839+0000","last_fullsized":"2024-09-12T01:27:16.830839+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:03:37.519530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"96'17","reported_seq":57,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:44.324198+0000","last_change":"2024-09-12T01:26:44.448843+0000","last_active":"2024-09-12T01:27:44.324198+0000","last_peered":"2024-09-12T01:27:44.324198+0000","last_clean":"2024-09-12T01:27:44.324198+0000","last_became_active":"2024-09-12T01:26:44.448451+0000","last_became_peered":"2024-09-12T01:26:44.448451+0000","last_unstale":"2024-09-12T01:27:44.324198+0000","last_undegraded":"2024-09-12T01:27:44.324198+0000","last_fullsized":"2024-09-12T01:27:44.324198+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:59:21.422174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.615765+0000","last_change":"2024-09-12T01:26:57.541164+0000","last_active":"2024-09-12T01:27:16.615765+0000","last_peered":"2024-09-12T01:27:16.615765+0000","last_clean":"2024-09-12T01:27:16.615765+0000","last_became_active":"2024-09-12T01:26:57.540746+0000","last_became_peered":"2024-09-12T01:26:57.540746+0000","last_unstale":"2024-09-12T01:27:16.615765+0000","last_undegraded":"2024-09-12T01:27:16.615765+0000","last_fullsized":"2024-09-12T01:27:16.615765+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:58:26.870141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.618651+0000","last_change":"2024-09-12T01:26:47.473261+0000","last_active":"2024-09-12T01:27:16.618651+0000","last_peered":"2024-09-12T01:27:16.618651+0000","last_clean":"2024-09-12T01:27:16.618651+0000","last_became_active":"2024-09-12T01:26:47.472902+0000","last_became_peered":"2024-09-12T01:26:47.472902+0000","last_unstale":"2024-09-12T01:27:16.618651+0000","last_undegraded":"2024-09-12T01:27:16.618651+0000","last_fullsized":"2024-09-12T01:27:16.618651+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:58:41.080240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"96'6","reported_seq":67,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:28:02.363629+0000","last_change":"2024-09-12T01:26:44.440503+0000","last_active":"2024-09-12T01:28:02.363629+0000","last_peered":"2024-09-12T01:28:02.363629+0000","last_clean":"2024-09-12T01:28:02.363629+0000","last_became_active":"2024-09-12T01:26:44.440128+0000","last_became_peered":"2024-09-12T01:26:44.440128+0000","last_unstale":"2024-09-12T01:28:02.363629+0000","last_undegraded":"2024-09-12T01:28:02.363629+0000","last_fullsized":"2024-09-12T01:28:02.363629+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:39:35.373682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.831878+0000","last_change":"2024-09-12T01:26:57.533883+0000","last_active":"2024-09-12T01:27:16.831878+0000","last_peered":"2024-09-12T01:27:16.831878+0000","last_clean":"2024-09-12T01:27:16.831878+0000","last_became_active":"2024-09-12T01:26:57.533329+0000","last_became_peered":"2024-09-12T01:26:57.533329+0000","last_unstale":"2024-09-12T01:27:16.831878+0000","last_undegraded":"2024-09-12T01:27:16.831878+0000","last_fullsized":"2024-09-12T01:27:16.831878+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:17.655219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.618886+0000","last_change":"2024-09-12T01:26:47.480276+0000","last_active":"2024-09-12T01:27:16.618886+0000","last_peered":"2024-09-12T01:27:16.618886+0000","last_clean":"2024-09-12T01:27:16.618886+0000","last_became_active":"2024-09-12T01:26:47.479965+0000","last_became_peered":"2024-09-12T01:26:47.479965+0000","last_unstale":"2024-09-12T01:27:16.618886+0000","last_undegraded":"2024-09-12T01:27:16.618886+0000","last_fullsized":"2024-09-12T01:27:16.618886+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:24:10.339000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"96'5","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:44.328224+0000","last_change":"2024-09-12T01:26:44.442346+0000","last_active":"2024-09-12T01:27:44.328224+0000","last_peered":"2024-09-12T01:27:44.328224+0000","last_clean":"2024-09-12T01:27:44.328224+0000","last_became_active":"2024-09-12T01:26:44.442044+0000","last_became_peered":"2024-09-12T01:26:44.442044+0000","last_unstale":"2024-09-12T01:27:44.328224+0000","last_undegraded":"2024-09-12T01:27:44.328224+0000","last_fullsized":"2024-09-12T01:27:44.328224+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:47:00.193554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.616223+0000","last_change":"2024-09-12T01:26:57.531650+0000","last_active":"2024-09-12T01:27:16.616223+0000","last_peered":"2024-09-12T01:27:16.616223+0000","last_clean":"2024-09-12T01:27:16.616223+0000","last_became_active":"2024-09-12T01:26:57.531184+0000","last_became_peered":"2024-09-12T01:26:57.531184+0000","last_unstale":"2024-09-12T01:27:16.616223+0000","last_undegraded":"2024-09-12T01:27:16.616223+0000","last_fullsized":"2024-09-12T01:27:16.616223+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:42:56.191490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.615182+0000","last_change":"2024-09-12T01:26:47.480344+0000","last_active":"2024-09-12T01:27:16.615182+0000","last_peered":"2024-09-12T01:27:16.615182+0000","last_clean":"2024-09-12T01:27:16.615182+0000","last_became_active":"2024-09-12T01:26:47.479966+0000","last_became_peered":"2024-09-12T01:26:47.479966+0000","last_unstale":"2024-09-12T01:27:16.615182+0000","last_undegraded":"2024-09-12T01:27:16.615182+0000","last_fullsized":"2024-09-12T01:27:16.615182+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:44:31.968977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"96'9","reported_seq":51,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:45.336523+0000","last_change":"2024-09-12T01:26:44.447708+0000","last_active":"2024-09-12T01:27:45.336523+0000","last_peered":"2024-09-12T01:27:45.336523+0000","last_clean":"2024-09-12T01:27:45.336523+0000","last_became_active":"2024-09-12T01:26:44.446974+0000","last_became_peered":"2024-09-12T01:26:44.446974+0000","last_unstale":"2024-09-12T01:27:45.336523+0000","last_undegraded":"2024-09-12T01:27:45.336523+0000","last_fullsized":"2024-09-12T01:27:45.336523+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:01:10.440044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.831215+0000","last_change":"2024-09-12T01:26:57.533128+0000","last_active":"2024-09-12T01:27:16.831215+0000","last_peered":"2024-09-12T01:27:16.831215+0000","last_clean":"2024-09-12T01:27:16.831215+0000","last_became_active":"2024-09-12T01:26:57.532731+0000","last_became_peered":"2024-09-12T01:26:57.532731+0000","last_unstale":"2024-09-12T01:27:16.831215+0000","last_undegraded":"2024-09-12T01:27:16.831215+0000","last_fullsized":"2024-09-12T01:27:16.831215+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:09:06.118313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.618798+0000","last_change":"2024-09-12T01:26:47.471545+0000","last_active":"2024-09-12T01:27:16.618798+0000","last_peered":"2024-09-12T01:27:16.618798+0000","last_clean":"2024-09-12T01:27:16.618798+0000","last_became_active":"2024-09-12T01:26:47.471234+0000","last_became_peered":"2024-09-12T01:26:47.471234+0000","last_unstale":"2024-09-12T01:27:16.618798+0000","last_undegraded":"2024-09-12T01:27:16.618798+0000","last_fullsized":"2024-09-12T01:27:16.618798+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:24:00.844203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"96'31","reported_seq":126,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:28:01.361940+0000","last_change":"2024-09-12T01:27:16.629235+0000","last_active":"2024-09-12T01:28:01.361940+0000","last_peered":"2024-09-12T01:28:01.361940+0000","last_clean":"2024-09-12T01:28:01.361940+0000","last_became_active":"2024-09-12T01:27:16.621719+0000","last_became_peered":"2024-09-12T01:27:16.621719+0000","last_unstale":"2024-09-12T01:28:01.361940+0000","last_undegraded":"2024-09-12T01:28:01.361940+0000","last_fullsized":"2024-09-12T01:28:01.361940+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:45:43.011452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56749,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":40,"num_read_kb":75,"num_write":33,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.831355+0000","last_change":"2024-09-12T01:26:57.521950+0000","last_active":"2024-09-12T01:27:16.831355+0000","last_peered":"2024-09-12T01:27:16.831355+0000","last_clean":"2024-09-12T01:27:16.831355+0000","last_became_active":"2024-09-12T01:26:57.521574+0000","last_became_peered":"2024-09-12T01:26:57.521574+0000","last_unstale":"2024-09-12T01:27:16.831355+0000","last_undegraded":"2024-09-12T01:27:16.831355+0000","last_fullsized":"2024-09-12T01:27:16.831355+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:23.086233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.616172+0000","last_change":"2024-09-12T01:26:47.478212+0000","last_active":"2024-09-12T01:27:16.616172+0000","last_peered":"2024-09-12T01:27:16.616172+0000","last_clean":"2024-09-12T01:27:16.616172+0000","last_became_active":"2024-09-12T01:26:47.477553+0000","last_became_peered":"2024-09-12T01:26:47.477553+0000","last_unstale":"2024-09-12T01:27:16.616172+0000","last_undegraded":"2024-09-12T01:27:16.616172+0000","last_fullsized":"2024-09-12T01:27:16.616172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:48:22.390040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"96'5","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:43.320016+0000","last_change":"2024-09-12T01:26:44.441558+0000","last_active":"2024-09-12T01:27:43.320016+0000","last_peered":"2024-09-12T01:27:43.320016+0000","last_clean":"2024-09-12T01:27:43.320016+0000","last_became_active":"2024-09-12T01:26:44.441099+0000","last_became_peered":"2024-09-12T01:26:44.441099+0000","last_unstale":"2024-09-12T01:27:43.320016+0000","last_undegraded":"2024-09-12T01:27:43.320016+0000","last_fullsized":"2024-09-12T01:27:43.320016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:07:40.566150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.617323+0000","last_change":"2024-09-12T01:26:57.524838+0000","last_active":"2024-09-12T01:27:16.617323+0000","last_peered":"2024-09-12T01:27:16.617323+0000","last_clean":"2024-09-12T01:27:16.617323+0000","last_became_active":"2024-09-12T01:26:57.524182+0000","last_became_peered":"2024-09-12T01:26:57.524182+0000","last_unstale":"2024-09-12T01:27:16.617323+0000","last_undegraded":"2024-09-12T01:27:16.617323+0000","last_fullsized":"2024-09-12T01:27:16.617323+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:18:55.493116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.612416+0000","last_change":"2024-09-12T01:26:47.478562+0000","last_active":"2024-09-12T01:27:16.612416+0000","last_peered":"2024-09-12T01:27:16.612416+0000","last_clean":"2024-09-12T01:27:16.612416+0000","last_became_active":"2024-09-12T01:26:47.478223+0000","last_became_peered":"2024-09-12T01:26:47.478223+0000","last_unstale":"2024-09-12T01:27:16.612416+0000","last_undegraded":"2024-09-12T01:27:16.612416+0000","last_fullsized":"2024-09-12T01:27:16.612416+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:32:20.472870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"96'4","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:44.325427+0000","last_change":"2024-09-12T01:26:44.443434+0000","last_active":"2024-09-12T01:27:44.325427+0000","last_peered":"2024-09-12T01:27:44.325427+0000","last_clean":"2024-09-12T01:27:44.325427+0000","last_became_active":"2024-09-12T01:26:44.443097+0000","last_became_peered":"2024-09-12T01:26:44.443097+0000","last_unstale":"2024-09-12T01:27:44.325427+0000","last_undegraded":"2024-09-12T01:27:44.325427+0000","last_fullsized":"2024-09-12T01:27:44.325427+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:48:56.626378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.831047+0000","last_change":"2024-09-12T01:26:57.532507+0000","last_active":"2024-09-12T01:27:16.831047+0000","last_peered":"2024-09-12T01:27:16.831047+0000","last_clean":"2024-09-12T01:27:16.831047+0000","last_became_active":"2024-09-12T01:26:57.529757+0000","last_became_peered":"2024-09-12T01:26:57.529757+0000","last_unstale":"2024-09-12T01:27:16.831047+0000","last_undegraded":"2024-09-12T01:27:16.831047+0000","last_fullsized":"2024-09-12T01:27:16.831047+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:35.589141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.616354+0000","last_change":"2024-09-12T01:26:47.474632+0000","last_active":"2024-09-12T01:27:16.616354+0000","last_peered":"2024-09-12T01:27:16.616354+0000","last_clean":"2024-09-12T01:27:16.616354+0000","last_became_active":"2024-09-12T01:26:47.473791+0000","last_became_peered":"2024-09-12T01:26:47.473791+0000","last_unstale":"2024-09-12T01:27:16.616354+0000","last_undegraded":"2024-09-12T01:27:16.616354+0000","last_fullsized":"2024-09-12T01:27:16.616354+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:58:45.241657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"96'4","reported_seq":77,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:28:01.360991+0000","last_change":"2024-09-12T01:26:44.445045+0000","last_active":"2024-09-12T01:28:01.360991+0000","last_peered":"2024-09-12T01:28:01.360991+0000","last_clean":"2024-09-12T01:28:01.360991+0000","last_became_active":"2024-09-12T01:26:44.444668+0000","last_became_peered":"2024-09-12T01:26:44.444668+0000","last_unstale":"2024-09-12T01:28:01.360991+0000","last_undegraded":"2024-09-12T01:28:01.360991+0000","last_fullsized":"2024-09-12T01:28:01.360991+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:05:14.112438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.727294+0000","last_change":"2024-09-12T01:26:57.523792+0000","last_active":"2024-09-12T01:27:16.727294+0000","last_peered":"2024-09-12T01:27:16.727294+0000","last_clean":"2024-09-12T01:27:16.727294+0000","last_became_active":"2024-09-12T01:26:57.523361+0000","last_became_peered":"2024-09-12T01:26:57.523361+0000","last_unstale":"2024-09-12T01:27:16.727294+0000","last_undegraded":"2024-09-12T01:27:16.727294+0000","last_fullsized":"2024-09-12T01:27:16.727294+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:42:46.177077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.611989+0000","last_change":"2024-09-12T01:26:57.534204+0000","last_active":"2024-09-12T01:27:16.611989+0000","last_peered":"2024-09-12T01:27:16.611989+0000","last_clean":"2024-09-12T01:27:16.611989+0000","last_became_active":"2024-09-12T01:26:57.532094+0000","last_became_peered":"2024-09-12T01:26:57.532094+0000","last_unstale":"2024-09-12T01:27:16.611989+0000","last_undegraded":"2024-09-12T01:27:16.611989+0000","last_fullsized":"2024-09-12T01:27:16.611989+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:43:17.714896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.831451+0000","last_change":"2024-09-12T01:26:47.480169+0000","last_active":"2024-09-12T01:27:16.831451+0000","last_peered":"2024-09-12T01:27:16.831451+0000","last_clean":"2024-09-12T01:27:16.831451+0000","last_became_active":"2024-09-12T01:26:47.479853+0000","last_became_peered":"2024-09-12T01:26:47.479853+0000","last_unstale":"2024-09-12T01:27:16.831451+0000","last_undegraded":"2024-09-12T01:27:16.831451+0000","last_fullsized":"2024-09-12T01:27:16.831451+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:58:10.578637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"96'3","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:44.326245+0000","last_change":"2024-09-12T01:26:44.440641+0000","last_active":"2024-09-12T01:27:44.326245+0000","last_peered":"2024-09-12T01:27:44.326245+0000","last_clean":"2024-09-12T01:27:44.326245+0000","last_became_active":"2024-09-12T01:26:44.440315+0000","last_became_peered":"2024-09-12T01:26:44.440315+0000","last_unstale":"2024-09-12T01:27:44.326245+0000","last_undegraded":"2024-09-12T01:27:44.326245+0000","last_fullsized":"2024-09-12T01:27:44.326245+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:21:52.919820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.616589+0000","last_change":"2024-09-12T01:26:57.532046+0000","last_active":"2024-09-12T01:27:16.616589+0000","last_peered":"2024-09-12T01:27:16.616589+0000","last_clean":"2024-09-12T01:27:16.616589+0000","last_became_active":"2024-09-12T01:26:57.531660+0000","last_became_peered":"2024-09-12T01:26:57.531660+0000","last_unstale":"2024-09-12T01:27:16.616589+0000","last_undegraded":"2024-09-12T01:27:16.616589+0000","last_fullsized":"2024-09-12T01:27:16.616589+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:26:12.650687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.727258+0000","last_change":"2024-09-12T01:26:47.476383+0000","last_active":"2024-09-12T01:27:16.727258+0000","last_peered":"2024-09-12T01:27:16.727258+0000","last_clean":"2024-09-12T01:27:16.727258+0000","last_became_active":"2024-09-12T01:26:47.475759+0000","last_became_peered":"2024-09-12T01:26:47.475759+0000","last_unstale":"2024-09-12T01:27:16.727258+0000","last_undegraded":"2024-09-12T01:27:16.727258+0000","last_fullsized":"2024-09-12T01:27:16.727258+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:14:38.898332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"96'3","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:42.309751+0000","last_change":"2024-09-12T01:26:44.445418+0000","last_active":"2024-09-12T01:27:42.309751+0000","last_peered":"2024-09-12T01:27:42.309751+0000","last_clean":"2024-09-12T01:27:42.309751+0000","last_became_active":"2024-09-12T01:26:44.444396+0000","last_became_peered":"2024-09-12T01:26:44.444396+0000","last_unstale":"2024-09-12T01:27:42.309751+0000","last_undegraded":"2024-09-12T01:27:42.309751+0000","last_fullsized":"2024-09-12T01:27:42.309751+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:16:16.595947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"96'7","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:43.319874+0000","last_change":"2024-09-12T01:27:16.620062+0000","last_active":"2024-09-12T01:27:43.319874+0000","last_peered":"2024-09-12T01:27:43.319874+0000","last_clean":"2024-09-12T01:27:43.319874+0000","last_became_active":"2024-09-12T01:27:16.619031+0000","last_became_peered":"2024-09-12T01:27:16.619031+0000","last_unstale":"2024-09-12T01:27:43.319874+0000","last_undegraded":"2024-09-12T01:27:43.319874+0000","last_fullsized":"2024-09-12T01:27:43.319874+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:04:04.329211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.831159+0000","last_change":"2024-09-12T01:26:47.478706+0000","last_active":"2024-09-12T01:27:16.831159+0000","last_peered":"2024-09-12T01:27:16.831159+0000","last_clean":"2024-09-12T01:27:16.831159+0000","last_became_active":"2024-09-12T01:26:47.478339+0000","last_became_peered":"2024-09-12T01:26:47.478339+0000","last_unstale":"2024-09-12T01:27:16.831159+0000","last_undegraded":"2024-09-12T01:27:16.831159+0000","last_fullsized":"2024-09-12T01:27:16.831159+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:20:37.180485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.614983+0000","last_change":"2024-09-12T01:26:57.533268+0000","last_active":"2024-09-12T01:27:16.614983+0000","last_peered":"2024-09-12T01:27:16.614983+0000","last_clean":"2024-09-12T01:27:16.614983+0000","last_became_active":"2024-09-12T01:26:57.532862+0000","last_became_peered":"2024-09-12T01:26:57.532862+0000","last_unstale":"2024-09-12T01:27:16.614983+0000","last_undegraded":"2024-09-12T01:27:16.614983+0000","last_fullsized":"2024-09-12T01:27:16.614983+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:53.659734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"96'5","reported_seq":61,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:58.357220+0000","last_change":"2024-09-12T01:26:44.443113+0000","last_active":"2024-09-12T01:27:58.357220+0000","last_peered":"2024-09-12T01:27:58.357220+0000","last_clean":"2024-09-12T01:27:58.357220+0000","last_became_active":"2024-09-12T01:26:44.442642+0000","last_became_peered":"2024-09-12T01:26:44.442642+0000","last_unstale":"2024-09-12T01:27:58.357220+0000","last_undegraded":"2024-09-12T01:27:58.357220+0000","last_fullsized":"2024-09-12T01:27:58.357220+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:16:20.143475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.618053+0000","last_change":"2024-09-12T01:26:47.476115+0000","last_active":"2024-09-12T01:27:16.618053+0000","last_peered":"2024-09-12T01:27:16.618053+0000","last_clean":"2024-09-12T01:27:16.618053+0000","last_became_active":"2024-09-12T01:26:47.475815+0000","last_became_peered":"2024-09-12T01:26:47.475815+0000","last_unstale":"2024-09-12T01:27:16.618053+0000","last_undegraded":"2024-09-12T01:27:16.618053+0000","last_fullsized":"2024-09-12T01:27:16.618053+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:50.775693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.615201+0000","last_change":"2024-09-12T01:26:57.518831+0000","last_active":"2024-09-12T01:27:16.615201+0000","last_peered":"2024-09-12T01:27:16.615201+0000","last_clean":"2024-09-12T01:27:16.615201+0000","last_became_active":"2024-09-12T01:26:57.518439+0000","last_became_peered":"2024-09-12T01:26:57.518439+0000","last_unstale":"2024-09-12T01:27:16.615201+0000","last_undegraded":"2024-09-12T01:27:16.615201+0000","last_fullsized":"2024-09-12T01:27:16.615201+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:49:14.845248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.832809+0000","last_change":"2024-09-12T01:27:16.832809+0000","last_active":"2024-09-12T01:27:16.832809+0000","last_peered":"2024-09-12T01:27:16.832809+0000","last_clean":"2024-09-12T01:27:16.832809+0000","last_became_active":"2024-09-12T01:27:16.832243+0000","last_became_peered":"2024-09-12T01:27:16.832243+0000","last_unstale":"2024-09-12T01:27:16.832809+0000","last_undegraded":"2024-09-12T01:27:16.832809+0000","last_fullsized":"2024-09-12T01:27:16.832809+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:55:00.002884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.614548+0000","last_change":"2024-09-12T01:26:47.475959+0000","last_active":"2024-09-12T01:27:16.614548+0000","last_peered":"2024-09-12T01:27:16.614548+0000","last_clean":"2024-09-12T01:27:16.614548+0000","last_became_active":"2024-09-12T01:26:47.475660+0000","last_became_peered":"2024-09-12T01:26:47.475660+0000","last_unstale":"2024-09-12T01:27:16.614548+0000","last_undegraded":"2024-09-12T01:27:16.614548+0000","last_fullsized":"2024-09-12T01:27:16.614548+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:42.556069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.614300+0000","last_change":"2024-09-12T01:26:47.474380+0000","last_active":"2024-09-12T01:27:16.614300+0000","last_peered":"2024-09-12T01:27:16.614300+0000","last_clean":"2024-09-12T01:27:16.614300+0000","last_became_active":"2024-09-12T01:26:47.474050+0000","last_became_peered":"2024-09-12T01:26:47.474050+0000","last_unstale":"2024-09-12T01:27:16.614300+0000","last_undegraded":"2024-09-12T01:27:16.614300+0000","last_fullsized":"2024-09-12T01:27:16.614300+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:19:05.032866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.612222+0000","last_change":"2024-09-12T01:26:47.477056+0000","last_active":"2024-09-12T01:27:16.612222+0000","last_peered":"2024-09-12T01:27:16.612222+0000","last_clean":"2024-09-12T01:27:16.612222+0000","last_became_active":"2024-09-12T01:26:47.476641+0000","last_became_peered":"2024-09-12T01:26:47.476641+0000","last_unstale":"2024-09-12T01:27:16.612222+0000","last_undegraded":"2024-09-12T01:27:16.612222+0000","last_fullsized":"2024-09-12T01:27:16.612222+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:52.924471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.832763+0000","last_change":"2024-09-12T01:27:16.832763+0000","last_active":"2024-09-12T01:27:16.832763+0000","last_peered":"2024-09-12T01:27:16.832763+0000","last_clean":"2024-09-12T01:27:16.832763+0000","last_became_active":"2024-09-12T01:27:16.832396+0000","last_became_peered":"2024-09-12T01:27:16.832396+0000","last_unstale":"2024-09-12T01:27:16.832763+0000","last_undegraded":"2024-09-12T01:27:16.832763+0000","last_fullsized":"2024-09-12T01:27:16.832763+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:59.444289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.616683+0000","last_change":"2024-09-12T01:26:47.472125+0000","last_active":"2024-09-12T01:27:16.616683+0000","last_peered":"2024-09-12T01:27:16.616683+0000","last_clean":"2024-09-12T01:27:16.616683+0000","last_became_active":"2024-09-12T01:26:47.471758+0000","last_became_peered":"2024-09-12T01:26:47.471758+0000","last_unstale":"2024-09-12T01:27:16.616683+0000","last_undegraded":"2024-09-12T01:27:16.616683+0000","last_fullsized":"2024-09-12T01:27:16.616683+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:38.949566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.615278+0000","last_change":"2024-09-12T01:26:47.473584+0000","last_active":"2024-09-12T01:27:16.615278+0000","last_peered":"2024-09-12T01:27:16.615278+0000","last_clean":"2024-09-12T01:27:16.615278+0000","last_became_active":"2024-09-12T01:26:47.470089+0000","last_became_peered":"2024-09-12T01:26:47.470089+0000","last_unstale":"2024-09-12T01:27:16.615278+0000","last_undegraded":"2024-09-12T01:27:16.615278+0000","last_fullsized":"2024-09-12T01:27:16.615278+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:44:58.878810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.727704+0000","last_change":"2024-09-12T01:26:47.486547+0000","last_active":"2024-09-12T01:27:16.727704+0000","last_peered":"2024-09-12T01:27:16.727704+0000","last_clean":"2024-09-12T01:27:16.727704+0000","last_became_active":"2024-09-12T01:26:47.486105+0000","last_became_peered":"2024-09-12T01:26:47.486105+0000","last_unstale":"2024-09-12T01:27:16.727704+0000","last_undegraded":"2024-09-12T01:27:16.727704+0000","last_fullsized":"2024-09-12T01:27:16.727704+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:21:18.820930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.830807+0000","last_change":"2024-09-12T01:26:47.480459+0000","last_active":"2024-09-12T01:27:16.830807+0000","last_peered":"2024-09-12T01:27:16.830807+0000","last_clean":"2024-09-12T01:27:16.830807+0000","last_became_active":"2024-09-12T01:26:47.478917+0000","last_became_peered":"2024-09-12T01:26:47.478917+0000","last_unstale":"2024-09-12T01:27:16.830807+0000","last_undegraded":"2024-09-12T01:27:16.830807+0000","last_fullsized":"2024-09-12T01:27:16.830807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:20:54.470888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.727398+0000","last_change":"2024-09-12T01:26:47.480744+0000","last_active":"2024-09-12T01:27:16.727398+0000","last_peered":"2024-09-12T01:27:16.727398+0000","last_clean":"2024-09-12T01:27:16.727398+0000","last_became_active":"2024-09-12T01:26:47.480354+0000","last_became_peered":"2024-09-12T01:26:47.480354+0000","last_unstale":"2024-09-12T01:27:16.727398+0000","last_undegraded":"2024-09-12T01:27:16.727398+0000","last_fullsized":"2024-09-12T01:27:16.727398+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:02:04.670945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.616718+0000","last_change":"2024-09-12T01:26:47.476431+0000","last_active":"2024-09-12T01:27:16.616718+0000","last_peered":"2024-09-12T01:27:16.616718+0000","last_clean":"2024-09-12T01:27:16.616718+0000","last_became_active":"2024-09-12T01:26:47.476110+0000","last_became_peered":"2024-09-12T01:26:47.476110+0000","last_unstale":"2024-09-12T01:27:16.616718+0000","last_undegraded":"2024-09-12T01:27:16.616718+0000","last_fullsized":"2024-09-12T01:27:16.616718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:34:24.086677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.615591+0000","last_change":"2024-09-12T01:26:47.484568+0000","last_active":"2024-09-12T01:27:16.615591+0000","last_peered":"2024-09-12T01:27:16.615591+0000","last_clean":"2024-09-12T01:27:16.615591+0000","last_became_active":"2024-09-12T01:26:47.484165+0000","last_became_peered":"2024-09-12T01:26:47.484165+0000","last_unstale":"2024-09-12T01:27:16.615591+0000","last_undegraded":"2024-09-12T01:27:16.615591+0000","last_fullsized":"2024-09-12T01:27:16.615591+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:26:51.324754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.616471+0000","last_change":"2024-09-12T01:26:47.482608+0000","last_active":"2024-09-12T01:27:16.616471+0000","last_peered":"2024-09-12T01:27:16.616471+0000","last_clean":"2024-09-12T01:27:16.616471+0000","last_became_active":"2024-09-12T01:26:47.482299+0000","last_became_peered":"2024-09-12T01:26:47.482299+0000","last_unstale":"2024-09-12T01:27:16.616471+0000","last_undegraded":"2024-09-12T01:27:16.616471+0000","last_fullsized":"2024-09-12T01:27:16.616471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:48:39.516539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.615626+0000","last_change":"2024-09-12T01:26:47.472976+0000","last_active":"2024-09-12T01:27:16.615626+0000","last_peered":"2024-09-12T01:27:16.615626+0000","last_clean":"2024-09-12T01:27:16.615626+0000","last_became_active":"2024-09-12T01:26:47.472495+0000","last_became_peered":"2024-09-12T01:26:47.472495+0000","last_unstale":"2024-09-12T01:27:16.615626+0000","last_undegraded":"2024-09-12T01:27:16.615626+0000","last_fullsized":"2024-09-12T01:27:16.615626+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:57.556675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.831973+0000","last_change":"2024-09-12T01:26:47.476004+0000","last_active":"2024-09-12T01:27:16.831973+0000","last_peered":"2024-09-12T01:27:16.831973+0000","last_clean":"2024-09-12T01:27:16.831973+0000","last_became_active":"2024-09-12T01:26:47.475440+0000","last_became_peered":"2024-09-12T01:26:47.475440+0000","last_unstale":"2024-09-12T01:27:16.831973+0000","last_undegraded":"2024-09-12T01:27:16.831973+0000","last_fullsized":"2024-09-12T01:27:16.831973+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:36:47.650577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.617378+0000","last_change":"2024-09-12T01:26:47.483848+0000","last_active":"2024-09-12T01:27:16.617378+0000","last_peered":"2024-09-12T01:27:16.617378+0000","last_clean":"2024-09-12T01:27:16.617378+0000","last_became_active":"2024-09-12T01:26:47.483446+0000","last_became_peered":"2024-09-12T01:26:47.483446+0000","last_unstale":"2024-09-12T01:27:16.617378+0000","last_undegraded":"2024-09-12T01:27:16.617378+0000","last_fullsized":"2024-09-12T01:27:16.617378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:47.844458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.615704+0000","last_change":"2024-09-12T01:26:47.473801+0000","last_active":"2024-09-12T01:27:16.615704+0000","last_peered":"2024-09-12T01:27:16.615704+0000","last_clean":"2024-09-12T01:27:16.615704+0000","last_became_active":"2024-09-12T01:26:47.473268+0000","last_became_peered":"2024-09-12T01:26:47.473268+0000","last_unstale":"2024-09-12T01:27:16.615704+0000","last_undegraded":"2024-09-12T01:27:16.615704+0000","last_fullsized":"2024-09-12T01:27:16.615704+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:13:30.595643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.611890+0000","last_change":"2024-09-12T01:26:47.475778+0000","last_active":"2024-09-12T01:27:16.611890+0000","last_peered":"2024-09-12T01:27:16.611890+0000","last_clean":"2024-09-12T01:27:16.611890+0000","last_became_active":"2024-09-12T01:26:47.475477+0000","last_became_peered":"2024-09-12T01:26:47.475477+0000","last_unstale":"2024-09-12T01:27:16.611890+0000","last_undegraded":"2024-09-12T01:27:16.611890+0000","last_fullsized":"2024-09-12T01:27:16.611890+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:56.555424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.611897+0000","last_change":"2024-09-12T01:26:47.474736+0000","last_active":"2024-09-12T01:27:16.611897+0000","last_peered":"2024-09-12T01:27:16.611897+0000","last_clean":"2024-09-12T01:27:16.611897+0000","last_became_active":"2024-09-12T01:26:47.474378+0000","last_became_peered":"2024-09-12T01:26:47.474378+0000","last_unstale":"2024-09-12T01:27:16.611897+0000","last_undegraded":"2024-09-12T01:27:16.611897+0000","last_fullsized":"2024-09-12T01:27:16.611897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:02:37.818552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.615491+0000","last_change":"2024-09-12T01:26:47.479586+0000","last_active":"2024-09-12T01:27:16.615491+0000","last_peered":"2024-09-12T01:27:16.615491+0000","last_clean":"2024-09-12T01:27:16.615491+0000","last_became_active":"2024-09-12T01:26:47.473216+0000","last_became_peered":"2024-09-12T01:26:47.473216+0000","last_unstale":"2024-09-12T01:27:16.615491+0000","last_undegraded":"2024-09-12T01:27:16.615491+0000","last_fullsized":"2024-09-12T01:27:16.615491+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:05:28.994081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.618661+0000","last_change":"2024-09-12T01:26:47.475634+0000","last_active":"2024-09-12T01:27:16.618661+0000","last_peered":"2024-09-12T01:27:16.618661+0000","last_clean":"2024-09-12T01:27:16.618661+0000","last_became_active":"2024-09-12T01:26:47.475319+0000","last_became_peered":"2024-09-12T01:26:47.475319+0000","last_unstale":"2024-09-12T01:27:16.618661+0000","last_undegraded":"2024-09-12T01:27:16.618661+0000","last_fullsized":"2024-09-12T01:27:16.618661+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:40.095276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.619362+0000","last_change":"2024-09-12T01:26:47.475721+0000","last_active":"2024-09-12T01:27:16.619362+0000","last_peered":"2024-09-12T01:27:16.619362+0000","last_clean":"2024-09-12T01:27:16.619362+0000","last_became_active":"2024-09-12T01:26:47.475149+0000","last_became_peered":"2024-09-12T01:26:47.475149+0000","last_unstale":"2024-09-12T01:27:16.619362+0000","last_undegraded":"2024-09-12T01:27:16.619362+0000","last_fullsized":"2024-09-12T01:27:16.619362+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:27:19.946756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.613490+0000","last_change":"2024-09-12T01:26:47.482667+0000","last_active":"2024-09-12T01:27:16.613490+0000","last_peered":"2024-09-12T01:27:16.613490+0000","last_clean":"2024-09-12T01:27:16.613490+0000","last_became_active":"2024-09-12T01:26:47.482258+0000","last_became_peered":"2024-09-12T01:26:47.482258+0000","last_unstale":"2024-09-12T01:27:16.613490+0000","last_undegraded":"2024-09-12T01:27:16.613490+0000","last_fullsized":"2024-09-12T01:27:16.613490+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:10.449000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.616427+0000","last_change":"2024-09-12T01:26:47.475406+0000","last_active":"2024-09-12T01:27:16.616427+0000","last_peered":"2024-09-12T01:27:16.616427+0000","last_clean":"2024-09-12T01:27:16.616427+0000","last_became_active":"2024-09-12T01:26:47.474846+0000","last_became_peered":"2024-09-12T01:26:47.474846+0000","last_unstale":"2024-09-12T01:27:16.616427+0000","last_undegraded":"2024-09-12T01:27:16.616427+0000","last_fullsized":"2024-09-12T01:27:16.616427+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:54:02.527691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.727210+0000","last_change":"2024-09-12T01:26:47.477383+0000","last_active":"2024-09-12T01:27:16.727210+0000","last_peered":"2024-09-12T01:27:16.727210+0000","last_clean":"2024-09-12T01:27:16.727210+0000","last_became_active":"2024-09-12T01:26:47.476977+0000","last_became_peered":"2024-09-12T01:26:47.476977+0000","last_unstale":"2024-09-12T01:27:16.727210+0000","last_undegraded":"2024-09-12T01:27:16.727210+0000","last_fullsized":"2024-09-12T01:27:16.727210+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:51:45.310556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.617664+0000","last_change":"2024-09-12T01:26:47.473686+0000","last_active":"2024-09-12T01:27:16.617664+0000","last_peered":"2024-09-12T01:27:16.617664+0000","last_clean":"2024-09-12T01:27:16.617664+0000","last_became_active":"2024-09-12T01:26:47.473363+0000","last_became_peered":"2024-09-12T01:26:47.473363+0000","last_unstale":"2024-09-12T01:27:16.617664+0000","last_undegraded":"2024-09-12T01:27:16.617664+0000","last_fullsized":"2024-09-12T01:27:16.617664+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:08:23.238434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.612749+0000","last_change":"2024-09-12T01:26:47.479310+0000","last_active":"2024-09-12T01:27:16.612749+0000","last_peered":"2024-09-12T01:27:16.612749+0000","last_clean":"2024-09-12T01:27:16.612749+0000","last_became_active":"2024-09-12T01:26:47.477416+0000","last_became_peered":"2024-09-12T01:26:47.477416+0000","last_unstale":"2024-09-12T01:27:16.612749+0000","last_undegraded":"2024-09-12T01:27:16.612749+0000","last_fullsized":"2024-09-12T01:27:16.612749+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:01:55.023455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.617943+0000","last_change":"2024-09-12T01:26:47.478455+0000","last_active":"2024-09-12T01:27:16.617943+0000","last_peered":"2024-09-12T01:27:16.617943+0000","last_clean":"2024-09-12T01:27:16.617943+0000","last_became_active":"2024-09-12T01:26:47.478093+0000","last_became_peered":"2024-09-12T01:26:47.478093+0000","last_unstale":"2024-09-12T01:27:16.617943+0000","last_undegraded":"2024-09-12T01:27:16.617943+0000","last_fullsized":"2024-09-12T01:27:16.617943+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:04:41.626677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.618563+0000","last_change":"2024-09-12T01:26:47.479385+0000","last_active":"2024-09-12T01:27:16.618563+0000","last_peered":"2024-09-12T01:27:16.618563+0000","last_clean":"2024-09-12T01:27:16.618563+0000","last_became_active":"2024-09-12T01:26:47.479025+0000","last_became_peered":"2024-09-12T01:26:47.479025+0000","last_unstale":"2024-09-12T01:27:16.618563+0000","last_undegraded":"2024-09-12T01:27:16.618563+0000","last_fullsized":"2024-09-12T01:27:16.618563+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:11.927111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.612094+0000","last_change":"2024-09-12T01:26:47.461567+0000","last_active":"2024-09-12T01:27:16.612094+0000","last_peered":"2024-09-12T01:27:16.612094+0000","last_clean":"2024-09-12T01:27:16.612094+0000","last_became_active":"2024-09-12T01:26:47.461230+0000","last_became_peered":"2024-09-12T01:26:47.461230+0000","last_unstale":"2024-09-12T01:27:16.612094+0000","last_undegraded":"2024-09-12T01:27:16.612094+0000","last_fullsized":"2024-09-12T01:27:16.612094+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:09:24.519448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.612091+0000","last_change":"2024-09-12T01:26:47.472323+0000","last_active":"2024-09-12T01:27:16.612091+0000","last_peered":"2024-09-12T01:27:16.612091+0000","last_clean":"2024-09-12T01:27:16.612091+0000","last_became_active":"2024-09-12T01:26:47.469837+0000","last_became_peered":"2024-09-12T01:26:47.469837+0000","last_unstale":"2024-09-12T01:27:16.612091+0000","last_undegraded":"2024-09-12T01:27:16.612091+0000","last_fullsized":"2024-09-12T01:27:16.612091+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:46.202656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T01:27:16.617266+0000","last_change":"2024-09-12T01:26:47.479707+0000","last_active":"2024-09-12T01:27:16.617266+0000","last_peered":"2024-09-12T01:27:16.617266+0000","last_clean":"2024-09-12T01:27:16.617266+0000","last_became_active":"2024-09-12T01:26:47.479368+0000","last_became_peered":"2024-09-12T01:26:47.479368+0000","last_unstale":"2024-09-12T01:27:16.617266+0000","last_undegraded":"2024-09-12T01:27:16.617266+0000","last_fullsized":"2024-09-12T01:27:16.617266+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:04:30.129381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64350,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":211,"num_read_kb":178,"num_write":243,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":193050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416412,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1101824,"data_stored":545474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645345,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37096,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708056,"statfs":{"total":95995035648,"available":95957049344,"internally_reserved":0,"allocated":1794048,"data_stored":1193135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808871,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1126400,"data_stored":545011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070508,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":545389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332145,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37024,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708128,"statfs":{"total":95995035648,"available":95957123072,"internally_reserved":0,"allocated":1720320,"data_stored":1142281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495672,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":540908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757309,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37032,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708120,"statfs":{"total":95995035648,"available":95957114880,"internally_reserved":0,"allocated":1728512,"data_stored":1137077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920836,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28172,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716980,"statfs":{"total":95995035648,"available":95966187520,"internally_reserved":0,"allocated":1175552,"data_stored":601731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182472,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1114112,"data_stored":540223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36436,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708716,"statfs":{"total":95995035648,"available":95957725184,"internally_reserved":0,"allocated":1118208,"data_stored":542158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36472,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708680,"statfs":{"total":95995035648,"available":95957688320,"internally_reserved":0,"allocated":1155072,"data_stored":591036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36444,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708708,"statfs":{"total":95995035648,"available":95957716992,"internally_reserved":0,"allocated":1126400,"data_stored":542364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":58894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":57842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":58878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T01:28:06.139 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-12T01:28:06.140 INFO:tasks.cephadm:Setting up client nodes... 2024-09-12T01:28:06.140 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-12T01:28:06.520 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:28:06.658 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:06 smithi001 ceph-mon[30387]: pgmap v351: 129 pgs: 129 active+clean; 640 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.4 KiB/s wr, 11 op/s 2024-09-12T01:28:06.659 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:06 smithi001 ceph-mon[30387]: from='client.24754 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:28:06.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:06 smithi053 ceph-mon[35158]: pgmap v351: 129 pgs: 129 active+clean; 640 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.4 KiB/s wr, 11 op/s 2024-09-12T01:28:06.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:06 smithi053 ceph-mon[35158]: from='client.24754 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:28:06.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:06 smithi104 ceph-mon[35219]: pgmap v351: 129 pgs: 129 active+clean; 640 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.4 KiB/s wr, 11 op/s 2024-09-12T01:28:06.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:06 smithi104 ceph-mon[35219]: from='client.24754 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:28:07.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.135+0000 7f9781600640 1 -- 172.21.15.1:0/2903480874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f977c074b40 msgr2=0x7f977c074fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:07.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.135+0000 7f9781600640 1 --2- 172.21.15.1:0/2903480874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f977c074b40 0x7f977c074fa0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9764009900 tx=0x7f976402a410 comp rx=0 tx=0).stop 2024-09-12T01:28:07.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.135+0000 7f9781600640 1 -- 172.21.15.1:0/2903480874 shutdown_connections 2024-09-12T01:28:07.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.135+0000 7f9781600640 1 --2- 172.21.15.1:0/2903480874 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f977c1098b0 0x7f977c10bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:07.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.135+0000 7f9781600640 1 --2- 172.21.15.1:0/2903480874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f977c074b40 0x7f977c074fa0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:07.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.135+0000 7f9781600640 1 --2- 172.21.15.1:0/2903480874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f977c0761a0 0x7f977c074600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:07.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.135+0000 7f9781600640 1 -- 172.21.15.1:0/2903480874 >> 172.21.15.1:0/2903480874 conn(0x7f977c0fd1f0 msgr2=0x7f977c0ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:07.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.135+0000 7f9781600640 1 -- 172.21.15.1:0/2903480874 shutdown_connections 2024-09-12T01:28:07.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.135+0000 7f9781600640 1 -- 172.21.15.1:0/2903480874 wait complete. 2024-09-12T01:28:07.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.136+0000 7f9781600640 1 Processor -- start 2024-09-12T01:28:07.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.136+0000 7f9781600640 1 -- start start 2024-09-12T01:28:07.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.136+0000 7f9781600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f977c074b40 0x7f977c199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:07.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.136+0000 7f9781600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f977c0761a0 0x7f977c19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:07.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.137+0000 7f9781600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f977c1098b0 0x7f977c1a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:07.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.137+0000 7f9781600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f977c10f8b0 con 0x7f977c1098b0 2024-09-12T01:28:07.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.137+0000 7f9781600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f977c10f730 con 0x7f977c0761a0 2024-09-12T01:28:07.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.137+0000 7f9781600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f977c10fa30 con 0x7f977c074b40 2024-09-12T01:28:07.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.137+0000 7f9773e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f977c0761a0 0x7f977c19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:07.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.137+0000 7f9773e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f977c0761a0 0x7f977c19a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:40126/0 (socket says 172.21.15.1:40126) 2024-09-12T01:28:07.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.137+0000 7f9780c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f977c1098b0 0x7f977c1a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:07.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.137+0000 7f9773e00640 1 -- 172.21.15.1:0/4050025261 learned_addr learned my addr 172.21.15.1:0/4050025261 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:28:07.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.137+0000 7f977be00640 1 --2- 172.21.15.1:0/4050025261 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f977c074b40 0x7f977c199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:07.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.137+0000 7f9773e00640 1 -- 172.21.15.1:0/4050025261 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f977c074b40 msgr2=0x7f977c199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:07.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.137+0000 7f9773e00640 1 --2- 172.21.15.1:0/4050025261 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f977c074b40 0x7f977c199f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:07.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.137+0000 7f9773e00640 1 -- 172.21.15.1:0/4050025261 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f977c1098b0 msgr2=0x7f977c1a14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:07.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.137+0000 7f9773e00640 1 --2- 172.21.15.1:0/4050025261 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f977c1098b0 0x7f977c1a14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:07.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.137+0000 7f9773e00640 1 -- 172.21.15.1:0/4050025261 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f977c1a1b40 con 0x7f977c0761a0 2024-09-12T01:28:07.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.138+0000 7f9773e00640 1 --2- 172.21.15.1:0/4050025261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f977c0761a0 0x7f977c19a450 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9764009230 tx=0x7f97640042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:07.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.138+0000 7f9779600640 1 -- 172.21.15.1:0/4050025261 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976403f030 con 0x7f977c0761a0 2024-09-12T01:28:07.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.138+0000 7f9779600640 1 -- 172.21.15.1:0/4050025261 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9764003ce0 con 0x7f977c0761a0 2024-09-12T01:28:07.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.138+0000 7f9781600640 1 -- 172.21.15.1:0/4050025261 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f977c1a1dd0 con 0x7f977c0761a0 2024-09-12T01:28:07.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.138+0000 7f9779600640 1 -- 172.21.15.1:0/4050025261 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9764035070 con 0x7f977c0761a0 2024-09-12T01:28:07.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.138+0000 7f9781600640 1 -- 172.21.15.1:0/4050025261 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f977c1a2250 con 0x7f977c0761a0 2024-09-12T01:28:07.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.139+0000 7f9781600640 1 -- 172.21.15.1:0/4050025261 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f977c1086c0 con 0x7f977c0761a0 2024-09-12T01:28:07.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.141+0000 7f9779600640 1 -- 172.21.15.1:0/4050025261 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f9764042030 con 0x7f977c0761a0 2024-09-12T01:28:07.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.142+0000 7f9779600640 1 --2- 172.21.15.1:0/4050025261 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f97600a2c40 0x7f97600a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:07.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.142+0000 7f9779600640 1 -- 172.21.15.1:0/4050025261 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9092+0+0 (secure 0 0 0) 0x7f97640efdd0 con 0x7f977c0761a0 2024-09-12T01:28:07.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.142+0000 7f977be00640 1 --2- 172.21.15.1:0/4050025261 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f97600a2c40 0x7f97600a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:07.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.143+0000 7f977be00640 1 --2- 172.21.15.1:0/4050025261 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f97600a2c40 0x7f97600a5100 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f976c009bd0 tx=0x7f976c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:07.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.144+0000 7f9779600640 1 -- 172.21.15.1:0/4050025261 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f97640bc1c0 con 0x7f977c0761a0 2024-09-12T01:28:07.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.401+0000 7f9781600640 1 -- 172.21.15.1:0/4050025261 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f977c02da30 con 0x7f977c0761a0 2024-09-12T01:28:07.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.407+0000 7f9779600640 1 -- 172.21.15.1:0/4050025261 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f97640c04c0 con 0x7f977c0761a0 2024-09-12T01:28:07.406 INFO:teuthology.orchestra.run.smithi001.stdout:[client.0] 2024-09-12T01:28:07.406 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQCnQ+Jm8UkNGBAAdm8DvSHKwWpydVtc69w9WA== 2024-09-12T01:28:07.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.409+0000 7f9781600640 1 -- 172.21.15.1:0/4050025261 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f97600a2c40 msgr2=0x7f97600a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:07.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.409+0000 7f9781600640 1 --2- 172.21.15.1:0/4050025261 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f97600a2c40 0x7f97600a5100 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f976c009bd0 tx=0x7f976c005f90 comp rx=0 tx=0).stop 2024-09-12T01:28:07.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.409+0000 7f9781600640 1 -- 172.21.15.1:0/4050025261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f977c0761a0 msgr2=0x7f977c19a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:07.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.409+0000 7f9781600640 1 --2- 172.21.15.1:0/4050025261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f977c0761a0 0x7f977c19a450 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9764009230 tx=0x7f97640042f0 comp rx=0 tx=0).stop 2024-09-12T01:28:07.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.409+0000 7f9781600640 1 -- 172.21.15.1:0/4050025261 shutdown_connections 2024-09-12T01:28:07.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.409+0000 7f9781600640 1 --2- 172.21.15.1:0/4050025261 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f977c1098b0 0x7f977c1a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:07.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.409+0000 7f9781600640 1 --2- 172.21.15.1:0/4050025261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f977c0761a0 0x7f977c19a450 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:07.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.409+0000 7f9781600640 1 --2- 172.21.15.1:0/4050025261 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f97600a2c40 0x7f97600a5100 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:07.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.409+0000 7f9781600640 1 --2- 172.21.15.1:0/4050025261 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f977c074b40 0x7f977c199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:07.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.409+0000 7f9781600640 1 -- 172.21.15.1:0/4050025261 >> 172.21.15.1:0/4050025261 conn(0x7f977c0fd1f0 msgr2=0x7f977c105890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:07.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.409+0000 7f9781600640 1 -- 172.21.15.1:0/4050025261 shutdown_connections 2024-09-12T01:28:07.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:07.409+0000 7f9781600640 1 -- 172.21.15.1:0/4050025261 wait complete. 2024-09-12T01:28:08.290 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-12T01:28:08.290 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-12T01:28:08.290 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-12T01:28:08.370 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-12T01:28:08.370 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-12T01:28:08.371 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-12T01:28:08.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:08 smithi001 ceph-mon[30387]: pgmap v352: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-09-12T01:28:08.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:08 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/4050025261' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T01:28:08.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:08 smithi001 ceph-mon[30387]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T01:28:08.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:08 smithi001 ceph-mon[30387]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-12T01:28:08.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:08 smithi053 ceph-mon[35158]: pgmap v352: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-09-12T01:28:08.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:08 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/4050025261' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T01:28:08.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:08 smithi053 ceph-mon[35158]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T01:28:08.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:08 smithi053 ceph-mon[35158]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-12T01:28:08.786 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:28:08.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:08 smithi104 ceph-mon[35219]: pgmap v352: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-09-12T01:28:08.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:08 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/4050025261' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T01:28:08.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:08 smithi104 ceph-mon[35219]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T01:28:08.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:08 smithi104 ceph-mon[35219]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-12T01:28:09.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.386+0000 7f8d0be00640 1 -- 172.21.15.1:0/1494633934 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8d04106f10 msgr2=0x7f8d0410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:09.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.386+0000 7f8d0be00640 1 --2- 172.21.15.1:0/1494633934 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8d04106f10 0x7f8d0410ea90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc009ca0 tx=0x7f8cfc02a8c0 comp rx=0 tx=0).stop 2024-09-12T01:28:09.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.386+0000 7f8d0be00640 1 -- 172.21.15.1:0/1494633934 shutdown_connections 2024-09-12T01:28:09.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.386+0000 7f8d0be00640 1 --2- 172.21.15.1:0/1494633934 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8d04106f10 0x7f8d0410ea90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:09.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.386+0000 7f8d0be00640 1 --2- 172.21.15.1:0/1494633934 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d04102790 0x7f8d041067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:09.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.386+0000 7f8d0be00640 1 --2- 172.21.15.1:0/1494633934 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d04101de0 0x7f8d041021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:09.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.386+0000 7f8d0be00640 1 -- 172.21.15.1:0/1494633934 >> 172.21.15.1:0/1494633934 conn(0x7f8d040fd8c0 msgr2=0x7f8d040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:09.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.386+0000 7f8d0be00640 1 -- 172.21.15.1:0/1494633934 shutdown_connections 2024-09-12T01:28:09.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.386+0000 7f8d0be00640 1 -- 172.21.15.1:0/1494633934 wait complete. 2024-09-12T01:28:09.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.387+0000 7f8d0be00640 1 Processor -- start 2024-09-12T01:28:09.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.387+0000 7f8d0be00640 1 -- start start 2024-09-12T01:28:09.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.387+0000 7f8d0be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8d04101de0 0x7f8d0419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:09.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.387+0000 7f8d0be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d04102790 0x7f8d0419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:09.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.387+0000 7f8d0be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d04106f10 0x7f8d0419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:09.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.387+0000 7f8d0be00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8d04111fc0 con 0x7f8d04101de0 2024-09-12T01:28:09.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.387+0000 7f8d0be00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8d04111e40 con 0x7f8d04102790 2024-09-12T01:28:09.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.387+0000 7f8d0be00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8d04112140 con 0x7f8d04106f10 2024-09-12T01:28:09.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.388+0000 7f8d09400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8d04101de0 0x7f8d0419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:09.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.388+0000 7f8d09400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8d04101de0 0x7f8d0419a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51250/0 (socket says 172.21.15.1:51250) 2024-09-12T01:28:09.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.388+0000 7f8d09400640 1 -- 172.21.15.1:0/1996344180 learned_addr learned my addr 172.21.15.1:0/1996344180 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:28:09.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.388+0000 7f8d09e00640 1 --2- 172.21.15.1:0/1996344180 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d04106f10 0x7f8d0419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:09.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.388+0000 7f8d08a00640 1 --2- 172.21.15.1:0/1996344180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d04102790 0x7f8d0419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:09.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.388+0000 7f8d09400640 1 -- 172.21.15.1:0/1996344180 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d04106f10 msgr2=0x7f8d0419ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:09.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.388+0000 7f8d09400640 1 --2- 172.21.15.1:0/1996344180 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d04106f10 0x7f8d0419ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:09.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.388+0000 7f8d09400640 1 -- 172.21.15.1:0/1996344180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d04102790 msgr2=0x7f8d0419ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:09.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.388+0000 7f8d09400640 1 --2- 172.21.15.1:0/1996344180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d04102790 0x7f8d0419ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:09.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.388+0000 7f8d09400640 1 -- 172.21.15.1:0/1996344180 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d0419f5f0 con 0x7f8d04101de0 2024-09-12T01:28:09.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.388+0000 7f8d09400640 1 --2- 172.21.15.1:0/1996344180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8d04101de0 0x7f8d0419a640 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8cf800e9f0 tx=0x7f8cf800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:09.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.389+0000 7f8d02000640 1 -- 172.21.15.1:0/1996344180 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf800cd80 con 0x7f8d04101de0 2024-09-12T01:28:09.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.389+0000 7f8d02000640 1 -- 172.21.15.1:0/1996344180 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cf8010910 con 0x7f8d04101de0 2024-09-12T01:28:09.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.389+0000 7f8d0be00640 1 -- 172.21.15.1:0/1996344180 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d0419f8e0 con 0x7f8d04101de0 2024-09-12T01:28:09.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.389+0000 7f8d02000640 1 -- 172.21.15.1:0/1996344180 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf8018990 con 0x7f8d04101de0 2024-09-12T01:28:09.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.389+0000 7f8d0be00640 1 -- 172.21.15.1:0/1996344180 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d041a71c0 con 0x7f8d04101de0 2024-09-12T01:28:09.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.390+0000 7f8d0be00640 1 -- 172.21.15.1:0/1996344180 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d041021c0 con 0x7f8d04101de0 2024-09-12T01:28:09.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.392+0000 7f8d02000640 1 -- 172.21.15.1:0/1996344180 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f8cf8010ac0 con 0x7f8d04101de0 2024-09-12T01:28:09.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.394+0000 7f8d02000640 1 --2- 172.21.15.1:0/1996344180 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f8cec0a2c90 0x7f8cec0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:09.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.394+0000 7f8d02000640 1 -- 172.21.15.1:0/1996344180 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9092+0+0 (secure 0 0 0) 0x7f8cf80cf800 con 0x7f8d04101de0 2024-09-12T01:28:09.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.394+0000 7f8d08a00640 1 --2- 172.21.15.1:0/1996344180 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f8cec0a2c90 0x7f8cec0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:09.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.395+0000 7f8d08a00640 1 --2- 172.21.15.1:0/1996344180 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f8cec0a2c90 0x7f8cec0a5150 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8d040ff6a0 tx=0x7f8cf0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:09.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.398+0000 7f8d02000640 1 -- 172.21.15.1:0/1996344180 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8cf809bbf0 con 0x7f8d04101de0 2024-09-12T01:28:09.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.637+0000 7f8d0be00640 1 -- 172.21.15.1:0/1996344180 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f8d041a7480 con 0x7f8d04101de0 2024-09-12T01:28:09.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.643+0000 7f8d02000640 1 -- 172.21.15.1:0/1996344180 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263191 (secure 0 0 0) 0x7f8cf809fef0 con 0x7f8d04101de0 2024-09-12T01:28:09.646 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:28:09.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.647+0000 7f8d0be00640 1 -- 172.21.15.1:0/1996344180 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f8cec0a2c90 msgr2=0x7f8cec0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:09.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.647+0000 7f8d0be00640 1 --2- 172.21.15.1:0/1996344180 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f8cec0a2c90 0x7f8cec0a5150 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8d040ff6a0 tx=0x7f8cf0005f90 comp rx=0 tx=0).stop 2024-09-12T01:28:09.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.647+0000 7f8d0be00640 1 -- 172.21.15.1:0/1996344180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8d04101de0 msgr2=0x7f8d0419a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:09.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.647+0000 7f8d0be00640 1 --2- 172.21.15.1:0/1996344180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8d04101de0 0x7f8d0419a640 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8cf800e9f0 tx=0x7f8cf800eeb0 comp rx=0 tx=0).stop 2024-09-12T01:28:09.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.647+0000 7f8d0be00640 1 -- 172.21.15.1:0/1996344180 shutdown_connections 2024-09-12T01:28:09.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.647+0000 7f8d0be00640 1 --2- 172.21.15.1:0/1996344180 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f8cec0a2c90 0x7f8cec0a5150 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:09.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.647+0000 7f8d0be00640 1 --2- 172.21.15.1:0/1996344180 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d04106f10 0x7f8d0419ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:09.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.647+0000 7f8d0be00640 1 --2- 172.21.15.1:0/1996344180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d04102790 0x7f8d0419ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:09.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.647+0000 7f8d0be00640 1 --2- 172.21.15.1:0/1996344180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8d04101de0 0x7f8d0419a640 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:09.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.647+0000 7f8d0be00640 1 -- 172.21.15.1:0/1996344180 >> 172.21.15.1:0/1996344180 conn(0x7f8d040fd8c0 msgr2=0x7f8d040fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:09.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.647+0000 7f8d0be00640 1 -- 172.21.15.1:0/1996344180 shutdown_connections 2024-09-12T01:28:09.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:09.647+0000 7f8d0be00640 1 -- 172.21.15.1:0/1996344180 wait complete. 2024-09-12T01:28:10.423 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":1928662456},{"type":"v1","addr":"172.21.15.1:6801","nonce":1928662456}]},"active_addr":"172.21.15.1:6801/1928662456","active_change":"2024-09-12T01:19:14.580745+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.1:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":743312396}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":4032339702}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":1385185651}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":2721395899}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":2846539740}]}]} 2024-09-12T01:28:10.428 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-12T01:28:10.428 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-12T01:28:10.429 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T01:28:10.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:10 smithi001 ceph-mon[30387]: pgmap v353: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-09-12T01:28:10.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:10 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/1996344180' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-12T01:28:10.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:10 smithi053 ceph-mon[35158]: pgmap v353: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-09-12T01:28:10.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:10 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/1996344180' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-12T01:28:10.810 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:28:10.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:10 smithi104 ceph-mon[35219]: pgmap v353: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-09-12T01:28:10.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:10 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/1996344180' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-12T01:28:11.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.423+0000 7efcca000640 1 -- 172.21.15.1:0/972616602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcc4103ff0 msgr2=0x7efcc41043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:11.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.423+0000 7efcca000640 1 --2- 172.21.15.1:0/972616602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcc4103ff0 0x7efcc41043d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efcb4009860 tx=0x7efcb402a340 comp rx=0 tx=0).stop 2024-09-12T01:28:11.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.423+0000 7efcca000640 1 -- 172.21.15.1:0/972616602 shutdown_connections 2024-09-12T01:28:11.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.423+0000 7efcca000640 1 --2- 172.21.15.1:0/972616602 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efcc4109120 0x7efcc4110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:11.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.423+0000 7efcca000640 1 --2- 172.21.15.1:0/972616602 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcc41049a0 0x7efcc41089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:11.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.423+0000 7efcca000640 1 --2- 172.21.15.1:0/972616602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcc4103ff0 0x7efcc41043d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:11.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.423+0000 7efcca000640 1 -- 172.21.15.1:0/972616602 >> 172.21.15.1:0/972616602 conn(0x7efcc40ffa90 msgr2=0x7efcc4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:11.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.423+0000 7efcca000640 1 -- 172.21.15.1:0/972616602 shutdown_connections 2024-09-12T01:28:11.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.424+0000 7efcca000640 1 -- 172.21.15.1:0/972616602 wait complete. 2024-09-12T01:28:11.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.424+0000 7efcca000640 1 Processor -- start 2024-09-12T01:28:11.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.425+0000 7efcca000640 1 -- start start 2024-09-12T01:28:11.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.425+0000 7efcca000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcc4103ff0 0x7efcc419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:11.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.425+0000 7efcca000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efcc41049a0 0x7efcc419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:11.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.425+0000 7efcca000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcc4109120 0x7efcc419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:11.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.425+0000 7efcca000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efcc4114150 con 0x7efcc4103ff0 2024-09-12T01:28:11.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.425+0000 7efcca000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7efcc4113fd0 con 0x7efcc4109120 2024-09-12T01:28:11.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.425+0000 7efcca000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efcc41142d0 con 0x7efcc41049a0 2024-09-12T01:28:11.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.425+0000 7efcc3400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcc4103ff0 0x7efcc419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:11.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.425+0000 7efcc3400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcc4103ff0 0x7efcc419a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51254/0 (socket says 172.21.15.1:51254) 2024-09-12T01:28:11.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.425+0000 7efcc3400640 1 -- 172.21.15.1:0/1677354237 learned_addr learned my addr 172.21.15.1:0/1677354237 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:28:11.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.425+0000 7efcc3e00640 1 --2- 172.21.15.1:0/1677354237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcc4109120 0x7efcc419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:11.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.426+0000 7efcbbe00640 1 --2- 172.21.15.1:0/1677354237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efcc41049a0 0x7efcc419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:11.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.426+0000 7efcc3400640 1 -- 172.21.15.1:0/1677354237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efcc41049a0 msgr2=0x7efcc419abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:11.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.426+0000 7efcc3400640 1 --2- 172.21.15.1:0/1677354237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efcc41049a0 0x7efcc419abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:11.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.426+0000 7efcc3400640 1 -- 172.21.15.1:0/1677354237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcc4109120 msgr2=0x7efcc419ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:11.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.426+0000 7efcc3400640 1 --2- 172.21.15.1:0/1677354237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcc4109120 0x7efcc419ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:11.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.426+0000 7efcc3400640 1 -- 172.21.15.1:0/1677354237 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc419f620 con 0x7efcc4103ff0 2024-09-12T01:28:11.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.426+0000 7efcbbe00640 1 --2- 172.21.15.1:0/1677354237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efcc41049a0 0x7efcc419abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:28:11.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.426+0000 7efcc3400640 1 --2- 172.21.15.1:0/1677354237 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcc4103ff0 0x7efcc419a670 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efcb4035040 tx=0x7efcb4033b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:11.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.427+0000 7efcc0c00640 1 -- 172.21.15.1:0/1677354237 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb4037040 con 0x7efcc4103ff0 2024-09-12T01:28:11.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.427+0000 7efcca000640 1 -- 172.21.15.1:0/1677354237 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcc419f8b0 con 0x7efcc4103ff0 2024-09-12T01:28:11.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.427+0000 7efcc0c00640 1 -- 172.21.15.1:0/1677354237 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcb4040070 con 0x7efcc4103ff0 2024-09-12T01:28:11.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.427+0000 7efcc0c00640 1 -- 172.21.15.1:0/1677354237 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb40039b0 con 0x7efcc4103ff0 2024-09-12T01:28:11.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.427+0000 7efcca000640 1 -- 172.21.15.1:0/1677354237 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcc41a71f0 con 0x7efcc4103ff0 2024-09-12T01:28:11.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.428+0000 7efcca000640 1 -- 172.21.15.1:0/1677354237 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcc41043d0 con 0x7efcc4103ff0 2024-09-12T01:28:11.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.430+0000 7efcc0c00640 1 -- 172.21.15.1:0/1677354237 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7efcb40305e0 con 0x7efcc4103ff0 2024-09-12T01:28:11.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.431+0000 7efcc0c00640 1 --2- 172.21.15.1:0/1677354237 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7efca80a2c10 0x7efca80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:11.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.431+0000 7efcc0c00640 1 -- 172.21.15.1:0/1677354237 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9092+0+0 (secure 0 0 0) 0x7efcb40ef500 con 0x7efcc4103ff0 2024-09-12T01:28:11.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.431+0000 7efcbbe00640 1 --2- 172.21.15.1:0/1677354237 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7efca80a2c10 0x7efca80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:11.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.432+0000 7efcbbe00640 1 --2- 172.21.15.1:0/1677354237 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7efca80a2c10 0x7efca80a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efcc40fff40 tx=0x7efcac009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:11.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.435+0000 7efcc0c00640 1 -- 172.21.15.1:0/1677354237 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efcb40bc050 con 0x7efcc4103ff0 2024-09-12T01:28:11.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.623+0000 7efcca000640 1 -- 172.21.15.1:0/1677354237 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efcc410c300 con 0x7efcc4103ff0 2024-09-12T01:28:11.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.624+0000 7efcc0c00640 1 -- 172.21.15.1:0/1677354237 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24657 (secure 0 0 0) 0x7efcb40bb950 con 0x7efcc4103ff0 2024-09-12T01:28:11.624 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:28:11.624 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":96,"fsid":"c0e4f180-70a4-11ef-bcea-c7b262605968","created":"2024-09-12T01:18:19.337406+0000","modified":"2024-09-12T01:27:16.608027+0000","last_up_change":"2024-09-12T01:25:50.601477+0000","last_in_change":"2024-09-12T01:25:27.674600+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:21:41.204242+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:26:43.066475+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:26:46.112771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T01:26:56.040130+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b9a396ae-bf97-462f-9ef9-b10ef6b9d337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6803","nonce":1125335136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6805","nonce":1125335136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6809","nonce":1125335136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6807","nonce":1125335136}]},"public_addr":"172.21.15.1:6803/1125335136","cluster_addr":"172.21.15.1:6805/1125335136","heartbeat_back_addr":"172.21.15.1:6809/1125335136","heartbeat_front_addr":"172.21.15.1:6807/1125335136","state":["exists","up"]},{"osd":1,"uuid":"dfe7a8a3-3700-4789-ac64-88660de9e0d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6801","nonce":3131049165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6803","nonce":3131049165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6807","nonce":3131049165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6805","nonce":3131049165}]},"public_addr":"172.21.15.53:6801/3131049165","cluster_addr":"172.21.15.53:6803/3131049165","heartbeat_back_addr":"172.21.15.53:6807/3131049165","heartbeat_front_addr":"172.21.15.53:6805/3131049165","state":["exists","up"]},{"osd":2,"uuid":"5d9d3603-ba7c-4115-9267-12abcb20d55b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6801","nonce":1133285636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6803","nonce":1133285636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6807","nonce":1133285636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6805","nonce":1133285636}]},"public_addr":"172.21.15.104:6801/1133285636","cluster_addr":"172.21.15.104:6803/1133285636","heartbeat_back_addr":"172.21.15.104:6807/1133285636","heartbeat_front_addr":"172.21.15.104:6805/1133285636","state":["exists","up"]},{"osd":3,"uuid":"1e063c66-41b4-4db1-a964-28e122ce628e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6811","nonce":1247468803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6813","nonce":1247468803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6817","nonce":1247468803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6815","nonce":1247468803}]},"public_addr":"172.21.15.1:6811/1247468803","cluster_addr":"172.21.15.1:6813/1247468803","heartbeat_back_addr":"172.21.15.1:6817/1247468803","heartbeat_front_addr":"172.21.15.1:6815/1247468803","state":["exists","up"]},{"osd":4,"uuid":"fe89ece0-2171-45d5-a06d-e400f77233e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6809","nonce":2673413639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6811","nonce":2673413639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6815","nonce":2673413639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6813","nonce":2673413639}]},"public_addr":"172.21.15.53:6809/2673413639","cluster_addr":"172.21.15.53:6811/2673413639","heartbeat_back_addr":"172.21.15.53:6815/2673413639","heartbeat_front_addr":"172.21.15.53:6813/2673413639","state":["exists","up"]},{"osd":5,"uuid":"daab795c-9942-4d16-aa34-8bba30a4b99b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6809","nonce":3131498790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6811","nonce":3131498790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6815","nonce":3131498790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6813","nonce":3131498790}]},"public_addr":"172.21.15.104:6809/3131498790","cluster_addr":"172.21.15.104:6811/3131498790","heartbeat_back_addr":"172.21.15.104:6815/3131498790","heartbeat_front_addr":"172.21.15.104:6813/3131498790","state":["exists","up"]},{"osd":6,"uuid":"f31d8633-e435-499b-9d46-0aa11a2481cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6819","nonce":3413067918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6821","nonce":3413067918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6825","nonce":3413067918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6823","nonce":3413067918}]},"public_addr":"172.21.15.1:6819/3413067918","cluster_addr":"172.21.15.1:6821/3413067918","heartbeat_back_addr":"172.21.15.1:6825/3413067918","heartbeat_front_addr":"172.21.15.1:6823/3413067918","state":["exists","up"]},{"osd":7,"uuid":"e64da49e-d31e-4801-b6cc-634a233b4761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6817","nonce":2841484136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6819","nonce":2841484136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6823","nonce":2841484136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6821","nonce":2841484136}]},"public_addr":"172.21.15.53:6817/2841484136","cluster_addr":"172.21.15.53:6819/2841484136","heartbeat_back_addr":"172.21.15.53:6823/2841484136","heartbeat_front_addr":"172.21.15.53:6821/2841484136","state":["exists","up"]},{"osd":8,"uuid":"fdfb5a42-dca7-4874-b479-176b4a191e21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6817","nonce":941453547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6819","nonce":941453547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6823","nonce":941453547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6821","nonce":941453547}]},"public_addr":"172.21.15.104:6817/941453547","cluster_addr":"172.21.15.104:6819/941453547","heartbeat_back_addr":"172.21.15.104:6823/941453547","heartbeat_front_addr":"172.21.15.104:6821/941453547","state":["exists","up"]},{"osd":9,"uuid":"7f81537a-cae5-4bb8-aec2-4eb07deb7958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6827","nonce":958287586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6829","nonce":958287586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6833","nonce":958287586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6831","nonce":958287586}]},"public_addr":"172.21.15.1:6827/958287586","cluster_addr":"172.21.15.1:6829/958287586","heartbeat_back_addr":"172.21.15.1:6833/958287586","heartbeat_front_addr":"172.21.15.1:6831/958287586","state":["exists","up"]},{"osd":10,"uuid":"e3c9c37d-46b9-4585-990d-69c447453bd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6825","nonce":3357909498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6827","nonce":3357909498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6831","nonce":3357909498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6829","nonce":3357909498}]},"public_addr":"172.21.15.53:6825/3357909498","cluster_addr":"172.21.15.53:6827/3357909498","heartbeat_back_addr":"172.21.15.53:6831/3357909498","heartbeat_front_addr":"172.21.15.53:6829/3357909498","state":["exists","up"]},{"osd":11,"uuid":"474b5345-f2ae-4e0e-a197-53ab11f58c04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6825","nonce":3381427987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6827","nonce":3381427987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6831","nonce":3381427987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6829","nonce":3381427987}]},"public_addr":"172.21.15.104:6825/3381427987","cluster_addr":"172.21.15.104:6827/3381427987","heartbeat_back_addr":"172.21.15.104:6831/3381427987","heartbeat_front_addr":"172.21.15.104:6829/3381427987","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:20:46.903181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:21:12.735556+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:21:37.868739+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:22:09.141228+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:22:36.438563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:23:02.220948+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:23:33.505881+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:24:00.982176+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:24:26.010444+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:24:56.711121+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:25:23.846495+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:25:48.815427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1373984726":"2024-09-13T01:19:14.580637+0000","172.21.15.1:0/1429963718":"2024-09-13T01:19:14.580637+0000","172.21.15.1:6801/4101281615":"2024-09-13T01:19:14.580637+0000","172.21.15.1:6800/4101281615":"2024-09-13T01:19:14.580637+0000","172.21.15.1:0/2564759586":"2024-09-13T01:19:14.580637+0000","172.21.15.1:6801/2935968453":"2024-09-13T01:18:45.882682+0000","172.21.15.1:6800/2935968453":"2024-09-13T01:18:45.882682+0000","172.21.15.1:0/1459848813":"2024-09-13T01:18:45.882682+0000","172.21.15.1:0/578605922":"2024-09-13T01:18:45.882682+0000","172.21.15.1:0/738035035":"2024-09-13T01:18:45.882682+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:28:11.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.627+0000 7efcca000640 1 -- 172.21.15.1:0/1677354237 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7efca80a2c10 msgr2=0x7efca80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:11.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.627+0000 7efcca000640 1 --2- 172.21.15.1:0/1677354237 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7efca80a2c10 0x7efca80a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efcc40fff40 tx=0x7efcac009260 comp rx=0 tx=0).stop 2024-09-12T01:28:11.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.627+0000 7efcca000640 1 -- 172.21.15.1:0/1677354237 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcc4103ff0 msgr2=0x7efcc419a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:11.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.627+0000 7efcca000640 1 --2- 172.21.15.1:0/1677354237 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcc4103ff0 0x7efcc419a670 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efcb4035040 tx=0x7efcb4033b60 comp rx=0 tx=0).stop 2024-09-12T01:28:11.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.627+0000 7efcca000640 1 -- 172.21.15.1:0/1677354237 shutdown_connections 2024-09-12T01:28:11.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.627+0000 7efcca000640 1 --2- 172.21.15.1:0/1677354237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcc4109120 0x7efcc419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:11.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.627+0000 7efcca000640 1 --2- 172.21.15.1:0/1677354237 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7efca80a2c10 0x7efca80a50d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:11.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.627+0000 7efcca000640 1 --2- 172.21.15.1:0/1677354237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efcc41049a0 0x7efcc419abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:11.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.627+0000 7efcca000640 1 --2- 172.21.15.1:0/1677354237 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcc4103ff0 0x7efcc419a670 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:11.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.627+0000 7efcca000640 1 -- 172.21.15.1:0/1677354237 >> 172.21.15.1:0/1677354237 conn(0x7efcc40ffa90 msgr2=0x7efcc4100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:11.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.627+0000 7efcca000640 1 -- 172.21.15.1:0/1677354237 shutdown_connections 2024-09-12T01:28:11.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:11.627+0000 7efcca000640 1 -- 172.21.15.1:0/1677354237 wait complete. 2024-09-12T01:28:12.434 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-12T01:28:12.435 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T01:28:12.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:12 smithi001 ceph-mon[30387]: pgmap v354: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-12T01:28:12.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:12 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/1677354237' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:28:12.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:12 smithi053 ceph-mon[35158]: pgmap v354: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-12T01:28:12.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:12 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/1677354237' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:28:12.815 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:28:12.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:12 smithi104 ceph-mon[35219]: pgmap v354: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-12T01:28:12.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:12 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/1677354237' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:28:13.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.397+0000 7f1f9b600640 1 -- 172.21.15.1:0/2220830856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f94103ff0 msgr2=0x7f1f941043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:13.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.397+0000 7f1f9b600640 1 --2- 172.21.15.1:0/2220830856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f94103ff0 0x7f1f941043d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1f88009900 tx=0x7f1f8802a410 comp rx=0 tx=0).stop 2024-09-12T01:28:13.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.397+0000 7f1f9b600640 1 -- 172.21.15.1:0/2220830856 shutdown_connections 2024-09-12T01:28:13.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.397+0000 7f1f9b600640 1 --2- 172.21.15.1:0/2220830856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f94109120 0x7f1f94110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:13.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.397+0000 7f1f9b600640 1 --2- 172.21.15.1:0/2220830856 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f941049a0 0x7f1f941089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:13.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.397+0000 7f1f9b600640 1 --2- 172.21.15.1:0/2220830856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f94103ff0 0x7f1f941043d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:13.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.397+0000 7f1f9b600640 1 -- 172.21.15.1:0/2220830856 >> 172.21.15.1:0/2220830856 conn(0x7f1f940ffa90 msgr2=0x7f1f94101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:13.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.397+0000 7f1f9b600640 1 -- 172.21.15.1:0/2220830856 shutdown_connections 2024-09-12T01:28:13.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.397+0000 7f1f9b600640 1 -- 172.21.15.1:0/2220830856 wait complete. 2024-09-12T01:28:13.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.398+0000 7f1f9b600640 1 Processor -- start 2024-09-12T01:28:13.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.398+0000 7f1f9b600640 1 -- start start 2024-09-12T01:28:13.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.399+0000 7f1f9b600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f94103ff0 0x7f1f9419a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:13.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.399+0000 7f1f9b600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f941049a0 0x7f1f9419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:13.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.399+0000 7f1f9b600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f94109120 0x7f1f9419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:13.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.399+0000 7f1f9b600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1f94114140 con 0x7f1f94103ff0 2024-09-12T01:28:13.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.399+0000 7f1f9b600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1f94113fc0 con 0x7f1f941049a0 2024-09-12T01:28:13.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.399+0000 7f1f9b600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1f941142c0 con 0x7f1f94109120 2024-09-12T01:28:13.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.399+0000 7f1f98c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f94103ff0 0x7f1f9419a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:13.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.399+0000 7f1f98c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f94103ff0 0x7f1f9419a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51278/0 (socket says 172.21.15.1:51278) 2024-09-12T01:28:13.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.399+0000 7f1f93e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f941049a0 0x7f1f9419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:13.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.399+0000 7f1f98c00640 1 -- 172.21.15.1:0/476173306 learned_addr learned my addr 172.21.15.1:0/476173306 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:28:13.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.399+0000 7f1f99600640 1 --2- 172.21.15.1:0/476173306 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f94109120 0x7f1f9419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:13.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.400+0000 7f1f93e00640 1 -- 172.21.15.1:0/476173306 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f94109120 msgr2=0x7f1f9419eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:13.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.400+0000 7f1f93e00640 1 --2- 172.21.15.1:0/476173306 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f94109120 0x7f1f9419eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:13.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.400+0000 7f1f93e00640 1 -- 172.21.15.1:0/476173306 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f94103ff0 msgr2=0x7f1f9419a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:13.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.400+0000 7f1f93e00640 1 --2- 172.21.15.1:0/476173306 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f94103ff0 0x7f1f9419a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:13.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.400+0000 7f1f93e00640 1 -- 172.21.15.1:0/476173306 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f9419f5b0 con 0x7f1f941049a0 2024-09-12T01:28:13.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.400+0000 7f1f93e00640 1 --2- 172.21.15.1:0/476173306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f941049a0 0x7f1f9419ab40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1f8000e9f0 tx=0x7f1f8000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:13.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.400+0000 7f1f91600640 1 -- 172.21.15.1:0/476173306 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8000ccc0 con 0x7f1f941049a0 2024-09-12T01:28:13.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.400+0000 7f1f91600640 1 -- 172.21.15.1:0/476173306 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f80005020 con 0x7f1f941049a0 2024-09-12T01:28:13.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.400+0000 7f1f9b600640 1 -- 172.21.15.1:0/476173306 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f9419f8a0 con 0x7f1f941049a0 2024-09-12T01:28:13.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.400+0000 7f1f91600640 1 -- 172.21.15.1:0/476173306 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f800049e0 con 0x7f1f941049a0 2024-09-12T01:28:13.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.401+0000 7f1f9b600640 1 -- 172.21.15.1:0/476173306 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f941a7180 con 0x7f1f941049a0 2024-09-12T01:28:13.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.402+0000 7f1f9b600640 1 -- 172.21.15.1:0/476173306 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f941043d0 con 0x7f1f941049a0 2024-09-12T01:28:13.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.403+0000 7f1f91600640 1 -- 172.21.15.1:0/476173306 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f1f80005560 con 0x7f1f941049a0 2024-09-12T01:28:13.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.405+0000 7f1f91600640 1 --2- 172.21.15.1:0/476173306 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1f7c0a2c10 0x7f1f7c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:13.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.405+0000 7f1f91600640 1 -- 172.21.15.1:0/476173306 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9092+0+0 (secure 0 0 0) 0x7f1f800cf500 con 0x7f1f941049a0 2024-09-12T01:28:13.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.405+0000 7f1f98c00640 1 --2- 172.21.15.1:0/476173306 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1f7c0a2c10 0x7f1f7c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:13.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.406+0000 7f1f98c00640 1 --2- 172.21.15.1:0/476173306 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1f7c0a2c10 0x7f1f7c0a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1f880096c0 tx=0x7f1f88005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:13.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.409+0000 7f1f91600640 1 -- 172.21.15.1:0/476173306 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1f800051d0 con 0x7f1f941049a0 2024-09-12T01:28:13.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.596+0000 7f1f9b600640 1 -- 172.21.15.1:0/476173306 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1f9402d7c0 con 0x7f1f941049a0 2024-09-12T01:28:13.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.598+0000 7f1f91600640 1 -- 172.21.15.1:0/476173306 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24657 (secure 0 0 0) 0x7f1f8009bf40 con 0x7f1f941049a0 2024-09-12T01:28:13.598 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:28:13.599 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":96,"fsid":"c0e4f180-70a4-11ef-bcea-c7b262605968","created":"2024-09-12T01:18:19.337406+0000","modified":"2024-09-12T01:27:16.608027+0000","last_up_change":"2024-09-12T01:25:50.601477+0000","last_in_change":"2024-09-12T01:25:27.674600+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:21:41.204242+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:26:43.066475+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:26:46.112771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T01:26:56.040130+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b9a396ae-bf97-462f-9ef9-b10ef6b9d337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6803","nonce":1125335136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6805","nonce":1125335136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6809","nonce":1125335136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6807","nonce":1125335136}]},"public_addr":"172.21.15.1:6803/1125335136","cluster_addr":"172.21.15.1:6805/1125335136","heartbeat_back_addr":"172.21.15.1:6809/1125335136","heartbeat_front_addr":"172.21.15.1:6807/1125335136","state":["exists","up"]},{"osd":1,"uuid":"dfe7a8a3-3700-4789-ac64-88660de9e0d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6801","nonce":3131049165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6803","nonce":3131049165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6807","nonce":3131049165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6805","nonce":3131049165}]},"public_addr":"172.21.15.53:6801/3131049165","cluster_addr":"172.21.15.53:6803/3131049165","heartbeat_back_addr":"172.21.15.53:6807/3131049165","heartbeat_front_addr":"172.21.15.53:6805/3131049165","state":["exists","up"]},{"osd":2,"uuid":"5d9d3603-ba7c-4115-9267-12abcb20d55b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6801","nonce":1133285636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6803","nonce":1133285636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6807","nonce":1133285636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6805","nonce":1133285636}]},"public_addr":"172.21.15.104:6801/1133285636","cluster_addr":"172.21.15.104:6803/1133285636","heartbeat_back_addr":"172.21.15.104:6807/1133285636","heartbeat_front_addr":"172.21.15.104:6805/1133285636","state":["exists","up"]},{"osd":3,"uuid":"1e063c66-41b4-4db1-a964-28e122ce628e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6811","nonce":1247468803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6813","nonce":1247468803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6817","nonce":1247468803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6815","nonce":1247468803}]},"public_addr":"172.21.15.1:6811/1247468803","cluster_addr":"172.21.15.1:6813/1247468803","heartbeat_back_addr":"172.21.15.1:6817/1247468803","heartbeat_front_addr":"172.21.15.1:6815/1247468803","state":["exists","up"]},{"osd":4,"uuid":"fe89ece0-2171-45d5-a06d-e400f77233e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6809","nonce":2673413639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6811","nonce":2673413639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6815","nonce":2673413639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6813","nonce":2673413639}]},"public_addr":"172.21.15.53:6809/2673413639","cluster_addr":"172.21.15.53:6811/2673413639","heartbeat_back_addr":"172.21.15.53:6815/2673413639","heartbeat_front_addr":"172.21.15.53:6813/2673413639","state":["exists","up"]},{"osd":5,"uuid":"daab795c-9942-4d16-aa34-8bba30a4b99b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6809","nonce":3131498790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6811","nonce":3131498790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6815","nonce":3131498790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6813","nonce":3131498790}]},"public_addr":"172.21.15.104:6809/3131498790","cluster_addr":"172.21.15.104:6811/3131498790","heartbeat_back_addr":"172.21.15.104:6815/3131498790","heartbeat_front_addr":"172.21.15.104:6813/3131498790","state":["exists","up"]},{"osd":6,"uuid":"f31d8633-e435-499b-9d46-0aa11a2481cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6819","nonce":3413067918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6821","nonce":3413067918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6825","nonce":3413067918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6823","nonce":3413067918}]},"public_addr":"172.21.15.1:6819/3413067918","cluster_addr":"172.21.15.1:6821/3413067918","heartbeat_back_addr":"172.21.15.1:6825/3413067918","heartbeat_front_addr":"172.21.15.1:6823/3413067918","state":["exists","up"]},{"osd":7,"uuid":"e64da49e-d31e-4801-b6cc-634a233b4761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6817","nonce":2841484136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6819","nonce":2841484136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6823","nonce":2841484136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6821","nonce":2841484136}]},"public_addr":"172.21.15.53:6817/2841484136","cluster_addr":"172.21.15.53:6819/2841484136","heartbeat_back_addr":"172.21.15.53:6823/2841484136","heartbeat_front_addr":"172.21.15.53:6821/2841484136","state":["exists","up"]},{"osd":8,"uuid":"fdfb5a42-dca7-4874-b479-176b4a191e21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6817","nonce":941453547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6819","nonce":941453547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6823","nonce":941453547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6821","nonce":941453547}]},"public_addr":"172.21.15.104:6817/941453547","cluster_addr":"172.21.15.104:6819/941453547","heartbeat_back_addr":"172.21.15.104:6823/941453547","heartbeat_front_addr":"172.21.15.104:6821/941453547","state":["exists","up"]},{"osd":9,"uuid":"7f81537a-cae5-4bb8-aec2-4eb07deb7958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6827","nonce":958287586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6829","nonce":958287586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6833","nonce":958287586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6831","nonce":958287586}]},"public_addr":"172.21.15.1:6827/958287586","cluster_addr":"172.21.15.1:6829/958287586","heartbeat_back_addr":"172.21.15.1:6833/958287586","heartbeat_front_addr":"172.21.15.1:6831/958287586","state":["exists","up"]},{"osd":10,"uuid":"e3c9c37d-46b9-4585-990d-69c447453bd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6825","nonce":3357909498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6827","nonce":3357909498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6831","nonce":3357909498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6829","nonce":3357909498}]},"public_addr":"172.21.15.53:6825/3357909498","cluster_addr":"172.21.15.53:6827/3357909498","heartbeat_back_addr":"172.21.15.53:6831/3357909498","heartbeat_front_addr":"172.21.15.53:6829/3357909498","state":["exists","up"]},{"osd":11,"uuid":"474b5345-f2ae-4e0e-a197-53ab11f58c04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6825","nonce":3381427987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6827","nonce":3381427987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6831","nonce":3381427987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6829","nonce":3381427987}]},"public_addr":"172.21.15.104:6825/3381427987","cluster_addr":"172.21.15.104:6827/3381427987","heartbeat_back_addr":"172.21.15.104:6831/3381427987","heartbeat_front_addr":"172.21.15.104:6829/3381427987","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:20:46.903181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:21:12.735556+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:21:37.868739+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:22:09.141228+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:22:36.438563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:23:02.220948+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:23:33.505881+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:24:00.982176+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:24:26.010444+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:24:56.711121+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:25:23.846495+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:25:48.815427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1373984726":"2024-09-13T01:19:14.580637+0000","172.21.15.1:0/1429963718":"2024-09-13T01:19:14.580637+0000","172.21.15.1:6801/4101281615":"2024-09-13T01:19:14.580637+0000","172.21.15.1:6800/4101281615":"2024-09-13T01:19:14.580637+0000","172.21.15.1:0/2564759586":"2024-09-13T01:19:14.580637+0000","172.21.15.1:6801/2935968453":"2024-09-13T01:18:45.882682+0000","172.21.15.1:6800/2935968453":"2024-09-13T01:18:45.882682+0000","172.21.15.1:0/1459848813":"2024-09-13T01:18:45.882682+0000","172.21.15.1:0/578605922":"2024-09-13T01:18:45.882682+0000","172.21.15.1:0/738035035":"2024-09-13T01:18:45.882682+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:28:13.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.601+0000 7f1f9b600640 1 -- 172.21.15.1:0/476173306 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1f7c0a2c10 msgr2=0x7f1f7c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:13.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.601+0000 7f1f9b600640 1 --2- 172.21.15.1:0/476173306 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1f7c0a2c10 0x7f1f7c0a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1f880096c0 tx=0x7f1f88005a20 comp rx=0 tx=0).stop 2024-09-12T01:28:13.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.601+0000 7f1f9b600640 1 -- 172.21.15.1:0/476173306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f941049a0 msgr2=0x7f1f9419ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:13.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.601+0000 7f1f9b600640 1 --2- 172.21.15.1:0/476173306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f941049a0 0x7f1f9419ab40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1f8000e9f0 tx=0x7f1f8000eeb0 comp rx=0 tx=0).stop 2024-09-12T01:28:13.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.601+0000 7f1f9b600640 1 -- 172.21.15.1:0/476173306 shutdown_connections 2024-09-12T01:28:13.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.601+0000 7f1f9b600640 1 --2- 172.21.15.1:0/476173306 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1f7c0a2c10 0x7f1f7c0a50d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:13.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.601+0000 7f1f9b600640 1 --2- 172.21.15.1:0/476173306 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f94109120 0x7f1f9419eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:13.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.601+0000 7f1f9b600640 1 --2- 172.21.15.1:0/476173306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f941049a0 0x7f1f9419ab40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:13.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.601+0000 7f1f9b600640 1 --2- 172.21.15.1:0/476173306 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f94103ff0 0x7f1f9419a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:13.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.601+0000 7f1f9b600640 1 -- 172.21.15.1:0/476173306 >> 172.21.15.1:0/476173306 conn(0x7f1f940ffa90 msgr2=0x7f1f94100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:13.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.601+0000 7f1f9b600640 1 -- 172.21.15.1:0/476173306 shutdown_connections 2024-09-12T01:28:13.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:13.601+0000 7f1f9b600640 1 -- 172.21.15.1:0/476173306 wait complete. 2024-09-12T01:28:14.559 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-12T01:28:14.560 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-12T01:28:14.560 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-12T01:28:14.561 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-12T01:28:14.561 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-12T01:28:14.561 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-12T01:28:14.562 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-12T01:28:14.562 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-12T01:28:14.563 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-12T01:28:14.564 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-12T01:28:14.564 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-12T01:28:14.565 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-12T01:28:14.658 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:14 smithi001 ceph-mon[30387]: pgmap v355: 129 pgs: 129 active+clean; 646 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-12T01:28:14.659 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:14 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/476173306' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:28:14.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:14 smithi053 ceph-mon[35158]: pgmap v355: 129 pgs: 129 active+clean; 646 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-12T01:28:14.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:14 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/476173306' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:28:14.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:14 smithi104 ceph-mon[35219]: pgmap v355: 129 pgs: 129 active+clean; 646 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-12T01:28:14.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:14 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/476173306' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:28:15.811 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:28:15.825 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:28:15.828 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:28:15.829 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:28:15.831 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:28:15.839 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:28:15.839 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:28:15.841 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:28:15.849 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:28:15.852 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:28:15.858 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:28:15.862 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:28:16.611 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:16 smithi001 ceph-mon[30387]: pgmap v356: 129 pgs: 129 active+clean; 646 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-12T01:28:16.611 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:16 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]: dispatch 2024-09-12T01:28:16.611 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:16 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-09-12T01:28:16.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:16 smithi053 ceph-mon[35158]: pgmap v356: 129 pgs: 129 active+clean; 646 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-12T01:28:16.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:16 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]: dispatch 2024-09-12T01:28:16.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:16 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-09-12T01:28:16.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:16 smithi104 ceph-mon[35219]: pgmap v356: 129 pgs: 129 active+clean; 646 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-12T01:28:16.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:16 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]: dispatch 2024-09-12T01:28:16.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:16 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-09-12T01:28:17.552 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:17 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]': finished 2024-09-12T01:28:17.552 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:17 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-09-12T01:28:17.552 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:17 smithi001 ceph-mon[30387]: osdmap e97: 12 total, 12 up, 12 in 2024-09-12T01:28:17.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:17 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]': finished 2024-09-12T01:28:17.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:17 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-09-12T01:28:17.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:17 smithi053 ceph-mon[35158]: osdmap e97: 12 total, 12 up, 12 in 2024-09-12T01:28:17.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:17 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]': finished 2024-09-12T01:28:17.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:17 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-09-12T01:28:17.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:17 smithi104 ceph-mon[35219]: osdmap e97: 12 total, 12 up, 12 in 2024-09-12T01:28:18.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.425+0000 7f0593c00640 1 -- 172.21.15.1:0/1324064171 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f058c1049a0 msgr2=0x7f058c1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:18.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.425+0000 7f0593c00640 1 --2- 172.21.15.1:0/1324064171 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f058c1049a0 0x7f058c1089f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0578009860 tx=0x7f057802a340 comp rx=0 tx=0).stop 2024-09-12T01:28:18.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.425+0000 7f0593c00640 1 -- 172.21.15.1:0/1324064171 shutdown_connections 2024-09-12T01:28:18.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.425+0000 7f0593c00640 1 --2- 172.21.15.1:0/1324064171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f058c109120 0x7f058c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:18.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.425+0000 7f0593c00640 1 --2- 172.21.15.1:0/1324064171 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f058c1049a0 0x7f058c1089f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:18.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.425+0000 7f0593c00640 1 --2- 172.21.15.1:0/1324064171 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f058c103ff0 0x7f058c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:18.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.425+0000 7f0593c00640 1 -- 172.21.15.1:0/1324064171 >> 172.21.15.1:0/1324064171 conn(0x7f058c0ffa90 msgr2=0x7f058c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:18.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.426+0000 7f0593c00640 1 -- 172.21.15.1:0/1324064171 shutdown_connections 2024-09-12T01:28:18.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.426+0000 7f0593c00640 1 -- 172.21.15.1:0/1324064171 wait complete. 2024-09-12T01:28:18.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.426+0000 7f0593c00640 1 Processor -- start 2024-09-12T01:28:18.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.427+0000 7f0593c00640 1 -- start start 2024-09-12T01:28:18.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.427+0000 7f0593c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f058c103ff0 0x7f058c19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:18.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.427+0000 7f0593c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f058c1049a0 0x7f058c19abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:18.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.427+0000 7f0593c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f058c109120 0x7f058c19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:18.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.427+0000 7f0593c00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f058c114140 con 0x7f058c1049a0 2024-09-12T01:28:18.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.427+0000 7f0593c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f058c113fc0 con 0x7f058c103ff0 2024-09-12T01:28:18.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.427+0000 7f0593c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f058c1142c0 con 0x7f058c109120 2024-09-12T01:28:18.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.427+0000 7f058be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f058c1049a0 0x7f058c19abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:18.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.427+0000 7f058be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f058c1049a0 0x7f058c19abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51316/0 (socket says 172.21.15.1:51316) 2024-09-12T01:28:18.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.427+0000 7f058be00640 1 -- 172.21.15.1:0/2627867212 learned_addr learned my addr 172.21.15.1:0/2627867212 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:28:18.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.427+0000 7f0591200640 1 --2- 172.21.15.1:0/2627867212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f058c103ff0 0x7f058c19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:18.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.427+0000 7f0591c00640 1 --2- 172.21.15.1:0/2627867212 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f058c109120 0x7f058c19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:18.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.428+0000 7f058be00640 1 -- 172.21.15.1:0/2627867212 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f058c109120 msgr2=0x7f058c19ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:18.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.428+0000 7f058be00640 1 --2- 172.21.15.1:0/2627867212 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f058c109120 0x7f058c19ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:18.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.428+0000 7f058be00640 1 -- 172.21.15.1:0/2627867212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f058c103ff0 msgr2=0x7f058c19a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:18.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.428+0000 7f058be00640 1 --2- 172.21.15.1:0/2627867212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f058c103ff0 0x7f058c19a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:18.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.428+0000 7f058be00640 1 -- 172.21.15.1:0/2627867212 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f058c19f650 con 0x7f058c1049a0 2024-09-12T01:28:18.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.428+0000 7f058be00640 1 --2- 172.21.15.1:0/2627867212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f058c1049a0 0x7f058c19abe0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0578035040 tx=0x7f0578033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:18.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.428+0000 7f0589600640 1 -- 172.21.15.1:0/2627867212 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057803f020 con 0x7f058c1049a0 2024-09-12T01:28:18.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.428+0000 7f0593c00640 1 -- 172.21.15.1:0/2627867212 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f058c19f8e0 con 0x7f058c1049a0 2024-09-12T01:28:18.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.428+0000 7f0589600640 1 -- 172.21.15.1:0/2627867212 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0578032770 con 0x7f058c1049a0 2024-09-12T01:28:18.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.428+0000 7f0589600640 1 -- 172.21.15.1:0/2627867212 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0578037040 con 0x7f058c1049a0 2024-09-12T01:28:18.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.429+0000 7f0593c00640 1 -- 172.21.15.1:0/2627867212 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f058c1a7220 con 0x7f058c1049a0 2024-09-12T01:28:18.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.429+0000 7f0593c00640 1 -- 172.21.15.1:0/2627867212 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f058c062900 con 0x7f058c1049a0 2024-09-12T01:28:18.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.431+0000 7f0589600640 1 -- 172.21.15.1:0/2627867212 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f0578042020 con 0x7f058c1049a0 2024-09-12T01:28:18.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.433+0000 7f0589600640 1 --2- 172.21.15.1:0/2627867212 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f05740a2c10 0x7f05740a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:18.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.433+0000 7f0589600640 1 -- 172.21.15.1:0/2627867212 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f05780efb90 con 0x7f058c1049a0 2024-09-12T01:28:18.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.433+0000 7f0591200640 1 --2- 172.21.15.1:0/2627867212 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f05740a2c10 0x7f05740a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:18.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.433+0000 7f0589600640 1 --2- 172.21.15.1:0/2627867212 >> [v2:172.21.15.1:6818/3413067918,v1:172.21.15.1:6819/3413067918] conn(0x7f05740adee0 0x7f05740b0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:18.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.433+0000 7f0589600640 1 -- 172.21.15.1:0/2627867212 --> [v2:172.21.15.1:6818/3413067918,v1:172.21.15.1:6819/3413067918] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0578007e80 con 0x7f05740adee0 2024-09-12T01:28:18.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.433+0000 7f0589600640 1 -- 172.21.15.1:0/2627867212 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7f05780eff40 con 0x7f058c1049a0 2024-09-12T01:28:18.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.433+0000 7f0591c00640 1 --2- 172.21.15.1:0/2627867212 >> [v2:172.21.15.1:6818/3413067918,v1:172.21.15.1:6819/3413067918] conn(0x7f05740adee0 0x7f05740b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:18.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.434+0000 7f0591200640 1 --2- 172.21.15.1:0/2627867212 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f05740a2c10 0x7f05740a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0580009530 tx=0x7f0580009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:18.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.434+0000 7f0591c00640 1 --2- 172.21.15.1:0/2627867212 >> [v2:172.21.15.1:6818/3413067918,v1:172.21.15.1:6819/3413067918] conn(0x7f05740adee0 0x7f05740b0380 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:18.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.435+0000 7f0589600640 1 -- 172.21.15.1:0/2627867212 <== osd.6 v2:172.21.15.1:6818/3413067918 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0578007e80 con 0x7f05740adee0 2024-09-12T01:28:18.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.449+0000 7f0593c00640 1 -- 172.21.15.1:0/2627867212 --> [v2:172.21.15.1:6818/3413067918,v1:172.21.15.1:6819/3413067918] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f058c062d00 con 0x7f05740adee0 2024-09-12T01:28:18.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.450+0000 7f0589600640 1 -- 172.21.15.1:0/2627867212 <== osd.6 v2:172.21.15.1:6818/3413067918 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f058c062d00 con 0x7f05740adee0 2024-09-12T01:28:18.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.450+0000 7f0593c00640 1 -- 172.21.15.1:0/2627867212 >> [v2:172.21.15.1:6818/3413067918,v1:172.21.15.1:6819/3413067918] conn(0x7f05740adee0 msgr2=0x7f05740b0380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:18.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.450+0000 7f0593c00640 1 --2- 172.21.15.1:0/2627867212 >> [v2:172.21.15.1:6818/3413067918,v1:172.21.15.1:6819/3413067918] conn(0x7f05740adee0 0x7f05740b0380 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:18.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.450+0000 7f0593c00640 1 -- 172.21.15.1:0/2627867212 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f05740a2c10 msgr2=0x7f05740a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:18.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.450+0000 7f0593c00640 1 --2- 172.21.15.1:0/2627867212 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f05740a2c10 0x7f05740a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0580009530 tx=0x7f0580009260 comp rx=0 tx=0).stop 2024-09-12T01:28:18.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.450+0000 7f0593c00640 1 -- 172.21.15.1:0/2627867212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f058c1049a0 msgr2=0x7f058c19abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:18.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.450+0000 7f0593c00640 1 --2- 172.21.15.1:0/2627867212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f058c1049a0 0x7f058c19abe0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0578035040 tx=0x7f0578033d90 comp rx=0 tx=0).stop 2024-09-12T01:28:18.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.450+0000 7f0591c00640 1 -- 172.21.15.1:0/2627867212 reap_dead start 2024-09-12T01:28:18.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.450+0000 7f0593c00640 1 -- 172.21.15.1:0/2627867212 shutdown_connections 2024-09-12T01:28:18.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.450+0000 7f0593c00640 1 -- 172.21.15.1:0/2627867212 >> 172.21.15.1:0/2627867212 conn(0x7f058c0ffa90 msgr2=0x7f058c100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:18.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.451+0000 7f0593c00640 1 -- 172.21.15.1:0/2627867212 shutdown_connections 2024-09-12T01:28:18.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.451+0000 7f0593c00640 1 -- 172.21.15.1:0/2627867212 wait complete. 2024-09-12T01:28:18.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:18 smithi001 ceph-mon[30387]: pgmap v358: 129 pgs: 129 active+clean; 646 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-12T01:28:18.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:18 smithi001 ceph-mon[30387]: osdmap e98: 12 total, 12 up, 12 in 2024-09-12T01:28:18.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:18 smithi053 ceph-mon[35158]: pgmap v358: 129 pgs: 129 active+clean; 646 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-12T01:28:18.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:18 smithi053 ceph-mon[35158]: osdmap e98: 12 total, 12 up, 12 in 2024-09-12T01:28:18.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.327+0000 7faf29800640 1 -- 172.21.15.1:0/3280988123 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf24076720 msgr2=0x7faf24076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:18.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.327+0000 7faf29800640 1 --2- 172.21.15.1:0/3280988123 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf24076720 0x7faf24076b00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7faf14009900 tx=0x7faf1402a3f0 comp rx=0 tx=0).stop 2024-09-12T01:28:18.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.327+0000 7faf29800640 1 -- 172.21.15.1:0/3280988123 shutdown_connections 2024-09-12T01:28:18.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.327+0000 7faf29800640 1 --2- 172.21.15.1:0/3280988123 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf24112a30 0x7faf24114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:18.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.327+0000 7faf29800640 1 --2- 172.21.15.1:0/3280988123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf24077040 0x7faf241124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:18.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.327+0000 7faf29800640 1 --2- 172.21.15.1:0/3280988123 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf24076720 0x7faf24076b00 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:18.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.327+0000 7faf29800640 1 -- 172.21.15.1:0/3280988123 >> 172.21.15.1:0/3280988123 conn(0x7faf240ffa90 msgr2=0x7faf24101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:18.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.327+0000 7faf29800640 1 -- 172.21.15.1:0/3280988123 shutdown_connections 2024-09-12T01:28:18.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.327+0000 7faf29800640 1 -- 172.21.15.1:0/3280988123 wait complete. 2024-09-12T01:28:18.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.328+0000 7faf29800640 1 Processor -- start 2024-09-12T01:28:18.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.328+0000 7faf29800640 1 -- start start 2024-09-12T01:28:18.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.328+0000 7faf29800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf24076720 0x7faf241a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:18.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.328+0000 7faf29800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf24077040 0x7faf241a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:18.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.328+0000 7faf29800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf24112a30 0x7faf2419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:18.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.328+0000 7faf29800640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7faf241185a0 con 0x7faf24077040 2024-09-12T01:28:18.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.328+0000 7faf29800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7faf24118420 con 0x7faf24076720 2024-09-12T01:28:18.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.328+0000 7faf29800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7faf24118720 con 0x7faf24112a30 2024-09-12T01:28:18.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.329+0000 7faf22a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf24076720 0x7faf241a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:18.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.329+0000 7faf22000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf24077040 0x7faf241a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:18.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.329+0000 7faf22a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf24076720 0x7faf241a46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:43668/0 (socket says 172.21.15.1:43668) 2024-09-12T01:28:18.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.329+0000 7faf22a00640 1 -- 172.21.15.1:0/1043320077 learned_addr learned my addr 172.21.15.1:0/1043320077 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:28:18.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.329+0000 7faf23400640 1 --2- 172.21.15.1:0/1043320077 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf24112a30 0x7faf2419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:18.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.329+0000 7faf22a00640 1 -- 172.21.15.1:0/1043320077 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf24112a30 msgr2=0x7faf2419e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:18.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.329+0000 7faf22a00640 1 --2- 172.21.15.1:0/1043320077 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf24112a30 0x7faf2419e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:18.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.329+0000 7faf22a00640 1 -- 172.21.15.1:0/1043320077 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf24077040 msgr2=0x7faf241a4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:18.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.329+0000 7faf22a00640 1 --2- 172.21.15.1:0/1043320077 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf24077040 0x7faf241a4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:18.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.329+0000 7faf22a00640 1 -- 172.21.15.1:0/1043320077 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf2419efe0 con 0x7faf24076720 2024-09-12T01:28:18.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.329+0000 7faf22a00640 1 --2- 172.21.15.1:0/1043320077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf24076720 0x7faf241a46f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faf14006010 tx=0x7faf14004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:18.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.330+0000 7faf1b400640 1 -- 172.21.15.1:0/1043320077 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf1403f030 con 0x7faf24076720 2024-09-12T01:28:18.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.330+0000 7faf29800640 1 -- 172.21.15.1:0/1043320077 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf2419f270 con 0x7faf24076720 2024-09-12T01:28:18.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.330+0000 7faf1b400640 1 -- 172.21.15.1:0/1043320077 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf14004440 con 0x7faf24076720 2024-09-12T01:28:18.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.330+0000 7faf1b400640 1 -- 172.21.15.1:0/1043320077 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf14035070 con 0x7faf24076720 2024-09-12T01:28:18.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.330+0000 7faf29800640 1 -- 172.21.15.1:0/1043320077 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf241ab5c0 con 0x7faf24076720 2024-09-12T01:28:18.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.331+0000 7faf29800640 1 -- 172.21.15.1:0/1043320077 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faf24062980 con 0x7faf24076720 2024-09-12T01:28:18.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.335+0000 7faf1b400640 1 -- 172.21.15.1:0/1043320077 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7faf14042030 con 0x7faf24076720 2024-09-12T01:28:18.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.336+0000 7faf1b400640 1 --2- 172.21.15.1:0/1043320077 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7faf080a2c10 0x7faf080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:18.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.336+0000 7faf1b400640 1 -- 172.21.15.1:0/1043320077 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9150+0+0 (secure 0 0 0) 0x7faf140f0130 con 0x7faf24076720 2024-09-12T01:28:18.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.337+0000 7faf22000640 1 --2- 172.21.15.1:0/1043320077 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7faf080a2c10 0x7faf080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:18.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.337+0000 7faf1b400640 1 --2- 172.21.15.1:0/1043320077 >> [v2:172.21.15.1:6810/1247468803,v1:172.21.15.1:6811/1247468803] conn(0x7faf080adee0 0x7faf080b0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:18.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.337+0000 7faf1b400640 1 -- 172.21.15.1:0/1043320077 --> [v2:172.21.15.1:6810/1247468803,v1:172.21.15.1:6811/1247468803] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faf080b0a50 con 0x7faf080adee0 2024-09-12T01:28:18.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.337+0000 7faf1b400640 1 -- 172.21.15.1:0/1043320077 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=97) ==== 24+0+0 (secure 0 0 0) 0x7faf140f0580 con 0x7faf24076720 2024-09-12T01:28:18.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.337+0000 7faf23400640 1 --2- 172.21.15.1:0/1043320077 >> [v2:172.21.15.1:6810/1247468803,v1:172.21.15.1:6811/1247468803] conn(0x7faf080adee0 0x7faf080b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:18.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.337+0000 7faf22000640 1 --2- 172.21.15.1:0/1043320077 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7faf080a2c10 0x7faf080a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7faf24079300 tx=0x7faf10005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:18.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.338+0000 7faf23400640 1 --2- 172.21.15.1:0/1043320077 >> [v2:172.21.15.1:6810/1247468803,v1:172.21.15.1:6811/1247468803] conn(0x7faf080adee0 0x7faf080b0380 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:18.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.339+0000 7faf1b400640 1 -- 172.21.15.1:0/1043320077 <== osd.3 v2:172.21.15.1:6810/1247468803 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faf080b0a50 con 0x7faf080adee0 2024-09-12T01:28:18.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.354+0000 7faf29800640 1 -- 172.21.15.1:0/1043320077 --> [v2:172.21.15.1:6810/1247468803,v1:172.21.15.1:6811/1247468803] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7faf24062d80 con 0x7faf080adee0 2024-09-12T01:28:18.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.354+0000 7faf1b400640 1 -- 172.21.15.1:0/1043320077 <== osd.3 v2:172.21.15.1:6810/1247468803 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7faf24062d80 con 0x7faf080adee0 2024-09-12T01:28:18.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.355+0000 7faf29800640 1 -- 172.21.15.1:0/1043320077 >> [v2:172.21.15.1:6810/1247468803,v1:172.21.15.1:6811/1247468803] conn(0x7faf080adee0 msgr2=0x7faf080b0380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:18.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.355+0000 7faf29800640 1 --2- 172.21.15.1:0/1043320077 >> [v2:172.21.15.1:6810/1247468803,v1:172.21.15.1:6811/1247468803] conn(0x7faf080adee0 0x7faf080b0380 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:18.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.355+0000 7faf29800640 1 -- 172.21.15.1:0/1043320077 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7faf080a2c10 msgr2=0x7faf080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:18.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.355+0000 7faf29800640 1 --2- 172.21.15.1:0/1043320077 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7faf080a2c10 0x7faf080a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7faf24079300 tx=0x7faf10005f90 comp rx=0 tx=0).stop 2024-09-12T01:28:18.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.355+0000 7faf29800640 1 -- 172.21.15.1:0/1043320077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf24076720 msgr2=0x7faf241a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:18.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.355+0000 7faf29800640 1 --2- 172.21.15.1:0/1043320077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf24076720 0x7faf241a46f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faf14006010 tx=0x7faf14004040 comp rx=0 tx=0).stop 2024-09-12T01:28:18.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.355+0000 7faf23400640 1 -- 172.21.15.1:0/1043320077 reap_dead start 2024-09-12T01:28:18.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.355+0000 7faf29800640 1 -- 172.21.15.1:0/1043320077 shutdown_connections 2024-09-12T01:28:18.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.355+0000 7faf29800640 1 -- 172.21.15.1:0/1043320077 >> 172.21.15.1:0/1043320077 conn(0x7faf240ffa90 msgr2=0x7faf24101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:18.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.355+0000 7faf29800640 1 -- 172.21.15.1:0/1043320077 shutdown_connections 2024-09-12T01:28:18.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:17.355+0000 7faf29800640 1 -- 172.21.15.1:0/1043320077 wait complete. 2024-09-12T01:28:18.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:18 smithi104 ceph-mon[35219]: pgmap v358: 129 pgs: 129 active+clean; 646 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-12T01:28:18.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:18 smithi104 ceph-mon[35219]: osdmap e98: 12 total, 12 up, 12 in 2024-09-12T01:28:19.717 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:19 smithi001 ceph-mon[30387]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-12T01:28:19.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:19 smithi053 ceph-mon[35158]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-12T01:28:19.754 INFO:teuthology.orchestra.run.smithi001.stdout:197568495676 2024-09-12T01:28:19.754 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-12T01:28:19.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:19 smithi104 ceph-mon[35219]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-12T01:28:20.509 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:20 smithi001 ceph-mon[30387]: pgmap v360: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-12T01:28:20.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:20 smithi053 ceph-mon[35158]: pgmap v360: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-12T01:28:20.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:20 smithi104 ceph-mon[35219]: pgmap v360: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-12T01:28:21.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.205+0000 7f5a87600640 1 -- 172.21.15.1:0/296975699 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a80112a30 msgr2=0x7f5a80114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:21.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.205+0000 7f5a87600640 1 --2- 172.21.15.1:0/296975699 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a80112a30 0x7f5a80114e20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5a78009ca0 tx=0x7f5a7802a8b0 comp rx=0 tx=0).stop 2024-09-12T01:28:21.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.205+0000 7f5a87600640 1 -- 172.21.15.1:0/296975699 shutdown_connections 2024-09-12T01:28:21.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.205+0000 7f5a87600640 1 --2- 172.21.15.1:0/296975699 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a80112a30 0x7f5a80114e20 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:21.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.205+0000 7f5a87600640 1 --2- 172.21.15.1:0/296975699 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a80077040 0x7f5a801124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:21.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.205+0000 7f5a87600640 1 --2- 172.21.15.1:0/296975699 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5a80076720 0x7f5a80076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:21.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.205+0000 7f5a87600640 1 -- 172.21.15.1:0/296975699 >> 172.21.15.1:0/296975699 conn(0x7f5a800ffa90 msgr2=0x7f5a80101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:21.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.205+0000 7f5a87600640 1 -- 172.21.15.1:0/296975699 shutdown_connections 2024-09-12T01:28:21.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.205+0000 7f5a87600640 1 -- 172.21.15.1:0/296975699 wait complete. 2024-09-12T01:28:21.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.206+0000 7f5a87600640 1 Processor -- start 2024-09-12T01:28:21.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.206+0000 7f5a87600640 1 -- start start 2024-09-12T01:28:21.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.207+0000 7f5a87600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a80076720 0x7f5a801a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:21.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.207+0000 7f5a87600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5a80077040 0x7f5a801a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:21.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.207+0000 7f5a87600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a80112a30 0x7f5a8019e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:21.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.207+0000 7f5a87600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5a80118580 con 0x7f5a80112a30 2024-09-12T01:28:21.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.207+0000 7f5a87600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5a80118400 con 0x7f5a80076720 2024-09-12T01:28:21.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.207+0000 7f5a87600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5a80118700 con 0x7f5a80077040 2024-09-12T01:28:21.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.207+0000 7f5a84c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a80076720 0x7f5a801a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:21.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.207+0000 7f5a84c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a80076720 0x7f5a801a46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:45684/0 (socket says 172.21.15.1:45684) 2024-09-12T01:28:21.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.207+0000 7f5a84c00640 1 -- 172.21.15.1:0/988216665 learned_addr learned my addr 172.21.15.1:0/988216665 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:28:21.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.207+0000 7f5a85600640 1 --2- 172.21.15.1:0/988216665 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a80112a30 0x7f5a8019e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:21.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.207+0000 7f5a7fe00640 1 --2- 172.21.15.1:0/988216665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5a80077040 0x7f5a801a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:21.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.207+0000 7f5a84c00640 1 -- 172.21.15.1:0/988216665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5a80077040 msgr2=0x7f5a801a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:21.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.207+0000 7f5a84c00640 1 --2- 172.21.15.1:0/988216665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5a80077040 0x7f5a801a4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:21.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.207+0000 7f5a84c00640 1 -- 172.21.15.1:0/988216665 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a80112a30 msgr2=0x7f5a8019e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:21.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.207+0000 7f5a84c00640 1 --2- 172.21.15.1:0/988216665 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a80112a30 0x7f5a8019e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:21.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.207+0000 7f5a84c00640 1 -- 172.21.15.1:0/988216665 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a8019efc0 con 0x7f5a80076720 2024-09-12T01:28:21.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.208+0000 7f5a84c00640 1 --2- 172.21.15.1:0/988216665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a80076720 0x7f5a801a46d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5a7400e9f0 tx=0x7f5a7400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:21.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.208+0000 7f5a7d600640 1 -- 172.21.15.1:0/988216665 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7400cd80 con 0x7f5a80076720 2024-09-12T01:28:21.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.208+0000 7f5a87600640 1 -- 172.21.15.1:0/988216665 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a8019f2b0 con 0x7f5a80076720 2024-09-12T01:28:21.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.208+0000 7f5a7d600640 1 -- 172.21.15.1:0/988216665 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a74005500 con 0x7f5a80076720 2024-09-12T01:28:21.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.208+0000 7f5a7d600640 1 -- 172.21.15.1:0/988216665 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a74010690 con 0x7f5a80076720 2024-09-12T01:28:21.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.208+0000 7f5a87600640 1 -- 172.21.15.1:0/988216665 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a801ab5a0 con 0x7f5a80076720 2024-09-12T01:28:21.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.209+0000 7f5a87600640 1 -- 172.21.15.1:0/988216665 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5a80062980 con 0x7f5a80076720 2024-09-12T01:28:21.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.211+0000 7f5a7d600640 1 -- 172.21.15.1:0/988216665 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f5a740046c0 con 0x7f5a80076720 2024-09-12T01:28:21.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.213+0000 7f5a7d600640 1 --2- 172.21.15.1:0/988216665 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f5a680a2c10 0x7f5a680a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:21.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.213+0000 7f5a7d600640 1 -- 172.21.15.1:0/988216665 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f5a740cfe00 con 0x7f5a80076720 2024-09-12T01:28:21.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.213+0000 7f5a7fe00640 1 --2- 172.21.15.1:0/988216665 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f5a680a2c10 0x7f5a680a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:21.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.213+0000 7f5a7d600640 1 --2- 172.21.15.1:0/988216665 >> [v2:172.21.15.104:6808/3131498790,v1:172.21.15.104:6809/3131498790] conn(0x7f5a680adee0 0x7f5a680b0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:21.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.213+0000 7f5a7d600640 1 -- 172.21.15.1:0/988216665 --> [v2:172.21.15.104:6808/3131498790,v1:172.21.15.104:6809/3131498790] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5a74005350 con 0x7f5a680adee0 2024-09-12T01:28:21.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.213+0000 7f5a7d600640 1 -- 172.21.15.1:0/988216665 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7f5a740d01b0 con 0x7f5a80076720 2024-09-12T01:28:21.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.214+0000 7f5a85600640 1 --2- 172.21.15.1:0/988216665 >> [v2:172.21.15.104:6808/3131498790,v1:172.21.15.104:6809/3131498790] conn(0x7f5a680adee0 0x7f5a680b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:21.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.214+0000 7f5a7fe00640 1 --2- 172.21.15.1:0/988216665 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f5a680a2c10 0x7f5a680a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5a80079290 tx=0x7f5a6c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:21.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.214+0000 7f5a85600640 1 --2- 172.21.15.1:0/988216665 >> [v2:172.21.15.104:6808/3131498790,v1:172.21.15.104:6809/3131498790] conn(0x7f5a680adee0 0x7f5a680b0380 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:21.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.215+0000 7f5a7d600640 1 -- 172.21.15.1:0/988216665 <== osd.5 v2:172.21.15.104:6808/3131498790 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5a74005350 con 0x7f5a680adee0 2024-09-12T01:28:21.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.230+0000 7f5a87600640 1 -- 172.21.15.1:0/988216665 --> [v2:172.21.15.104:6808/3131498790,v1:172.21.15.104:6809/3131498790] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5a80062e00 con 0x7f5a680adee0 2024-09-12T01:28:21.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.230+0000 7f5a7d600640 1 -- 172.21.15.1:0/988216665 <== osd.5 v2:172.21.15.104:6808/3131498790 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5a80062e00 con 0x7f5a680adee0 2024-09-12T01:28:21.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.231+0000 7f5a87600640 1 -- 172.21.15.1:0/988216665 >> [v2:172.21.15.104:6808/3131498790,v1:172.21.15.104:6809/3131498790] conn(0x7f5a680adee0 msgr2=0x7f5a680b0380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:21.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.231+0000 7f5a87600640 1 --2- 172.21.15.1:0/988216665 >> [v2:172.21.15.104:6808/3131498790,v1:172.21.15.104:6809/3131498790] conn(0x7f5a680adee0 0x7f5a680b0380 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:21.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.231+0000 7f5a87600640 1 -- 172.21.15.1:0/988216665 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f5a680a2c10 msgr2=0x7f5a680a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:21.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.231+0000 7f5a87600640 1 --2- 172.21.15.1:0/988216665 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f5a680a2c10 0x7f5a680a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5a80079290 tx=0x7f5a6c005f90 comp rx=0 tx=0).stop 2024-09-12T01:28:21.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.231+0000 7f5a87600640 1 -- 172.21.15.1:0/988216665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a80076720 msgr2=0x7f5a801a46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:21.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.231+0000 7f5a87600640 1 --2- 172.21.15.1:0/988216665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a80076720 0x7f5a801a46d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5a7400e9f0 tx=0x7f5a7400edf0 comp rx=0 tx=0).stop 2024-09-12T01:28:21.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.231+0000 7f5a85600640 1 -- 172.21.15.1:0/988216665 reap_dead start 2024-09-12T01:28:21.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.231+0000 7f5a87600640 1 -- 172.21.15.1:0/988216665 shutdown_connections 2024-09-12T01:28:21.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.231+0000 7f5a87600640 1 -- 172.21.15.1:0/988216665 >> 172.21.15.1:0/988216665 conn(0x7f5a800ffa90 msgr2=0x7f5a80101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:21.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.231+0000 7f5a87600640 1 -- 172.21.15.1:0/988216665 shutdown_connections 2024-09-12T01:28:21.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.231+0000 7f5a87600640 1 -- 172.21.15.1:0/988216665 wait complete. 2024-09-12T01:28:21.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.721+0000 7fc44aa00640 1 -- 172.21.15.1:0/2470963593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc444112a30 msgr2=0x7fc444114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:21.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.721+0000 7fc44aa00640 1 --2- 172.21.15.1:0/2470963593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc444112a30 0x7fc444114e20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc43c009ca0 tx=0x7fc43c02a8c0 comp rx=0 tx=0).stop 2024-09-12T01:28:21.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.721+0000 7fc44aa00640 1 -- 172.21.15.1:0/2470963593 shutdown_connections 2024-09-12T01:28:21.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.721+0000 7fc44aa00640 1 --2- 172.21.15.1:0/2470963593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc444112a30 0x7fc444114e20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:21.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.721+0000 7fc44aa00640 1 --2- 172.21.15.1:0/2470963593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc444077040 0x7fc4441124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:21.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.721+0000 7fc44aa00640 1 --2- 172.21.15.1:0/2470963593 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc444076720 0x7fc444076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:21.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.721+0000 7fc44aa00640 1 -- 172.21.15.1:0/2470963593 >> 172.21.15.1:0/2470963593 conn(0x7fc4440ffa90 msgr2=0x7fc444101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:21.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.721+0000 7fc44aa00640 1 -- 172.21.15.1:0/2470963593 shutdown_connections 2024-09-12T01:28:21.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.722+0000 7fc44aa00640 1 -- 172.21.15.1:0/2470963593 wait complete. 2024-09-12T01:28:21.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.722+0000 7fc44aa00640 1 Processor -- start 2024-09-12T01:28:21.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.723+0000 7fc44aa00640 1 -- start start 2024-09-12T01:28:21.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.723+0000 7fc44aa00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc444076720 0x7fc4441a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:21.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.723+0000 7fc44aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc444077040 0x7fc4441a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:21.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.723+0000 7fc44aa00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc444112a30 0x7fc44419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:21.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.723+0000 7fc44aa00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc4441185d0 con 0x7fc444112a30 2024-09-12T01:28:21.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.723+0000 7fc44aa00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc444118450 con 0x7fc444077040 2024-09-12T01:28:21.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.723+0000 7fc44aa00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc444118750 con 0x7fc444076720 2024-09-12T01:28:21.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.723+0000 7fc448a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc444112a30 0x7fc44419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:21.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.723+0000 7fc448a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc444112a30 0x7fc44419e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37656/0 (socket says 172.21.15.1:37656) 2024-09-12T01:28:21.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.723+0000 7fc448a00640 1 -- 172.21.15.1:0/1093720744 learned_addr learned my addr 172.21.15.1:0/1093720744 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:28:21.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.724+0000 7fc443400640 1 --2- 172.21.15.1:0/1093720744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc444077040 0x7fc4441a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:21.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.724+0000 7fc443e00640 1 --2- 172.21.15.1:0/1093720744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc444076720 0x7fc4441a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:21.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.724+0000 7fc448a00640 1 -- 172.21.15.1:0/1093720744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc444076720 msgr2=0x7fc4441a46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:21.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.724+0000 7fc448a00640 1 --2- 172.21.15.1:0/1093720744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc444076720 0x7fc4441a46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:21.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.724+0000 7fc448a00640 1 -- 172.21.15.1:0/1093720744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc444077040 msgr2=0x7fc4441a4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:21.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.724+0000 7fc448a00640 1 --2- 172.21.15.1:0/1093720744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc444077040 0x7fc4441a4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:21.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.724+0000 7fc448a00640 1 -- 172.21.15.1:0/1093720744 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc44419ef90 con 0x7fc444112a30 2024-09-12T01:28:21.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.725+0000 7fc448a00640 1 --2- 172.21.15.1:0/1093720744 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc444112a30 0x7fc44419e860 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc43c005e30 tx=0x7fc43c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:21.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.725+0000 7fc440c00640 1 -- 172.21.15.1:0/1093720744 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43c031650 con 0x7fc444112a30 2024-09-12T01:28:21.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.725+0000 7fc440c00640 1 -- 172.21.15.1:0/1093720744 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc43c0317f0 con 0x7fc444112a30 2024-09-12T01:28:21.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.725+0000 7fc44aa00640 1 -- 172.21.15.1:0/1093720744 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc44419f220 con 0x7fc444112a30 2024-09-12T01:28:21.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.725+0000 7fc440c00640 1 -- 172.21.15.1:0/1093720744 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43c02fa10 con 0x7fc444112a30 2024-09-12T01:28:21.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.725+0000 7fc44aa00640 1 -- 172.21.15.1:0/1093720744 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4441ab570 con 0x7fc444112a30 2024-09-12T01:28:21.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.727+0000 7fc44aa00640 1 -- 172.21.15.1:0/1093720744 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc444062980 con 0x7fc444112a30 2024-09-12T01:28:21.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.729+0000 7fc440c00640 1 -- 172.21.15.1:0/1093720744 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fc43c02fbb0 con 0x7fc444112a30 2024-09-12T01:28:21.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.731+0000 7fc440c00640 1 --2- 172.21.15.1:0/1093720744 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fc4280a2c90 0x7fc4280a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:21.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.731+0000 7fc440c00640 1 -- 172.21.15.1:0/1093720744 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fc43c0f0a30 con 0x7fc444112a30 2024-09-12T01:28:21.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.731+0000 7fc443e00640 1 --2- 172.21.15.1:0/1093720744 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fc4280a2c90 0x7fc4280a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:21.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.731+0000 7fc440c00640 1 --2- 172.21.15.1:0/1093720744 >> [v2:172.21.15.104:6800/1133285636,v1:172.21.15.104:6801/1133285636] conn(0x7fc4280adf60 0x7fc4280b0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:21.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.731+0000 7fc440c00640 1 -- 172.21.15.1:0/1093720744 --> [v2:172.21.15.104:6800/1133285636,v1:172.21.15.104:6801/1133285636] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc43c0020b0 con 0x7fc4280adf60 2024-09-12T01:28:21.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.731+0000 7fc440c00640 1 -- 172.21.15.1:0/1093720744 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7fc43c0f0de0 con 0x7fc444112a30 2024-09-12T01:28:21.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.732+0000 7fc443400640 1 --2- 172.21.15.1:0/1093720744 >> [v2:172.21.15.104:6800/1133285636,v1:172.21.15.104:6801/1133285636] conn(0x7fc4280adf60 0x7fc4280b0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:21.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.732+0000 7fc443e00640 1 --2- 172.21.15.1:0/1093720744 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fc4280a2c90 0x7fc4280a5150 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc44419f9a0 tx=0x7fc43400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:21.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.732+0000 7fc443400640 1 --2- 172.21.15.1:0/1093720744 >> [v2:172.21.15.104:6800/1133285636,v1:172.21.15.104:6801/1133285636] conn(0x7fc4280adf60 0x7fc4280b0400 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:21.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.733+0000 7fc440c00640 1 -- 172.21.15.1:0/1093720744 <== osd.2 v2:172.21.15.104:6800/1133285636 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc43c0020b0 con 0x7fc4280adf60 2024-09-12T01:28:21.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.747+0000 7fc44aa00640 1 -- 172.21.15.1:0/1093720744 --> [v2:172.21.15.104:6800/1133285636,v1:172.21.15.104:6801/1133285636] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc444062d80 con 0x7fc4280adf60 2024-09-12T01:28:21.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.748+0000 7fc440c00640 1 -- 172.21.15.1:0/1093720744 <== osd.2 v2:172.21.15.104:6800/1133285636 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc444062d80 con 0x7fc4280adf60 2024-09-12T01:28:21.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.749+0000 7fc44aa00640 1 -- 172.21.15.1:0/1093720744 >> [v2:172.21.15.104:6800/1133285636,v1:172.21.15.104:6801/1133285636] conn(0x7fc4280adf60 msgr2=0x7fc4280b0400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:21.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.749+0000 7fc44aa00640 1 --2- 172.21.15.1:0/1093720744 >> [v2:172.21.15.104:6800/1133285636,v1:172.21.15.104:6801/1133285636] conn(0x7fc4280adf60 0x7fc4280b0400 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:21.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.749+0000 7fc44aa00640 1 -- 172.21.15.1:0/1093720744 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fc4280a2c90 msgr2=0x7fc4280a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:21.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.749+0000 7fc44aa00640 1 --2- 172.21.15.1:0/1093720744 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fc4280a2c90 0x7fc4280a5150 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc44419f9a0 tx=0x7fc43400b390 comp rx=0 tx=0).stop 2024-09-12T01:28:21.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.749+0000 7fc44aa00640 1 -- 172.21.15.1:0/1093720744 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc444112a30 msgr2=0x7fc44419e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:21.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.749+0000 7fc44aa00640 1 --2- 172.21.15.1:0/1093720744 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc444112a30 0x7fc44419e860 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc43c005e30 tx=0x7fc43c003f10 comp rx=0 tx=0).stop 2024-09-12T01:28:21.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.749+0000 7fc448a00640 1 -- 172.21.15.1:0/1093720744 reap_dead start 2024-09-12T01:28:21.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.749+0000 7fc44aa00640 1 -- 172.21.15.1:0/1093720744 shutdown_connections 2024-09-12T01:28:21.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.749+0000 7fc44aa00640 1 -- 172.21.15.1:0/1093720744 >> 172.21.15.1:0/1093720744 conn(0x7fc4440ffa90 msgr2=0x7fc444101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:21.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.749+0000 7fc44aa00640 1 -- 172.21.15.1:0/1093720744 shutdown_connections 2024-09-12T01:28:21.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.749+0000 7fc44aa00640 1 -- 172.21.15.1:0/1093720744 wait complete. 2024-09-12T01:28:21.815 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:21 smithi001 ceph-mon[30387]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-12T01:28:21.815 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:21 smithi001 ceph-mon[30387]: Cluster is now healthy 2024-09-12T01:28:21.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:21 smithi104 ceph-mon[35219]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-12T01:28:21.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:21 smithi104 ceph-mon[35219]: Cluster is now healthy 2024-09-12T01:28:21.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:21 smithi053 ceph-mon[35158]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-12T01:28:21.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:21 smithi053 ceph-mon[35158]: Cluster is now healthy 2024-09-12T01:28:22.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.758+0000 7f7809200640 1 -- 172.21.15.1:0/591450818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7804103ff0 msgr2=0x7f78041043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:22.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.758+0000 7f7809200640 1 --2- 172.21.15.1:0/591450818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7804103ff0 0x7f78041043d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f77f4009900 tx=0x7f77f402a410 comp rx=0 tx=0).stop 2024-09-12T01:28:22.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.758+0000 7f7809200640 1 -- 172.21.15.1:0/591450818 shutdown_connections 2024-09-12T01:28:22.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.758+0000 7f7809200640 1 --2- 172.21.15.1:0/591450818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7804109120 0x7f7804110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:22.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.758+0000 7f7809200640 1 --2- 172.21.15.1:0/591450818 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78041049a0 0x7f78041089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:22.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.758+0000 7f7809200640 1 --2- 172.21.15.1:0/591450818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7804103ff0 0x7f78041043d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:22.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.758+0000 7f7809200640 1 -- 172.21.15.1:0/591450818 >> 172.21.15.1:0/591450818 conn(0x7f78040ffa90 msgr2=0x7f7804101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:22.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.758+0000 7f7809200640 1 -- 172.21.15.1:0/591450818 shutdown_connections 2024-09-12T01:28:22.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.758+0000 7f7809200640 1 -- 172.21.15.1:0/591450818 wait complete. 2024-09-12T01:28:22.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.759+0000 7f7809200640 1 Processor -- start 2024-09-12T01:28:22.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.759+0000 7f7809200640 1 -- start start 2024-09-12T01:28:22.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.760+0000 7f7809200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7804103ff0 0x7f780419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:22.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.760+0000 7f7809200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78041049a0 0x7f780419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:22.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.760+0000 7f7809200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7804109120 0x7f780419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:22.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.760+0000 7f7809200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7804114190 con 0x7f7804103ff0 2024-09-12T01:28:22.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.760+0000 7f7809200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7804114010 con 0x7f78041049a0 2024-09-12T01:28:22.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.760+0000 7f7809200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7804114310 con 0x7f7804109120 2024-09-12T01:28:22.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.760+0000 7f7801e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7804103ff0 0x7f780419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:22.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.760+0000 7f7801e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7804103ff0 0x7f780419a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51340/0 (socket says 172.21.15.1:51340) 2024-09-12T01:28:22.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.760+0000 7f7801e00640 1 -- 172.21.15.1:0/2403848846 learned_addr learned my addr 172.21.15.1:0/2403848846 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:28:22.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.760+0000 7f77fbe00640 1 --2- 172.21.15.1:0/2403848846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78041049a0 0x7f780419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:22.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.760+0000 7f7802800640 1 --2- 172.21.15.1:0/2403848846 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7804109120 0x7f780419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:22.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.761+0000 7f7801e00640 1 -- 172.21.15.1:0/2403848846 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7804109120 msgr2=0x7f780419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:22.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.761+0000 7f7801e00640 1 --2- 172.21.15.1:0/2403848846 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7804109120 0x7f780419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:22.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.761+0000 7f7801e00640 1 -- 172.21.15.1:0/2403848846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78041049a0 msgr2=0x7f780419ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:22.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.761+0000 7f7801e00640 1 --2- 172.21.15.1:0/2403848846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78041049a0 0x7f780419ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:22.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.761+0000 7f7801e00640 1 -- 172.21.15.1:0/2403848846 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f780419f5e0 con 0x7f7804103ff0 2024-09-12T01:28:22.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.761+0000 7f7802800640 1 --2- 172.21.15.1:0/2403848846 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7804109120 0x7f780419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:28:22.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.761+0000 7f77fbe00640 1 --2- 172.21.15.1:0/2403848846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78041049a0 0x7f780419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:28:22.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.761+0000 7f7801e00640 1 --2- 172.21.15.1:0/2403848846 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7804103ff0 0x7f780419a630 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f77f4004770 tx=0x7f77f4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:22.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.761+0000 7f77faa00640 1 -- 172.21.15.1:0/2403848846 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f403f070 con 0x7f7804103ff0 2024-09-12T01:28:22.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.761+0000 7f77faa00640 1 -- 172.21.15.1:0/2403848846 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77f4031ca0 con 0x7f7804103ff0 2024-09-12T01:28:22.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.761+0000 7f77faa00640 1 -- 172.21.15.1:0/2403848846 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f4035030 con 0x7f7804103ff0 2024-09-12T01:28:22.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.761+0000 7f7809200640 1 -- 172.21.15.1:0/2403848846 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f780419f870 con 0x7f7804103ff0 2024-09-12T01:28:22.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.762+0000 7f7809200640 1 -- 172.21.15.1:0/2403848846 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78041a71b0 con 0x7f7804103ff0 2024-09-12T01:28:22.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.762+0000 7f7809200640 1 -- 172.21.15.1:0/2403848846 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7804062980 con 0x7f7804103ff0 2024-09-12T01:28:22.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.764+0000 7f77faa00640 1 -- 172.21.15.1:0/2403848846 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f77f4042070 con 0x7f7804103ff0 2024-09-12T01:28:22.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.766+0000 7f77faa00640 1 --2- 172.21.15.1:0/2403848846 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f77e80a2c10 0x7f77e80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:22.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.766+0000 7f77faa00640 1 -- 172.21.15.1:0/2403848846 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f77f40effc0 con 0x7f7804103ff0 2024-09-12T01:28:22.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.766+0000 7f77fbe00640 1 --2- 172.21.15.1:0/2403848846 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f77e80a2c10 0x7f77e80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:22.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.766+0000 7f77faa00640 1 --2- 172.21.15.1:0/2403848846 >> [v2:172.21.15.53:6816/2841484136,v1:172.21.15.53:6817/2841484136] conn(0x7f77e80adee0 0x7f77e80b0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:22.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.766+0000 7f77faa00640 1 -- 172.21.15.1:0/2403848846 --> [v2:172.21.15.53:6816/2841484136,v1:172.21.15.53:6817/2841484136] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f77e80b0a50 con 0x7f77e80adee0 2024-09-12T01:28:22.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.766+0000 7f77faa00640 1 -- 172.21.15.1:0/2403848846 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7f77f40f0370 con 0x7f7804103ff0 2024-09-12T01:28:22.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.766+0000 7f7802800640 1 --2- 172.21.15.1:0/2403848846 >> [v2:172.21.15.53:6816/2841484136,v1:172.21.15.53:6817/2841484136] conn(0x7f77e80adee0 0x7f77e80b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:22.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.767+0000 7f77fbe00640 1 --2- 172.21.15.1:0/2403848846 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f77e80a2c10 0x7f77e80a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f78040fff20 tx=0x7f77ec005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:22.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.767+0000 7f7802800640 1 --2- 172.21.15.1:0/2403848846 >> [v2:172.21.15.53:6816/2841484136,v1:172.21.15.53:6817/2841484136] conn(0x7f77e80adee0 0x7f77e80b0380 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:22.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.768+0000 7f77faa00640 1 -- 172.21.15.1:0/2403848846 <== osd.7 v2:172.21.15.53:6816/2841484136 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f77e80b0a50 con 0x7f77e80adee0 2024-09-12T01:28:22.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.783+0000 7f7809200640 1 -- 172.21.15.1:0/2403848846 --> [v2:172.21.15.53:6816/2841484136,v1:172.21.15.53:6817/2841484136] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7804062e00 con 0x7f77e80adee0 2024-09-12T01:28:22.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.784+0000 7f77faa00640 1 -- 172.21.15.1:0/2403848846 <== osd.7 v2:172.21.15.53:6816/2841484136 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7804062e00 con 0x7f77e80adee0 2024-09-12T01:28:22.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.784+0000 7f7809200640 1 -- 172.21.15.1:0/2403848846 >> [v2:172.21.15.53:6816/2841484136,v1:172.21.15.53:6817/2841484136] conn(0x7f77e80adee0 msgr2=0x7f77e80b0380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:22.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.784+0000 7f7809200640 1 --2- 172.21.15.1:0/2403848846 >> [v2:172.21.15.53:6816/2841484136,v1:172.21.15.53:6817/2841484136] conn(0x7f77e80adee0 0x7f77e80b0380 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:22.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.784+0000 7f7809200640 1 -- 172.21.15.1:0/2403848846 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f77e80a2c10 msgr2=0x7f77e80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:22.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.784+0000 7f7809200640 1 --2- 172.21.15.1:0/2403848846 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f77e80a2c10 0x7f77e80a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f78040fff20 tx=0x7f77ec005f90 comp rx=0 tx=0).stop 2024-09-12T01:28:22.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.784+0000 7f7809200640 1 -- 172.21.15.1:0/2403848846 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7804103ff0 msgr2=0x7f780419a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:22.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.784+0000 7f7809200640 1 --2- 172.21.15.1:0/2403848846 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7804103ff0 0x7f780419a630 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f77f4004770 tx=0x7f77f4003c10 comp rx=0 tx=0).stop 2024-09-12T01:28:22.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.784+0000 7f7802800640 1 -- 172.21.15.1:0/2403848846 reap_dead start 2024-09-12T01:28:22.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.784+0000 7f7809200640 1 -- 172.21.15.1:0/2403848846 shutdown_connections 2024-09-12T01:28:22.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.784+0000 7f7809200640 1 -- 172.21.15.1:0/2403848846 >> 172.21.15.1:0/2403848846 conn(0x7f78040ffa90 msgr2=0x7f7804100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:22.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.784+0000 7f7809200640 1 -- 172.21.15.1:0/2403848846 shutdown_connections 2024-09-12T01:28:22.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:18.784+0000 7f7809200640 1 -- 172.21.15.1:0/2403848846 wait complete. 2024-09-12T01:28:22.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.297+0000 7fe518600640 1 -- 172.21.15.1:0/1739830660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe510109120 msgr2=0x7fe510110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:22.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.297+0000 7fe518600640 1 --2- 172.21.15.1:0/1739830660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe510109120 0x7fe510110ca0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe508009ca0 tx=0x7fe50802a8c0 comp rx=0 tx=0).stop 2024-09-12T01:28:22.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.297+0000 7fe518600640 1 -- 172.21.15.1:0/1739830660 shutdown_connections 2024-09-12T01:28:22.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.297+0000 7fe518600640 1 --2- 172.21.15.1:0/1739830660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe510109120 0x7fe510110ca0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:22.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.297+0000 7fe518600640 1 --2- 172.21.15.1:0/1739830660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5101049a0 0x7fe5101089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:22.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.297+0000 7fe518600640 1 --2- 172.21.15.1:0/1739830660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe510103ff0 0x7fe5101043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:22.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.297+0000 7fe518600640 1 -- 172.21.15.1:0/1739830660 >> 172.21.15.1:0/1739830660 conn(0x7fe5100ffa90 msgr2=0x7fe510101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:22.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.297+0000 7fe518600640 1 -- 172.21.15.1:0/1739830660 shutdown_connections 2024-09-12T01:28:22.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.297+0000 7fe518600640 1 -- 172.21.15.1:0/1739830660 wait complete. 2024-09-12T01:28:22.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.298+0000 7fe518600640 1 Processor -- start 2024-09-12T01:28:22.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.298+0000 7fe518600640 1 -- start start 2024-09-12T01:28:22.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.299+0000 7fe518600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe510103ff0 0x7fe51019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:22.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.299+0000 7fe518600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5101049a0 0x7fe51019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:22.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.299+0000 7fe518600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe510109120 0x7fe51019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:22.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.299+0000 7fe518600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe510114150 con 0x7fe510103ff0 2024-09-12T01:28:22.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.299+0000 7fe518600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe510113fd0 con 0x7fe510109120 2024-09-12T01:28:22.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.299+0000 7fe518600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe5101142d0 con 0x7fe5101049a0 2024-09-12T01:28:22.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.299+0000 7fe515c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe510103ff0 0x7fe51019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:22.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.299+0000 7fe515c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe510103ff0 0x7fe51019a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37662/0 (socket says 172.21.15.1:37662) 2024-09-12T01:28:22.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.299+0000 7fe515c00640 1 -- 172.21.15.1:0/1791967185 learned_addr learned my addr 172.21.15.1:0/1791967185 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:28:22.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.299+0000 7fe515200640 1 --2- 172.21.15.1:0/1791967185 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5101049a0 0x7fe51019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:22.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.299+0000 7fe516600640 1 --2- 172.21.15.1:0/1791967185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe510109120 0x7fe51019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:22.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.299+0000 7fe515c00640 1 -- 172.21.15.1:0/1791967185 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5101049a0 msgr2=0x7fe51019ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:22.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.299+0000 7fe515c00640 1 --2- 172.21.15.1:0/1791967185 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5101049a0 0x7fe51019ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:22.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.299+0000 7fe515c00640 1 -- 172.21.15.1:0/1791967185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe510109120 msgr2=0x7fe51019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:22.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.299+0000 7fe515c00640 1 --2- 172.21.15.1:0/1791967185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe510109120 0x7fe51019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:22.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.300+0000 7fe515c00640 1 -- 172.21.15.1:0/1791967185 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe51019f5e0 con 0x7fe510103ff0 2024-09-12T01:28:22.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.300+0000 7fe515c00640 1 --2- 172.21.15.1:0/1791967185 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe510103ff0 0x7fe51019a630 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe50400e9f0 tx=0x7fe50400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:22.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.300+0000 7fe50e000640 1 -- 172.21.15.1:0/1791967185 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50400cd80 con 0x7fe510103ff0 2024-09-12T01:28:22.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.300+0000 7fe50e000640 1 -- 172.21.15.1:0/1791967185 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe504005500 con 0x7fe510103ff0 2024-09-12T01:28:22.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.300+0000 7fe50e000640 1 -- 172.21.15.1:0/1791967185 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe504010690 con 0x7fe510103ff0 2024-09-12T01:28:22.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.300+0000 7fe518600640 1 -- 172.21.15.1:0/1791967185 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe51019f8d0 con 0x7fe510103ff0 2024-09-12T01:28:22.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.300+0000 7fe518600640 1 -- 172.21.15.1:0/1791967185 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5101a71b0 con 0x7fe510103ff0 2024-09-12T01:28:22.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.301+0000 7fe518600640 1 -- 172.21.15.1:0/1791967185 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe510062980 con 0x7fe510103ff0 2024-09-12T01:28:22.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.303+0000 7fe50e000640 1 -- 172.21.15.1:0/1791967185 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fe5040046c0 con 0x7fe510103ff0 2024-09-12T01:28:22.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.304+0000 7fe50e000640 1 --2- 172.21.15.1:0/1791967185 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe4f80a2c10 0x7fe4f80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:22.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.305+0000 7fe50e000640 1 -- 172.21.15.1:0/1791967185 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fe5040cfe00 con 0x7fe510103ff0 2024-09-12T01:28:22.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.305+0000 7fe515200640 1 --2- 172.21.15.1:0/1791967185 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe4f80a2c10 0x7fe4f80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:22.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.305+0000 7fe50e000640 1 --2- 172.21.15.1:0/1791967185 >> [v2:172.21.15.53:6800/3131049165,v1:172.21.15.53:6801/3131049165] conn(0x7fe4f80adee0 0x7fe4f80b0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:22.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.305+0000 7fe50e000640 1 -- 172.21.15.1:0/1791967185 --> [v2:172.21.15.53:6800/3131049165,v1:172.21.15.53:6801/3131049165] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe504005350 con 0x7fe4f80adee0 2024-09-12T01:28:22.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.305+0000 7fe50e000640 1 -- 172.21.15.1:0/1791967185 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7fe5040d01b0 con 0x7fe510103ff0 2024-09-12T01:28:22.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.305+0000 7fe516600640 1 --2- 172.21.15.1:0/1791967185 >> [v2:172.21.15.53:6800/3131049165,v1:172.21.15.53:6801/3131049165] conn(0x7fe4f80adee0 0x7fe4f80b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:22.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.306+0000 7fe515200640 1 --2- 172.21.15.1:0/1791967185 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe4f80a2c10 0x7fe4f80a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc0095f0 tx=0x7fe4fc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:22.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.306+0000 7fe516600640 1 --2- 172.21.15.1:0/1791967185 >> [v2:172.21.15.53:6800/3131049165,v1:172.21.15.53:6801/3131049165] conn(0x7fe4f80adee0 0x7fe4f80b0380 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:22.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.307+0000 7fe50e000640 1 -- 172.21.15.1:0/1791967185 <== osd.1 v2:172.21.15.53:6800/3131049165 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe504005350 con 0x7fe4f80adee0 2024-09-12T01:28:22.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.322+0000 7fe518600640 1 -- 172.21.15.1:0/1791967185 --> [v2:172.21.15.53:6800/3131049165,v1:172.21.15.53:6801/3131049165] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe510062d80 con 0x7fe4f80adee0 2024-09-12T01:28:22.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.322+0000 7fe50e000640 1 -- 172.21.15.1:0/1791967185 <== osd.1 v2:172.21.15.53:6800/3131049165 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe510062d80 con 0x7fe4f80adee0 2024-09-12T01:28:22.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.323+0000 7fe518600640 1 -- 172.21.15.1:0/1791967185 >> [v2:172.21.15.53:6800/3131049165,v1:172.21.15.53:6801/3131049165] conn(0x7fe4f80adee0 msgr2=0x7fe4f80b0380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:22.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.323+0000 7fe518600640 1 --2- 172.21.15.1:0/1791967185 >> [v2:172.21.15.53:6800/3131049165,v1:172.21.15.53:6801/3131049165] conn(0x7fe4f80adee0 0x7fe4f80b0380 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:22.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.323+0000 7fe518600640 1 -- 172.21.15.1:0/1791967185 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe4f80a2c10 msgr2=0x7fe4f80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:22.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.323+0000 7fe518600640 1 --2- 172.21.15.1:0/1791967185 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe4f80a2c10 0x7fe4f80a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc0095f0 tx=0x7fe4fc005f90 comp rx=0 tx=0).stop 2024-09-12T01:28:22.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.323+0000 7fe518600640 1 -- 172.21.15.1:0/1791967185 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe510103ff0 msgr2=0x7fe51019a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:22.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.323+0000 7fe518600640 1 --2- 172.21.15.1:0/1791967185 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe510103ff0 0x7fe51019a630 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe50400e9f0 tx=0x7fe50400edf0 comp rx=0 tx=0).stop 2024-09-12T01:28:22.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.323+0000 7fe516600640 1 -- 172.21.15.1:0/1791967185 reap_dead start 2024-09-12T01:28:22.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.323+0000 7fe518600640 1 -- 172.21.15.1:0/1791967185 shutdown_connections 2024-09-12T01:28:22.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.323+0000 7fe518600640 1 -- 172.21.15.1:0/1791967185 >> 172.21.15.1:0/1791967185 conn(0x7fe5100ffa90 msgr2=0x7fe510100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:22.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.323+0000 7fe518600640 1 -- 172.21.15.1:0/1791967185 shutdown_connections 2024-09-12T01:28:22.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.323+0000 7fe518600640 1 -- 172.21.15.1:0/1791967185 wait complete. 2024-09-12T01:28:22.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:22 smithi104 ceph-mon[35219]: pgmap v361: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 9 op/s 2024-09-12T01:28:22.876 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:22 smithi001 ceph-mon[30387]: pgmap v361: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 9 op/s 2024-09-12T01:28:22.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:22 smithi053 ceph-mon[35158]: pgmap v361: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 9 op/s 2024-09-12T01:28:23.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.876+0000 7feb74c00640 1 -- 172.21.15.1:0/1965931714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb70112a50 msgr2=0x7feb70114e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:23.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.876+0000 7feb74c00640 1 --2- 172.21.15.1:0/1965931714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb70112a50 0x7feb70114e40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7feb68009ca0 tx=0x7feb6802a8c0 comp rx=0 tx=0).stop 2024-09-12T01:28:23.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.876+0000 7feb74c00640 1 -- 172.21.15.1:0/1965931714 shutdown_connections 2024-09-12T01:28:23.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.876+0000 7feb74c00640 1 --2- 172.21.15.1:0/1965931714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb70112a50 0x7feb70114e40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:23.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.876+0000 7feb74c00640 1 --2- 172.21.15.1:0/1965931714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb70077040 0x7feb70112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:23.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.876+0000 7feb74c00640 1 --2- 172.21.15.1:0/1965931714 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb70076720 0x7feb70076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:23.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.876+0000 7feb74c00640 1 -- 172.21.15.1:0/1965931714 >> 172.21.15.1:0/1965931714 conn(0x7feb700ffa50 msgr2=0x7feb70101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:23.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.876+0000 7feb74c00640 1 -- 172.21.15.1:0/1965931714 shutdown_connections 2024-09-12T01:28:23.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.877+0000 7feb74c00640 1 -- 172.21.15.1:0/1965931714 wait complete. 2024-09-12T01:28:23.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.878+0000 7feb74c00640 1 Processor -- start 2024-09-12T01:28:23.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.878+0000 7feb74c00640 1 -- start start 2024-09-12T01:28:23.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.878+0000 7feb74c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb70076720 0x7feb701a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:23.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.878+0000 7feb74c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb70077040 0x7feb701a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:23.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.878+0000 7feb74c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb70112a50 0x7feb7019e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:23.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.878+0000 7feb74c00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7feb701185f0 con 0x7feb70112a50 2024-09-12T01:28:23.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.878+0000 7feb74c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7feb70118470 con 0x7feb70077040 2024-09-12T01:28:23.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.878+0000 7feb74c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7feb70118770 con 0x7feb70076720 2024-09-12T01:28:23.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.879+0000 7feb6de00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb70076720 0x7feb701a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:23.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.879+0000 7feb6d400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb70077040 0x7feb701a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:23.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.879+0000 7feb6e800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb70112a50 0x7feb7019e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:23.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.879+0000 7feb6de00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb70076720 0x7feb701a46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:40394/0 (socket says 172.21.15.1:40394) 2024-09-12T01:28:23.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.879+0000 7feb6de00640 1 -- 172.21.15.1:0/1319308068 learned_addr learned my addr 172.21.15.1:0/1319308068 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:28:23.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.879+0000 7feb6de00640 1 -- 172.21.15.1:0/1319308068 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb70077040 msgr2=0x7feb701a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:23.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.879+0000 7feb6de00640 1 --2- 172.21.15.1:0/1319308068 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb70077040 0x7feb701a4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:23.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.879+0000 7feb6de00640 1 -- 172.21.15.1:0/1319308068 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb70112a50 msgr2=0x7feb7019e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:23.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.879+0000 7feb6de00640 1 --2- 172.21.15.1:0/1319308068 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb70112a50 0x7feb7019e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:23.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.879+0000 7feb6de00640 1 -- 172.21.15.1:0/1319308068 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb7019efb0 con 0x7feb70076720 2024-09-12T01:28:23.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.879+0000 7feb6de00640 1 --2- 172.21.15.1:0/1319308068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb70076720 0x7feb701a46c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7feb6000e9f0 tx=0x7feb6000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:23.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.880+0000 7feb66a00640 1 -- 172.21.15.1:0/1319308068 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6000cd80 con 0x7feb70076720 2024-09-12T01:28:23.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.880+0000 7feb66a00640 1 -- 172.21.15.1:0/1319308068 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb60005500 con 0x7feb70076720 2024-09-12T01:28:23.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.880+0000 7feb74c00640 1 -- 172.21.15.1:0/1319308068 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb7019f2a0 con 0x7feb70076720 2024-09-12T01:28:23.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.880+0000 7feb66a00640 1 -- 172.21.15.1:0/1319308068 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb60010690 con 0x7feb70076720 2024-09-12T01:28:23.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.880+0000 7feb74c00640 1 -- 172.21.15.1:0/1319308068 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb701ab590 con 0x7feb70076720 2024-09-12T01:28:23.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.881+0000 7feb74c00640 1 -- 172.21.15.1:0/1319308068 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7feb70114a10 con 0x7feb70076720 2024-09-12T01:28:23.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.883+0000 7feb66a00640 1 -- 172.21.15.1:0/1319308068 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7feb600046c0 con 0x7feb70076720 2024-09-12T01:28:23.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.884+0000 7feb66a00640 1 --2- 172.21.15.1:0/1319308068 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7feb540a2c10 0x7feb540a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:23.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.884+0000 7feb66a00640 1 -- 172.21.15.1:0/1319308068 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7feb600cfe00 con 0x7feb70076720 2024-09-12T01:28:23.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.884+0000 7feb6d400640 1 --2- 172.21.15.1:0/1319308068 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7feb540a2c10 0x7feb540a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:23.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.884+0000 7feb66a00640 1 --2- 172.21.15.1:0/1319308068 >> [v2:172.21.15.104:6816/941453547,v1:172.21.15.104:6817/941453547] conn(0x7feb540adee0 0x7feb540b0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:23.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.884+0000 7feb66a00640 1 -- 172.21.15.1:0/1319308068 --> [v2:172.21.15.104:6816/941453547,v1:172.21.15.104:6817/941453547] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feb60005350 con 0x7feb540adee0 2024-09-12T01:28:23.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.884+0000 7feb66a00640 1 -- 172.21.15.1:0/1319308068 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7feb600d01b0 con 0x7feb70076720 2024-09-12T01:28:23.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.885+0000 7feb6e800640 1 --2- 172.21.15.1:0/1319308068 >> [v2:172.21.15.104:6816/941453547,v1:172.21.15.104:6817/941453547] conn(0x7feb540adee0 0x7feb540b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:23.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.885+0000 7feb6d400640 1 --2- 172.21.15.1:0/1319308068 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7feb540a2c10 0x7feb540a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7feb70079290 tx=0x7feb5c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:23.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.885+0000 7feb6e800640 1 --2- 172.21.15.1:0/1319308068 >> [v2:172.21.15.104:6816/941453547,v1:172.21.15.104:6817/941453547] conn(0x7feb540adee0 0x7feb540b0380 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:23.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.886+0000 7feb66a00640 1 -- 172.21.15.1:0/1319308068 <== osd.8 v2:172.21.15.104:6816/941453547 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feb60005350 con 0x7feb540adee0 2024-09-12T01:28:23.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.901+0000 7feb74c00640 1 -- 172.21.15.1:0/1319308068 --> [v2:172.21.15.104:6816/941453547,v1:172.21.15.104:6817/941453547] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7feb70077e60 con 0x7feb540adee0 2024-09-12T01:28:23.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.902+0000 7feb66a00640 1 -- 172.21.15.1:0/1319308068 <== osd.8 v2:172.21.15.104:6816/941453547 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7feb70077e60 con 0x7feb540adee0 2024-09-12T01:28:23.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.902+0000 7feb74c00640 1 -- 172.21.15.1:0/1319308068 >> [v2:172.21.15.104:6816/941453547,v1:172.21.15.104:6817/941453547] conn(0x7feb540adee0 msgr2=0x7feb540b0380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:23.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.902+0000 7feb74c00640 1 --2- 172.21.15.1:0/1319308068 >> [v2:172.21.15.104:6816/941453547,v1:172.21.15.104:6817/941453547] conn(0x7feb540adee0 0x7feb540b0380 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:23.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.902+0000 7feb74c00640 1 -- 172.21.15.1:0/1319308068 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7feb540a2c10 msgr2=0x7feb540a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:23.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.902+0000 7feb74c00640 1 --2- 172.21.15.1:0/1319308068 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7feb540a2c10 0x7feb540a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7feb70079290 tx=0x7feb5c005f90 comp rx=0 tx=0).stop 2024-09-12T01:28:23.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.902+0000 7feb74c00640 1 -- 172.21.15.1:0/1319308068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb70076720 msgr2=0x7feb701a46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:23.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.902+0000 7feb74c00640 1 --2- 172.21.15.1:0/1319308068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb70076720 0x7feb701a46c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7feb6000e9f0 tx=0x7feb6000edf0 comp rx=0 tx=0).stop 2024-09-12T01:28:23.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.902+0000 7feb6e800640 1 -- 172.21.15.1:0/1319308068 reap_dead start 2024-09-12T01:28:23.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.902+0000 7feb74c00640 1 -- 172.21.15.1:0/1319308068 shutdown_connections 2024-09-12T01:28:23.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.902+0000 7feb74c00640 1 -- 172.21.15.1:0/1319308068 >> 172.21.15.1:0/1319308068 conn(0x7feb700ffa50 msgr2=0x7feb70101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:23.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.902+0000 7feb74c00640 1 -- 172.21.15.1:0/1319308068 shutdown_connections 2024-09-12T01:28:23.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:22.903+0000 7feb74c00640 1 -- 172.21.15.1:0/1319308068 wait complete. 2024-09-12T01:28:23.952 INFO:teuthology.orchestra.run.smithi001.stdout:163208757314 2024-09-12T01:28:23.952 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-12T01:28:24.062 INFO:teuthology.orchestra.run.smithi001.stdout:107374182476 2024-09-12T01:28:24.063 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-12T01:28:24.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.098+0000 7fd710000640 1 -- 172.21.15.1:0/562339517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd708109140 msgr2=0x7fd708110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:24.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.098+0000 7fd710000640 1 --2- 172.21.15.1:0/562339517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd708109140 0x7fd708110cc0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd700009ca0 tx=0x7fd70002a8c0 comp rx=0 tx=0).stop 2024-09-12T01:28:24.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.098+0000 7fd710000640 1 -- 172.21.15.1:0/562339517 shutdown_connections 2024-09-12T01:28:24.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.098+0000 7fd710000640 1 --2- 172.21.15.1:0/562339517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd708109140 0x7fd708110cc0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:24.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.098+0000 7fd710000640 1 --2- 172.21.15.1:0/562339517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7081049c0 0x7fd708108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:24.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.098+0000 7fd710000640 1 --2- 172.21.15.1:0/562339517 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd708104010 0x7fd7081043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:24.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.098+0000 7fd710000640 1 -- 172.21.15.1:0/562339517 >> 172.21.15.1:0/562339517 conn(0x7fd7080ffa50 msgr2=0x7fd708101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:24.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.098+0000 7fd710000640 1 -- 172.21.15.1:0/562339517 shutdown_connections 2024-09-12T01:28:24.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.098+0000 7fd710000640 1 -- 172.21.15.1:0/562339517 wait complete. 2024-09-12T01:28:24.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.099+0000 7fd710000640 1 Processor -- start 2024-09-12T01:28:24.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.099+0000 7fd710000640 1 -- start start 2024-09-12T01:28:24.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.099+0000 7fd710000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd708104010 0x7fd70819a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:24.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.099+0000 7fd710000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7081049c0 0x7fd70819ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:24.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.099+0000 7fd710000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd708109140 0x7fd70819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:24.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.099+0000 7fd710000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd708114170 con 0x7fd708104010 2024-09-12T01:28:24.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.099+0000 7fd710000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd708113ff0 con 0x7fd7081049c0 2024-09-12T01:28:24.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.099+0000 7fd710000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd7081142f0 con 0x7fd708109140 2024-09-12T01:28:24.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.100+0000 7fd70cc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7081049c0 0x7fd70819ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:24.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.100+0000 7fd70e000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd708109140 0x7fd70819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:24.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.100+0000 7fd70e000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd708109140 0x7fd70819efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:40326/0 (socket says 172.21.15.1:40326) 2024-09-12T01:28:24.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.100+0000 7fd70e000640 1 -- 172.21.15.1:0/842140949 learned_addr learned my addr 172.21.15.1:0/842140949 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:28:24.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.100+0000 7fd70d600640 1 --2- 172.21.15.1:0/842140949 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd708104010 0x7fd70819a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:24.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.100+0000 7fd70e000640 1 -- 172.21.15.1:0/842140949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7081049c0 msgr2=0x7fd70819ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:24.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.100+0000 7fd70e000640 1 --2- 172.21.15.1:0/842140949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7081049c0 0x7fd70819ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:24.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.100+0000 7fd70e000640 1 -- 172.21.15.1:0/842140949 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd708104010 msgr2=0x7fd70819a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:24.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.100+0000 7fd70e000640 1 --2- 172.21.15.1:0/842140949 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd708104010 0x7fd70819a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:24.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.100+0000 7fd70e000640 1 -- 172.21.15.1:0/842140949 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd70819f680 con 0x7fd708109140 2024-09-12T01:28:24.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.100+0000 7fd70cc00640 1 --2- 172.21.15.1:0/842140949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7081049c0 0x7fd70819ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:28:24.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.100+0000 7fd70d600640 1 --2- 172.21.15.1:0/842140949 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd708104010 0x7fd70819a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:28:24.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.101+0000 7fd70e000640 1 --2- 172.21.15.1:0/842140949 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd708109140 0x7fd70819efa0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd700031a20 tx=0x7fd700031a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:24.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.101+0000 7fd706000640 1 -- 172.21.15.1:0/842140949 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd700003c30 con 0x7fd708109140 2024-09-12T01:28:24.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.101+0000 7fd710000640 1 -- 172.21.15.1:0/842140949 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd70819f910 con 0x7fd708109140 2024-09-12T01:28:24.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.101+0000 7fd706000640 1 -- 172.21.15.1:0/842140949 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd700003dd0 con 0x7fd708109140 2024-09-12T01:28:24.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.101+0000 7fd706000640 1 -- 172.21.15.1:0/842140949 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd70002fc40 con 0x7fd708109140 2024-09-12T01:28:24.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.101+0000 7fd710000640 1 -- 172.21.15.1:0/842140949 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7081a7250 con 0x7fd708109140 2024-09-12T01:28:24.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.102+0000 7fd710000640 1 -- 172.21.15.1:0/842140949 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd708062900 con 0x7fd708109140 2024-09-12T01:28:24.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.104+0000 7fd706000640 1 -- 172.21.15.1:0/842140949 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fd700004050 con 0x7fd708109140 2024-09-12T01:28:24.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.105+0000 7fd706000640 1 --2- 172.21.15.1:0/842140949 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fd6f00a2c10 0x7fd6f00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:24.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.105+0000 7fd706000640 1 -- 172.21.15.1:0/842140949 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fd7000f08e0 con 0x7fd708109140 2024-09-12T01:28:24.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.106+0000 7fd70d600640 1 --2- 172.21.15.1:0/842140949 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fd6f00a2c10 0x7fd6f00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:24.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.106+0000 7fd706000640 1 --2- 172.21.15.1:0/842140949 >> [v2:172.21.15.53:6824/3357909498,v1:172.21.15.53:6825/3357909498] conn(0x7fd6f00addb0 0x7fd6f00b0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:24.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.106+0000 7fd706000640 1 -- 172.21.15.1:0/842140949 --> [v2:172.21.15.53:6824/3357909498,v1:172.21.15.53:6825/3357909498] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd6f00b0920 con 0x7fd6f00addb0 2024-09-12T01:28:24.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.106+0000 7fd706000640 1 -- 172.21.15.1:0/842140949 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7fd7000f0c90 con 0x7fd708109140 2024-09-12T01:28:24.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.106+0000 7fd70cc00640 1 --2- 172.21.15.1:0/842140949 >> [v2:172.21.15.53:6824/3357909498,v1:172.21.15.53:6825/3357909498] conn(0x7fd6f00addb0 0x7fd6f00b0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:24.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.106+0000 7fd70d600640 1 --2- 172.21.15.1:0/842140949 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fd6f00a2c10 0x7fd6f00a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc009fc0 tx=0x7fd6fc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:24.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.107+0000 7fd70cc00640 1 --2- 172.21.15.1:0/842140949 >> [v2:172.21.15.53:6824/3357909498,v1:172.21.15.53:6825/3357909498] conn(0x7fd6f00addb0 0x7fd6f00b0250 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:24.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.108+0000 7fd706000640 1 -- 172.21.15.1:0/842140949 <== osd.10 v2:172.21.15.53:6824/3357909498 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd6f00b0920 con 0x7fd6f00addb0 2024-09-12T01:28:24.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.122+0000 7fd710000640 1 -- 172.21.15.1:0/842140949 --> [v2:172.21.15.53:6824/3357909498,v1:172.21.15.53:6825/3357909498] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd708062d80 con 0x7fd6f00addb0 2024-09-12T01:28:24.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.123+0000 7fd706000640 1 -- 172.21.15.1:0/842140949 <== osd.10 v2:172.21.15.53:6824/3357909498 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd708062d80 con 0x7fd6f00addb0 2024-09-12T01:28:24.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.123+0000 7fd710000640 1 -- 172.21.15.1:0/842140949 >> [v2:172.21.15.53:6824/3357909498,v1:172.21.15.53:6825/3357909498] conn(0x7fd6f00addb0 msgr2=0x7fd6f00b0250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:24.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.123+0000 7fd710000640 1 --2- 172.21.15.1:0/842140949 >> [v2:172.21.15.53:6824/3357909498,v1:172.21.15.53:6825/3357909498] conn(0x7fd6f00addb0 0x7fd6f00b0250 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:24.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.123+0000 7fd710000640 1 -- 172.21.15.1:0/842140949 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fd6f00a2c10 msgr2=0x7fd6f00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:24.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.123+0000 7fd710000640 1 --2- 172.21.15.1:0/842140949 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fd6f00a2c10 0x7fd6f00a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc009fc0 tx=0x7fd6fc005f90 comp rx=0 tx=0).stop 2024-09-12T01:28:24.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.123+0000 7fd710000640 1 -- 172.21.15.1:0/842140949 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd708109140 msgr2=0x7fd70819efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:24.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.123+0000 7fd710000640 1 --2- 172.21.15.1:0/842140949 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd708109140 0x7fd70819efa0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd700031a20 tx=0x7fd700031a50 comp rx=0 tx=0).stop 2024-09-12T01:28:24.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.123+0000 7fd70e000640 1 -- 172.21.15.1:0/842140949 reap_dead start 2024-09-12T01:28:24.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.123+0000 7fd710000640 1 -- 172.21.15.1:0/842140949 shutdown_connections 2024-09-12T01:28:24.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.123+0000 7fd710000640 1 -- 172.21.15.1:0/842140949 >> 172.21.15.1:0/842140949 conn(0x7fd7080ffa50 msgr2=0x7fd708100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:24.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.124+0000 7fd710000640 1 -- 172.21.15.1:0/842140949 shutdown_connections 2024-09-12T01:28:24.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:21.124+0000 7fd710000640 1 -- 172.21.15.1:0/842140949 wait complete. 2024-09-12T01:28:24.563 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:24 smithi001 ceph-mon[30387]: pgmap v362: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-12T01:28:24.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:24 smithi104 ceph-mon[35219]: pgmap v362: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-12T01:28:24.992 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:24 smithi053 ceph-mon[35158]: pgmap v362: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-12T01:28:25.834 INFO:teuthology.orchestra.run.smithi001.stdout:253403070512 2024-09-12T01:28:25.834 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-12T01:28:26.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.045+0000 7f1ff0600640 1 -- 172.21.15.1:0/908265795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fe81049c0 msgr2=0x7f1fe8108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:26.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.045+0000 7f1ff0600640 1 --2- 172.21.15.1:0/908265795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fe81049c0 0x7f1fe8108a10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4009900 tx=0x7f1fd402a400 comp rx=0 tx=0).stop 2024-09-12T01:28:26.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.045+0000 7f1ff0600640 1 -- 172.21.15.1:0/908265795 shutdown_connections 2024-09-12T01:28:26.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.045+0000 7f1ff0600640 1 --2- 172.21.15.1:0/908265795 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fe8109140 0x7f1fe8110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:26.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.045+0000 7f1ff0600640 1 --2- 172.21.15.1:0/908265795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fe81049c0 0x7f1fe8108a10 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:26.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.045+0000 7f1ff0600640 1 --2- 172.21.15.1:0/908265795 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1fe8104010 0x7f1fe81043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:26.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.045+0000 7f1ff0600640 1 -- 172.21.15.1:0/908265795 >> 172.21.15.1:0/908265795 conn(0x7f1fe80ffa50 msgr2=0x7f1fe8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:26.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.045+0000 7f1ff0600640 1 -- 172.21.15.1:0/908265795 shutdown_connections 2024-09-12T01:28:26.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.045+0000 7f1ff0600640 1 -- 172.21.15.1:0/908265795 wait complete. 2024-09-12T01:28:26.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.046+0000 7f1ff0600640 1 Processor -- start 2024-09-12T01:28:26.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.046+0000 7f1ff0600640 1 -- start start 2024-09-12T01:28:26.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.047+0000 7f1ff0600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fe8104010 0x7f1fe8076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:26.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.047+0000 7f1ff0600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fe81049c0 0x7f1fe8076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:26.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.047+0000 7f1ff0600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1fe8109140 0x7f1fe8077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:26.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.047+0000 7f1ff0600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe8114160 con 0x7f1fe81049c0 2024-09-12T01:28:26.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.047+0000 7f1ff0600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe8113fe0 con 0x7f1fe8104010 2024-09-12T01:28:26.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.047+0000 7f1ff0600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe81142e0 con 0x7f1fe8109140 2024-09-12T01:28:26.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.047+0000 7f1fedc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fe8104010 0x7f1fe8076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:26.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.047+0000 7f1fedc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fe8104010 0x7f1fe8076700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:45778/0 (socket says 172.21.15.1:45778) 2024-09-12T01:28:26.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.047+0000 7f1fedc00640 1 -- 172.21.15.1:0/3966045419 learned_addr learned my addr 172.21.15.1:0/3966045419 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:28:26.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.047+0000 7f1fed200640 1 --2- 172.21.15.1:0/3966045419 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fe81049c0 0x7f1fe8076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:26.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.047+0000 7f1fee600640 1 --2- 172.21.15.1:0/3966045419 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1fe8109140 0x7f1fe8077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:26.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.048+0000 7f1fedc00640 1 -- 172.21.15.1:0/3966045419 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1fe8109140 msgr2=0x7f1fe8077180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:26.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.048+0000 7f1fedc00640 1 --2- 172.21.15.1:0/3966045419 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1fe8109140 0x7f1fe8077180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:26.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.048+0000 7f1fedc00640 1 -- 172.21.15.1:0/3966045419 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fe81049c0 msgr2=0x7f1fe8076c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:26.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.048+0000 7f1fedc00640 1 --2- 172.21.15.1:0/3966045419 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fe81049c0 0x7f1fe8076c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:26.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.048+0000 7f1fedc00640 1 -- 172.21.15.1:0/3966045419 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe81aad30 con 0x7f1fe8104010 2024-09-12T01:28:26.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.048+0000 7f1fed200640 1 --2- 172.21.15.1:0/3966045419 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fe81049c0 0x7f1fe8076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:28:26.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.048+0000 7f1fedc00640 1 --2- 172.21.15.1:0/3966045419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fe8104010 0x7f1fe8076700 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc00ea50 tx=0x7f1fdc00ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:26.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.049+0000 7f1fe6000640 1 -- 172.21.15.1:0/3966045419 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc00ccc0 con 0x7f1fe8104010 2024-09-12T01:28:26.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.049+0000 7f1fe6000640 1 -- 172.21.15.1:0/3966045419 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fdc00ce60 con 0x7f1fe8104010 2024-09-12T01:28:26.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.049+0000 7f1ff0600640 1 -- 172.21.15.1:0/3966045419 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fe81aafc0 con 0x7f1fe8104010 2024-09-12T01:28:26.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.049+0000 7f1fe6000640 1 -- 172.21.15.1:0/3966045419 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc010690 con 0x7f1fe8104010 2024-09-12T01:28:26.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.049+0000 7f1ff0600640 1 -- 172.21.15.1:0/3966045419 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fe81ab480 con 0x7f1fe8104010 2024-09-12T01:28:26.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.050+0000 7f1ff0600640 1 -- 172.21.15.1:0/3966045419 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1fe8062980 con 0x7f1fe8104010 2024-09-12T01:28:26.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.053+0000 7f1fe6000640 1 -- 172.21.15.1:0/3966045419 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f1fdc0046c0 con 0x7f1fe8104010 2024-09-12T01:28:26.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.055+0000 7f1fe6000640 1 --2- 172.21.15.1:0/3966045419 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1fd00a2c10 0x7f1fd00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:26.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.055+0000 7f1fe6000640 1 -- 172.21.15.1:0/3966045419 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f1fdc0cfd80 con 0x7f1fe8104010 2024-09-12T01:28:26.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.055+0000 7f1fed200640 1 --2- 172.21.15.1:0/3966045419 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1fd00a2c10 0x7f1fd00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:26.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.055+0000 7f1fe6000640 1 --2- 172.21.15.1:0/3966045419 >> [v2:172.21.15.104:6824/3381427987,v1:172.21.15.104:6825/3381427987] conn(0x7f1fd00adee0 0x7f1fd00b0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:26.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.055+0000 7f1fe6000640 1 -- 172.21.15.1:0/3966045419 --> [v2:172.21.15.104:6824/3381427987,v1:172.21.15.104:6825/3381427987] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1fd00b0a50 con 0x7f1fd00adee0 2024-09-12T01:28:26.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.055+0000 7f1fe6000640 1 -- 172.21.15.1:0/3966045419 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7f1fdc0d0130 con 0x7f1fe8104010 2024-09-12T01:28:26.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.056+0000 7f1fee600640 1 --2- 172.21.15.1:0/3966045419 >> [v2:172.21.15.104:6824/3381427987,v1:172.21.15.104:6825/3381427987] conn(0x7f1fd00adee0 0x7f1fd00b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:26.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.056+0000 7f1fed200640 1 --2- 172.21.15.1:0/3966045419 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1fd00a2c10 0x7f1fd00a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4009230 tx=0x7f1fd40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:26.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.056+0000 7f1fee600640 1 --2- 172.21.15.1:0/3966045419 >> [v2:172.21.15.104:6824/3381427987,v1:172.21.15.104:6825/3381427987] conn(0x7f1fd00adee0 0x7f1fd00b0380 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:26.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.057+0000 7f1fe6000640 1 -- 172.21.15.1:0/3966045419 <== osd.11 v2:172.21.15.104:6824/3381427987 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1fd00b0a50 con 0x7f1fd00adee0 2024-09-12T01:28:26.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.072+0000 7f1ff0600640 1 -- 172.21.15.1:0/3966045419 --> [v2:172.21.15.104:6824/3381427987,v1:172.21.15.104:6825/3381427987] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1fe8062e00 con 0x7f1fd00adee0 2024-09-12T01:28:26.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.072+0000 7f1fe6000640 1 -- 172.21.15.1:0/3966045419 <== osd.11 v2:172.21.15.104:6824/3381427987 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1fe8062e00 con 0x7f1fd00adee0 2024-09-12T01:28:26.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.073+0000 7f1ff0600640 1 -- 172.21.15.1:0/3966045419 >> [v2:172.21.15.104:6824/3381427987,v1:172.21.15.104:6825/3381427987] conn(0x7f1fd00adee0 msgr2=0x7f1fd00b0380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:26.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.073+0000 7f1ff0600640 1 --2- 172.21.15.1:0/3966045419 >> [v2:172.21.15.104:6824/3381427987,v1:172.21.15.104:6825/3381427987] conn(0x7f1fd00adee0 0x7f1fd00b0380 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:26.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.073+0000 7f1ff0600640 1 -- 172.21.15.1:0/3966045419 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1fd00a2c10 msgr2=0x7f1fd00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:26.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.073+0000 7f1ff0600640 1 --2- 172.21.15.1:0/3966045419 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1fd00a2c10 0x7f1fd00a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4009230 tx=0x7f1fd40058d0 comp rx=0 tx=0).stop 2024-09-12T01:28:26.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.073+0000 7f1ff0600640 1 -- 172.21.15.1:0/3966045419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fe8104010 msgr2=0x7f1fe8076700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:26.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.073+0000 7f1ff0600640 1 --2- 172.21.15.1:0/3966045419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fe8104010 0x7f1fe8076700 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc00ea50 tx=0x7f1fdc00ee50 comp rx=0 tx=0).stop 2024-09-12T01:28:26.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.073+0000 7f1fee600640 1 -- 172.21.15.1:0/3966045419 reap_dead start 2024-09-12T01:28:26.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.073+0000 7f1ff0600640 1 -- 172.21.15.1:0/3966045419 shutdown_connections 2024-09-12T01:28:26.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.073+0000 7f1ff0600640 1 -- 172.21.15.1:0/3966045419 >> 172.21.15.1:0/3966045419 conn(0x7f1fe80ffa50 msgr2=0x7f1fe8100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:26.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.074+0000 7f1ff0600640 1 -- 172.21.15.1:0/3966045419 shutdown_connections 2024-09-12T01:28:26.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.074+0000 7f1ff0600640 1 -- 172.21.15.1:0/3966045419 wait complete. 2024-09-12T01:28:26.555 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:26 smithi001 ceph-mon[30387]: pgmap v363: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-12T01:28:26.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.673+0000 7f40cce00640 1 -- 172.21.15.1:0/294799039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40c8103ff0 msgr2=0x7f40c81043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:26.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.673+0000 7f40cce00640 1 --2- 172.21.15.1:0/294799039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40c8103ff0 0x7f40c81043d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f40b8009860 tx=0x7f40b802a340 comp rx=0 tx=0).stop 2024-09-12T01:28:26.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.673+0000 7f40cce00640 1 -- 172.21.15.1:0/294799039 shutdown_connections 2024-09-12T01:28:26.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.673+0000 7f40cce00640 1 --2- 172.21.15.1:0/294799039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40c8109120 0x7f40c8110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:26.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.673+0000 7f40cce00640 1 --2- 172.21.15.1:0/294799039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40c81049a0 0x7f40c81089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:26.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.673+0000 7f40cce00640 1 --2- 172.21.15.1:0/294799039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40c8103ff0 0x7f40c81043d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:26.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.673+0000 7f40cce00640 1 -- 172.21.15.1:0/294799039 >> 172.21.15.1:0/294799039 conn(0x7f40c80ffa90 msgr2=0x7f40c8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:26.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.673+0000 7f40cce00640 1 -- 172.21.15.1:0/294799039 shutdown_connections 2024-09-12T01:28:26.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.673+0000 7f40cce00640 1 -- 172.21.15.1:0/294799039 wait complete. 2024-09-12T01:28:26.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.674+0000 7f40cce00640 1 Processor -- start 2024-09-12T01:28:26.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.674+0000 7f40cce00640 1 -- start start 2024-09-12T01:28:26.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.675+0000 7f40cce00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40c8103ff0 0x7f40c819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:26.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.675+0000 7f40cce00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40c81049a0 0x7f40c819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:26.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.675+0000 7f40cce00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40c8109120 0x7f40c819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:26.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.675+0000 7f40cce00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f40c8114150 con 0x7f40c8109120 2024-09-12T01:28:26.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.675+0000 7f40cce00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f40c8113fd0 con 0x7f40c8103ff0 2024-09-12T01:28:26.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.675+0000 7f40cce00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f40c81142d0 con 0x7f40c81049a0 2024-09-12T01:28:26.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.675+0000 7f40c6800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40c8109120 0x7f40c819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:26.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.675+0000 7f40c5e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40c8103ff0 0x7f40c819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:26.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.675+0000 7f40c6800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40c8109120 0x7f40c819eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37722/0 (socket says 172.21.15.1:37722) 2024-09-12T01:28:26.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.675+0000 7f40c6800640 1 -- 172.21.15.1:0/719773216 learned_addr learned my addr 172.21.15.1:0/719773216 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:28:26.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.675+0000 7f40c5e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40c8103ff0 0x7f40c819a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:45824/0 (socket says 172.21.15.1:45824) 2024-09-12T01:28:26.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.675+0000 7f40c5400640 1 --2- 172.21.15.1:0/719773216 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40c81049a0 0x7f40c819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:26.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.675+0000 7f40c5e00640 1 -- 172.21.15.1:0/719773216 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40c81049a0 msgr2=0x7f40c819ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:26.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.675+0000 7f40c5e00640 1 --2- 172.21.15.1:0/719773216 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40c81049a0 0x7f40c819ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:26.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.675+0000 7f40c5e00640 1 -- 172.21.15.1:0/719773216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40c8109120 msgr2=0x7f40c819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:26.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.675+0000 7f40c5e00640 1 --2- 172.21.15.1:0/719773216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40c8109120 0x7f40c819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:26.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.675+0000 7f40c5e00640 1 -- 172.21.15.1:0/719773216 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40c819f5c0 con 0x7f40c8103ff0 2024-09-12T01:28:26.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.676+0000 7f40c5e00640 1 --2- 172.21.15.1:0/719773216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40c8103ff0 0x7f40c819a610 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f40b802a860 tx=0x7f40b8033b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:26.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.676+0000 7f40bea00640 1 -- 172.21.15.1:0/719773216 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b8037040 con 0x7f40c8103ff0 2024-09-12T01:28:26.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.676+0000 7f40bea00640 1 -- 172.21.15.1:0/719773216 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40b8033d50 con 0x7f40c8103ff0 2024-09-12T01:28:26.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.676+0000 7f40bea00640 1 -- 172.21.15.1:0/719773216 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b8003810 con 0x7f40c8103ff0 2024-09-12T01:28:26.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.676+0000 7f40cce00640 1 -- 172.21.15.1:0/719773216 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40c819f850 con 0x7f40c8103ff0 2024-09-12T01:28:26.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.676+0000 7f40cce00640 1 -- 172.21.15.1:0/719773216 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40c81a7190 con 0x7f40c8103ff0 2024-09-12T01:28:26.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.677+0000 7f40cce00640 1 -- 172.21.15.1:0/719773216 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f40c8062940 con 0x7f40c8103ff0 2024-09-12T01:28:26.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.681+0000 7f40bea00640 1 -- 172.21.15.1:0/719773216 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f40b8042020 con 0x7f40c8103ff0 2024-09-12T01:28:26.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.682+0000 7f40bea00640 1 --2- 172.21.15.1:0/719773216 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f40ac0a2c10 0x7f40ac0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:26.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.682+0000 7f40bea00640 1 -- 172.21.15.1:0/719773216 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f40b80ef400 con 0x7f40c8103ff0 2024-09-12T01:28:26.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.682+0000 7f40c5400640 1 --2- 172.21.15.1:0/719773216 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f40ac0a2c10 0x7f40ac0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:26.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.683+0000 7f40bea00640 1 --2- 172.21.15.1:0/719773216 >> [v2:172.21.15.53:6808/2673413639,v1:172.21.15.53:6809/2673413639] conn(0x7f40ac0adee0 0x7f40ac0b0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:26.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.683+0000 7f40bea00640 1 -- 172.21.15.1:0/719773216 --> [v2:172.21.15.53:6808/2673413639,v1:172.21.15.53:6809/2673413639] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f40b8034ec0 con 0x7f40ac0adee0 2024-09-12T01:28:26.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.683+0000 7f40bea00640 1 -- 172.21.15.1:0/719773216 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7f40b80ef7b0 con 0x7f40c8103ff0 2024-09-12T01:28:26.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.683+0000 7f40c6800640 1 --2- 172.21.15.1:0/719773216 >> [v2:172.21.15.53:6808/2673413639,v1:172.21.15.53:6809/2673413639] conn(0x7f40ac0adee0 0x7f40ac0b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:26.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.683+0000 7f40c5400640 1 --2- 172.21.15.1:0/719773216 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f40ac0a2c10 0x7f40ac0a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f40c80fff90 tx=0x7f40b4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:26.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.683+0000 7f40c6800640 1 --2- 172.21.15.1:0/719773216 >> [v2:172.21.15.53:6808/2673413639,v1:172.21.15.53:6809/2673413639] conn(0x7f40ac0adee0 0x7f40ac0b0380 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:26.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.684+0000 7f40bea00640 1 -- 172.21.15.1:0/719773216 <== osd.4 v2:172.21.15.53:6808/2673413639 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f40b8034ec0 con 0x7f40ac0adee0 2024-09-12T01:28:26.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.700+0000 7f40cce00640 1 -- 172.21.15.1:0/719773216 --> [v2:172.21.15.53:6808/2673413639,v1:172.21.15.53:6809/2673413639] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f40c8062dc0 con 0x7f40ac0adee0 2024-09-12T01:28:26.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.700+0000 7f40bea00640 1 -- 172.21.15.1:0/719773216 <== osd.4 v2:172.21.15.53:6808/2673413639 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f40c8062dc0 con 0x7f40ac0adee0 2024-09-12T01:28:26.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.701+0000 7f40cce00640 1 -- 172.21.15.1:0/719773216 >> [v2:172.21.15.53:6808/2673413639,v1:172.21.15.53:6809/2673413639] conn(0x7f40ac0adee0 msgr2=0x7f40ac0b0380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:26.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.701+0000 7f40cce00640 1 --2- 172.21.15.1:0/719773216 >> [v2:172.21.15.53:6808/2673413639,v1:172.21.15.53:6809/2673413639] conn(0x7f40ac0adee0 0x7f40ac0b0380 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:26.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.701+0000 7f40cce00640 1 -- 172.21.15.1:0/719773216 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f40ac0a2c10 msgr2=0x7f40ac0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:26.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.701+0000 7f40cce00640 1 --2- 172.21.15.1:0/719773216 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f40ac0a2c10 0x7f40ac0a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f40c80fff90 tx=0x7f40b4009260 comp rx=0 tx=0).stop 2024-09-12T01:28:26.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.701+0000 7f40cce00640 1 -- 172.21.15.1:0/719773216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40c8103ff0 msgr2=0x7f40c819a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:26.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.701+0000 7f40cce00640 1 --2- 172.21.15.1:0/719773216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40c8103ff0 0x7f40c819a610 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f40b802a860 tx=0x7f40b8033b60 comp rx=0 tx=0).stop 2024-09-12T01:28:26.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.701+0000 7f40c6800640 1 -- 172.21.15.1:0/719773216 reap_dead start 2024-09-12T01:28:26.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.701+0000 7f40cce00640 1 -- 172.21.15.1:0/719773216 shutdown_connections 2024-09-12T01:28:26.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.701+0000 7f40cce00640 1 -- 172.21.15.1:0/719773216 >> 172.21.15.1:0/719773216 conn(0x7f40c80ffa90 msgr2=0x7f40c8100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:26.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.701+0000 7f40cce00640 1 -- 172.21.15.1:0/719773216 shutdown_connections 2024-09-12T01:28:26.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.701+0000 7f40cce00640 1 -- 172.21.15.1:0/719773216 wait complete. 2024-09-12T01:28:26.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:26 smithi104 ceph-mon[35219]: pgmap v363: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-12T01:28:26.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.287+0000 7f0543e00640 1 -- 172.21.15.1:0/1278570150 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f053c106f20 msgr2=0x7f053c10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:26.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.287+0000 7f0543e00640 1 --2- 172.21.15.1:0/1278570150 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f053c106f20 0x7f053c10eaa0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0534009ca0 tx=0x7f053402a8c0 comp rx=0 tx=0).stop 2024-09-12T01:28:26.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.287+0000 7f0543e00640 1 -- 172.21.15.1:0/1278570150 shutdown_connections 2024-09-12T01:28:26.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.287+0000 7f0543e00640 1 --2- 172.21.15.1:0/1278570150 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f053c106f20 0x7f053c10eaa0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:26.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.287+0000 7f0543e00640 1 --2- 172.21.15.1:0/1278570150 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f053c1027a0 0x7f053c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:26.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.287+0000 7f0543e00640 1 --2- 172.21.15.1:0/1278570150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f053c101df0 0x7f053c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:26.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.287+0000 7f0543e00640 1 -- 172.21.15.1:0/1278570150 >> 172.21.15.1:0/1278570150 conn(0x7f053c0fd8b0 msgr2=0x7f053c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:26.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.287+0000 7f0543e00640 1 -- 172.21.15.1:0/1278570150 shutdown_connections 2024-09-12T01:28:26.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.287+0000 7f0543e00640 1 -- 172.21.15.1:0/1278570150 wait complete. 2024-09-12T01:28:26.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.288+0000 7f0543e00640 1 Processor -- start 2024-09-12T01:28:26.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.289+0000 7f0543e00640 1 -- start start 2024-09-12T01:28:26.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.289+0000 7f0543e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f053c101df0 0x7f053c19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:26.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.289+0000 7f0543e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f053c1027a0 0x7f053c19aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:26.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.289+0000 7f0543e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f053c106f20 0x7f053c19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:26.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.289+0000 7f0543e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f053c111f80 con 0x7f053c1027a0 2024-09-12T01:28:26.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.289+0000 7f0543e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f053c111e00 con 0x7f053c106f20 2024-09-12T01:28:26.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.289+0000 7f0543e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f053c112100 con 0x7f053c101df0 2024-09-12T01:28:26.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.289+0000 7f0540a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f053c1027a0 0x7f053c19aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:26.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.289+0000 7f0541400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f053c101df0 0x7f053c19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:26.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.289+0000 7f0540a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f053c1027a0 0x7f053c19aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37708/0 (socket says 172.21.15.1:37708) 2024-09-12T01:28:26.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.289+0000 7f0540a00640 1 -- 172.21.15.1:0/3925693357 learned_addr learned my addr 172.21.15.1:0/3925693357 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:28:26.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.289+0000 7f0541e00640 1 --2- 172.21.15.1:0/3925693357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f053c106f20 0x7f053c19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:26.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.290+0000 7f0541400640 1 -- 172.21.15.1:0/3925693357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f053c106f20 msgr2=0x7f053c19ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:26.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.290+0000 7f0541400640 1 --2- 172.21.15.1:0/3925693357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f053c106f20 0x7f053c19ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:26.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.290+0000 7f0541400640 1 -- 172.21.15.1:0/3925693357 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f053c1027a0 msgr2=0x7f053c19aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:26.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.290+0000 7f0541400640 1 --2- 172.21.15.1:0/3925693357 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f053c1027a0 0x7f053c19aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:26.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.290+0000 7f0541400640 1 -- 172.21.15.1:0/3925693357 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f053c19f610 con 0x7f053c101df0 2024-09-12T01:28:26.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.290+0000 7f0540a00640 1 --2- 172.21.15.1:0/3925693357 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f053c1027a0 0x7f053c19aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:28:26.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.290+0000 7f0541400640 1 --2- 172.21.15.1:0/3925693357 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f053c101df0 0x7f053c19a660 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f053000e9f0 tx=0x7f053000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:26.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.290+0000 7f053a000640 1 -- 172.21.15.1:0/3925693357 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053000cd80 con 0x7f053c101df0 2024-09-12T01:28:26.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.290+0000 7f053a000640 1 -- 172.21.15.1:0/3925693357 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05300035f0 con 0x7f053c101df0 2024-09-12T01:28:26.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.290+0000 7f053a000640 1 -- 172.21.15.1:0/3925693357 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0530010690 con 0x7f053c101df0 2024-09-12T01:28:26.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.290+0000 7f0543e00640 1 -- 172.21.15.1:0/3925693357 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f053c19f900 con 0x7f053c101df0 2024-09-12T01:28:26.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.291+0000 7f0543e00640 1 -- 172.21.15.1:0/3925693357 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f053c1a71e0 con 0x7f053c101df0 2024-09-12T01:28:26.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.291+0000 7f0543e00640 1 -- 172.21.15.1:0/3925693357 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f053c10e140 con 0x7f053c101df0 2024-09-12T01:28:26.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.294+0000 7f053a000640 1 -- 172.21.15.1:0/3925693357 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f0530003b30 con 0x7f053c101df0 2024-09-12T01:28:26.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.295+0000 7f053a000640 1 --2- 172.21.15.1:0/3925693357 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f05240a2c10 0x7f05240a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:26.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.295+0000 7f053a000640 1 -- 172.21.15.1:0/3925693357 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f05300cfb00 con 0x7f053c101df0 2024-09-12T01:28:26.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.295+0000 7f0540a00640 1 --2- 172.21.15.1:0/3925693357 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f05240a2c10 0x7f05240a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:26.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.295+0000 7f053a000640 1 --2- 172.21.15.1:0/3925693357 >> [v2:172.21.15.1:6826/958287586,v1:172.21.15.1:6827/958287586] conn(0x7f05240adee0 0x7f05240b0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:26.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.295+0000 7f053a000640 1 -- 172.21.15.1:0/3925693357 --> [v2:172.21.15.1:6826/958287586,v1:172.21.15.1:6827/958287586] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0530013ec0 con 0x7f05240adee0 2024-09-12T01:28:26.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.295+0000 7f053a000640 1 -- 172.21.15.1:0/3925693357 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7f05300cfeb0 con 0x7f053c101df0 2024-09-12T01:28:26.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.296+0000 7f0541e00640 1 --2- 172.21.15.1:0/3925693357 >> [v2:172.21.15.1:6826/958287586,v1:172.21.15.1:6827/958287586] conn(0x7f05240adee0 0x7f05240b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:26.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.296+0000 7f0540a00640 1 --2- 172.21.15.1:0/3925693357 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f05240a2c10 0x7f05240a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f05280095f0 tx=0x7f0528005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:26.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.296+0000 7f0541e00640 1 --2- 172.21.15.1:0/3925693357 >> [v2:172.21.15.1:6826/958287586,v1:172.21.15.1:6827/958287586] conn(0x7f05240adee0 0x7f05240b0380 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:26.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.298+0000 7f053a000640 1 -- 172.21.15.1:0/3925693357 <== osd.9 v2:172.21.15.1:6826/958287586 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0530013ec0 con 0x7f05240adee0 2024-09-12T01:28:26.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.313+0000 7f0543e00640 1 -- 172.21.15.1:0/3925693357 --> [v2:172.21.15.1:6826/958287586,v1:172.21.15.1:6827/958287586] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f053c1032d0 con 0x7f05240adee0 2024-09-12T01:28:26.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.314+0000 7f053a000640 1 -- 172.21.15.1:0/3925693357 <== osd.9 v2:172.21.15.1:6826/958287586 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f053c1032d0 con 0x7f05240adee0 2024-09-12T01:28:26.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.314+0000 7f0543e00640 1 -- 172.21.15.1:0/3925693357 >> [v2:172.21.15.1:6826/958287586,v1:172.21.15.1:6827/958287586] conn(0x7f05240adee0 msgr2=0x7f05240b0380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:26.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.314+0000 7f0543e00640 1 --2- 172.21.15.1:0/3925693357 >> [v2:172.21.15.1:6826/958287586,v1:172.21.15.1:6827/958287586] conn(0x7f05240adee0 0x7f05240b0380 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:26.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.314+0000 7f0543e00640 1 -- 172.21.15.1:0/3925693357 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f05240a2c10 msgr2=0x7f05240a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:26.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.314+0000 7f0543e00640 1 --2- 172.21.15.1:0/3925693357 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f05240a2c10 0x7f05240a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f05280095f0 tx=0x7f0528005f90 comp rx=0 tx=0).stop 2024-09-12T01:28:26.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.314+0000 7f0543e00640 1 -- 172.21.15.1:0/3925693357 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f053c101df0 msgr2=0x7f053c19a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:26.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.314+0000 7f0543e00640 1 --2- 172.21.15.1:0/3925693357 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f053c101df0 0x7f053c19a660 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f053000e9f0 tx=0x7f053000eeb0 comp rx=0 tx=0).stop 2024-09-12T01:28:26.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.314+0000 7f0541e00640 1 -- 172.21.15.1:0/3925693357 reap_dead start 2024-09-12T01:28:26.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.314+0000 7f0543e00640 1 -- 172.21.15.1:0/3925693357 shutdown_connections 2024-09-12T01:28:26.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.314+0000 7f0543e00640 1 -- 172.21.15.1:0/3925693357 >> 172.21.15.1:0/3925693357 conn(0x7f053c0fd8b0 msgr2=0x7f053c0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:26.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.314+0000 7f0543e00640 1 -- 172.21.15.1:0/3925693357 shutdown_connections 2024-09-12T01:28:26.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:26.315+0000 7f0543e00640 1 -- 172.21.15.1:0/3925693357 wait complete. 2024-09-12T01:28:26.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:26 smithi053 ceph-mon[35158]: pgmap v363: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-12T01:28:27.387 INFO:teuthology.orchestra.run.smithi001.stdout:219043332149 2024-09-12T01:28:27.387 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-12T01:28:27.960 INFO:teuthology.orchestra.run.smithi001.stdout:287762808876 2024-09-12T01:28:27.960 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-12T01:28:28.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.239+0000 7f0438400640 1 -- 172.21.15.1:0/478682701 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0430109120 msgr2=0x7f0430110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:28.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.239+0000 7f0438400640 1 --2- 172.21.15.1:0/478682701 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0430109120 0x7f0430110ca0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0428009ca0 tx=0x7f042802a8c0 comp rx=0 tx=0).stop 2024-09-12T01:28:28.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.239+0000 7f0438400640 1 -- 172.21.15.1:0/478682701 shutdown_connections 2024-09-12T01:28:28.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.239+0000 7f0438400640 1 --2- 172.21.15.1:0/478682701 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0430109120 0x7f0430110ca0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:28.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.239+0000 7f0438400640 1 --2- 172.21.15.1:0/478682701 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04301049a0 0x7f04301089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:28.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.239+0000 7f0438400640 1 --2- 172.21.15.1:0/478682701 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0430103ff0 0x7f04301043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:28.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.239+0000 7f0438400640 1 -- 172.21.15.1:0/478682701 >> 172.21.15.1:0/478682701 conn(0x7f04300ffa90 msgr2=0x7f0430101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:28.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.239+0000 7f0438400640 1 -- 172.21.15.1:0/478682701 shutdown_connections 2024-09-12T01:28:28.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.239+0000 7f0438400640 1 -- 172.21.15.1:0/478682701 wait complete. 2024-09-12T01:28:28.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.240+0000 7f0438400640 1 Processor -- start 2024-09-12T01:28:28.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.240+0000 7f0438400640 1 -- start start 2024-09-12T01:28:28.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.241+0000 7f0438400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0430103ff0 0x7f043019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:28.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.241+0000 7f0438400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04301049a0 0x7f043019ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:28.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.241+0000 7f0438400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0430109120 0x7f043019eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:28.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.241+0000 7f0438400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0430114130 con 0x7f0430103ff0 2024-09-12T01:28:28.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.241+0000 7f0438400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0430113fb0 con 0x7f04301049a0 2024-09-12T01:28:28.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.241+0000 7f0438400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f04301142b0 con 0x7f0430109120 2024-09-12T01:28:28.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.241+0000 7f0435a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0430103ff0 0x7f043019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:28.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.241+0000 7f0435000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04301049a0 0x7f043019ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:28.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.241+0000 7f0435a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0430103ff0 0x7f043019a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37606/0 (socket says 172.21.15.1:37606) 2024-09-12T01:28:28.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.241+0000 7f0435a00640 1 -- 172.21.15.1:0/1541562235 learned_addr learned my addr 172.21.15.1:0/1541562235 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:28:28.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.241+0000 7f0436400640 1 --2- 172.21.15.1:0/1541562235 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0430109120 0x7f043019eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:28.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.242+0000 7f0435000640 1 -- 172.21.15.1:0/1541562235 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0430109120 msgr2=0x7f043019eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:28.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.242+0000 7f0435000640 1 --2- 172.21.15.1:0/1541562235 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0430109120 0x7f043019eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:28.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.242+0000 7f0435000640 1 -- 172.21.15.1:0/1541562235 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0430103ff0 msgr2=0x7f043019a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:28.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.242+0000 7f0435000640 1 --2- 172.21.15.1:0/1541562235 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0430103ff0 0x7f043019a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:28.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.242+0000 7f0435000640 1 -- 172.21.15.1:0/1541562235 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f043019f580 con 0x7f04301049a0 2024-09-12T01:28:28.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.242+0000 7f0435a00640 1 --2- 172.21.15.1:0/1541562235 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0430103ff0 0x7f043019a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:28:28.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.242+0000 7f0435000640 1 --2- 172.21.15.1:0/1541562235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04301049a0 0x7f043019ab10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f041c00e9f0 tx=0x7f041c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:28.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.242+0000 7f042e000640 1 -- 172.21.15.1:0/1541562235 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c00ccc0 con 0x7f04301049a0 2024-09-12T01:28:28.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.242+0000 7f042e000640 1 -- 172.21.15.1:0/1541562235 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f041c010910 con 0x7f04301049a0 2024-09-12T01:28:28.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.242+0000 7f0438400640 1 -- 172.21.15.1:0/1541562235 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f043019f870 con 0x7f04301049a0 2024-09-12T01:28:28.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.242+0000 7f042e000640 1 -- 172.21.15.1:0/1541562235 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c018960 con 0x7f04301049a0 2024-09-12T01:28:28.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.243+0000 7f0438400640 1 -- 172.21.15.1:0/1541562235 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04301a7150 con 0x7f04301049a0 2024-09-12T01:28:28.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.243+0000 7f0438400640 1 -- 172.21.15.1:0/1541562235 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0430062940 con 0x7f04301049a0 2024-09-12T01:28:28.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.246+0000 7f042e000640 1 -- 172.21.15.1:0/1541562235 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f041c010ac0 con 0x7f04301049a0 2024-09-12T01:28:28.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.247+0000 7f042e000640 1 --2- 172.21.15.1:0/1541562235 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f04180a2c90 0x7f04180a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:28.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.247+0000 7f042e000640 1 -- 172.21.15.1:0/1541562235 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f041c0cfff0 con 0x7f04301049a0 2024-09-12T01:28:28.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.247+0000 7f0435a00640 1 --2- 172.21.15.1:0/1541562235 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f04180a2c90 0x7f04180a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:28.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.247+0000 7f042e000640 1 --2- 172.21.15.1:0/1541562235 >> [v2:172.21.15.1:6802/1125335136,v1:172.21.15.1:6803/1125335136] conn(0x7f04180adf60 0x7f04180b0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:28.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.247+0000 7f042e000640 1 -- 172.21.15.1:0/1541562235 --> [v2:172.21.15.1:6802/1125335136,v1:172.21.15.1:6803/1125335136] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f041c028ea0 con 0x7f04180adf60 2024-09-12T01:28:28.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.247+0000 7f042e000640 1 -- 172.21.15.1:0/1541562235 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7f041c0d0440 con 0x7f04301049a0 2024-09-12T01:28:28.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.248+0000 7f0436400640 1 --2- 172.21.15.1:0/1541562235 >> [v2:172.21.15.1:6802/1125335136,v1:172.21.15.1:6803/1125335136] conn(0x7f04180adf60 0x7f04180b0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:28.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.248+0000 7f0435a00640 1 --2- 172.21.15.1:0/1541562235 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f04180a2c90 0x7f04180a5150 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0424011550 tx=0x7f0424005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:28.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.248+0000 7f0436400640 1 --2- 172.21.15.1:0/1541562235 >> [v2:172.21.15.1:6802/1125335136,v1:172.21.15.1:6803/1125335136] conn(0x7f04180adf60 0x7f04180b0400 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:28.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.249+0000 7f042e000640 1 -- 172.21.15.1:0/1541562235 <== osd.0 v2:172.21.15.1:6802/1125335136 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f041c028ea0 con 0x7f04180adf60 2024-09-12T01:28:28.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.264+0000 7f0438400640 1 -- 172.21.15.1:0/1541562235 --> [v2:172.21.15.1:6802/1125335136,v1:172.21.15.1:6803/1125335136] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0430062d40 con 0x7f04180adf60 2024-09-12T01:28:28.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.264+0000 7f042e000640 1 -- 172.21.15.1:0/1541562235 <== osd.0 v2:172.21.15.1:6802/1125335136 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0430062d40 con 0x7f04180adf60 2024-09-12T01:28:28.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.264+0000 7f0438400640 1 -- 172.21.15.1:0/1541562235 >> [v2:172.21.15.1:6802/1125335136,v1:172.21.15.1:6803/1125335136] conn(0x7f04180adf60 msgr2=0x7f04180b0400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:28.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.264+0000 7f0438400640 1 --2- 172.21.15.1:0/1541562235 >> [v2:172.21.15.1:6802/1125335136,v1:172.21.15.1:6803/1125335136] conn(0x7f04180adf60 0x7f04180b0400 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:28.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.265+0000 7f0438400640 1 -- 172.21.15.1:0/1541562235 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f04180a2c90 msgr2=0x7f04180a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:28.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.265+0000 7f0438400640 1 --2- 172.21.15.1:0/1541562235 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f04180a2c90 0x7f04180a5150 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0424011550 tx=0x7f0424005f90 comp rx=0 tx=0).stop 2024-09-12T01:28:28.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.265+0000 7f0438400640 1 -- 172.21.15.1:0/1541562235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04301049a0 msgr2=0x7f043019ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:28.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.265+0000 7f0438400640 1 --2- 172.21.15.1:0/1541562235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04301049a0 0x7f043019ab10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f041c00e9f0 tx=0x7f041c00eeb0 comp rx=0 tx=0).stop 2024-09-12T01:28:28.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.265+0000 7f0436400640 1 -- 172.21.15.1:0/1541562235 reap_dead start 2024-09-12T01:28:28.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.265+0000 7f0438400640 1 -- 172.21.15.1:0/1541562235 shutdown_connections 2024-09-12T01:28:28.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.265+0000 7f0438400640 1 -- 172.21.15.1:0/1541562235 >> 172.21.15.1:0/1541562235 conn(0x7f04300ffa90 msgr2=0x7f0430100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:28.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.265+0000 7f0438400640 1 -- 172.21.15.1:0/1541562235 shutdown_connections 2024-09-12T01:28:28.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:20.265+0000 7f0438400640 1 -- 172.21.15.1:0/1541562235 wait complete. 2024-09-12T01:28:28.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:28 smithi104 ceph-mon[35219]: pgmap v364: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-09-12T01:28:28.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:28 smithi001 ceph-mon[30387]: pgmap v364: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-09-12T01:28:28.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:28 smithi053 ceph-mon[35158]: pgmap v364: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-09-12T01:28:29.071 INFO:teuthology.orchestra.run.smithi001.stdout:34359738462 2024-09-12T01:28:29.071 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-12T01:28:29.723 INFO:teuthology.orchestra.run.smithi001.stdout:309237645349 2024-09-12T01:28:29.723 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-12T01:28:30.252 INFO:teuthology.orchestra.run.smithi001.stdout:77309411412 2024-09-12T01:28:30.252 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-12T01:28:30.714 INFO:teuthology.orchestra.run.smithi001.stdout:339302416418 2024-09-12T01:28:30.715 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-12T01:28:30.820 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:30 smithi001 ceph-mon[30387]: pgmap v365: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-12T01:28:30.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:30 smithi104 ceph-mon[35219]: pgmap v365: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-12T01:28:30.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:30 smithi053 ceph-mon[35158]: pgmap v365: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-12T01:28:31.216 INFO:teuthology.orchestra.run.smithi001.stdout:55834574937 2024-09-12T01:28:31.217 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-12T01:28:31.888 INFO:teuthology.orchestra.run.smithi001.stdout:141733920841 2024-09-12T01:28:31.889 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-12T01:28:31.959 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:28:31.960 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:28:31.960 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:28:31.972 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:28:31.972 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:28:32.299 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:28:32.300 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:28:32.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:32 smithi001 ceph-mon[30387]: pgmap v366: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-09-12T01:28:32.792 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:32 smithi053 ceph-mon[35158]: pgmap v366: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-09-12T01:28:32.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:32 smithi104 ceph-mon[35219]: pgmap v366: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-09-12T01:28:32.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.843+0000 7f1d9f200640 1 -- 172.21.15.1:0/3802157251 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1da01019c0 msgr2=0x7f1da0101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:32.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.843+0000 7f1d9f200640 1 --2- 172.21.15.1:0/3802157251 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1da01019c0 0x7f1da0101e20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1d88009930 tx=0x7f1d8802a420 comp rx=0 tx=0).stop 2024-09-12T01:28:32.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.843+0000 7f1d9f200640 1 -- 172.21.15.1:0/3802157251 shutdown_connections 2024-09-12T01:28:32.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.843+0000 7f1d9f200640 1 --2- 172.21.15.1:0/3802157251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1da0102360 0x7f1da010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:32.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.843+0000 7f1d9f200640 1 --2- 172.21.15.1:0/3802157251 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1da01019c0 0x7f1da0101e20 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:32.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.843+0000 7f1d9f200640 1 --2- 172.21.15.1:0/3802157251 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1da01079c0 0x7f1da0107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:32.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.843+0000 7f1d9f200640 1 -- 172.21.15.1:0/3802157251 >> 172.21.15.1:0/3802157251 conn(0x7f1da00fd740 msgr2=0x7f1da00ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:32.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.843+0000 7f1d9f200640 1 -- 172.21.15.1:0/3802157251 shutdown_connections 2024-09-12T01:28:32.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.843+0000 7f1d9f200640 1 -- 172.21.15.1:0/3802157251 wait complete. 2024-09-12T01:28:32.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.844+0000 7f1d9f200640 1 Processor -- start 2024-09-12T01:28:32.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.844+0000 7f1d9f200640 1 -- start start 2024-09-12T01:28:32.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.845+0000 7f1d9f200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1da01019c0 0x7f1da01982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:32.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.845+0000 7f1d9f200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1da0102360 0x7f1da0198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:32.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.845+0000 7f1d9f200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1da01079c0 0x7f1da019cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:32.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.845+0000 7f1d9f200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1da0111e10 con 0x7f1da01019c0 2024-09-12T01:28:32.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.845+0000 7f1d9f200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1da0111c90 con 0x7f1da0102360 2024-09-12T01:28:32.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.845+0000 7f1d9f200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1da0111f90 con 0x7f1da01079c0 2024-09-12T01:28:32.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.845+0000 7f1d9de00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1da01019c0 0x7f1da01982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:32.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.845+0000 7f1d95400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1da0102360 0x7f1da0198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:32.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.845+0000 7f1d95400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1da0102360 0x7f1da0198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:45404/0 (socket says 172.21.15.1:45404) 2024-09-12T01:28:32.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.845+0000 7f1d95400640 1 -- 172.21.15.1:0/786791368 learned_addr learned my addr 172.21.15.1:0/786791368 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:28:32.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.845+0000 7f1d9e800640 1 --2- 172.21.15.1:0/786791368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1da01079c0 0x7f1da019cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:32.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.846+0000 7f1d9de00640 1 -- 172.21.15.1:0/786791368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1da01079c0 msgr2=0x7f1da019cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:32.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.846+0000 7f1d9de00640 1 --2- 172.21.15.1:0/786791368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1da01079c0 0x7f1da019cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:32.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.846+0000 7f1d9de00640 1 -- 172.21.15.1:0/786791368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1da0102360 msgr2=0x7f1da0198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:32.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.846+0000 7f1d9de00640 1 --2- 172.21.15.1:0/786791368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1da0102360 0x7f1da0198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:32.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.846+0000 7f1d9de00640 1 -- 172.21.15.1:0/786791368 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da019d2a0 con 0x7f1da01019c0 2024-09-12T01:28:32.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.846+0000 7f1d95400640 1 --2- 172.21.15.1:0/786791368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1da0102360 0x7f1da0198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:28:32.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.846+0000 7f1d9de00640 1 --2- 172.21.15.1:0/786791368 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1da01019c0 0x7f1da01982f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1d90009fd0 tx=0x7f1d9000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:32.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.846+0000 7f1d97400640 1 -- 172.21.15.1:0/786791368 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d90004b20 con 0x7f1da01019c0 2024-09-12T01:28:32.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.846+0000 7f1d97400640 1 -- 172.21.15.1:0/786791368 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d90004cc0 con 0x7f1da01019c0 2024-09-12T01:28:32.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.846+0000 7f1d97400640 1 -- 172.21.15.1:0/786791368 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d900107b0 con 0x7f1da01019c0 2024-09-12T01:28:32.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.846+0000 7f1d9f200640 1 -- 172.21.15.1:0/786791368 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1da019d590 con 0x7f1da01019c0 2024-09-12T01:28:32.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.846+0000 7f1d9f200640 1 -- 172.21.15.1:0/786791368 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1da01a4e70 con 0x7f1da01019c0 2024-09-12T01:28:32.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.847+0000 7f1d9f200640 1 -- 172.21.15.1:0/786791368 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1da0101e20 con 0x7f1da01019c0 2024-09-12T01:28:32.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.849+0000 7f1d97400640 1 -- 172.21.15.1:0/786791368 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f1d90005020 con 0x7f1da01019c0 2024-09-12T01:28:32.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.851+0000 7f1d97400640 1 --2- 172.21.15.1:0/786791368 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1d840a2c90 0x7f1d840a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:32.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.851+0000 7f1d97400640 1 -- 172.21.15.1:0/786791368 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f1d900cfc30 con 0x7f1da01019c0 2024-09-12T01:28:32.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.851+0000 7f1d95400640 1 --2- 172.21.15.1:0/786791368 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1d840a2c90 0x7f1d840a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:32.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.851+0000 7f1d95400640 1 --2- 172.21.15.1:0/786791368 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1d840a2c90 0x7f1d840a5150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1da00ffd00 tx=0x7f1d88031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:32.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.855+0000 7f1d97400640 1 -- 172.21.15.1:0/786791368 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1d9009c640 con 0x7f1da01019c0 2024-09-12T01:28:32.967 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:28:32.968 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:28:33.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:33.056+0000 7f1d9f200640 1 -- 172.21.15.1:0/786791368 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f1da0109fb0 con 0x7f1da01019c0 2024-09-12T01:28:33.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:33.057+0000 7f1d97400640 1 -- 172.21.15.1:0/786791368 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1d9009bf40 con 0x7f1da01019c0 2024-09-12T01:28:33.057 INFO:teuthology.orchestra.run.smithi001.stdout:287762808877 2024-09-12T01:28:33.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:33.059+0000 7f1d9f200640 1 -- 172.21.15.1:0/786791368 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1d840a2c90 msgr2=0x7f1d840a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:33.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:33.059+0000 7f1d9f200640 1 --2- 172.21.15.1:0/786791368 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1d840a2c90 0x7f1d840a5150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1da00ffd00 tx=0x7f1d88031510 comp rx=0 tx=0).stop 2024-09-12T01:28:33.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:33.059+0000 7f1d9f200640 1 -- 172.21.15.1:0/786791368 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1da01019c0 msgr2=0x7f1da01982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:33.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:33.059+0000 7f1d9f200640 1 --2- 172.21.15.1:0/786791368 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1da01019c0 0x7f1da01982f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1d90009fd0 tx=0x7f1d9000ef90 comp rx=0 tx=0).stop 2024-09-12T01:28:33.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:33.059+0000 7f1d9f200640 1 -- 172.21.15.1:0/786791368 shutdown_connections 2024-09-12T01:28:33.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:33.059+0000 7f1d9f200640 1 --2- 172.21.15.1:0/786791368 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1d840a2c90 0x7f1d840a5150 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:33.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:33.059+0000 7f1d9f200640 1 --2- 172.21.15.1:0/786791368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1da01079c0 0x7f1da019cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:33.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:33.059+0000 7f1d9f200640 1 --2- 172.21.15.1:0/786791368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1da0102360 0x7f1da0198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:33.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:33.059+0000 7f1d9f200640 1 --2- 172.21.15.1:0/786791368 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1da01019c0 0x7f1da01982f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:33.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:33.059+0000 7f1d9f200640 1 -- 172.21.15.1:0/786791368 >> 172.21.15.1:0/786791368 conn(0x7f1da00fd740 msgr2=0x7f1da00fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:33.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:33.059+0000 7f1d9f200640 1 -- 172.21.15.1:0/786791368 shutdown_connections 2024-09-12T01:28:33.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:33.060+0000 7f1d9f200640 1 -- 172.21.15.1:0/786791368 wait complete. 2024-09-12T01:28:33.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.982+0000 7fa744c00640 1 -- 172.21.15.1:0/3219293426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7401019c0 msgr2=0x7fa740101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:33.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.982+0000 7fa744c00640 1 --2- 172.21.15.1:0/3219293426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7401019c0 0x7fa740101e20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa72c009900 tx=0x7fa72c02a3f0 comp rx=0 tx=0).stop 2024-09-12T01:28:33.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.982+0000 7fa744c00640 1 -- 172.21.15.1:0/3219293426 shutdown_connections 2024-09-12T01:28:33.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.982+0000 7fa744c00640 1 --2- 172.21.15.1:0/3219293426 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa740102360 0x7fa74010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:33.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.982+0000 7fa744c00640 1 --2- 172.21.15.1:0/3219293426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7401019c0 0x7fa740101e20 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:33.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.982+0000 7fa744c00640 1 --2- 172.21.15.1:0/3219293426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7401079c0 0x7fa740107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:33.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.982+0000 7fa744c00640 1 -- 172.21.15.1:0/3219293426 >> 172.21.15.1:0/3219293426 conn(0x7fa7400fd740 msgr2=0x7fa7400ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:33.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.982+0000 7fa744c00640 1 -- 172.21.15.1:0/3219293426 shutdown_connections 2024-09-12T01:28:33.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.982+0000 7fa744c00640 1 -- 172.21.15.1:0/3219293426 wait complete. 2024-09-12T01:28:33.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.984+0000 7fa744c00640 1 Processor -- start 2024-09-12T01:28:33.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.984+0000 7fa744c00640 1 -- start start 2024-09-12T01:28:33.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.984+0000 7fa744c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7401019c0 0x7fa7400739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:33.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.984+0000 7fa744c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa740102360 0x7fa74006ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:33.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.984+0000 7fa744c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7401079c0 0x7fa74006cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:33.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.985+0000 7fa744c00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa740111e10 con 0x7fa7401019c0 2024-09-12T01:28:33.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.985+0000 7fa744c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa740111c90 con 0x7fa7401079c0 2024-09-12T01:28:33.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.985+0000 7fa744c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa740111f90 con 0x7fa740102360 2024-09-12T01:28:33.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.985+0000 7fa73ea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa740102360 0x7fa74006ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:33.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.985+0000 7fa73f400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7401019c0 0x7fa7400739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:33.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.985+0000 7fa73ea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa740102360 0x7fa74006ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:56268/0 (socket says 172.21.15.1:56268) 2024-09-12T01:28:33.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.985+0000 7fa73ea00640 1 -- 172.21.15.1:0/2322339958 learned_addr learned my addr 172.21.15.1:0/2322339958 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:28:33.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.985+0000 7fa73fe00640 1 --2- 172.21.15.1:0/2322339958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7401079c0 0x7fa74006cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:33.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.985+0000 7fa73ea00640 1 -- 172.21.15.1:0/2322339958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7401079c0 msgr2=0x7fa74006cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:33.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.985+0000 7fa73ea00640 1 --2- 172.21.15.1:0/2322339958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7401079c0 0x7fa74006cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:33.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.985+0000 7fa73ea00640 1 -- 172.21.15.1:0/2322339958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7401019c0 msgr2=0x7fa7400739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:33.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.986+0000 7fa73ea00640 1 --2- 172.21.15.1:0/2322339958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7401019c0 0x7fa7400739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:33.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.986+0000 7fa73ea00640 1 -- 172.21.15.1:0/2322339958 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa74006d550 con 0x7fa740102360 2024-09-12T01:28:33.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.986+0000 7fa73f400640 1 --2- 172.21.15.1:0/2322339958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7401019c0 0x7fa7400739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:28:33.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.986+0000 7fa73ea00640 1 --2- 172.21.15.1:0/2322339958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa740102360 0x7fa74006ca50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa72c02a9e0 tx=0x7fa72c003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:33.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.986+0000 7fa737e00640 1 -- 172.21.15.1:0/2322339958 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72c03f070 con 0x7fa740102360 2024-09-12T01:28:33.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.986+0000 7fa737e00640 1 -- 172.21.15.1:0/2322339958 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa72c031c30 con 0x7fa740102360 2024-09-12T01:28:33.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.986+0000 7fa737e00640 1 -- 172.21.15.1:0/2322339958 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72c035030 con 0x7fa740102360 2024-09-12T01:28:33.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.986+0000 7fa744c00640 1 -- 172.21.15.1:0/2322339958 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa74006d7e0 con 0x7fa740102360 2024-09-12T01:28:33.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.987+0000 7fa744c00640 1 -- 172.21.15.1:0/2322339958 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa740071720 con 0x7fa740102360 2024-09-12T01:28:33.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.988+0000 7fa744c00640 1 -- 172.21.15.1:0/2322339958 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa740103100 con 0x7fa740102360 2024-09-12T01:28:33.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.994+0000 7fa737e00640 1 -- 172.21.15.1:0/2322339958 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fa72c042070 con 0x7fa740102360 2024-09-12T01:28:33.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.995+0000 7fa737e00640 1 --2- 172.21.15.1:0/2322339958 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fa7240a2c90 0x7fa7240a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:33.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.995+0000 7fa737e00640 1 -- 172.21.15.1:0/2322339958 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fa72c0f05b0 con 0x7fa740102360 2024-09-12T01:28:33.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.996+0000 7fa737e00640 1 -- 172.21.15.1:0/2322339958 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa72c049030 con 0x7fa740102360 2024-09-12T01:28:33.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.996+0000 7fa73f400640 1 --2- 172.21.15.1:0/2322339958 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fa7240a2c90 0x7fa7240a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:33.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:32.997+0000 7fa73f400640 1 --2- 172.21.15.1:0/2322339958 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fa7240a2c90 0x7fa7240a5150 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa730009f40 tx=0x7fa730005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:33.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:33.201+0000 7fa744c00640 1 -- 172.21.15.1:0/2322339958 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fa740072200 con 0x7fa740102360 2024-09-12T01:28:33.201 INFO:teuthology.orchestra.run.smithi001.stdout:107374182479 2024-09-12T01:28:33.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:33.202+0000 7fa737e00640 1 -- 172.21.15.1:0/2322339958 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa72c0bcfc0 con 0x7fa740102360 2024-09-12T01:28:33.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:33.204+0000 7fa744c00640 1 -- 172.21.15.1:0/2322339958 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fa7240a2c90 msgr2=0x7fa7240a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:33.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:33.204+0000 7fa744c00640 1 --2- 172.21.15.1:0/2322339958 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fa7240a2c90 0x7fa7240a5150 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa730009f40 tx=0x7fa730005f90 comp rx=0 tx=0).stop 2024-09-12T01:28:33.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:33.204+0000 7fa744c00640 1 -- 172.21.15.1:0/2322339958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa740102360 msgr2=0x7fa74006ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:33.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:33.204+0000 7fa744c00640 1 --2- 172.21.15.1:0/2322339958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa740102360 0x7fa74006ca50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa72c02a9e0 tx=0x7fa72c003c10 comp rx=0 tx=0).stop 2024-09-12T01:28:33.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:33.204+0000 7fa744c00640 1 -- 172.21.15.1:0/2322339958 shutdown_connections 2024-09-12T01:28:33.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:33.204+0000 7fa744c00640 1 --2- 172.21.15.1:0/2322339958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7401079c0 0x7fa74006cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:33.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:33.204+0000 7fa744c00640 1 --2- 172.21.15.1:0/2322339958 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fa7240a2c90 0x7fa7240a5150 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:33.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:33.204+0000 7fa744c00640 1 --2- 172.21.15.1:0/2322339958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa740102360 0x7fa74006ca50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:33.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:33.204+0000 7fa744c00640 1 --2- 172.21.15.1:0/2322339958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7401019c0 0x7fa7400739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:33.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:33.204+0000 7fa744c00640 1 -- 172.21.15.1:0/2322339958 >> 172.21.15.1:0/2322339958 conn(0x7fa7400fd740 msgr2=0x7fa7400fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:33.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:33.204+0000 7fa744c00640 1 -- 172.21.15.1:0/2322339958 shutdown_connections 2024-09-12T01:28:33.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:33.205+0000 7fa744c00640 1 -- 172.21.15.1:0/2322339958 wait complete. 2024-09-12T01:28:33.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:33 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/786791368' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-12T01:28:33.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:33 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/2322339958' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-12T01:28:33.849 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:33 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/786791368' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-12T01:28:33.849 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:33 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/2322339958' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-12T01:28:33.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:33 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/786791368' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-12T01:28:33.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:33 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/2322339958' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-12T01:28:34.070 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:28:34.646 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182476 got 107374182479 for osd.3 2024-09-12T01:28:34.646 DEBUG:teuthology.parallel:result is None 2024-09-12T01:28:34.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:34 smithi104 ceph-mon[35219]: pgmap v367: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-12T01:28:34.883 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:34 smithi001 ceph-mon[30387]: pgmap v367: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-12T01:28:34.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:34 smithi053 ceph-mon[35158]: pgmap v367: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-12T01:28:35.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.195+0000 7f9adee00640 1 -- 172.21.15.1:0/2586406904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ad8109140 msgr2=0x7f9ad8110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:35.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.195+0000 7f9adee00640 1 --2- 172.21.15.1:0/2586406904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ad8109140 0x7f9ad8110cc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0009ca0 tx=0x7f9ad002a8c0 comp rx=0 tx=0).stop 2024-09-12T01:28:35.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.195+0000 7f9adee00640 1 -- 172.21.15.1:0/2586406904 shutdown_connections 2024-09-12T01:28:35.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.195+0000 7f9adee00640 1 --2- 172.21.15.1:0/2586406904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ad8109140 0x7f9ad8110cc0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:35.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.195+0000 7f9adee00640 1 --2- 172.21.15.1:0/2586406904 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ad81049c0 0x7f9ad8108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:35.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.195+0000 7f9adee00640 1 --2- 172.21.15.1:0/2586406904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ad8104010 0x7f9ad81043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:35.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.195+0000 7f9adee00640 1 -- 172.21.15.1:0/2586406904 >> 172.21.15.1:0/2586406904 conn(0x7f9ad80ffa50 msgr2=0x7f9ad8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:35.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.195+0000 7f9adee00640 1 -- 172.21.15.1:0/2586406904 shutdown_connections 2024-09-12T01:28:35.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.195+0000 7f9adee00640 1 -- 172.21.15.1:0/2586406904 wait complete. 2024-09-12T01:28:35.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.196+0000 7f9adee00640 1 Processor -- start 2024-09-12T01:28:35.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.196+0000 7f9adee00640 1 -- start start 2024-09-12T01:28:35.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.197+0000 7f9adee00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ad8104010 0x7f9ad819a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:35.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.197+0000 7f9adee00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ad81049c0 0x7f9ad819ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:35.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.197+0000 7f9adee00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ad8109140 0x7f9ad819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:35.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.197+0000 7f9adee00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad8114170 con 0x7f9ad8104010 2024-09-12T01:28:35.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.197+0000 7f9adee00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad8113ff0 con 0x7f9ad81049c0 2024-09-12T01:28:35.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.197+0000 7f9adee00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad81142f0 con 0x7f9ad8109140 2024-09-12T01:28:35.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.197+0000 7f9ad7e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ad8104010 0x7f9ad819a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:35.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.197+0000 7f9ad7e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ad8104010 0x7f9ad819a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38158/0 (socket says 172.21.15.1:38158) 2024-09-12T01:28:35.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.197+0000 7f9ad7e00640 1 -- 172.21.15.1:0/1102019320 learned_addr learned my addr 172.21.15.1:0/1102019320 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:28:35.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.197+0000 7f9ad7400640 1 --2- 172.21.15.1:0/1102019320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ad81049c0 0x7f9ad819ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:35.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.197+0000 7f9adce00640 1 --2- 172.21.15.1:0/1102019320 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ad8109140 0x7f9ad819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:35.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.197+0000 7f9ad7400640 1 -- 172.21.15.1:0/1102019320 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ad8109140 msgr2=0x7f9ad819efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:35.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.197+0000 7f9ad7400640 1 --2- 172.21.15.1:0/1102019320 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ad8109140 0x7f9ad819efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:35.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.197+0000 7f9ad7400640 1 -- 172.21.15.1:0/1102019320 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ad8104010 msgr2=0x7f9ad819a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:35.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.197+0000 7f9ad7400640 1 --2- 172.21.15.1:0/1102019320 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ad8104010 0x7f9ad819a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:35.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.197+0000 7f9ad7400640 1 -- 172.21.15.1:0/1102019320 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ad819f680 con 0x7f9ad81049c0 2024-09-12T01:28:35.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.198+0000 7f9ad7400640 1 --2- 172.21.15.1:0/1102019320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ad81049c0 0x7f9ad819ac10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9ac400e9f0 tx=0x7f9ac400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:35.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.198+0000 7f9ad4c00640 1 -- 172.21.15.1:0/1102019320 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac400cd80 con 0x7f9ad81049c0 2024-09-12T01:28:35.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.198+0000 7f9ad4c00640 1 -- 172.21.15.1:0/1102019320 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ac40035f0 con 0x7f9ad81049c0 2024-09-12T01:28:35.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.198+0000 7f9adee00640 1 -- 172.21.15.1:0/1102019320 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ad819f970 con 0x7f9ad81049c0 2024-09-12T01:28:35.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.198+0000 7f9ad4c00640 1 -- 172.21.15.1:0/1102019320 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac4010690 con 0x7f9ad81049c0 2024-09-12T01:28:35.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.198+0000 7f9adee00640 1 -- 172.21.15.1:0/1102019320 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ad81a7250 con 0x7f9ad81049c0 2024-09-12T01:28:35.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.199+0000 7f9adee00640 1 -- 172.21.15.1:0/1102019320 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ad81043f0 con 0x7f9ad81049c0 2024-09-12T01:28:35.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.202+0000 7f9ad4c00640 1 -- 172.21.15.1:0/1102019320 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f9ac4003b30 con 0x7f9ad81049c0 2024-09-12T01:28:35.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.203+0000 7f9ad4c00640 1 --2- 172.21.15.1:0/1102019320 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f9abc0a2c10 0x7f9abc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:35.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.203+0000 7f9ad4c00640 1 -- 172.21.15.1:0/1102019320 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f9ac40cf840 con 0x7f9ad81049c0 2024-09-12T01:28:35.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.204+0000 7f9ad7e00640 1 --2- 172.21.15.1:0/1102019320 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f9abc0a2c10 0x7f9abc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:35.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.204+0000 7f9ad7e00640 1 --2- 172.21.15.1:0/1102019320 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f9abc0a2c10 0x7f9abc0a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8009cd0 tx=0x7f9ac8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:35.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.208+0000 7f9ad4c00640 1 -- 172.21.15.1:0/1102019320 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9ac409c250 con 0x7f9ad81049c0 2024-09-12T01:28:35.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.403+0000 7f9adee00640 1 -- 172.21.15.1:0/1102019320 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f9ad810bdf0 con 0x7f9ad81049c0 2024-09-12T01:28:35.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.404+0000 7f9ad4c00640 1 -- 172.21.15.1:0/1102019320 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9ac409bb50 con 0x7f9ad81049c0 2024-09-12T01:28:35.403 INFO:teuthology.orchestra.run.smithi001.stdout:34359738464 2024-09-12T01:28:35.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.406+0000 7f9adee00640 1 -- 172.21.15.1:0/1102019320 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f9abc0a2c10 msgr2=0x7f9abc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:35.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.406+0000 7f9adee00640 1 --2- 172.21.15.1:0/1102019320 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f9abc0a2c10 0x7f9abc0a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8009cd0 tx=0x7f9ac8005f90 comp rx=0 tx=0).stop 2024-09-12T01:28:35.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.406+0000 7f9adee00640 1 -- 172.21.15.1:0/1102019320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ad81049c0 msgr2=0x7f9ad819ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:35.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.406+0000 7f9adee00640 1 --2- 172.21.15.1:0/1102019320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ad81049c0 0x7f9ad819ac10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9ac400e9f0 tx=0x7f9ac400eeb0 comp rx=0 tx=0).stop 2024-09-12T01:28:35.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.406+0000 7f9adee00640 1 -- 172.21.15.1:0/1102019320 shutdown_connections 2024-09-12T01:28:35.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.406+0000 7f9adee00640 1 --2- 172.21.15.1:0/1102019320 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f9abc0a2c10 0x7f9abc0a50d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:35.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.406+0000 7f9adee00640 1 --2- 172.21.15.1:0/1102019320 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ad8109140 0x7f9ad819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:35.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.406+0000 7f9adee00640 1 --2- 172.21.15.1:0/1102019320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ad81049c0 0x7f9ad819ac10 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:35.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.406+0000 7f9adee00640 1 --2- 172.21.15.1:0/1102019320 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ad8104010 0x7f9ad819a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:35.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.406+0000 7f9adee00640 1 -- 172.21.15.1:0/1102019320 >> 172.21.15.1:0/1102019320 conn(0x7f9ad80ffa50 msgr2=0x7f9ad8100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:35.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.406+0000 7f9adee00640 1 -- 172.21.15.1:0/1102019320 shutdown_connections 2024-09-12T01:28:35.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.406+0000 7f9adee00640 1 -- 172.21.15.1:0/1102019320 wait complete. 2024-09-12T01:28:35.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.431+0000 7fedf9600640 1 -- 172.21.15.1:0/400562847 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fedf41079c0 msgr2=0x7fedf4107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:35.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.431+0000 7fedf9600640 1 --2- 172.21.15.1:0/400562847 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fedf41079c0 0x7fedf4107da0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fede4009930 tx=0x7fede402a440 comp rx=0 tx=0).stop 2024-09-12T01:28:35.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.432+0000 7fedf9600640 1 -- 172.21.15.1:0/400562847 shutdown_connections 2024-09-12T01:28:35.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.432+0000 7fedf9600640 1 --2- 172.21.15.1:0/400562847 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fedf4102360 0x7fedf410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:35.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.432+0000 7fedf9600640 1 --2- 172.21.15.1:0/400562847 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fedf41019c0 0x7fedf4101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:35.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.432+0000 7fedf9600640 1 --2- 172.21.15.1:0/400562847 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fedf41079c0 0x7fedf4107da0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:35.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.432+0000 7fedf9600640 1 -- 172.21.15.1:0/400562847 >> 172.21.15.1:0/400562847 conn(0x7fedf40fd740 msgr2=0x7fedf40ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:35.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.432+0000 7fedf9600640 1 -- 172.21.15.1:0/400562847 shutdown_connections 2024-09-12T01:28:35.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.432+0000 7fedf9600640 1 -- 172.21.15.1:0/400562847 wait complete. 2024-09-12T01:28:35.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.433+0000 7fedf9600640 1 Processor -- start 2024-09-12T01:28:35.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.433+0000 7fedf9600640 1 -- start start 2024-09-12T01:28:35.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.433+0000 7fedf9600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fedf41019c0 0x7fedf40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:35.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.433+0000 7fedf9600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fedf4102360 0x7fedf4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:35.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.433+0000 7fedf9600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fedf41079c0 0x7fedf4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:35.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.433+0000 7fedf9600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fedf4111dd0 con 0x7fedf41079c0 2024-09-12T01:28:35.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.433+0000 7fedf9600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fedf4111c50 con 0x7fedf4102360 2024-09-12T01:28:35.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.433+0000 7fedf9600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fedf4111f50 con 0x7fedf41019c0 2024-09-12T01:28:35.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.434+0000 7fedf3400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fedf4102360 0x7fedf4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:35.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.434+0000 7fedf3400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fedf4102360 0x7fedf4074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:45448/0 (socket says 172.21.15.1:45448) 2024-09-12T01:28:35.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.434+0000 7fedf3400640 1 -- 172.21.15.1:0/1312624210 learned_addr learned my addr 172.21.15.1:0/1312624210 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:28:35.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.434+0000 7fedf8c00640 1 --2- 172.21.15.1:0/1312624210 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fedf41079c0 0x7fedf4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:35.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.434+0000 7fedf3e00640 1 --2- 172.21.15.1:0/1312624210 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fedf41019c0 0x7fedf40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:35.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.434+0000 7fedf3400640 1 -- 172.21.15.1:0/1312624210 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fedf41019c0 msgr2=0x7fedf40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:35.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.434+0000 7fedf3400640 1 --2- 172.21.15.1:0/1312624210 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fedf41019c0 0x7fedf40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:35.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.434+0000 7fedf3400640 1 -- 172.21.15.1:0/1312624210 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fedf41079c0 msgr2=0x7fedf4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:35.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.434+0000 7fedf3400640 1 --2- 172.21.15.1:0/1312624210 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fedf41079c0 0x7fedf4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:35.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.434+0000 7fedf3400640 1 -- 172.21.15.1:0/1312624210 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedf41a6790 con 0x7fedf4102360 2024-09-12T01:28:35.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.434+0000 7fedf3400640 1 --2- 172.21.15.1:0/1312624210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fedf4102360 0x7fedf4074a30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fede000d950 tx=0x7fede000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:35.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.435+0000 7fedf0c00640 1 -- 172.21.15.1:0/1312624210 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede0017070 con 0x7fedf4102360 2024-09-12T01:28:35.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.435+0000 7fedf0c00640 1 -- 172.21.15.1:0/1312624210 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fede0009c30 con 0x7fedf4102360 2024-09-12T01:28:35.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.435+0000 7fedf9600640 1 -- 172.21.15.1:0/1312624210 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedf41a6a80 con 0x7fedf4102360 2024-09-12T01:28:35.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.435+0000 7fedf0c00640 1 -- 172.21.15.1:0/1312624210 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede000f6b0 con 0x7fedf4102360 2024-09-12T01:28:35.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.435+0000 7fedf9600640 1 -- 172.21.15.1:0/1312624210 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedf41a7010 con 0x7fedf4102360 2024-09-12T01:28:35.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.436+0000 7fedf9600640 1 -- 172.21.15.1:0/1312624210 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedf4101e20 con 0x7fedf4102360 2024-09-12T01:28:35.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.439+0000 7fedf0c00640 1 -- 172.21.15.1:0/1312624210 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fede00109d0 con 0x7fedf4102360 2024-09-12T01:28:35.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.440+0000 7fedf0c00640 1 --2- 172.21.15.1:0/1312624210 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fedd80a2c90 0x7fedd80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:35.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.440+0000 7fedf0c00640 1 -- 172.21.15.1:0/1312624210 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fede00cf180 con 0x7fedf4102360 2024-09-12T01:28:35.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.440+0000 7fedf3e00640 1 --2- 172.21.15.1:0/1312624210 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fedd80a2c90 0x7fedd80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:35.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.441+0000 7fedf3e00640 1 --2- 172.21.15.1:0/1312624210 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fedd80a2c90 0x7fedd80a5150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fede4006010 tx=0x7fede4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:35.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.442+0000 7fedf0c00640 1 -- 172.21.15.1:0/1312624210 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fede0009de0 con 0x7fedf4102360 2024-09-12T01:28:35.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.643+0000 7fedf9600640 1 -- 172.21.15.1:0/1312624210 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fedf4075a20 con 0x7fedf4102360 2024-09-12T01:28:35.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.643+0000 7fedf0c00640 1 -- 172.21.15.1:0/1312624210 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fede009bc10 con 0x7fedf4102360 2024-09-12T01:28:35.643 INFO:teuthology.orchestra.run.smithi001.stdout:197568495679 2024-09-12T01:28:35.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.645+0000 7fedf9600640 1 -- 172.21.15.1:0/1312624210 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fedd80a2c90 msgr2=0x7fedd80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:35.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.645+0000 7fedf9600640 1 --2- 172.21.15.1:0/1312624210 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fedd80a2c90 0x7fedd80a5150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fede4006010 tx=0x7fede4005a20 comp rx=0 tx=0).stop 2024-09-12T01:28:35.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.645+0000 7fedf9600640 1 -- 172.21.15.1:0/1312624210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fedf4102360 msgr2=0x7fedf4074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:35.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.645+0000 7fedf9600640 1 --2- 172.21.15.1:0/1312624210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fedf4102360 0x7fedf4074a30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fede000d950 tx=0x7fede000de10 comp rx=0 tx=0).stop 2024-09-12T01:28:35.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.645+0000 7fedf9600640 1 -- 172.21.15.1:0/1312624210 shutdown_connections 2024-09-12T01:28:35.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.645+0000 7fedf9600640 1 --2- 172.21.15.1:0/1312624210 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fedf41079c0 0x7fedf4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:35.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.645+0000 7fedf9600640 1 --2- 172.21.15.1:0/1312624210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fedf4102360 0x7fedf4074a30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:35.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.645+0000 7fedf9600640 1 --2- 172.21.15.1:0/1312624210 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fedd80a2c90 0x7fedd80a5150 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:35.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.645+0000 7fedf9600640 1 --2- 172.21.15.1:0/1312624210 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fedf41019c0 0x7fedf40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:35.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.645+0000 7fedf9600640 1 -- 172.21.15.1:0/1312624210 >> 172.21.15.1:0/1312624210 conn(0x7fedf40fd740 msgr2=0x7fedf40fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:35.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.646+0000 7fedf9600640 1 -- 172.21.15.1:0/1312624210 shutdown_connections 2024-09-12T01:28:35.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:35.646+0000 7fedf9600640 1 -- 172.21.15.1:0/1312624210 wait complete. 2024-09-12T01:28:35.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:35 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/1102019320' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-12T01:28:35.828 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:35 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/1102019320' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-12T01:28:35.865 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808876 got 287762808877 for osd.9 2024-09-12T01:28:35.865 DEBUG:teuthology.parallel:result is None 2024-09-12T01:28:35.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:35 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/1102019320' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-12T01:28:36.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.042+0000 7fafa5c00640 1 -- 172.21.15.1:0/3026775806 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fafa01019c0 msgr2=0x7fafa0101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:36.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.042+0000 7fafa5c00640 1 --2- 172.21.15.1:0/3026775806 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fafa01019c0 0x7fafa0101e20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faf8c009890 tx=0x7faf8c02a360 comp rx=0 tx=0).stop 2024-09-12T01:28:36.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.043+0000 7fafa5c00640 1 -- 172.21.15.1:0/3026775806 shutdown_connections 2024-09-12T01:28:36.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.043+0000 7fafa5c00640 1 --2- 172.21.15.1:0/3026775806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafa0102360 0x7fafa010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:36.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.043+0000 7fafa5c00640 1 --2- 172.21.15.1:0/3026775806 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fafa01019c0 0x7fafa0101e20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:36.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.043+0000 7fafa5c00640 1 --2- 172.21.15.1:0/3026775806 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafa01079c0 0x7fafa0107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:36.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.043+0000 7fafa5c00640 1 -- 172.21.15.1:0/3026775806 >> 172.21.15.1:0/3026775806 conn(0x7fafa00fd740 msgr2=0x7fafa00ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:36.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.043+0000 7fafa5c00640 1 -- 172.21.15.1:0/3026775806 shutdown_connections 2024-09-12T01:28:36.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.043+0000 7fafa5c00640 1 -- 172.21.15.1:0/3026775806 wait complete. 2024-09-12T01:28:36.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.044+0000 7fafa5c00640 1 Processor -- start 2024-09-12T01:28:36.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.044+0000 7fafa5c00640 1 -- start start 2024-09-12T01:28:36.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.045+0000 7fafa5c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fafa01019c0 0x7fafa01a34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:36.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.045+0000 7fafa5c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafa0102360 0x7fafa01a39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:36.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.045+0000 7fafa5c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafa01079c0 0x7fafa010da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:36.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.045+0000 7fafa5c00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fafa0111d90 con 0x7fafa01079c0 2024-09-12T01:28:36.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.045+0000 7fafa5c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fafa0111c10 con 0x7fafa0102360 2024-09-12T01:28:36.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.045+0000 7fafa5c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fafa0111f10 con 0x7fafa01019c0 2024-09-12T01:28:36.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.045+0000 7faf9fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fafa01019c0 0x7fafa01a34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:36.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.045+0000 7fafa5200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafa01079c0 0x7fafa010da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:36.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.045+0000 7faf9f400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafa0102360 0x7fafa01a39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:36.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.045+0000 7faf9fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fafa01019c0 0x7fafa01a34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:56340/0 (socket says 172.21.15.1:56340) 2024-09-12T01:28:36.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.045+0000 7faf9fe00640 1 -- 172.21.15.1:0/3898808387 learned_addr learned my addr 172.21.15.1:0/3898808387 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:28:36.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.046+0000 7faf9fe00640 1 -- 172.21.15.1:0/3898808387 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafa0102360 msgr2=0x7fafa01a39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:36.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.046+0000 7faf9fe00640 1 --2- 172.21.15.1:0/3898808387 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafa0102360 0x7fafa01a39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:36.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.046+0000 7faf9fe00640 1 -- 172.21.15.1:0/3898808387 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafa01079c0 msgr2=0x7fafa010da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:36.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.046+0000 7faf9fe00640 1 --2- 172.21.15.1:0/3898808387 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafa01079c0 0x7fafa010da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:36.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.046+0000 7faf9fe00640 1 -- 172.21.15.1:0/3898808387 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa010e190 con 0x7fafa01019c0 2024-09-12T01:28:36.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.046+0000 7faf9fe00640 1 --2- 172.21.15.1:0/3898808387 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fafa01019c0 0x7fafa01a34b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7faf9000e9c0 tx=0x7faf9000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:36.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.046+0000 7faf9cc00640 1 -- 172.21.15.1:0/3898808387 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9000cde0 con 0x7fafa01019c0 2024-09-12T01:28:36.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.047+0000 7faf9cc00640 1 -- 172.21.15.1:0/3898808387 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf900035f0 con 0x7fafa01019c0 2024-09-12T01:28:36.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.047+0000 7fafa5c00640 1 -- 172.21.15.1:0/3898808387 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafa010e480 con 0x7fafa01019c0 2024-09-12T01:28:36.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.047+0000 7faf9cc00640 1 -- 172.21.15.1:0/3898808387 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf90010690 con 0x7fafa01019c0 2024-09-12T01:28:36.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.047+0000 7fafa5c00640 1 -- 172.21.15.1:0/3898808387 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafa01ab3e0 con 0x7fafa01019c0 2024-09-12T01:28:36.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.048+0000 7fafa5c00640 1 -- 172.21.15.1:0/3898808387 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf80005290 con 0x7fafa01019c0 2024-09-12T01:28:36.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.051+0000 7faf9cc00640 1 -- 172.21.15.1:0/3898808387 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7faf90010830 con 0x7fafa01019c0 2024-09-12T01:28:36.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.053+0000 7faf9cc00640 1 --2- 172.21.15.1:0/3898808387 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7faf840a2c90 0x7faf840a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:36.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.053+0000 7faf9cc00640 1 -- 172.21.15.1:0/3898808387 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7faf900cf690 con 0x7fafa01019c0 2024-09-12T01:28:36.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.053+0000 7faf9f400640 1 --2- 172.21.15.1:0/3898808387 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7faf840a2c90 0x7faf840a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:36.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.054+0000 7faf9f400640 1 --2- 172.21.15.1:0/3898808387 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7faf840a2c90 0x7faf840a5150 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7faf8c02a950 tx=0x7faf8c0056d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:36.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.057+0000 7faf9cc00640 1 -- 172.21.15.1:0/3898808387 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faf9009c100 con 0x7fafa01019c0 2024-09-12T01:28:36.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.250+0000 7fafa5c00640 1 -- 172.21.15.1:0/3898808387 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7faf80005850 con 0x7fafa01019c0 2024-09-12T01:28:36.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.250+0000 7faf9cc00640 1 -- 172.21.15.1:0/3898808387 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7faf9009ba00 con 0x7fafa01019c0 2024-09-12T01:28:36.250 INFO:teuthology.orchestra.run.smithi001.stdout:77309411414 2024-09-12T01:28:36.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.252+0000 7fafa5c00640 1 -- 172.21.15.1:0/3898808387 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7faf840a2c90 msgr2=0x7faf840a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:36.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.252+0000 7fafa5c00640 1 --2- 172.21.15.1:0/3898808387 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7faf840a2c90 0x7faf840a5150 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7faf8c02a950 tx=0x7faf8c0056d0 comp rx=0 tx=0).stop 2024-09-12T01:28:36.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.252+0000 7fafa5c00640 1 -- 172.21.15.1:0/3898808387 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fafa01019c0 msgr2=0x7fafa01a34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:36.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.252+0000 7fafa5c00640 1 --2- 172.21.15.1:0/3898808387 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fafa01019c0 0x7fafa01a34b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7faf9000e9c0 tx=0x7faf9000ee80 comp rx=0 tx=0).stop 2024-09-12T01:28:36.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.252+0000 7fafa5c00640 1 -- 172.21.15.1:0/3898808387 shutdown_connections 2024-09-12T01:28:36.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.252+0000 7fafa5c00640 1 --2- 172.21.15.1:0/3898808387 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafa01079c0 0x7fafa010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:36.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.252+0000 7fafa5c00640 1 --2- 172.21.15.1:0/3898808387 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafa0102360 0x7fafa01a39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:36.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.252+0000 7fafa5c00640 1 --2- 172.21.15.1:0/3898808387 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7faf840a2c90 0x7faf840a5150 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:36.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.252+0000 7fafa5c00640 1 --2- 172.21.15.1:0/3898808387 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fafa01019c0 0x7fafa01a34b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:36.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.252+0000 7fafa5c00640 1 -- 172.21.15.1:0/3898808387 >> 172.21.15.1:0/3898808387 conn(0x7fafa00fd740 msgr2=0x7fafa00fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:36.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.253+0000 7fafa5c00640 1 -- 172.21.15.1:0/3898808387 shutdown_connections 2024-09-12T01:28:36.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:36.253+0000 7fafa5c00640 1 -- 172.21.15.1:0/3898808387 wait complete. 2024-09-12T01:28:36.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:36 smithi104 ceph-mon[35219]: pgmap v368: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T01:28:36.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:36 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/1312624210' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-12T01:28:36.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:36 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/3898808387' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-12T01:28:36.872 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:36 smithi001 ceph-mon[30387]: pgmap v368: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T01:28:36.872 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:36 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/1312624210' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-12T01:28:36.872 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:36 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/3898808387' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-12T01:28:36.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:36 smithi053 ceph-mon[35158]: pgmap v368: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T01:28:37.004 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:36 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/1312624210' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-12T01:28:37.004 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:36 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/3898808387' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-12T01:28:37.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.765+0000 7f8efce00640 1 -- 172.21.15.1:0/884599702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ef8102790 msgr2=0x7f8ef81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:37.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.765+0000 7f8efce00640 1 --2- 172.21.15.1:0/884599702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ef8102790 0x7f8ef81067e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0009900 tx=0x7f8ee002a3f0 comp rx=0 tx=0).stop 2024-09-12T01:28:37.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.766+0000 7f8efce00640 1 -- 172.21.15.1:0/884599702 shutdown_connections 2024-09-12T01:28:37.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.766+0000 7f8efce00640 1 --2- 172.21.15.1:0/884599702 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ef8106f10 0x7f8ef810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:37.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.766+0000 7f8efce00640 1 --2- 172.21.15.1:0/884599702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ef8102790 0x7f8ef81067e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:37.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.766+0000 7f8efce00640 1 --2- 172.21.15.1:0/884599702 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ef8101de0 0x7f8ef81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:37.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.766+0000 7f8efce00640 1 -- 172.21.15.1:0/884599702 >> 172.21.15.1:0/884599702 conn(0x7f8ef80fd8c0 msgr2=0x7f8ef80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:37.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.766+0000 7f8efce00640 1 -- 172.21.15.1:0/884599702 shutdown_connections 2024-09-12T01:28:37.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.766+0000 7f8efce00640 1 -- 172.21.15.1:0/884599702 wait complete. 2024-09-12T01:28:37.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.767+0000 7f8efce00640 1 Processor -- start 2024-09-12T01:28:37.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.767+0000 7f8efce00640 1 -- start start 2024-09-12T01:28:37.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.767+0000 7f8efce00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ef8101de0 0x7f8ef819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:37.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.767+0000 7f8efce00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ef8102790 0x7f8ef819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:37.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.767+0000 7f8efce00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ef8106f10 0x7f8ef819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:37.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.767+0000 7f8efce00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef8111f70 con 0x7f8ef8102790 2024-09-12T01:28:37.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.767+0000 7f8efce00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef8111df0 con 0x7f8ef8106f10 2024-09-12T01:28:37.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.767+0000 7f8efce00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef81120f0 con 0x7f8ef8101de0 2024-09-12T01:28:37.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.768+0000 7f8eed400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ef8102790 0x7f8ef819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:37.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.768+0000 7f8eed400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ef8102790 0x7f8ef819ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38210/0 (socket says 172.21.15.1:38210) 2024-09-12T01:28:37.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.768+0000 7f8eed400640 1 -- 172.21.15.1:0/92110906 learned_addr learned my addr 172.21.15.1:0/92110906 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:28:37.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.768+0000 7f8ef5e00640 1 --2- 172.21.15.1:0/92110906 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ef8101de0 0x7f8ef819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:37.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.768+0000 7f8ef6800640 1 --2- 172.21.15.1:0/92110906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ef8106f10 0x7f8ef819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:37.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.768+0000 7f8eed400640 1 -- 172.21.15.1:0/92110906 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ef8101de0 msgr2=0x7f8ef819a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:37.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.768+0000 7f8eed400640 1 --2- 172.21.15.1:0/92110906 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ef8101de0 0x7f8ef819a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:37.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.768+0000 7f8eed400640 1 -- 172.21.15.1:0/92110906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ef8106f10 msgr2=0x7f8ef819ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:37.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.768+0000 7f8eed400640 1 --2- 172.21.15.1:0/92110906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ef8106f10 0x7f8ef819ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:37.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.768+0000 7f8eed400640 1 -- 172.21.15.1:0/92110906 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef819f5e0 con 0x7f8ef8102790 2024-09-12T01:28:37.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.768+0000 7f8ef5e00640 1 --2- 172.21.15.1:0/92110906 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ef8101de0 0x7f8ef819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:28:37.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.769+0000 7f8eed400640 1 --2- 172.21.15.1:0/92110906 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ef8102790 0x7f8ef819ab70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0006010 tx=0x7f8ee0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:37.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.769+0000 7f8eef400640 1 -- 172.21.15.1:0/92110906 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee003f030 con 0x7f8ef8102790 2024-09-12T01:28:37.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.769+0000 7f8eef400640 1 -- 172.21.15.1:0/92110906 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ee0004440 con 0x7f8ef8102790 2024-09-12T01:28:37.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.769+0000 7f8efce00640 1 -- 172.21.15.1:0/92110906 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ef819f870 con 0x7f8ef8102790 2024-09-12T01:28:37.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.769+0000 7f8eef400640 1 -- 172.21.15.1:0/92110906 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0035070 con 0x7f8ef8102790 2024-09-12T01:28:37.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.769+0000 7f8efce00640 1 -- 172.21.15.1:0/92110906 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ef81a71b0 con 0x7f8ef8102790 2024-09-12T01:28:37.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.770+0000 7f8efce00640 1 -- 172.21.15.1:0/92110906 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ef81032c0 con 0x7f8ef8102790 2024-09-12T01:28:37.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.772+0000 7f8eef400640 1 -- 172.21.15.1:0/92110906 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f8ee0042030 con 0x7f8ef8102790 2024-09-12T01:28:37.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.774+0000 7f8eef400640 1 --2- 172.21.15.1:0/92110906 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f8edc0a2c10 0x7f8edc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:37.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.774+0000 7f8eef400640 1 -- 172.21.15.1:0/92110906 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f8ee00efb40 con 0x7f8ef8102790 2024-09-12T01:28:37.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.774+0000 7f8ef5e00640 1 --2- 172.21.15.1:0/92110906 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f8edc0a2c10 0x7f8edc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:37.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.775+0000 7f8ef5e00640 1 --2- 172.21.15.1:0/92110906 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f8edc0a2c10 0x7f8edc0a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8009fc0 tx=0x7f8ee8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:37.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.776+0000 7f8eef400640 1 -- 172.21.15.1:0/92110906 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8ee00bbf00 con 0x7f8ef8102790 2024-09-12T01:28:37.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.965+0000 7f8efce00640 1 -- 172.21.15.1:0/92110906 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f8ef81a7380 con 0x7f8ef8102790 2024-09-12T01:28:37.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.965+0000 7f8eef400640 1 -- 172.21.15.1:0/92110906 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8ee00c0200 con 0x7f8ef8102790 2024-09-12T01:28:37.965 INFO:teuthology.orchestra.run.smithi001.stdout:55834574940 2024-09-12T01:28:37.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.968+0000 7f8efce00640 1 -- 172.21.15.1:0/92110906 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f8edc0a2c10 msgr2=0x7f8edc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:37.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.968+0000 7f8efce00640 1 --2- 172.21.15.1:0/92110906 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f8edc0a2c10 0x7f8edc0a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8009fc0 tx=0x7f8ee8005f90 comp rx=0 tx=0).stop 2024-09-12T01:28:37.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.968+0000 7f8efce00640 1 -- 172.21.15.1:0/92110906 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ef8102790 msgr2=0x7f8ef819ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:37.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.968+0000 7f8efce00640 1 --2- 172.21.15.1:0/92110906 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ef8102790 0x7f8ef819ab70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0006010 tx=0x7f8ee0004040 comp rx=0 tx=0).stop 2024-09-12T01:28:37.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.968+0000 7f8efce00640 1 -- 172.21.15.1:0/92110906 shutdown_connections 2024-09-12T01:28:37.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.968+0000 7f8efce00640 1 --2- 172.21.15.1:0/92110906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ef8106f10 0x7f8ef819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:37.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.968+0000 7f8efce00640 1 --2- 172.21.15.1:0/92110906 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ef8102790 0x7f8ef819ab70 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:37.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.968+0000 7f8efce00640 1 --2- 172.21.15.1:0/92110906 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f8edc0a2c10 0x7f8edc0a50d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:37.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.968+0000 7f8efce00640 1 --2- 172.21.15.1:0/92110906 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ef8101de0 0x7f8ef819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:37.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.968+0000 7f8efce00640 1 -- 172.21.15.1:0/92110906 >> 172.21.15.1:0/92110906 conn(0x7f8ef80fd8c0 msgr2=0x7f8ef80feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:37.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.968+0000 7f8efce00640 1 -- 172.21.15.1:0/92110906 shutdown_connections 2024-09-12T01:28:37.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:37.968+0000 7f8efce00640 1 -- 172.21.15.1:0/92110906 wait complete. 2024-09-12T01:28:38.801 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:38 smithi001 ceph-mon[30387]: pgmap v369: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-12T01:28:38.802 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:38 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/92110906' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-12T01:28:38.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:38 smithi104 ceph-mon[35219]: pgmap v369: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-12T01:28:38.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:38 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/92110906' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-12T01:28:38.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:38 smithi053 ceph-mon[35158]: pgmap v369: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-12T01:28:38.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:38 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/92110906' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-12T01:28:39.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.019+0000 7f872f200640 1 -- 172.21.15.1:0/443028840 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8730074510 msgr2=0x7f87300748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:39.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.019+0000 7f872f200640 1 --2- 172.21.15.1:0/443028840 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8730074510 0x7f87300748f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8720009860 tx=0x7f872002a3d0 comp rx=0 tx=0).stop 2024-09-12T01:28:39.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.019+0000 7f872f200640 1 -- 172.21.15.1:0/443028840 shutdown_connections 2024-09-12T01:28:39.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.019+0000 7f872f200640 1 --2- 172.21.15.1:0/443028840 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f873010e280 0x7f8730110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:39.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.019+0000 7f872f200640 1 --2- 172.21.15.1:0/443028840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8730074e30 0x7f873010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:39.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.019+0000 7f872f200640 1 --2- 172.21.15.1:0/443028840 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8730074510 0x7f87300748f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:39.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.019+0000 7f872f200640 1 -- 172.21.15.1:0/443028840 >> 172.21.15.1:0/443028840 conn(0x7f87300fd740 msgr2=0x7f87300ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:39.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.019+0000 7f872f200640 1 -- 172.21.15.1:0/443028840 shutdown_connections 2024-09-12T01:28:39.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.020+0000 7f872f200640 1 -- 172.21.15.1:0/443028840 wait complete. 2024-09-12T01:28:39.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.020+0000 7f872f200640 1 Processor -- start 2024-09-12T01:28:39.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.021+0000 7f872f200640 1 -- start start 2024-09-12T01:28:39.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.021+0000 7f872f200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8730074510 0x7f873019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:39.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.021+0000 7f872f200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8730074e30 0x7f873019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:39.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.021+0000 7f872f200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f873010e280 0x7f873019ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:39.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.021+0000 7f872f200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8730113fb0 con 0x7f873010e280 2024-09-12T01:28:39.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.021+0000 7f872f200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8730113e30 con 0x7f8730074510 2024-09-12T01:28:39.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.021+0000 7f872f200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8730114130 con 0x7f8730074e30 2024-09-12T01:28:39.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.021+0000 7f872de00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8730074510 0x7f873019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:39.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.021+0000 7f872de00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8730074510 0x7f873019a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:46166/0 (socket says 172.21.15.1:46166) 2024-09-12T01:28:39.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.021+0000 7f872de00640 1 -- 172.21.15.1:0/3509603542 learned_addr learned my addr 172.21.15.1:0/3509603542 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:28:39.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.021+0000 7f872e800640 1 --2- 172.21.15.1:0/3509603542 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f873010e280 0x7f873019ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:39.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.021+0000 7f872d400640 1 --2- 172.21.15.1:0/3509603542 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8730074e30 0x7f873019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:39.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.022+0000 7f872de00640 1 -- 172.21.15.1:0/3509603542 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8730074e30 msgr2=0x7f873019aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:39.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.022+0000 7f872de00640 1 --2- 172.21.15.1:0/3509603542 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8730074e30 0x7f873019aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:39.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.022+0000 7f872de00640 1 -- 172.21.15.1:0/3509603542 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f873010e280 msgr2=0x7f873019ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:39.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.022+0000 7f872de00640 1 --2- 172.21.15.1:0/3509603542 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f873010e280 0x7f873019ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:39.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.022+0000 7f872de00640 1 -- 172.21.15.1:0/3509603542 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f873019f520 con 0x7f8730074510 2024-09-12T01:28:39.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.022+0000 7f872d400640 1 --2- 172.21.15.1:0/3509603542 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8730074e30 0x7f873019aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:28:39.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.022+0000 7f872de00640 1 --2- 172.21.15.1:0/3509603542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8730074510 0x7f873019a570 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f872002a8f0 tx=0x7f8720030650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:39.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.022+0000 7f8726a00640 1 -- 172.21.15.1:0/3509603542 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8720040020 con 0x7f8730074510 2024-09-12T01:28:39.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.022+0000 7f8726a00640 1 -- 172.21.15.1:0/3509603542 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8720004460 con 0x7f8730074510 2024-09-12T01:28:39.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.022+0000 7f872f200640 1 -- 172.21.15.1:0/3509603542 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f873019f7b0 con 0x7f8730074510 2024-09-12T01:28:39.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.023+0000 7f8726a00640 1 -- 172.21.15.1:0/3509603542 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87200046c0 con 0x7f8730074510 2024-09-12T01:28:39.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.023+0000 7f872f200640 1 -- 172.21.15.1:0/3509603542 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87301a70f0 con 0x7f8730074510 2024-09-12T01:28:39.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.024+0000 7f872f200640 1 -- 172.21.15.1:0/3509603542 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8730075c50 con 0x7f8730074510 2024-09-12T01:28:39.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.026+0000 7f8726a00640 1 -- 172.21.15.1:0/3509603542 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f8720043020 con 0x7f8730074510 2024-09-12T01:28:39.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.027+0000 7f8726a00640 1 --2- 172.21.15.1:0/3509603542 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f87140a2c10 0x7f87140a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:39.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.027+0000 7f8726a00640 1 -- 172.21.15.1:0/3509603542 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f87200f04e0 con 0x7f8730074510 2024-09-12T01:28:39.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.027+0000 7f872d400640 1 --2- 172.21.15.1:0/3509603542 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f87140a2c10 0x7f87140a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:39.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.028+0000 7f872d400640 1 --2- 172.21.15.1:0/3509603542 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f87140a2c10 0x7f87140a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f87300ffd60 tx=0x7f871c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:39.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.031+0000 7f8726a00640 1 -- 172.21.15.1:0/3509603542 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f872004a020 con 0x7f8730074510 2024-09-12T01:28:39.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.229+0000 7f872f200640 1 -- 172.21.15.1:0/3509603542 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f8730109400 con 0x7f8730074510 2024-09-12T01:28:39.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.230+0000 7f8726a00640 1 -- 172.21.15.1:0/3509603542 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f87200bcef0 con 0x7f8730074510 2024-09-12T01:28:39.229 INFO:teuthology.orchestra.run.smithi001.stdout:253403070515 2024-09-12T01:28:39.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.232+0000 7f872f200640 1 -- 172.21.15.1:0/3509603542 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f87140a2c10 msgr2=0x7f87140a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:39.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.232+0000 7f872f200640 1 --2- 172.21.15.1:0/3509603542 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f87140a2c10 0x7f87140a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f87300ffd60 tx=0x7f871c005f90 comp rx=0 tx=0).stop 2024-09-12T01:28:39.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.232+0000 7f872f200640 1 -- 172.21.15.1:0/3509603542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8730074510 msgr2=0x7f873019a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:39.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.232+0000 7f872f200640 1 --2- 172.21.15.1:0/3509603542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8730074510 0x7f873019a570 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f872002a8f0 tx=0x7f8720030650 comp rx=0 tx=0).stop 2024-09-12T01:28:39.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.232+0000 7f872f200640 1 -- 172.21.15.1:0/3509603542 shutdown_connections 2024-09-12T01:28:39.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.232+0000 7f872f200640 1 --2- 172.21.15.1:0/3509603542 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f873010e280 0x7f873019ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:39.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.232+0000 7f872f200640 1 --2- 172.21.15.1:0/3509603542 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f87140a2c10 0x7f87140a50d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:39.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.232+0000 7f872f200640 1 --2- 172.21.15.1:0/3509603542 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8730074e30 0x7f873019aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:39.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.232+0000 7f872f200640 1 --2- 172.21.15.1:0/3509603542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8730074510 0x7f873019a570 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:39.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.232+0000 7f872f200640 1 -- 172.21.15.1:0/3509603542 >> 172.21.15.1:0/3509603542 conn(0x7f87300fd740 msgr2=0x7f87300feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:39.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.232+0000 7f872f200640 1 -- 172.21.15.1:0/3509603542 shutdown_connections 2024-09-12T01:28:39.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.233+0000 7f872f200640 1 -- 172.21.15.1:0/3509603542 wait complete. 2024-09-12T01:28:39.562 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738462 got 34359738464 for osd.0 2024-09-12T01:28:39.562 DEBUG:teuthology.parallel:result is None 2024-09-12T01:28:39.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:38.919+0000 7f48dbe00640 1 -- 172.21.15.1:0/70683863 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48dc10e260 msgr2=0x7f48dc110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:39.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:38.919+0000 7f48dbe00640 1 --2- 172.21.15.1:0/70683863 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48dc10e260 0x7f48dc110650 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f48d4009be0 tx=0x7f48d402a750 comp rx=0 tx=0).stop 2024-09-12T01:28:39.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:38.919+0000 7f48dbe00640 1 -- 172.21.15.1:0/70683863 shutdown_connections 2024-09-12T01:28:39.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:38.919+0000 7f48dbe00640 1 --2- 172.21.15.1:0/70683863 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48dc10e260 0x7f48dc110650 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:39.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:38.919+0000 7f48dbe00640 1 --2- 172.21.15.1:0/70683863 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f48dc074ee0 0x7f48dc10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:39.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:38.919+0000 7f48dbe00640 1 --2- 172.21.15.1:0/70683863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48dc0745c0 0x7f48dc0749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:39.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:38.919+0000 7f48dbe00640 1 -- 172.21.15.1:0/70683863 >> 172.21.15.1:0/70683863 conn(0x7f48dc0fd740 msgr2=0x7f48dc0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:39.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:38.920+0000 7f48dbe00640 1 -- 172.21.15.1:0/70683863 shutdown_connections 2024-09-12T01:28:39.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:38.920+0000 7f48dbe00640 1 -- 172.21.15.1:0/70683863 wait complete. 2024-09-12T01:28:39.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:38.920+0000 7f48dbe00640 1 Processor -- start 2024-09-12T01:28:39.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:38.921+0000 7f48dbe00640 1 -- start start 2024-09-12T01:28:39.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:38.921+0000 7f48dbe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f48dc0745c0 0x7f48dc19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:39.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:38.921+0000 7f48dbe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48dc074ee0 0x7f48dc19a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:39.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:38.921+0000 7f48dbe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48dc10e260 0x7f48dc19ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:39.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:38.921+0000 7f48dbe00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f48dc113f20 con 0x7f48dc0745c0 2024-09-12T01:28:39.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:38.921+0000 7f48dbe00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f48dc113da0 con 0x7f48dc074ee0 2024-09-12T01:28:39.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:38.921+0000 7f48dbe00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f48dc1140a0 con 0x7f48dc10e260 2024-09-12T01:28:39.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:38.921+0000 7f48daa00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f48dc0745c0 0x7f48dc19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:39.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:38.921+0000 7f48daa00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f48dc0745c0 0x7f48dc19a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48166/0 (socket says 172.21.15.1:48166) 2024-09-12T01:28:39.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:38.921+0000 7f48daa00640 1 -- 172.21.15.1:0/1116801815 learned_addr learned my addr 172.21.15.1:0/1116801815 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:28:39.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:38.921+0000 7f48db400640 1 --2- 172.21.15.1:0/1116801815 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48dc10e260 0x7f48dc19ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:39.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:38.921+0000 7f48da000640 1 --2- 172.21.15.1:0/1116801815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48dc074ee0 0x7f48dc19a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:39.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:38.922+0000 7f48daa00640 1 -- 172.21.15.1:0/1116801815 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48dc10e260 msgr2=0x7f48dc19ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:39.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:38.922+0000 7f48daa00640 1 --2- 172.21.15.1:0/1116801815 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48dc10e260 0x7f48dc19ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:39.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:38.922+0000 7f48daa00640 1 -- 172.21.15.1:0/1116801815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48dc074ee0 msgr2=0x7f48dc19a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:39.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:38.922+0000 7f48daa00640 1 --2- 172.21.15.1:0/1116801815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48dc074ee0 0x7f48dc19a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:39.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:38.922+0000 7f48daa00640 1 -- 172.21.15.1:0/1116801815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48dc19f3f0 con 0x7f48dc0745c0 2024-09-12T01:28:39.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:38.922+0000 7f48daa00640 1 --2- 172.21.15.1:0/1116801815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f48dc0745c0 0x7f48dc19a440 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f48cc00e9c0 tx=0x7f48cc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:39.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:38.922+0000 7f48d3400640 1 -- 172.21.15.1:0/1116801815 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48cc00cde0 con 0x7f48dc0745c0 2024-09-12T01:28:39.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:38.922+0000 7f48d3400640 1 -- 172.21.15.1:0/1116801815 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48cc0035f0 con 0x7f48dc0745c0 2024-09-12T01:28:39.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:38.922+0000 7f48d3400640 1 -- 172.21.15.1:0/1116801815 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48cc010690 con 0x7f48dc0745c0 2024-09-12T01:28:39.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:38.922+0000 7f48dbe00640 1 -- 172.21.15.1:0/1116801815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48dc19f6e0 con 0x7f48dc0745c0 2024-09-12T01:28:39.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:38.923+0000 7f48dbe00640 1 -- 172.21.15.1:0/1116801815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48dc1a6fc0 con 0x7f48dc0745c0 2024-09-12T01:28:39.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:38.924+0000 7f48dbe00640 1 -- 172.21.15.1:0/1116801815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48dc0749a0 con 0x7f48dc0745c0 2024-09-12T01:28:39.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:38.925+0000 7f48d3400640 1 -- 172.21.15.1:0/1116801815 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f48cc0037a0 con 0x7f48dc0745c0 2024-09-12T01:28:39.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:38.927+0000 7f48d3400640 1 --2- 172.21.15.1:0/1116801815 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f48c00a2c10 0x7f48c00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:39.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:38.927+0000 7f48d3400640 1 -- 172.21.15.1:0/1116801815 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f48cc0cf8f0 con 0x7f48dc0745c0 2024-09-12T01:28:39.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:38.927+0000 7f48da000640 1 --2- 172.21.15.1:0/1116801815 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f48c00a2c10 0x7f48c00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:39.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:38.928+0000 7f48da000640 1 --2- 172.21.15.1:0/1116801815 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f48c00a2c10 0x7f48c00a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f48dc10f680 tx=0x7f48c8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:39.643 INFO:teuthology.orchestra.run.smithi001.stdout:163208757317 2024-09-12T01:28:39.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:38.931+0000 7f48d3400640 1 -- 172.21.15.1:0/1116801815 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f48cc09c300 con 0x7f48dc0745c0 2024-09-12T01:28:39.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.134+0000 7f48dbe00640 1 -- 172.21.15.1:0/1116801815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f48dc108e50 con 0x7f48dc0745c0 2024-09-12T01:28:39.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.134+0000 7f48d3400640 1 -- 172.21.15.1:0/1116801815 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f48cc09bc00 con 0x7f48dc0745c0 2024-09-12T01:28:39.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.137+0000 7f48dbe00640 1 -- 172.21.15.1:0/1116801815 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f48c00a2c10 msgr2=0x7f48c00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:39.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.137+0000 7f48dbe00640 1 --2- 172.21.15.1:0/1116801815 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f48c00a2c10 0x7f48c00a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f48dc10f680 tx=0x7f48c8005f90 comp rx=0 tx=0).stop 2024-09-12T01:28:39.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.137+0000 7f48dbe00640 1 -- 172.21.15.1:0/1116801815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f48dc0745c0 msgr2=0x7f48dc19a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:39.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.137+0000 7f48dbe00640 1 --2- 172.21.15.1:0/1116801815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f48dc0745c0 0x7f48dc19a440 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f48cc00e9c0 tx=0x7f48cc00ee80 comp rx=0 tx=0).stop 2024-09-12T01:28:39.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.137+0000 7f48dbe00640 1 -- 172.21.15.1:0/1116801815 shutdown_connections 2024-09-12T01:28:39.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.137+0000 7f48dbe00640 1 --2- 172.21.15.1:0/1116801815 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f48c00a2c10 0x7f48c00a50d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:39.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.137+0000 7f48dbe00640 1 --2- 172.21.15.1:0/1116801815 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48dc10e260 0x7f48dc19ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:39.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.137+0000 7f48dbe00640 1 --2- 172.21.15.1:0/1116801815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48dc074ee0 0x7f48dc19a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:39.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.137+0000 7f48dbe00640 1 --2- 172.21.15.1:0/1116801815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f48dc0745c0 0x7f48dc19a440 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:39.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.137+0000 7f48dbe00640 1 -- 172.21.15.1:0/1116801815 >> 172.21.15.1:0/1116801815 conn(0x7f48dc0fd740 msgr2=0x7f48dc0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:39.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.137+0000 7f48dbe00640 1 -- 172.21.15.1:0/1116801815 shutdown_connections 2024-09-12T01:28:39.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:39.137+0000 7f48dbe00640 1 -- 172.21.15.1:0/1116801815 wait complete. 2024-09-12T01:28:39.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:39 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/1116801815' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-12T01:28:39.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:39 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/3509603542' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-12T01:28:39.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:39 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/1116801815' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-12T01:28:39.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:39 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/3509603542' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-12T01:28:39.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:39 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/1116801815' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-12T01:28:39.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:39 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/3509603542' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-12T01:28:40.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.094+0000 7f1cff200640 1 -- 172.21.15.1:0/885478834 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d00103e30 msgr2=0x7f1d00104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:40.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.094+0000 7f1cff200640 1 --2- 172.21.15.1:0/885478834 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d00103e30 0x7f1d00104210 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0009900 tx=0x7f1cf002a410 comp rx=0 tx=0).stop 2024-09-12T01:28:40.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.094+0000 7f1cff200640 1 -- 172.21.15.1:0/885478834 shutdown_connections 2024-09-12T01:28:40.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.094+0000 7f1cff200640 1 --2- 172.21.15.1:0/885478834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d00108f60 0x7f1d00110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:40.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.094+0000 7f1cff200640 1 --2- 172.21.15.1:0/885478834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d001047e0 0x7f1d00108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:40.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.094+0000 7f1cff200640 1 --2- 172.21.15.1:0/885478834 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d00103e30 0x7f1d00104210 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:40.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.094+0000 7f1cff200640 1 -- 172.21.15.1:0/885478834 >> 172.21.15.1:0/885478834 conn(0x7f1d000ff910 msgr2=0x7f1d00101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:40.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.094+0000 7f1cff200640 1 -- 172.21.15.1:0/885478834 shutdown_connections 2024-09-12T01:28:40.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.094+0000 7f1cff200640 1 -- 172.21.15.1:0/885478834 wait complete. 2024-09-12T01:28:40.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.095+0000 7f1cff200640 1 Processor -- start 2024-09-12T01:28:40.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.095+0000 7f1cff200640 1 -- start start 2024-09-12T01:28:40.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.095+0000 7f1cff200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d00103e30 0x7f1d0019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:40.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.095+0000 7f1cff200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d001047e0 0x7f1d0019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:40.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.095+0000 7f1cff200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d00108f60 0x7f1d0019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:40.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.095+0000 7f1cff200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1d00113f80 con 0x7f1d00108f60 2024-09-12T01:28:40.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.095+0000 7f1cff200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1d00113e00 con 0x7f1d001047e0 2024-09-12T01:28:40.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.095+0000 7f1cff200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1d00114100 con 0x7f1d00103e30 2024-09-12T01:28:40.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.096+0000 7f1cf7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d001047e0 0x7f1d0019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:40.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.096+0000 7f1cfde00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d00103e30 0x7f1d0019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:40.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.096+0000 7f1cfe800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d00108f60 0x7f1d0019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:40.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.096+0000 7f1cfde00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d00103e30 0x7f1d0019a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:51854/0 (socket says 172.21.15.1:51854) 2024-09-12T01:28:40.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.096+0000 7f1cf7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d001047e0 0x7f1d0019a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:46182/0 (socket says 172.21.15.1:46182) 2024-09-12T01:28:40.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.096+0000 7f1cfde00640 1 -- 172.21.15.1:0/2994958823 learned_addr learned my addr 172.21.15.1:0/2994958823 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:28:40.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.096+0000 7f1cf7e00640 1 -- 172.21.15.1:0/2994958823 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d00103e30 msgr2=0x7f1d0019a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:40.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.096+0000 7f1cf7e00640 1 --2- 172.21.15.1:0/2994958823 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d00103e30 0x7f1d0019a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:40.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.096+0000 7f1cf7e00640 1 -- 172.21.15.1:0/2994958823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d00108f60 msgr2=0x7f1d0019ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:40.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.096+0000 7f1cf7e00640 1 --2- 172.21.15.1:0/2994958823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d00108f60 0x7f1d0019ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:40.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.096+0000 7f1cf7e00640 1 -- 172.21.15.1:0/2994958823 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d0019f430 con 0x7f1d001047e0 2024-09-12T01:28:40.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.096+0000 7f1cfde00640 1 --2- 172.21.15.1:0/2994958823 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d00103e30 0x7f1d0019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:28:40.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.097+0000 7f1cf7e00640 1 --2- 172.21.15.1:0/2994958823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d001047e0 0x7f1d0019a9c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1ce800e9f0 tx=0x7f1ce800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:40.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.097+0000 7f1cf6a00640 1 -- 172.21.15.1:0/2994958823 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce800ccc0 con 0x7f1d001047e0 2024-09-12T01:28:40.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.097+0000 7f1cff200640 1 -- 172.21.15.1:0/2994958823 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d0019f720 con 0x7f1d001047e0 2024-09-12T01:28:40.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.097+0000 7f1cf6a00640 1 -- 172.21.15.1:0/2994958823 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ce8010910 con 0x7f1d001047e0 2024-09-12T01:28:40.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.097+0000 7f1cf6a00640 1 -- 172.21.15.1:0/2994958823 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8018960 con 0x7f1d001047e0 2024-09-12T01:28:40.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.097+0000 7f1cff200640 1 -- 172.21.15.1:0/2994958823 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d001a7000 con 0x7f1d001047e0 2024-09-12T01:28:40.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.098+0000 7f1cff200640 1 -- 172.21.15.1:0/2994958823 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d00105310 con 0x7f1d001047e0 2024-09-12T01:28:40.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.100+0000 7f1cf6a00640 1 -- 172.21.15.1:0/2994958823 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f1ce8010ac0 con 0x7f1d001047e0 2024-09-12T01:28:40.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.101+0000 7f1cf6a00640 1 --2- 172.21.15.1:0/2994958823 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1ce40a2c10 0x7f1ce40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:40.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.102+0000 7f1cf6a00640 1 -- 172.21.15.1:0/2994958823 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f1ce80cfe10 con 0x7f1d001047e0 2024-09-12T01:28:40.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.102+0000 7f1cfde00640 1 --2- 172.21.15.1:0/2994958823 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1ce40a2c10 0x7f1ce40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:40.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.102+0000 7f1cfde00640 1 --2- 172.21.15.1:0/2994958823 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1ce40a2c10 0x7f1ce40a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1cf00096c0 tx=0x7f1cf0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:40.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.103+0000 7f1cf6a00640 1 -- 172.21.15.1:0/2994958823 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1ce809c1d0 con 0x7f1d001047e0 2024-09-12T01:28:40.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.295+0000 7f1cff200640 1 -- 172.21.15.1:0/2994958823 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f1d00104210 con 0x7f1d001047e0 2024-09-12T01:28:40.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.296+0000 7f1cf6a00640 1 -- 172.21.15.1:0/2994958823 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f1d00104210 con 0x7f1d001047e0 2024-09-12T01:28:40.295 INFO:teuthology.orchestra.run.smithi001.stdout:309237645353 2024-09-12T01:28:40.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.298+0000 7f1cff200640 1 -- 172.21.15.1:0/2994958823 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1ce40a2c10 msgr2=0x7f1ce40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:40.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.298+0000 7f1cff200640 1 --2- 172.21.15.1:0/2994958823 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1ce40a2c10 0x7f1ce40a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1cf00096c0 tx=0x7f1cf0005a20 comp rx=0 tx=0).stop 2024-09-12T01:28:40.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.298+0000 7f1cff200640 1 -- 172.21.15.1:0/2994958823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d001047e0 msgr2=0x7f1d0019a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:40.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.298+0000 7f1cff200640 1 --2- 172.21.15.1:0/2994958823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d001047e0 0x7f1d0019a9c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1ce800e9f0 tx=0x7f1ce800eeb0 comp rx=0 tx=0).stop 2024-09-12T01:28:40.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.298+0000 7f1cff200640 1 -- 172.21.15.1:0/2994958823 shutdown_connections 2024-09-12T01:28:40.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.298+0000 7f1cff200640 1 --2- 172.21.15.1:0/2994958823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d00108f60 0x7f1d0019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:40.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.298+0000 7f1cff200640 1 --2- 172.21.15.1:0/2994958823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d001047e0 0x7f1d0019a9c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:40.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.298+0000 7f1cff200640 1 --2- 172.21.15.1:0/2994958823 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1ce40a2c10 0x7f1ce40a50d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:40.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.298+0000 7f1cff200640 1 --2- 172.21.15.1:0/2994958823 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d00103e30 0x7f1d0019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:40.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.298+0000 7f1cff200640 1 -- 172.21.15.1:0/2994958823 >> 172.21.15.1:0/2994958823 conn(0x7f1d000ff910 msgr2=0x7f1d00100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:40.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.298+0000 7f1cff200640 1 -- 172.21.15.1:0/2994958823 shutdown_connections 2024-09-12T01:28:40.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:40.298+0000 7f1cff200640 1 -- 172.21.15.1:0/2994958823 wait complete. 2024-09-12T01:28:40.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:40 smithi104 ceph-mon[35219]: pgmap v370: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-12T01:28:40.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:40 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/2994958823' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-12T01:28:40.825 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:40 smithi001 ceph-mon[30387]: pgmap v370: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-12T01:28:40.825 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:40 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/2994958823' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-12T01:28:40.864 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757314 got 163208757317 for osd.5 2024-09-12T01:28:40.864 DEBUG:teuthology.parallel:result is None 2024-09-12T01:28:40.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:40 smithi053 ceph-mon[35158]: pgmap v370: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-12T01:28:40.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:40 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/2994958823' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-12T01:28:42.013 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411412 got 77309411414 for osd.2 2024-09-12T01:28:42.013 DEBUG:teuthology.parallel:result is None 2024-09-12T01:28:42.343 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645349 got 309237645353 for osd.10 2024-09-12T01:28:42.343 DEBUG:teuthology.parallel:result is None 2024-09-12T01:28:42.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.571+0000 7fdff5000640 1 -- 172.21.15.1:0/2459952822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdff0103ff0 msgr2=0x7fdff01043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:42.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.571+0000 7fdff5000640 1 --2- 172.21.15.1:0/2459952822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdff0103ff0 0x7fdff01043d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0009900 tx=0x7fdfe002a3f0 comp rx=0 tx=0).stop 2024-09-12T01:28:42.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.571+0000 7fdff5000640 1 -- 172.21.15.1:0/2459952822 shutdown_connections 2024-09-12T01:28:42.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.571+0000 7fdff5000640 1 --2- 172.21.15.1:0/2459952822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdff0109120 0x7fdff0110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:42.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.571+0000 7fdff5000640 1 --2- 172.21.15.1:0/2459952822 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdff01049a0 0x7fdff01089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:42.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.571+0000 7fdff5000640 1 --2- 172.21.15.1:0/2459952822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdff0103ff0 0x7fdff01043d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:42.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.571+0000 7fdff5000640 1 -- 172.21.15.1:0/2459952822 >> 172.21.15.1:0/2459952822 conn(0x7fdff00ffa90 msgr2=0x7fdff0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:42.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.571+0000 7fdff5000640 1 -- 172.21.15.1:0/2459952822 shutdown_connections 2024-09-12T01:28:42.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.571+0000 7fdff5000640 1 -- 172.21.15.1:0/2459952822 wait complete. 2024-09-12T01:28:42.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.572+0000 7fdff5000640 1 Processor -- start 2024-09-12T01:28:42.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.572+0000 7fdff5000640 1 -- start start 2024-09-12T01:28:42.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.572+0000 7fdff5000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdff0103ff0 0x7fdff019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:42.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.572+0000 7fdff5000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdff01049a0 0x7fdff019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:42.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.572+0000 7fdff5000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdff0109120 0x7fdff019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:42.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.572+0000 7fdff5000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdff01141c0 con 0x7fdff01049a0 2024-09-12T01:28:42.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.572+0000 7fdff5000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdff0114040 con 0x7fdff0109120 2024-09-12T01:28:42.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.572+0000 7fdff5000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdff0114340 con 0x7fdff0103ff0 2024-09-12T01:28:42.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.572+0000 7fdfe5400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdff01049a0 0x7fdff019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:42.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.572+0000 7fdfe5400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdff01049a0 0x7fdff019ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48204/0 (socket says 172.21.15.1:48204) 2024-09-12T01:28:42.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.572+0000 7fdfe5400640 1 -- 172.21.15.1:0/157661187 learned_addr learned my addr 172.21.15.1:0/157661187 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:28:42.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.572+0000 7fdfede00640 1 --2- 172.21.15.1:0/157661187 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdff0103ff0 0x7fdff019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:42.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.573+0000 7fdfee800640 1 --2- 172.21.15.1:0/157661187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdff0109120 0x7fdff019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:42.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.573+0000 7fdfe5400640 1 -- 172.21.15.1:0/157661187 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdff0103ff0 msgr2=0x7fdff019a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:42.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.573+0000 7fdfe5400640 1 --2- 172.21.15.1:0/157661187 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdff0103ff0 0x7fdff019a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:42.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.573+0000 7fdfe5400640 1 -- 172.21.15.1:0/157661187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdff0109120 msgr2=0x7fdff019ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:42.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.573+0000 7fdfe5400640 1 --2- 172.21.15.1:0/157661187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdff0109120 0x7fdff019ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:42.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.573+0000 7fdfe5400640 1 -- 172.21.15.1:0/157661187 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdff019f5f0 con 0x7fdff01049a0 2024-09-12T01:28:42.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.573+0000 7fdfee800640 1 --2- 172.21.15.1:0/157661187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdff0109120 0x7fdff019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:28:42.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.573+0000 7fdfe5400640 1 --2- 172.21.15.1:0/157661187 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdff01049a0 0x7fdff019ab80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdfd800e9f0 tx=0x7fdfd800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:42.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.573+0000 7fdfe7400640 1 -- 172.21.15.1:0/157661187 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd800cd80 con 0x7fdff01049a0 2024-09-12T01:28:42.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.574+0000 7fdfe7400640 1 -- 172.21.15.1:0/157661187 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfd8005500 con 0x7fdff01049a0 2024-09-12T01:28:42.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.574+0000 7fdfe7400640 1 -- 172.21.15.1:0/157661187 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd8010690 con 0x7fdff01049a0 2024-09-12T01:28:42.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.574+0000 7fdff5000640 1 -- 172.21.15.1:0/157661187 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdff019f8e0 con 0x7fdff01049a0 2024-09-12T01:28:42.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.574+0000 7fdff5000640 1 -- 172.21.15.1:0/157661187 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdff01a71c0 con 0x7fdff01049a0 2024-09-12T01:28:42.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.575+0000 7fdff5000640 1 -- 172.21.15.1:0/157661187 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdff01043d0 con 0x7fdff01049a0 2024-09-12T01:28:42.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.577+0000 7fdfe7400640 1 -- 172.21.15.1:0/157661187 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fdfd80046c0 con 0x7fdff01049a0 2024-09-12T01:28:42.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.578+0000 7fdfe7400640 1 --2- 172.21.15.1:0/157661187 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fdfd40a2c10 0x7fdfd40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:42.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.578+0000 7fdfe7400640 1 -- 172.21.15.1:0/157661187 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fdfd80cf9a0 con 0x7fdff01049a0 2024-09-12T01:28:42.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.578+0000 7fdfede00640 1 --2- 172.21.15.1:0/157661187 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fdfd40a2c10 0x7fdfd40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:42.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.579+0000 7fdfede00640 1 --2- 172.21.15.1:0/157661187 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fdfd40a2c10 0x7fdfd40a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0000c00 tx=0x7fdfe0005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:42.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.585+0000 7fdfe7400640 1 -- 172.21.15.1:0/157661187 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdfd809c3b0 con 0x7fdff01049a0 2024-09-12T01:28:42.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.773+0000 7fdff5000640 1 -- 172.21.15.1:0/157661187 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fdff002d7c0 con 0x7fdff01049a0 2024-09-12T01:28:42.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.773+0000 7fdfe7400640 1 -- 172.21.15.1:0/157661187 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fdfd809bcb0 con 0x7fdff01049a0 2024-09-12T01:28:42.773 INFO:teuthology.orchestra.run.smithi001.stdout:339302416420 2024-09-12T01:28:42.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.775+0000 7fdff5000640 1 -- 172.21.15.1:0/157661187 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fdfd40a2c10 msgr2=0x7fdfd40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:42.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.775+0000 7fdff5000640 1 --2- 172.21.15.1:0/157661187 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fdfd40a2c10 0x7fdfd40a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0000c00 tx=0x7fdfe0005b50 comp rx=0 tx=0).stop 2024-09-12T01:28:42.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.775+0000 7fdff5000640 1 -- 172.21.15.1:0/157661187 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdff01049a0 msgr2=0x7fdff019ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:42.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.775+0000 7fdff5000640 1 --2- 172.21.15.1:0/157661187 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdff01049a0 0x7fdff019ab80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdfd800e9f0 tx=0x7fdfd800edf0 comp rx=0 tx=0).stop 2024-09-12T01:28:42.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.776+0000 7fdff5000640 1 -- 172.21.15.1:0/157661187 shutdown_connections 2024-09-12T01:28:42.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.776+0000 7fdff5000640 1 --2- 172.21.15.1:0/157661187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdff0109120 0x7fdff019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:42.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.776+0000 7fdff5000640 1 --2- 172.21.15.1:0/157661187 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdff01049a0 0x7fdff019ab80 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:42.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.776+0000 7fdff5000640 1 --2- 172.21.15.1:0/157661187 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fdfd40a2c10 0x7fdfd40a50d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:42.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.776+0000 7fdff5000640 1 --2- 172.21.15.1:0/157661187 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdff0103ff0 0x7fdff019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:42.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.776+0000 7fdff5000640 1 -- 172.21.15.1:0/157661187 >> 172.21.15.1:0/157661187 conn(0x7fdff00ffa90 msgr2=0x7fdff0100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:42.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.776+0000 7fdff5000640 1 -- 172.21.15.1:0/157661187 shutdown_connections 2024-09-12T01:28:42.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:42.776+0000 7fdff5000640 1 -- 172.21.15.1:0/157661187 wait complete. 2024-09-12T01:28:42.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:42 smithi104 ceph-mon[35219]: pgmap v371: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-12T01:28:42.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:42 smithi001 ceph-mon[30387]: pgmap v371: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-12T01:28:42.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:42 smithi053 ceph-mon[35158]: pgmap v371: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-12T01:28:43.101 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070512 got 253403070515 for osd.8 2024-09-12T01:28:43.101 DEBUG:teuthology.parallel:result is None 2024-09-12T01:28:43.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:43 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/157661187' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-12T01:28:43.903 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:43 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/157661187' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-12T01:28:43.943 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574937 got 55834574940 for osd.1 2024-09-12T01:28:43.944 DEBUG:teuthology.parallel:result is None 2024-09-12T01:28:43.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:43 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/157661187' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-12T01:28:44.709 INFO:tasks.cephadm.ceph_manager.ceph:need seq 339302416418 got 339302416420 for osd.11 2024-09-12T01:28:44.709 DEBUG:teuthology.parallel:result is None 2024-09-12T01:28:44.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:44 smithi104 ceph-mon[35219]: pgmap v372: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T01:28:44.884 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:44 smithi001 ceph-mon[30387]: pgmap v372: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T01:28:44.897 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:28:44.948 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495676 got 197568495679 for osd.6 2024-09-12T01:28:44.949 DEBUG:teuthology.parallel:result is None 2024-09-12T01:28:44.965 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:28:44.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:44 smithi053 ceph-mon[35158]: pgmap v372: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T01:28:45.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.679+0000 7f5effe00640 1 -- 172.21.15.1:0/3796191017 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ef8109140 msgr2=0x7f5ef8110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:45.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.679+0000 7f5effe00640 1 --2- 172.21.15.1:0/3796191017 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ef8109140 0x7f5ef8110cc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0009ca0 tx=0x7f5ef002a8c0 comp rx=0 tx=0).stop 2024-09-12T01:28:45.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.679+0000 7f5effe00640 1 -- 172.21.15.1:0/3796191017 shutdown_connections 2024-09-12T01:28:45.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.679+0000 7f5effe00640 1 --2- 172.21.15.1:0/3796191017 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ef8109140 0x7f5ef8110cc0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:45.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.679+0000 7f5effe00640 1 --2- 172.21.15.1:0/3796191017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ef81049c0 0x7f5ef8108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:45.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.679+0000 7f5effe00640 1 --2- 172.21.15.1:0/3796191017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ef8104010 0x7f5ef81043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:45.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.679+0000 7f5effe00640 1 -- 172.21.15.1:0/3796191017 >> 172.21.15.1:0/3796191017 conn(0x7f5ef80ffa50 msgr2=0x7f5ef8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:45.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.679+0000 7f5effe00640 1 -- 172.21.15.1:0/3796191017 shutdown_connections 2024-09-12T01:28:45.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.679+0000 7f5effe00640 1 -- 172.21.15.1:0/3796191017 wait complete. 2024-09-12T01:28:45.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.680+0000 7f5effe00640 1 Processor -- start 2024-09-12T01:28:45.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.680+0000 7f5effe00640 1 -- start start 2024-09-12T01:28:45.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.680+0000 7f5effe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ef8104010 0x7f5ef819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:45.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.680+0000 7f5effe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ef81049c0 0x7f5ef819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:45.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.680+0000 7f5effe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ef8109140 0x7f5ef819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:45.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.681+0000 7f5effe00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef8114150 con 0x7f5ef81049c0 2024-09-12T01:28:45.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.681+0000 7f5effe00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef8113fd0 con 0x7f5ef8109140 2024-09-12T01:28:45.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.681+0000 7f5effe00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef81142d0 con 0x7f5ef8104010 2024-09-12T01:28:45.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.681+0000 7f5efd400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ef8104010 0x7f5ef819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:45.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.681+0000 7f5efd400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ef8104010 0x7f5ef819a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:51896/0 (socket says 172.21.15.1:51896) 2024-09-12T01:28:45.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.681+0000 7f5efd400640 1 -- 172.21.15.1:0/3155369206 learned_addr learned my addr 172.21.15.1:0/3155369206 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:28:45.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.681+0000 7f5efca00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ef81049c0 0x7f5ef819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:45.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.681+0000 7f5efde00640 1 --2- 172.21.15.1:0/3155369206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ef8109140 0x7f5ef819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:45.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.681+0000 7f5efd400640 1 -- 172.21.15.1:0/3155369206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ef8109140 msgr2=0x7f5ef819ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:45.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.681+0000 7f5efd400640 1 --2- 172.21.15.1:0/3155369206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ef8109140 0x7f5ef819ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:45.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.681+0000 7f5efd400640 1 -- 172.21.15.1:0/3155369206 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ef81049c0 msgr2=0x7f5ef819ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:45.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.681+0000 7f5efd400640 1 --2- 172.21.15.1:0/3155369206 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ef81049c0 0x7f5ef819ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:45.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.681+0000 7f5efd400640 1 -- 172.21.15.1:0/3155369206 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ef819f5e0 con 0x7f5ef8104010 2024-09-12T01:28:45.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.682+0000 7f5efd400640 1 --2- 172.21.15.1:0/3155369206 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ef8104010 0x7f5ef819a630 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5eec00ea60 tx=0x7f5eec00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:45.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.682+0000 7f5ef6000640 1 -- 172.21.15.1:0/3155369206 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eec00cde0 con 0x7f5ef8104010 2024-09-12T01:28:45.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.682+0000 7f5ef6000640 1 -- 172.21.15.1:0/3155369206 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5eec0035f0 con 0x7f5ef8104010 2024-09-12T01:28:45.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.682+0000 7f5ef6000640 1 -- 172.21.15.1:0/3155369206 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eec010690 con 0x7f5ef8104010 2024-09-12T01:28:45.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.682+0000 7f5effe00640 1 -- 172.21.15.1:0/3155369206 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ef819f8d0 con 0x7f5ef8104010 2024-09-12T01:28:45.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.682+0000 7f5effe00640 1 -- 172.21.15.1:0/3155369206 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ef81a71b0 con 0x7f5ef8104010 2024-09-12T01:28:45.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.683+0000 7f5effe00640 1 -- 172.21.15.1:0/3155369206 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ef81043f0 con 0x7f5ef8104010 2024-09-12T01:28:45.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.685+0000 7f5ef6000640 1 -- 172.21.15.1:0/3155369206 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f5eec0037a0 con 0x7f5ef8104010 2024-09-12T01:28:45.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.687+0000 7f5ef6000640 1 --2- 172.21.15.1:0/3155369206 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f5ee00a2c10 0x7f5ee00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:45.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.687+0000 7f5ef6000640 1 -- 172.21.15.1:0/3155369206 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f5eec0cf890 con 0x7f5ef8104010 2024-09-12T01:28:45.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.687+0000 7f5efca00640 1 --2- 172.21.15.1:0/3155369206 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f5ee00a2c10 0x7f5ee00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:45.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.688+0000 7f5efca00640 1 --2- 172.21.15.1:0/3155369206 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f5ee00a2c10 0x7f5ee00a50d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5ef810ab10 tx=0x7f5ee400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:45.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.691+0000 7f5ef6000640 1 -- 172.21.15.1:0/3155369206 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5eec09c2a0 con 0x7f5ef8104010 2024-09-12T01:28:45.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.899+0000 7f5effe00640 1 -- 172.21.15.1:0/3155369206 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f5ef802d7c0 con 0x7f5ef8104010 2024-09-12T01:28:45.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.900+0000 7f5ef6000640 1 -- 172.21.15.1:0/3155369206 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5eec09bba0 con 0x7f5ef8104010 2024-09-12T01:28:45.899 INFO:teuthology.orchestra.run.smithi001.stdout:219043332154 2024-09-12T01:28:45.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.902+0000 7f5effe00640 1 -- 172.21.15.1:0/3155369206 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f5ee00a2c10 msgr2=0x7f5ee00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:45.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.902+0000 7f5effe00640 1 --2- 172.21.15.1:0/3155369206 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f5ee00a2c10 0x7f5ee00a50d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5ef810ab10 tx=0x7f5ee400b410 comp rx=0 tx=0).stop 2024-09-12T01:28:45.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.902+0000 7f5effe00640 1 -- 172.21.15.1:0/3155369206 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ef8104010 msgr2=0x7f5ef819a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:45.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.902+0000 7f5effe00640 1 --2- 172.21.15.1:0/3155369206 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ef8104010 0x7f5ef819a630 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5eec00ea60 tx=0x7f5eec00ef20 comp rx=0 tx=0).stop 2024-09-12T01:28:45.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.902+0000 7f5effe00640 1 -- 172.21.15.1:0/3155369206 shutdown_connections 2024-09-12T01:28:45.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.902+0000 7f5effe00640 1 --2- 172.21.15.1:0/3155369206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ef8109140 0x7f5ef819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:45.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.902+0000 7f5effe00640 1 --2- 172.21.15.1:0/3155369206 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ef81049c0 0x7f5ef819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:45.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.902+0000 7f5effe00640 1 --2- 172.21.15.1:0/3155369206 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f5ee00a2c10 0x7f5ee00a50d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:45.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.902+0000 7f5effe00640 1 --2- 172.21.15.1:0/3155369206 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ef8104010 0x7f5ef819a630 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:45.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.902+0000 7f5effe00640 1 -- 172.21.15.1:0/3155369206 >> 172.21.15.1:0/3155369206 conn(0x7f5ef80ffa50 msgr2=0x7f5ef8100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:45.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.903+0000 7f5effe00640 1 -- 172.21.15.1:0/3155369206 shutdown_connections 2024-09-12T01:28:45.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.903+0000 7f5effe00640 1 -- 172.21.15.1:0/3155369206 wait complete. 2024-09-12T01:28:45.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.943+0000 7feeade00640 1 -- 172.21.15.1:0/526487847 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feea8077040 msgr2=0x7feea8112510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:45.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.943+0000 7feeade00640 1 --2- 172.21.15.1:0/526487847 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feea8077040 0x7feea8112510 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fee94009900 tx=0x7fee9402a3f0 comp rx=0 tx=0).stop 2024-09-12T01:28:45.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.943+0000 7feeade00640 1 -- 172.21.15.1:0/526487847 shutdown_connections 2024-09-12T01:28:45.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.943+0000 7feeade00640 1 --2- 172.21.15.1:0/526487847 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feea8112a50 0x7feea8114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:45.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.943+0000 7feeade00640 1 --2- 172.21.15.1:0/526487847 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feea8077040 0x7feea8112510 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:45.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.943+0000 7feeade00640 1 --2- 172.21.15.1:0/526487847 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feea8076720 0x7feea8076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:45.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.943+0000 7feeade00640 1 -- 172.21.15.1:0/526487847 >> 172.21.15.1:0/526487847 conn(0x7feea80ffa50 msgr2=0x7feea8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:45.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.943+0000 7feeade00640 1 -- 172.21.15.1:0/526487847 shutdown_connections 2024-09-12T01:28:45.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.943+0000 7feeade00640 1 -- 172.21.15.1:0/526487847 wait complete. 2024-09-12T01:28:45.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.944+0000 7feeade00640 1 Processor -- start 2024-09-12T01:28:45.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.944+0000 7feeade00640 1 -- start start 2024-09-12T01:28:45.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.945+0000 7feeade00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feea8076720 0x7feea81a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:45.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.945+0000 7feeade00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feea8077040 0x7feea81a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:45.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.945+0000 7feeade00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feea8112a50 0x7feea819e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:45.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.945+0000 7feeade00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7feea81185d0 con 0x7feea8112a50 2024-09-12T01:28:45.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.945+0000 7feeade00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7feea8118450 con 0x7feea8077040 2024-09-12T01:28:45.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.945+0000 7feeade00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7feea8118750 con 0x7feea8076720 2024-09-12T01:28:45.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.945+0000 7feea6a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feea8076720 0x7feea81a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:45.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.945+0000 7feea6000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feea8077040 0x7feea81a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:45.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.945+0000 7feea6a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feea8076720 0x7feea81a4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:51920/0 (socket says 172.21.15.1:51920) 2024-09-12T01:28:45.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.945+0000 7feea6a00640 1 -- 172.21.15.1:0/3191542423 learned_addr learned my addr 172.21.15.1:0/3191542423 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:28:45.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.945+0000 7feea7400640 1 --2- 172.21.15.1:0/3191542423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feea8112a50 0x7feea819e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:45.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.946+0000 7feea6a00640 1 -- 172.21.15.1:0/3191542423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feea8077040 msgr2=0x7feea81a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:45.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.946+0000 7feea6a00640 1 --2- 172.21.15.1:0/3191542423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feea8077040 0x7feea81a4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:45.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.946+0000 7feea6a00640 1 -- 172.21.15.1:0/3191542423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feea8112a50 msgr2=0x7feea819e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:45.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.946+0000 7feea6a00640 1 --2- 172.21.15.1:0/3191542423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feea8112a50 0x7feea819e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:45.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.946+0000 7feea6a00640 1 -- 172.21.15.1:0/3191542423 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea819eff0 con 0x7feea8076720 2024-09-12T01:28:45.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.946+0000 7feea6a00640 1 --2- 172.21.15.1:0/3191542423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feea8076720 0x7feea81a4700 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fee9800e9f0 tx=0x7fee9800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:45.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.946+0000 7feeaca00640 1 -- 172.21.15.1:0/3191542423 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9800cd80 con 0x7feea8076720 2024-09-12T01:28:45.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.946+0000 7feeaca00640 1 -- 172.21.15.1:0/3191542423 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee98010910 con 0x7feea8076720 2024-09-12T01:28:45.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.946+0000 7feeade00640 1 -- 172.21.15.1:0/3191542423 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feea819f2e0 con 0x7feea8076720 2024-09-12T01:28:45.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.947+0000 7feeaca00640 1 -- 172.21.15.1:0/3191542423 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee98018990 con 0x7feea8076720 2024-09-12T01:28:45.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.947+0000 7feeade00640 1 -- 172.21.15.1:0/3191542423 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feea81ab5d0 con 0x7feea8076720 2024-09-12T01:28:45.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.948+0000 7feeade00640 1 -- 172.21.15.1:0/3191542423 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feea8076b00 con 0x7feea8076720 2024-09-12T01:28:45.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.951+0000 7feeaca00640 1 -- 172.21.15.1:0/3191542423 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fee98010ac0 con 0x7feea8076720 2024-09-12T01:28:45.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.953+0000 7feeaca00640 1 --2- 172.21.15.1:0/3191542423 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fee8c0a2c90 0x7fee8c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:45.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.954+0000 7feeaca00640 1 -- 172.21.15.1:0/3191542423 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fee980cf8e0 con 0x7feea8076720 2024-09-12T01:28:45.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.954+0000 7feea6000640 1 --2- 172.21.15.1:0/3191542423 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fee8c0a2c90 0x7fee8c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:45.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.955+0000 7feea6000640 1 --2- 172.21.15.1:0/3191542423 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fee8c0a2c90 0x7fee8c0a5150 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fee94009230 tx=0x7fee940058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:45.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:45.956+0000 7feeaca00640 1 -- 172.21.15.1:0/3191542423 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fee9809c2f0 con 0x7feea8076720 2024-09-12T01:28:46.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:46.144+0000 7feeade00640 1 -- 172.21.15.1:0/3191542423 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7feea81ab840 con 0x7feea8076720 2024-09-12T01:28:46.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:46.144+0000 7feeaca00640 1 -- 172.21.15.1:0/3191542423 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fee9809bbf0 con 0x7feea8076720 2024-09-12T01:28:46.144 INFO:teuthology.orchestra.run.smithi001.stdout:141733920845 2024-09-12T01:28:46.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:46.147+0000 7feeade00640 1 -- 172.21.15.1:0/3191542423 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fee8c0a2c90 msgr2=0x7fee8c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:46.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:46.147+0000 7feeade00640 1 --2- 172.21.15.1:0/3191542423 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fee8c0a2c90 0x7fee8c0a5150 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fee94009230 tx=0x7fee940058d0 comp rx=0 tx=0).stop 2024-09-12T01:28:46.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:46.147+0000 7feeade00640 1 -- 172.21.15.1:0/3191542423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feea8076720 msgr2=0x7feea81a4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:46.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:46.147+0000 7feeade00640 1 --2- 172.21.15.1:0/3191542423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feea8076720 0x7feea81a4700 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fee9800e9f0 tx=0x7fee9800eeb0 comp rx=0 tx=0).stop 2024-09-12T01:28:46.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:46.147+0000 7feeade00640 1 -- 172.21.15.1:0/3191542423 shutdown_connections 2024-09-12T01:28:46.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:46.147+0000 7feeade00640 1 --2- 172.21.15.1:0/3191542423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feea8112a50 0x7feea819e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:46.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:46.147+0000 7feeade00640 1 --2- 172.21.15.1:0/3191542423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feea8077040 0x7feea81a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:46.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:46.147+0000 7feeade00640 1 --2- 172.21.15.1:0/3191542423 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fee8c0a2c90 0x7fee8c0a5150 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:46.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:46.147+0000 7feeade00640 1 --2- 172.21.15.1:0/3191542423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feea8076720 0x7feea81a4700 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:46.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:46.147+0000 7feeade00640 1 -- 172.21.15.1:0/3191542423 >> 172.21.15.1:0/3191542423 conn(0x7feea80ffa50 msgr2=0x7feea8101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:46.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:46.147+0000 7feeade00640 1 -- 172.21.15.1:0/3191542423 shutdown_connections 2024-09-12T01:28:46.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:46.148+0000 7feeade00640 1 -- 172.21.15.1:0/3191542423 wait complete. 2024-09-12T01:28:46.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:46 smithi104 ceph-mon[35219]: pgmap v373: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T01:28:46.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:46 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/3155369206' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-12T01:28:46.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:46 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/3191542423' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-12T01:28:46.848 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:46 smithi001 ceph-mon[30387]: pgmap v373: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T01:28:46.848 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:46 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/3155369206' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-12T01:28:46.848 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:46 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/3191542423' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-12T01:28:46.887 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332149 got 219043332154 for osd.7 2024-09-12T01:28:46.887 DEBUG:teuthology.parallel:result is None 2024-09-12T01:28:46.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:46 smithi053 ceph-mon[35158]: pgmap v373: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T01:28:46.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:46 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/3155369206' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-12T01:28:46.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:46 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/3191542423' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-12T01:28:47.339 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920841 got 141733920845 for osd.4 2024-09-12T01:28:47.340 DEBUG:teuthology.parallel:result is None 2024-09-12T01:28:47.340 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-12T01:28:47.340 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T01:28:47.726 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:28:48.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.336+0000 7f4291c00640 1 -- 172.21.15.1:0/69613865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f428c1049c0 msgr2=0x7f428c108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:48.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.336+0000 7f4291c00640 1 --2- 172.21.15.1:0/69613865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f428c1049c0 0x7f428c108a10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4278009900 tx=0x7f427802a3f0 comp rx=0 tx=0).stop 2024-09-12T01:28:48.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.336+0000 7f4291c00640 1 -- 172.21.15.1:0/69613865 shutdown_connections 2024-09-12T01:28:48.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.336+0000 7f4291c00640 1 --2- 172.21.15.1:0/69613865 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f428c109140 0x7f428c110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:48.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.336+0000 7f4291c00640 1 --2- 172.21.15.1:0/69613865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f428c1049c0 0x7f428c108a10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:48.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.336+0000 7f4291c00640 1 --2- 172.21.15.1:0/69613865 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f428c104010 0x7f428c1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:48.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.336+0000 7f4291c00640 1 -- 172.21.15.1:0/69613865 >> 172.21.15.1:0/69613865 conn(0x7f428c0ffa50 msgr2=0x7f428c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:48.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.336+0000 7f4291c00640 1 -- 172.21.15.1:0/69613865 shutdown_connections 2024-09-12T01:28:48.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.336+0000 7f4291c00640 1 -- 172.21.15.1:0/69613865 wait complete. 2024-09-12T01:28:48.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.338+0000 7f4291c00640 1 Processor -- start 2024-09-12T01:28:48.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.338+0000 7f4291c00640 1 -- start start 2024-09-12T01:28:48.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.338+0000 7f4291c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f428c104010 0x7f428c19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:48.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.338+0000 7f4291c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f428c1049c0 0x7f428c19ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:48.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.339+0000 7f428aa00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f428c104010 0x7f428c19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:48.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.339+0000 7f428aa00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f428c104010 0x7f428c19a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48266/0 (socket says 172.21.15.1:48266) 2024-09-12T01:28:48.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.339+0000 7f428a000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f428c1049c0 0x7f428c19ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:48.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.339+0000 7f428a000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f428c1049c0 0x7f428c19ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:46268/0 (socket says 172.21.15.1:46268) 2024-09-12T01:28:48.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.339+0000 7f4291c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f428c109140 0x7f428c19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:48.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.339+0000 7f4291c00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f428c114140 con 0x7f428c104010 2024-09-12T01:28:48.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.339+0000 7f4291c00640 1 -- 172.21.15.1:0/2918289875 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f428c113fc0 con 0x7f428c1049c0 2024-09-12T01:28:48.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.339+0000 7f4291c00640 1 -- 172.21.15.1:0/2918289875 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f428c1142c0 con 0x7f428c109140 2024-09-12T01:28:48.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.339+0000 7f428aa00640 1 -- 172.21.15.1:0/2918289875 learned_addr learned my addr 172.21.15.1:0/2918289875 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:28:48.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.339+0000 7f428aa00640 1 -- 172.21.15.1:0/2918289875 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f428c109140 msgr2=0x7f428c19efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:48.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.339+0000 7f428aa00640 1 --2- 172.21.15.1:0/2918289875 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f428c109140 0x7f428c19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:48.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.339+0000 7f428aa00640 1 -- 172.21.15.1:0/2918289875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f428c1049c0 msgr2=0x7f428c19ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:48.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.339+0000 7f428aa00640 1 --2- 172.21.15.1:0/2918289875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f428c1049c0 0x7f428c19ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:48.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.339+0000 7f428aa00640 1 -- 172.21.15.1:0/2918289875 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f428c19f690 con 0x7f428c104010 2024-09-12T01:28:48.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.340+0000 7f428aa00640 1 --2- 172.21.15.1:0/2918289875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f428c104010 0x7f428c19a6e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f427c00ea60 tx=0x7f427c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:48.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.340+0000 7f4283400640 1 -- 172.21.15.1:0/2918289875 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c00cde0 con 0x7f428c104010 2024-09-12T01:28:48.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.340+0000 7f4283400640 1 -- 172.21.15.1:0/2918289875 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f427c0035f0 con 0x7f428c104010 2024-09-12T01:28:48.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.340+0000 7f4291c00640 1 -- 172.21.15.1:0/2918289875 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f428c19f980 con 0x7f428c104010 2024-09-12T01:28:48.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.340+0000 7f4283400640 1 -- 172.21.15.1:0/2918289875 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c010670 con 0x7f428c104010 2024-09-12T01:28:48.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.340+0000 7f4291c00640 1 -- 172.21.15.1:0/2918289875 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f428c1a7260 con 0x7f428c104010 2024-09-12T01:28:48.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.341+0000 7f4291c00640 1 -- 172.21.15.1:0/2918289875 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f428c1043f0 con 0x7f428c104010 2024-09-12T01:28:48.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.343+0000 7f4283400640 1 -- 172.21.15.1:0/2918289875 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f427c0037a0 con 0x7f428c104010 2024-09-12T01:28:48.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.345+0000 7f4283400640 1 --2- 172.21.15.1:0/2918289875 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f42700a2c90 0x7f42700a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:48.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.345+0000 7f4283400640 1 -- 172.21.15.1:0/2918289875 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f427c0cf8e0 con 0x7f428c104010 2024-09-12T01:28:48.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.345+0000 7f428a000640 1 --2- 172.21.15.1:0/2918289875 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f42700a2c90 0x7f42700a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:48.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.346+0000 7f428a000640 1 --2- 172.21.15.1:0/2918289875 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f42700a2c90 0x7f42700a5150 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4278009230 tx=0x7f42780058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:48.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.349+0000 7f4283400640 1 -- 172.21.15.1:0/2918289875 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f427c09c2f0 con 0x7f428c104010 2024-09-12T01:28:48.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.542+0000 7f4291c00640 1 -- 172.21.15.1:0/2918289875 --> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f428c062630 con 0x7f42700a2c90 2024-09-12T01:28:48.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.552+0000 7f4283400640 1 -- 172.21.15.1:0/2918289875 <== mgr.14150 v2:172.21.15.1:6800/1928662456 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340428 (secure 0 0 0) 0x7f428c062630 con 0x7f42700a2c90 2024-09-12T01:28:48.557 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:28:48.559 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-12T01:28:48.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.557+0000 7f4291c00640 1 -- 172.21.15.1:0/2918289875 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f42700a2c90 msgr2=0x7f42700a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:48.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.557+0000 7f4291c00640 1 --2- 172.21.15.1:0/2918289875 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f42700a2c90 0x7f42700a5150 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4278009230 tx=0x7f42780058d0 comp rx=0 tx=0).stop 2024-09-12T01:28:48.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.557+0000 7f4291c00640 1 -- 172.21.15.1:0/2918289875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f428c104010 msgr2=0x7f428c19a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:48.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.557+0000 7f4291c00640 1 --2- 172.21.15.1:0/2918289875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f428c104010 0x7f428c19a6e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f427c00ea60 tx=0x7f427c00ef20 comp rx=0 tx=0).stop 2024-09-12T01:28:48.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.557+0000 7f4291c00640 1 -- 172.21.15.1:0/2918289875 shutdown_connections 2024-09-12T01:28:48.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.557+0000 7f4291c00640 1 --2- 172.21.15.1:0/2918289875 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f42700a2c90 0x7f42700a5150 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:48.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.557+0000 7f4291c00640 1 --2- 172.21.15.1:0/2918289875 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f428c109140 0x7f428c19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:48.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.557+0000 7f4291c00640 1 --2- 172.21.15.1:0/2918289875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f428c1049c0 0x7f428c19ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:48.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.557+0000 7f4291c00640 1 --2- 172.21.15.1:0/2918289875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f428c104010 0x7f428c19a6e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:48.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.557+0000 7f4291c00640 1 -- 172.21.15.1:0/2918289875 >> 172.21.15.1:0/2918289875 conn(0x7f428c0ffa50 msgr2=0x7f428c100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:48.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.557+0000 7f4291c00640 1 -- 172.21.15.1:0/2918289875 shutdown_connections 2024-09-12T01:28:48.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:48.557+0000 7f4291c00640 1 -- 172.21.15.1:0/2918289875 wait complete. 2024-09-12T01:28:48.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:48 smithi053 ceph-mon[35158]: pgmap v374: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-12T01:28:48.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:48 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:28:48.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:48 smithi104 ceph-mon[35219]: pgmap v374: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-12T01:28:48.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:48 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:28:48.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:48 smithi001 ceph-mon[30387]: pgmap v374: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-12T01:28:48.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:48 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:28:49.568 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":374,"stamp":"2024-09-12T01:28:46.704484+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661205,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":774,"num_read_kb":514,"num_write":429,"num_write_kb":3081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1182348,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":352,"ondisk_log_size":352,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":402208,"kb_used_data":15440,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124539616,"statfs":{"total":1151940427776,"available":1151528566784,"internally_reserved":0,"allocated":15810560,"data_stored":8718485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":395752762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":135,"num_read_kb":70,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003521"},"pg_stats":[{"pgid":"2.1f","version":"96'13","reported_seq":117,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:43.420559+0000","last_change":"2024-09-12T01:26:44.440383+0000","last_active":"2024-09-12T01:28:43.420559+0000","last_peered":"2024-09-12T01:28:43.420559+0000","last_clean":"2024-09-12T01:28:43.420559+0000","last_became_active":"2024-09-12T01:26:44.440048+0000","last_became_peered":"2024-09-12T01:26:44.440048+0000","last_unstale":"2024-09-12T01:28:43.420559+0000","last_undegraded":"2024-09-12T01:28:43.420559+0000","last_fullsized":"2024-09-12T01:28:43.420559+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:05:26.956556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":61,"num_read_kb":61,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.417952+0000","last_change":"2024-09-12T01:26:47.483952+0000","last_active":"2024-09-12T01:28:17.417952+0000","last_peered":"2024-09-12T01:28:17.417952+0000","last_clean":"2024-09-12T01:28:17.417952+0000","last_became_active":"2024-09-12T01:26:47.483679+0000","last_became_peered":"2024-09-12T01:26:47.483679+0000","last_unstale":"2024-09-12T01:28:17.417952+0000","last_undegraded":"2024-09-12T01:28:17.417952+0000","last_fullsized":"2024-09-12T01:28:17.417952+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:46:53.997935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.419822+0000","last_change":"2024-09-12T01:26:57.529904+0000","last_active":"2024-09-12T01:28:17.419822+0000","last_peered":"2024-09-12T01:28:17.419822+0000","last_clean":"2024-09-12T01:28:17.419822+0000","last_became_active":"2024-09-12T01:26:57.529332+0000","last_became_peered":"2024-09-12T01:26:57.529332+0000","last_unstale":"2024-09-12T01:28:17.419822+0000","last_undegraded":"2024-09-12T01:28:17.419822+0000","last_fullsized":"2024-09-12T01:28:17.419822+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:54.733150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"96'2","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.418199+0000","last_change":"2024-09-12T01:26:44.448605+0000","last_active":"2024-09-12T01:28:17.418199+0000","last_peered":"2024-09-12T01:28:17.418199+0000","last_clean":"2024-09-12T01:28:17.418199+0000","last_became_active":"2024-09-12T01:26:44.447035+0000","last_became_peered":"2024-09-12T01:26:44.447035+0000","last_unstale":"2024-09-12T01:28:17.418199+0000","last_undegraded":"2024-09-12T01:28:17.418199+0000","last_fullsized":"2024-09-12T01:28:17.418199+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:29:44.222912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.415243+0000","last_change":"2024-09-12T01:26:47.475058+0000","last_active":"2024-09-12T01:28:17.415243+0000","last_peered":"2024-09-12T01:28:17.415243+0000","last_clean":"2024-09-12T01:28:17.415243+0000","last_became_active":"2024-09-12T01:26:47.474057+0000","last_became_peered":"2024-09-12T01:26:47.474057+0000","last_unstale":"2024-09-12T01:28:17.415243+0000","last_undegraded":"2024-09-12T01:28:17.415243+0000","last_fullsized":"2024-09-12T01:28:17.415243+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:58:20.039831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.505782+0000","last_change":"2024-09-12T01:26:57.532139+0000","last_active":"2024-09-12T01:28:17.505782+0000","last_peered":"2024-09-12T01:28:17.505782+0000","last_clean":"2024-09-12T01:28:17.505782+0000","last_became_active":"2024-09-12T01:26:57.531718+0000","last_became_peered":"2024-09-12T01:26:57.531718+0000","last_unstale":"2024-09-12T01:28:17.505782+0000","last_undegraded":"2024-09-12T01:28:17.505782+0000","last_fullsized":"2024-09-12T01:28:17.505782+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:10.414782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"96'12","reported_seq":66,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.508075+0000","last_change":"2024-09-12T01:27:16.631523+0000","last_active":"2024-09-12T01:28:17.508075+0000","last_peered":"2024-09-12T01:28:17.508075+0000","last_clean":"2024-09-12T01:28:17.508075+0000","last_became_active":"2024-09-12T01:27:16.622591+0000","last_became_peered":"2024-09-12T01:27:16.622591+0000","last_unstale":"2024-09-12T01:28:17.508075+0000","last_undegraded":"2024-09-12T01:28:17.508075+0000","last_fullsized":"2024-09-12T01:28:17.508075+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:14:12.793775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506754+0000","last_change":"2024-09-12T01:26:47.477891+0000","last_active":"2024-09-12T01:28:17.506754+0000","last_peered":"2024-09-12T01:28:17.506754+0000","last_clean":"2024-09-12T01:28:17.506754+0000","last_became_active":"2024-09-12T01:26:47.477107+0000","last_became_peered":"2024-09-12T01:26:47.477107+0000","last_unstale":"2024-09-12T01:28:17.506754+0000","last_undegraded":"2024-09-12T01:28:17.506754+0000","last_fullsized":"2024-09-12T01:28:17.506754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:35:17.262896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":35,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.507397+0000","last_change":"2024-09-12T01:27:16.832962+0000","last_active":"2024-09-12T01:28:17.507397+0000","last_peered":"2024-09-12T01:28:17.507397+0000","last_clean":"2024-09-12T01:28:17.507397+0000","last_became_active":"2024-09-12T01:27:16.832463+0000","last_became_peered":"2024-09-12T01:27:16.832463+0000","last_unstale":"2024-09-12T01:28:17.507397+0000","last_undegraded":"2024-09-12T01:28:17.507397+0000","last_fullsized":"2024-09-12T01:28:17.507397+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:10:24.321133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"96'5","reported_seq":49,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.508051+0000","last_change":"2024-09-12T01:26:44.445375+0000","last_active":"2024-09-12T01:28:17.508051+0000","last_peered":"2024-09-12T01:28:17.508051+0000","last_clean":"2024-09-12T01:28:17.508051+0000","last_became_active":"2024-09-12T01:26:44.444820+0000","last_became_peered":"2024-09-12T01:26:44.444820+0000","last_unstale":"2024-09-12T01:28:17.508051+0000","last_undegraded":"2024-09-12T01:28:17.508051+0000","last_fullsized":"2024-09-12T01:28:17.508051+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:05:40.790222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506747+0000","last_change":"2024-09-12T01:26:47.480650+0000","last_active":"2024-09-12T01:28:17.506747+0000","last_peered":"2024-09-12T01:28:17.506747+0000","last_clean":"2024-09-12T01:28:17.506747+0000","last_became_active":"2024-09-12T01:26:47.480266+0000","last_became_peered":"2024-09-12T01:26:47.480266+0000","last_unstale":"2024-09-12T01:28:17.506747+0000","last_undegraded":"2024-09-12T01:28:17.506747+0000","last_fullsized":"2024-09-12T01:28:17.506747+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:00.218397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506101+0000","last_change":"2024-09-12T01:26:57.532582+0000","last_active":"2024-09-12T01:28:17.506101+0000","last_peered":"2024-09-12T01:28:17.506101+0000","last_clean":"2024-09-12T01:28:17.506101+0000","last_became_active":"2024-09-12T01:26:57.531966+0000","last_became_peered":"2024-09-12T01:26:57.531966+0000","last_unstale":"2024-09-12T01:28:17.506101+0000","last_undegraded":"2024-09-12T01:28:17.506101+0000","last_fullsized":"2024-09-12T01:28:17.506101+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:01:13.886678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.415608+0000","last_change":"2024-09-12T01:26:47.486567+0000","last_active":"2024-09-12T01:28:17.415608+0000","last_peered":"2024-09-12T01:28:17.415608+0000","last_clean":"2024-09-12T01:28:17.415608+0000","last_became_active":"2024-09-12T01:26:47.486219+0000","last_became_peered":"2024-09-12T01:26:47.486219+0000","last_unstale":"2024-09-12T01:28:17.415608+0000","last_undegraded":"2024-09-12T01:28:17.415608+0000","last_fullsized":"2024-09-12T01:28:17.415608+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:48:53.889111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"96'4","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.419178+0000","last_change":"2024-09-12T01:27:16.624095+0000","last_active":"2024-09-12T01:28:17.419178+0000","last_peered":"2024-09-12T01:28:17.419178+0000","last_clean":"2024-09-12T01:28:17.419178+0000","last_became_active":"2024-09-12T01:27:16.620392+0000","last_became_peered":"2024-09-12T01:27:16.620392+0000","last_unstale":"2024-09-12T01:28:17.419178+0000","last_undegraded":"2024-09-12T01:28:17.419178+0000","last_fullsized":"2024-09-12T01:28:17.419178+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:56:05.516283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506581+0000","last_change":"2024-09-12T01:26:57.729865+0000","last_active":"2024-09-12T01:28:17.506581+0000","last_peered":"2024-09-12T01:28:17.506581+0000","last_clean":"2024-09-12T01:28:17.506581+0000","last_became_active":"2024-09-12T01:26:57.728819+0000","last_became_peered":"2024-09-12T01:26:57.728819+0000","last_unstale":"2024-09-12T01:28:17.506581+0000","last_undegraded":"2024-09-12T01:28:17.506581+0000","last_fullsized":"2024-09-12T01:28:17.506581+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:11.337683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.417310+0000","last_change":"2024-09-12T01:26:47.485503+0000","last_active":"2024-09-12T01:28:17.417310+0000","last_peered":"2024-09-12T01:28:17.417310+0000","last_clean":"2024-09-12T01:28:17.417310+0000","last_became_active":"2024-09-12T01:26:47.485187+0000","last_became_peered":"2024-09-12T01:26:47.485187+0000","last_unstale":"2024-09-12T01:28:17.417310+0000","last_undegraded":"2024-09-12T01:28:17.417310+0000","last_fullsized":"2024-09-12T01:28:17.417310+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:24:52.544761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"96'3","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.419977+0000","last_change":"2024-09-12T01:26:44.447010+0000","last_active":"2024-09-12T01:28:17.419977+0000","last_peered":"2024-09-12T01:28:17.419977+0000","last_clean":"2024-09-12T01:28:17.419977+0000","last_became_active":"2024-09-12T01:26:44.446643+0000","last_became_peered":"2024-09-12T01:26:44.446643+0000","last_unstale":"2024-09-12T01:28:17.419977+0000","last_undegraded":"2024-09-12T01:28:17.419977+0000","last_fullsized":"2024-09-12T01:28:17.419977+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:56:33.298732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"96'1","reported_seq":29,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506880+0000","last_change":"2024-09-12T01:26:57.524687+0000","last_active":"2024-09-12T01:28:17.506880+0000","last_peered":"2024-09-12T01:28:17.506880+0000","last_clean":"2024-09-12T01:28:17.506880+0000","last_became_active":"2024-09-12T01:26:57.524201+0000","last_became_peered":"2024-09-12T01:26:57.524201+0000","last_unstale":"2024-09-12T01:28:17.506880+0000","last_undegraded":"2024-09-12T01:28:17.506880+0000","last_fullsized":"2024-09-12T01:28:17.506880+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:51:49.147460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.418610+0000","last_change":"2024-09-12T01:26:47.478432+0000","last_active":"2024-09-12T01:28:17.418610+0000","last_peered":"2024-09-12T01:28:17.418610+0000","last_clean":"2024-09-12T01:28:17.418610+0000","last_became_active":"2024-09-12T01:26:47.478109+0000","last_became_peered":"2024-09-12T01:26:47.478109+0000","last_unstale":"2024-09-12T01:28:17.418610+0000","last_undegraded":"2024-09-12T01:28:17.418610+0000","last_fullsized":"2024-09-12T01:28:17.418610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:30:19.917875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"96'7","reported_seq":51,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.417657+0000","last_change":"2024-09-12T01:26:44.441961+0000","last_active":"2024-09-12T01:28:17.417657+0000","last_peered":"2024-09-12T01:28:17.417657+0000","last_clean":"2024-09-12T01:28:17.417657+0000","last_became_active":"2024-09-12T01:26:44.441644+0000","last_became_peered":"2024-09-12T01:26:44.441644+0000","last_unstale":"2024-09-12T01:28:17.417657+0000","last_undegraded":"2024-09-12T01:28:17.417657+0000","last_fullsized":"2024-09-12T01:28:17.417657+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:40:48.561119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506845+0000","last_change":"2024-09-12T01:26:57.531164+0000","last_active":"2024-09-12T01:28:17.506845+0000","last_peered":"2024-09-12T01:28:17.506845+0000","last_clean":"2024-09-12T01:28:17.506845+0000","last_became_active":"2024-09-12T01:26:57.530513+0000","last_became_peered":"2024-09-12T01:26:57.530513+0000","last_unstale":"2024-09-12T01:28:17.506845+0000","last_undegraded":"2024-09-12T01:28:17.506845+0000","last_fullsized":"2024-09-12T01:28:17.506845+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:08:55.385620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.507365+0000","last_change":"2024-09-12T01:26:47.475562+0000","last_active":"2024-09-12T01:28:17.507365+0000","last_peered":"2024-09-12T01:28:17.507365+0000","last_clean":"2024-09-12T01:28:17.507365+0000","last_became_active":"2024-09-12T01:26:47.475167+0000","last_became_peered":"2024-09-12T01:26:47.475167+0000","last_unstale":"2024-09-12T01:28:17.507365+0000","last_undegraded":"2024-09-12T01:28:17.507365+0000","last_fullsized":"2024-09-12T01:28:17.507365+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:39:34.706811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"96'2","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506301+0000","last_change":"2024-09-12T01:26:44.447264+0000","last_active":"2024-09-12T01:28:17.506301+0000","last_peered":"2024-09-12T01:28:17.506301+0000","last_clean":"2024-09-12T01:28:17.506301+0000","last_became_active":"2024-09-12T01:26:44.446885+0000","last_became_peered":"2024-09-12T01:26:44.446885+0000","last_unstale":"2024-09-12T01:28:17.506301+0000","last_undegraded":"2024-09-12T01:28:17.506301+0000","last_fullsized":"2024-09-12T01:28:17.506301+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:56:27.229949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506780+0000","last_change":"2024-09-12T01:26:57.729792+0000","last_active":"2024-09-12T01:28:17.506780+0000","last_peered":"2024-09-12T01:28:17.506780+0000","last_clean":"2024-09-12T01:28:17.506780+0000","last_became_active":"2024-09-12T01:26:57.729252+0000","last_became_peered":"2024-09-12T01:26:57.729252+0000","last_unstale":"2024-09-12T01:28:17.506780+0000","last_undegraded":"2024-09-12T01:28:17.506780+0000","last_fullsized":"2024-09-12T01:28:17.506780+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:34:57.106270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.415242+0000","last_change":"2024-09-12T01:26:47.479179+0000","last_active":"2024-09-12T01:28:17.415242+0000","last_peered":"2024-09-12T01:28:17.415242+0000","last_clean":"2024-09-12T01:28:17.415242+0000","last_became_active":"2024-09-12T01:26:47.478783+0000","last_became_peered":"2024-09-12T01:26:47.478783+0000","last_unstale":"2024-09-12T01:28:17.415242+0000","last_undegraded":"2024-09-12T01:28:17.415242+0000","last_fullsized":"2024-09-12T01:28:17.415242+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:23:22.548509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"96'7","reported_seq":51,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.507766+0000","last_change":"2024-09-12T01:26:44.471854+0000","last_active":"2024-09-12T01:28:17.507766+0000","last_peered":"2024-09-12T01:28:17.507766+0000","last_clean":"2024-09-12T01:28:17.507766+0000","last_became_active":"2024-09-12T01:26:44.471511+0000","last_became_peered":"2024-09-12T01:26:44.471511+0000","last_unstale":"2024-09-12T01:28:17.507766+0000","last_undegraded":"2024-09-12T01:28:17.507766+0000","last_fullsized":"2024-09-12T01:28:17.507766+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:00:26.639984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.419824+0000","last_change":"2024-09-12T01:26:57.520467+0000","last_active":"2024-09-12T01:28:17.419824+0000","last_peered":"2024-09-12T01:28:17.419824+0000","last_clean":"2024-09-12T01:28:17.419824+0000","last_became_active":"2024-09-12T01:26:57.520048+0000","last_became_peered":"2024-09-12T01:26:57.520048+0000","last_unstale":"2024-09-12T01:28:17.419824+0000","last_undegraded":"2024-09-12T01:28:17.419824+0000","last_fullsized":"2024-09-12T01:28:17.419824+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:42:05.303804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.418170+0000","last_change":"2024-09-12T01:26:47.476484+0000","last_active":"2024-09-12T01:28:17.418170+0000","last_peered":"2024-09-12T01:28:17.418170+0000","last_clean":"2024-09-12T01:28:17.418170+0000","last_became_active":"2024-09-12T01:26:47.475610+0000","last_became_peered":"2024-09-12T01:26:47.475610+0000","last_unstale":"2024-09-12T01:28:17.418170+0000","last_undegraded":"2024-09-12T01:28:17.418170+0000","last_fullsized":"2024-09-12T01:28:17.418170+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:44.637986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"96'14","reported_seq":57,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.507202+0000","last_change":"2024-09-12T01:26:44.442988+0000","last_active":"2024-09-12T01:28:17.507202+0000","last_peered":"2024-09-12T01:28:17.507202+0000","last_clean":"2024-09-12T01:28:17.507202+0000","last_became_active":"2024-09-12T01:26:44.442612+0000","last_became_peered":"2024-09-12T01:26:44.442612+0000","last_unstale":"2024-09-12T01:28:17.507202+0000","last_undegraded":"2024-09-12T01:28:17.507202+0000","last_fullsized":"2024-09-12T01:28:17.507202+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:38:25.486470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.416883+0000","last_change":"2024-09-12T01:26:57.729762+0000","last_active":"2024-09-12T01:28:17.416883+0000","last_peered":"2024-09-12T01:28:17.416883+0000","last_clean":"2024-09-12T01:28:17.416883+0000","last_became_active":"2024-09-12T01:26:57.728944+0000","last_became_peered":"2024-09-12T01:26:57.728944+0000","last_unstale":"2024-09-12T01:28:17.416883+0000","last_undegraded":"2024-09-12T01:28:17.416883+0000","last_fullsized":"2024-09-12T01:28:17.416883+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:33:58.030332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506573+0000","last_change":"2024-09-12T01:26:47.482512+0000","last_active":"2024-09-12T01:28:17.506573+0000","last_peered":"2024-09-12T01:28:17.506573+0000","last_clean":"2024-09-12T01:28:17.506573+0000","last_became_active":"2024-09-12T01:26:47.481344+0000","last_became_peered":"2024-09-12T01:26:47.481344+0000","last_unstale":"2024-09-12T01:28:17.506573+0000","last_undegraded":"2024-09-12T01:28:17.506573+0000","last_fullsized":"2024-09-12T01:28:17.506573+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:52:22.920248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"96'4","reported_seq":48,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.508081+0000","last_change":"2024-09-12T01:26:44.447654+0000","last_active":"2024-09-12T01:28:17.508081+0000","last_peered":"2024-09-12T01:28:17.508081+0000","last_clean":"2024-09-12T01:28:17.508081+0000","last_became_active":"2024-09-12T01:26:44.447148+0000","last_became_peered":"2024-09-12T01:26:44.447148+0000","last_unstale":"2024-09-12T01:28:17.508081+0000","last_undegraded":"2024-09-12T01:28:17.508081+0000","last_fullsized":"2024-09-12T01:28:17.508081+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:40:02.928329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.419100+0000","last_change":"2024-09-12T01:26:57.530191+0000","last_active":"2024-09-12T01:28:17.419100+0000","last_peered":"2024-09-12T01:28:17.419100+0000","last_clean":"2024-09-12T01:28:17.419100+0000","last_became_active":"2024-09-12T01:26:57.529752+0000","last_became_peered":"2024-09-12T01:26:57.529752+0000","last_unstale":"2024-09-12T01:28:17.419100+0000","last_undegraded":"2024-09-12T01:28:17.419100+0000","last_fullsized":"2024-09-12T01:28:17.419100+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:00:16.639297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.507519+0000","last_change":"2024-09-12T01:26:47.478991+0000","last_active":"2024-09-12T01:28:17.507519+0000","last_peered":"2024-09-12T01:28:17.507519+0000","last_clean":"2024-09-12T01:28:17.507519+0000","last_became_active":"2024-09-12T01:26:47.478228+0000","last_became_peered":"2024-09-12T01:26:47.478228+0000","last_unstale":"2024-09-12T01:28:17.507519+0000","last_undegraded":"2024-09-12T01:28:17.507519+0000","last_fullsized":"2024-09-12T01:28:17.507519+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:45.236546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"96'1","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506987+0000","last_change":"2024-09-12T01:26:44.441143+0000","last_active":"2024-09-12T01:28:17.506987+0000","last_peered":"2024-09-12T01:28:17.506987+0000","last_clean":"2024-09-12T01:28:17.506987+0000","last_became_active":"2024-09-12T01:26:44.439361+0000","last_became_peered":"2024-09-12T01:26:44.439361+0000","last_unstale":"2024-09-12T01:28:17.506987+0000","last_undegraded":"2024-09-12T01:28:17.506987+0000","last_fullsized":"2024-09-12T01:28:17.506987+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:22:34.257977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.507368+0000","last_change":"2024-09-12T01:26:57.729010+0000","last_active":"2024-09-12T01:28:17.507368+0000","last_peered":"2024-09-12T01:28:17.507368+0000","last_clean":"2024-09-12T01:28:17.507368+0000","last_became_active":"2024-09-12T01:26:57.728576+0000","last_became_peered":"2024-09-12T01:26:57.728576+0000","last_unstale":"2024-09-12T01:28:17.507368+0000","last_undegraded":"2024-09-12T01:28:17.507368+0000","last_fullsized":"2024-09-12T01:28:17.507368+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:48:48.798028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.507275+0000","last_change":"2024-09-12T01:27:16.832723+0000","last_active":"2024-09-12T01:28:17.507275+0000","last_peered":"2024-09-12T01:28:17.507275+0000","last_clean":"2024-09-12T01:28:17.507275+0000","last_became_active":"2024-09-12T01:27:16.832241+0000","last_became_peered":"2024-09-12T01:27:16.832241+0000","last_unstale":"2024-09-12T01:28:17.507275+0000","last_undegraded":"2024-09-12T01:28:17.507275+0000","last_fullsized":"2024-09-12T01:28:17.507275+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:26:09.610468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"96'6","reported_seq":110,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:41.418335+0000","last_change":"2024-09-12T01:26:44.440802+0000","last_active":"2024-09-12T01:28:41.418335+0000","last_peered":"2024-09-12T01:28:41.418335+0000","last_clean":"2024-09-12T01:28:41.418335+0000","last_became_active":"2024-09-12T01:26:44.439686+0000","last_became_peered":"2024-09-12T01:26:44.439686+0000","last_unstale":"2024-09-12T01:28:41.418335+0000","last_undegraded":"2024-09-12T01:28:41.418335+0000","last_fullsized":"2024-09-12T01:28:41.418335+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:56.608975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":60,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506438+0000","last_change":"2024-09-12T01:26:57.537046+0000","last_active":"2024-09-12T01:28:17.506438+0000","last_peered":"2024-09-12T01:28:17.506438+0000","last_clean":"2024-09-12T01:28:17.506438+0000","last_became_active":"2024-09-12T01:26:57.536589+0000","last_became_peered":"2024-09-12T01:26:57.536589+0000","last_unstale":"2024-09-12T01:28:17.506438+0000","last_undegraded":"2024-09-12T01:28:17.506438+0000","last_fullsized":"2024-09-12T01:28:17.506438+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:24:54.232963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506200+0000","last_change":"2024-09-12T01:26:47.476288+0000","last_active":"2024-09-12T01:28:17.506200+0000","last_peered":"2024-09-12T01:28:17.506200+0000","last_clean":"2024-09-12T01:28:17.506200+0000","last_became_active":"2024-09-12T01:26:47.475836+0000","last_became_peered":"2024-09-12T01:26:47.475836+0000","last_unstale":"2024-09-12T01:28:17.506200+0000","last_undegraded":"2024-09-12T01:28:17.506200+0000","last_fullsized":"2024-09-12T01:28:17.506200+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:39:43.017253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"96'5","reported_seq":110,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:44.422289+0000","last_change":"2024-09-12T01:26:44.446539+0000","last_active":"2024-09-12T01:28:44.422289+0000","last_peered":"2024-09-12T01:28:44.422289+0000","last_clean":"2024-09-12T01:28:44.422289+0000","last_became_active":"2024-09-12T01:26:44.446219+0000","last_became_peered":"2024-09-12T01:26:44.446219+0000","last_unstale":"2024-09-12T01:28:44.422289+0000","last_undegraded":"2024-09-12T01:28:44.422289+0000","last_fullsized":"2024-09-12T01:28:44.422289+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:50:32.584078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":62,"num_read_kb":62,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506648+0000","last_change":"2024-09-12T01:26:57.729445+0000","last_active":"2024-09-12T01:28:17.506648+0000","last_peered":"2024-09-12T01:28:17.506648+0000","last_clean":"2024-09-12T01:28:17.506648+0000","last_became_active":"2024-09-12T01:26:57.728948+0000","last_became_peered":"2024-09-12T01:26:57.728948+0000","last_unstale":"2024-09-12T01:28:17.506648+0000","last_undegraded":"2024-09-12T01:28:17.506648+0000","last_fullsized":"2024-09-12T01:28:17.506648+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:33:27.017565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.505831+0000","last_change":"2024-09-12T01:26:47.481965+0000","last_active":"2024-09-12T01:28:17.505831+0000","last_peered":"2024-09-12T01:28:17.505831+0000","last_clean":"2024-09-12T01:28:17.505831+0000","last_became_active":"2024-09-12T01:26:47.481600+0000","last_became_peered":"2024-09-12T01:26:47.481600+0000","last_unstale":"2024-09-12T01:28:17.505831+0000","last_undegraded":"2024-09-12T01:28:17.505831+0000","last_fullsized":"2024-09-12T01:28:17.505831+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:20.078494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"96'4","reported_seq":19,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.418328+0000","last_change":"2024-09-12T01:27:16.832522+0000","last_active":"2024-09-12T01:28:17.418328+0000","last_peered":"2024-09-12T01:28:17.418328+0000","last_clean":"2024-09-12T01:28:17.418328+0000","last_became_active":"2024-09-12T01:27:16.832090+0000","last_became_peered":"2024-09-12T01:27:16.832090+0000","last_unstale":"2024-09-12T01:28:17.418328+0000","last_undegraded":"2024-09-12T01:28:17.418328+0000","last_fullsized":"2024-09-12T01:28:17.418328+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:15:48.061559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506492+0000","last_change":"2024-09-12T01:26:57.729622+0000","last_active":"2024-09-12T01:28:17.506492+0000","last_peered":"2024-09-12T01:28:17.506492+0000","last_clean":"2024-09-12T01:28:17.506492+0000","last_became_active":"2024-09-12T01:26:57.729175+0000","last_became_peered":"2024-09-12T01:26:57.729175+0000","last_unstale":"2024-09-12T01:28:17.506492+0000","last_undegraded":"2024-09-12T01:28:17.506492+0000","last_fullsized":"2024-09-12T01:28:17.506492+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:41.691685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.417373+0000","last_change":"2024-09-12T01:26:47.479155+0000","last_active":"2024-09-12T01:28:17.417373+0000","last_peered":"2024-09-12T01:28:17.417373+0000","last_clean":"2024-09-12T01:28:17.417373+0000","last_became_active":"2024-09-12T01:26:47.478781+0000","last_became_peered":"2024-09-12T01:26:47.478781+0000","last_unstale":"2024-09-12T01:28:17.417373+0000","last_undegraded":"2024-09-12T01:28:17.417373+0000","last_fullsized":"2024-09-12T01:28:17.417373+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:40:49.730016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"96'6","reported_seq":52,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.507634+0000","last_change":"2024-09-12T01:26:44.437048+0000","last_active":"2024-09-12T01:28:17.507634+0000","last_peered":"2024-09-12T01:28:17.507634+0000","last_clean":"2024-09-12T01:28:17.507634+0000","last_became_active":"2024-09-12T01:26:44.436702+0000","last_became_peered":"2024-09-12T01:26:44.436702+0000","last_unstale":"2024-09-12T01:28:17.507634+0000","last_undegraded":"2024-09-12T01:28:17.507634+0000","last_fullsized":"2024-09-12T01:28:17.507634+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:15:18.857093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":363,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.415480+0000","last_change":"2024-09-12T01:25:28.683724+0000","last_active":"2024-09-12T01:28:17.415480+0000","last_peered":"2024-09-12T01:28:17.415480+0000","last_clean":"2024-09-12T01:28:17.415480+0000","last_became_active":"2024-09-12T01:25:28.683155+0000","last_became_peered":"2024-09-12T01:25:28.683155+0000","last_unstale":"2024-09-12T01:28:17.415480+0000","last_undegraded":"2024-09-12T01:28:17.415480+0000","last_fullsized":"2024-09-12T01:28:17.415480+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:21:41.529611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:21:41.529611+0000","last_clean_scrub_stamp":"2024-09-12T01:21:41.529611+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:13:02.724915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"96'1","reported_seq":36,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.415877+0000","last_change":"2024-09-12T01:27:16.833028+0000","last_active":"2024-09-12T01:28:17.415877+0000","last_peered":"2024-09-12T01:28:17.415877+0000","last_clean":"2024-09-12T01:28:17.415877+0000","last_became_active":"2024-09-12T01:27:16.832416+0000","last_became_peered":"2024-09-12T01:27:16.832416+0000","last_unstale":"2024-09-12T01:28:17.415877+0000","last_undegraded":"2024-09-12T01:28:17.415877+0000","last_fullsized":"2024-09-12T01:28:17.415877+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:49:43.008107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.416164+0000","last_change":"2024-09-12T01:26:47.475467+0000","last_active":"2024-09-12T01:28:17.416164+0000","last_peered":"2024-09-12T01:28:17.416164+0000","last_clean":"2024-09-12T01:28:17.416164+0000","last_became_active":"2024-09-12T01:26:47.474724+0000","last_became_peered":"2024-09-12T01:26:47.474724+0000","last_unstale":"2024-09-12T01:28:17.416164+0000","last_undegraded":"2024-09-12T01:28:17.416164+0000","last_fullsized":"2024-09-12T01:28:17.416164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:56:54.651180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"96'6","reported_seq":114,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:44.436144+0000","last_change":"2024-09-12T01:26:44.472048+0000","last_active":"2024-09-12T01:28:44.436144+0000","last_peered":"2024-09-12T01:28:44.436144+0000","last_clean":"2024-09-12T01:28:44.436144+0000","last_became_active":"2024-09-12T01:26:44.471697+0000","last_became_peered":"2024-09-12T01:26:44.471697+0000","last_unstale":"2024-09-12T01:28:44.436144+0000","last_undegraded":"2024-09-12T01:28:44.436144+0000","last_fullsized":"2024-09-12T01:28:44.436144+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:44:13.071983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":64,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506525+0000","last_change":"2024-09-12T01:26:57.528837+0000","last_active":"2024-09-12T01:28:17.506525+0000","last_peered":"2024-09-12T01:28:17.506525+0000","last_clean":"2024-09-12T01:28:17.506525+0000","last_became_active":"2024-09-12T01:26:57.528285+0000","last_became_peered":"2024-09-12T01:26:57.528285+0000","last_unstale":"2024-09-12T01:28:17.506525+0000","last_undegraded":"2024-09-12T01:28:17.506525+0000","last_fullsized":"2024-09-12T01:28:17.506525+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:12:13.534002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.415874+0000","last_change":"2024-09-12T01:26:47.483456+0000","last_active":"2024-09-12T01:28:17.415874+0000","last_peered":"2024-09-12T01:28:17.415874+0000","last_clean":"2024-09-12T01:28:17.415874+0000","last_became_active":"2024-09-12T01:26:47.482189+0000","last_became_peered":"2024-09-12T01:26:47.482189+0000","last_unstale":"2024-09-12T01:28:17.415874+0000","last_undegraded":"2024-09-12T01:28:17.415874+0000","last_fullsized":"2024-09-12T01:28:17.415874+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:18:55.466266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"96'3","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506871+0000","last_change":"2024-09-12T01:26:44.443507+0000","last_active":"2024-09-12T01:28:17.506871+0000","last_peered":"2024-09-12T01:28:17.506871+0000","last_clean":"2024-09-12T01:28:17.506871+0000","last_became_active":"2024-09-12T01:26:44.443147+0000","last_became_peered":"2024-09-12T01:26:44.443147+0000","last_unstale":"2024-09-12T01:28:17.506871+0000","last_undegraded":"2024-09-12T01:28:17.506871+0000","last_fullsized":"2024-09-12T01:28:17.506871+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:38:40.433581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.415124+0000","last_change":"2024-09-12T01:26:57.523029+0000","last_active":"2024-09-12T01:28:17.415124+0000","last_peered":"2024-09-12T01:28:17.415124+0000","last_clean":"2024-09-12T01:28:17.415124+0000","last_became_active":"2024-09-12T01:26:57.522565+0000","last_became_peered":"2024-09-12T01:26:57.522565+0000","last_unstale":"2024-09-12T01:28:17.415124+0000","last_undegraded":"2024-09-12T01:28:17.415124+0000","last_fullsized":"2024-09-12T01:28:17.415124+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:36:27.432126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.416019+0000","last_change":"2024-09-12T01:26:47.471958+0000","last_active":"2024-09-12T01:28:17.416019+0000","last_peered":"2024-09-12T01:28:17.416019+0000","last_clean":"2024-09-12T01:28:17.416019+0000","last_became_active":"2024-09-12T01:26:47.471621+0000","last_became_peered":"2024-09-12T01:26:47.471621+0000","last_unstale":"2024-09-12T01:28:17.416019+0000","last_undegraded":"2024-09-12T01:28:17.416019+0000","last_fullsized":"2024-09-12T01:28:17.416019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:46:38.004188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"96'5","reported_seq":48,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506660+0000","last_change":"2024-09-12T01:26:44.442852+0000","last_active":"2024-09-12T01:28:17.506660+0000","last_peered":"2024-09-12T01:28:17.506660+0000","last_clean":"2024-09-12T01:28:17.506660+0000","last_became_active":"2024-09-12T01:26:44.442487+0000","last_became_peered":"2024-09-12T01:26:44.442487+0000","last_unstale":"2024-09-12T01:28:17.506660+0000","last_undegraded":"2024-09-12T01:28:17.506660+0000","last_fullsized":"2024-09-12T01:28:17.506660+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:04:32.688925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506049+0000","last_change":"2024-09-12T01:26:57.521155+0000","last_active":"2024-09-12T01:28:17.506049+0000","last_peered":"2024-09-12T01:28:17.506049+0000","last_clean":"2024-09-12T01:28:17.506049+0000","last_became_active":"2024-09-12T01:26:57.520721+0000","last_became_peered":"2024-09-12T01:26:57.520721+0000","last_unstale":"2024-09-12T01:28:17.506049+0000","last_undegraded":"2024-09-12T01:28:17.506049+0000","last_fullsized":"2024-09-12T01:28:17.506049+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:58:52.004697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.419297+0000","last_change":"2024-09-12T01:26:47.482564+0000","last_active":"2024-09-12T01:28:17.419297+0000","last_peered":"2024-09-12T01:28:17.419297+0000","last_clean":"2024-09-12T01:28:17.419297+0000","last_became_active":"2024-09-12T01:26:47.482215+0000","last_became_peered":"2024-09-12T01:26:47.482215+0000","last_unstale":"2024-09-12T01:28:17.419297+0000","last_undegraded":"2024-09-12T01:28:17.419297+0000","last_fullsized":"2024-09-12T01:28:17.419297+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:27:40.882473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"96'2","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.508036+0000","last_change":"2024-09-12T01:26:44.442774+0000","last_active":"2024-09-12T01:28:17.508036+0000","last_peered":"2024-09-12T01:28:17.508036+0000","last_clean":"2024-09-12T01:28:17.508036+0000","last_became_active":"2024-09-12T01:26:44.442389+0000","last_became_peered":"2024-09-12T01:26:44.442389+0000","last_unstale":"2024-09-12T01:28:17.508036+0000","last_undegraded":"2024-09-12T01:28:17.508036+0000","last_fullsized":"2024-09-12T01:28:17.508036+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:22.628392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506306+0000","last_change":"2024-09-12T01:26:57.526182+0000","last_active":"2024-09-12T01:28:17.506306+0000","last_peered":"2024-09-12T01:28:17.506306+0000","last_clean":"2024-09-12T01:28:17.506306+0000","last_became_active":"2024-09-12T01:26:57.525869+0000","last_became_peered":"2024-09-12T01:26:57.525869+0000","last_unstale":"2024-09-12T01:28:17.506306+0000","last_undegraded":"2024-09-12T01:28:17.506306+0000","last_fullsized":"2024-09-12T01:28:17.506306+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:40:09.994361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506628+0000","last_change":"2024-09-12T01:26:47.475872+0000","last_active":"2024-09-12T01:28:17.506628+0000","last_peered":"2024-09-12T01:28:17.506628+0000","last_clean":"2024-09-12T01:28:17.506628+0000","last_became_active":"2024-09-12T01:26:47.475536+0000","last_became_peered":"2024-09-12T01:26:47.475536+0000","last_unstale":"2024-09-12T01:28:17.506628+0000","last_undegraded":"2024-09-12T01:28:17.506628+0000","last_fullsized":"2024-09-12T01:28:17.506628+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:03:37.519530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"96'17","reported_seq":61,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506976+0000","last_change":"2024-09-12T01:26:44.448843+0000","last_active":"2024-09-12T01:28:17.506976+0000","last_peered":"2024-09-12T01:28:17.506976+0000","last_clean":"2024-09-12T01:28:17.506976+0000","last_became_active":"2024-09-12T01:26:44.448451+0000","last_became_peered":"2024-09-12T01:26:44.448451+0000","last_unstale":"2024-09-12T01:28:17.506976+0000","last_undegraded":"2024-09-12T01:28:17.506976+0000","last_fullsized":"2024-09-12T01:28:17.506976+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:59:21.422174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506602+0000","last_change":"2024-09-12T01:26:57.541164+0000","last_active":"2024-09-12T01:28:17.506602+0000","last_peered":"2024-09-12T01:28:17.506602+0000","last_clean":"2024-09-12T01:28:17.506602+0000","last_became_active":"2024-09-12T01:26:57.540746+0000","last_became_peered":"2024-09-12T01:26:57.540746+0000","last_unstale":"2024-09-12T01:28:17.506602+0000","last_undegraded":"2024-09-12T01:28:17.506602+0000","last_fullsized":"2024-09-12T01:28:17.506602+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:58:26.870141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.415354+0000","last_change":"2024-09-12T01:26:47.473261+0000","last_active":"2024-09-12T01:28:17.415354+0000","last_peered":"2024-09-12T01:28:17.415354+0000","last_clean":"2024-09-12T01:28:17.415354+0000","last_became_active":"2024-09-12T01:26:47.472902+0000","last_became_peered":"2024-09-12T01:26:47.472902+0000","last_unstale":"2024-09-12T01:28:17.415354+0000","last_undegraded":"2024-09-12T01:28:17.415354+0000","last_fullsized":"2024-09-12T01:28:17.415354+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:58:41.080240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"96'6","reported_seq":111,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:42.420306+0000","last_change":"2024-09-12T01:26:44.440503+0000","last_active":"2024-09-12T01:28:42.420306+0000","last_peered":"2024-09-12T01:28:42.420306+0000","last_clean":"2024-09-12T01:28:42.420306+0000","last_became_active":"2024-09-12T01:26:44.440128+0000","last_became_peered":"2024-09-12T01:26:44.440128+0000","last_unstale":"2024-09-12T01:28:42.420306+0000","last_undegraded":"2024-09-12T01:28:42.420306+0000","last_fullsized":"2024-09-12T01:28:42.420306+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:39:35.373682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":63,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.507000+0000","last_change":"2024-09-12T01:26:57.533883+0000","last_active":"2024-09-12T01:28:17.507000+0000","last_peered":"2024-09-12T01:28:17.507000+0000","last_clean":"2024-09-12T01:28:17.507000+0000","last_became_active":"2024-09-12T01:26:57.533329+0000","last_became_peered":"2024-09-12T01:26:57.533329+0000","last_unstale":"2024-09-12T01:28:17.507000+0000","last_undegraded":"2024-09-12T01:28:17.507000+0000","last_fullsized":"2024-09-12T01:28:17.507000+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:17.655219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.415790+0000","last_change":"2024-09-12T01:26:47.480276+0000","last_active":"2024-09-12T01:28:17.415790+0000","last_peered":"2024-09-12T01:28:17.415790+0000","last_clean":"2024-09-12T01:28:17.415790+0000","last_became_active":"2024-09-12T01:26:47.479965+0000","last_became_peered":"2024-09-12T01:26:47.479965+0000","last_unstale":"2024-09-12T01:28:17.415790+0000","last_undegraded":"2024-09-12T01:28:17.415790+0000","last_fullsized":"2024-09-12T01:28:17.415790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:24:10.339000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"96'5","reported_seq":49,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506650+0000","last_change":"2024-09-12T01:26:44.442346+0000","last_active":"2024-09-12T01:28:17.506650+0000","last_peered":"2024-09-12T01:28:17.506650+0000","last_clean":"2024-09-12T01:28:17.506650+0000","last_became_active":"2024-09-12T01:26:44.442044+0000","last_became_peered":"2024-09-12T01:26:44.442044+0000","last_unstale":"2024-09-12T01:28:17.506650+0000","last_undegraded":"2024-09-12T01:28:17.506650+0000","last_fullsized":"2024-09-12T01:28:17.506650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:47:00.193554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.419065+0000","last_change":"2024-09-12T01:26:57.531650+0000","last_active":"2024-09-12T01:28:17.419065+0000","last_peered":"2024-09-12T01:28:17.419065+0000","last_clean":"2024-09-12T01:28:17.419065+0000","last_became_active":"2024-09-12T01:26:57.531184+0000","last_became_peered":"2024-09-12T01:26:57.531184+0000","last_unstale":"2024-09-12T01:28:17.419065+0000","last_undegraded":"2024-09-12T01:28:17.419065+0000","last_fullsized":"2024-09-12T01:28:17.419065+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:42:56.191490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506152+0000","last_change":"2024-09-12T01:26:47.480344+0000","last_active":"2024-09-12T01:28:17.506152+0000","last_peered":"2024-09-12T01:28:17.506152+0000","last_clean":"2024-09-12T01:28:17.506152+0000","last_became_active":"2024-09-12T01:26:47.479966+0000","last_became_peered":"2024-09-12T01:26:47.479966+0000","last_unstale":"2024-09-12T01:28:17.506152+0000","last_undegraded":"2024-09-12T01:28:17.506152+0000","last_fullsized":"2024-09-12T01:28:17.506152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:44:31.968977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"96'9","reported_seq":55,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.420176+0000","last_change":"2024-09-12T01:26:44.447708+0000","last_active":"2024-09-12T01:28:17.420176+0000","last_peered":"2024-09-12T01:28:17.420176+0000","last_clean":"2024-09-12T01:28:17.420176+0000","last_became_active":"2024-09-12T01:26:44.446974+0000","last_became_peered":"2024-09-12T01:26:44.446974+0000","last_unstale":"2024-09-12T01:28:17.420176+0000","last_undegraded":"2024-09-12T01:28:17.420176+0000","last_fullsized":"2024-09-12T01:28:17.420176+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:01:10.440044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506451+0000","last_change":"2024-09-12T01:26:57.533128+0000","last_active":"2024-09-12T01:28:17.506451+0000","last_peered":"2024-09-12T01:28:17.506451+0000","last_clean":"2024-09-12T01:28:17.506451+0000","last_became_active":"2024-09-12T01:26:57.532731+0000","last_became_peered":"2024-09-12T01:26:57.532731+0000","last_unstale":"2024-09-12T01:28:17.506451+0000","last_undegraded":"2024-09-12T01:28:17.506451+0000","last_fullsized":"2024-09-12T01:28:17.506451+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:09:06.118313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.507882+0000","last_change":"2024-09-12T01:26:47.471545+0000","last_active":"2024-09-12T01:28:17.507882+0000","last_peered":"2024-09-12T01:28:17.507882+0000","last_clean":"2024-09-12T01:28:17.507882+0000","last_became_active":"2024-09-12T01:26:47.471234+0000","last_became_peered":"2024-09-12T01:26:47.471234+0000","last_unstale":"2024-09-12T01:28:17.507882+0000","last_undegraded":"2024-09-12T01:28:17.507882+0000","last_fullsized":"2024-09-12T01:28:17.507882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:24:00.844203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"96'34","reported_seq":225,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:46.425561+0000","last_change":"2024-09-12T01:27:16.629235+0000","last_active":"2024-09-12T01:28:46.425561+0000","last_peered":"2024-09-12T01:28:46.425561+0000","last_clean":"2024-09-12T01:28:46.425561+0000","last_became_active":"2024-09-12T01:27:16.621719+0000","last_became_peered":"2024-09-12T01:27:16.621719+0000","last_unstale":"2024-09-12T01:28:46.425561+0000","last_undegraded":"2024-09-12T01:28:46.425561+0000","last_fullsized":"2024-09-12T01:28:46.425561+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:45:43.011452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63002,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":132,"num_read_kb":127,"num_write":36,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506245+0000","last_change":"2024-09-12T01:26:57.521950+0000","last_active":"2024-09-12T01:28:17.506245+0000","last_peered":"2024-09-12T01:28:17.506245+0000","last_clean":"2024-09-12T01:28:17.506245+0000","last_became_active":"2024-09-12T01:26:57.521574+0000","last_became_peered":"2024-09-12T01:26:57.521574+0000","last_unstale":"2024-09-12T01:28:17.506245+0000","last_undegraded":"2024-09-12T01:28:17.506245+0000","last_fullsized":"2024-09-12T01:28:17.506245+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:23.086233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506860+0000","last_change":"2024-09-12T01:26:47.478212+0000","last_active":"2024-09-12T01:28:17.506860+0000","last_peered":"2024-09-12T01:28:17.506860+0000","last_clean":"2024-09-12T01:28:17.506860+0000","last_became_active":"2024-09-12T01:26:47.477553+0000","last_became_peered":"2024-09-12T01:26:47.477553+0000","last_unstale":"2024-09-12T01:28:17.506860+0000","last_undegraded":"2024-09-12T01:28:17.506860+0000","last_fullsized":"2024-09-12T01:28:17.506860+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:48:22.390040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"96'5","reported_seq":49,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506863+0000","last_change":"2024-09-12T01:26:44.441558+0000","last_active":"2024-09-12T01:28:17.506863+0000","last_peered":"2024-09-12T01:28:17.506863+0000","last_clean":"2024-09-12T01:28:17.506863+0000","last_became_active":"2024-09-12T01:26:44.441099+0000","last_became_peered":"2024-09-12T01:26:44.441099+0000","last_unstale":"2024-09-12T01:28:17.506863+0000","last_undegraded":"2024-09-12T01:28:17.506863+0000","last_fullsized":"2024-09-12T01:28:17.506863+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:07:40.566150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.507261+0000","last_change":"2024-09-12T01:26:57.524838+0000","last_active":"2024-09-12T01:28:17.507261+0000","last_peered":"2024-09-12T01:28:17.507261+0000","last_clean":"2024-09-12T01:28:17.507261+0000","last_became_active":"2024-09-12T01:26:57.524182+0000","last_became_peered":"2024-09-12T01:26:57.524182+0000","last_unstale":"2024-09-12T01:28:17.507261+0000","last_undegraded":"2024-09-12T01:28:17.507261+0000","last_fullsized":"2024-09-12T01:28:17.507261+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:18:55.493116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.415331+0000","last_change":"2024-09-12T01:26:47.478562+0000","last_active":"2024-09-12T01:28:17.415331+0000","last_peered":"2024-09-12T01:28:17.415331+0000","last_clean":"2024-09-12T01:28:17.415331+0000","last_became_active":"2024-09-12T01:26:47.478223+0000","last_became_peered":"2024-09-12T01:26:47.478223+0000","last_unstale":"2024-09-12T01:28:17.415331+0000","last_undegraded":"2024-09-12T01:28:17.415331+0000","last_fullsized":"2024-09-12T01:28:17.415331+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:32:20.472870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"96'4","reported_seq":48,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.415988+0000","last_change":"2024-09-12T01:26:44.443434+0000","last_active":"2024-09-12T01:28:17.415988+0000","last_peered":"2024-09-12T01:28:17.415988+0000","last_clean":"2024-09-12T01:28:17.415988+0000","last_became_active":"2024-09-12T01:26:44.443097+0000","last_became_peered":"2024-09-12T01:26:44.443097+0000","last_unstale":"2024-09-12T01:28:17.415988+0000","last_undegraded":"2024-09-12T01:28:17.415988+0000","last_fullsized":"2024-09-12T01:28:17.415988+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:48:56.626378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.505831+0000","last_change":"2024-09-12T01:26:57.532507+0000","last_active":"2024-09-12T01:28:17.505831+0000","last_peered":"2024-09-12T01:28:17.505831+0000","last_clean":"2024-09-12T01:28:17.505831+0000","last_became_active":"2024-09-12T01:26:57.529757+0000","last_became_peered":"2024-09-12T01:26:57.529757+0000","last_unstale":"2024-09-12T01:28:17.505831+0000","last_undegraded":"2024-09-12T01:28:17.505831+0000","last_fullsized":"2024-09-12T01:28:17.505831+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:35.589141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506805+0000","last_change":"2024-09-12T01:26:47.474632+0000","last_active":"2024-09-12T01:28:17.506805+0000","last_peered":"2024-09-12T01:28:17.506805+0000","last_clean":"2024-09-12T01:28:17.506805+0000","last_became_active":"2024-09-12T01:26:47.473791+0000","last_became_peered":"2024-09-12T01:26:47.473791+0000","last_unstale":"2024-09-12T01:28:17.506805+0000","last_undegraded":"2024-09-12T01:28:17.506805+0000","last_fullsized":"2024-09-12T01:28:17.506805+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:58:45.241657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"96'4","reported_seq":171,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:46.425565+0000","last_change":"2024-09-12T01:26:44.445045+0000","last_active":"2024-09-12T01:28:46.425565+0000","last_peered":"2024-09-12T01:28:46.425565+0000","last_clean":"2024-09-12T01:28:46.425565+0000","last_became_active":"2024-09-12T01:26:44.444668+0000","last_became_peered":"2024-09-12T01:26:44.444668+0000","last_unstale":"2024-09-12T01:28:46.425565+0000","last_undegraded":"2024-09-12T01:28:46.425565+0000","last_fullsized":"2024-09-12T01:28:46.425565+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:05:14.112438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":124,"num_read_kb":124,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":35,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.508728+0000","last_change":"2024-09-12T01:28:17.508728+0000","last_active":"2024-09-12T01:28:17.508728+0000","last_peered":"2024-09-12T01:28:17.508728+0000","last_clean":"2024-09-12T01:28:17.508728+0000","last_became_active":"2024-09-12T01:28:17.508333+0000","last_became_peered":"2024-09-12T01:28:17.508333+0000","last_unstale":"2024-09-12T01:28:17.508728+0000","last_undegraded":"2024-09-12T01:28:17.508728+0000","last_fullsized":"2024-09-12T01:28:17.508728+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:26:00.670909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8],"acting":[3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.415143+0000","last_change":"2024-09-12T01:26:57.534204+0000","last_active":"2024-09-12T01:28:17.415143+0000","last_peered":"2024-09-12T01:28:17.415143+0000","last_clean":"2024-09-12T01:28:17.415143+0000","last_became_active":"2024-09-12T01:26:57.532094+0000","last_became_peered":"2024-09-12T01:26:57.532094+0000","last_unstale":"2024-09-12T01:28:17.415143+0000","last_undegraded":"2024-09-12T01:28:17.415143+0000","last_fullsized":"2024-09-12T01:28:17.415143+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:43:17.714896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506094+0000","last_change":"2024-09-12T01:26:47.480169+0000","last_active":"2024-09-12T01:28:17.506094+0000","last_peered":"2024-09-12T01:28:17.506094+0000","last_clean":"2024-09-12T01:28:17.506094+0000","last_became_active":"2024-09-12T01:26:47.479853+0000","last_became_peered":"2024-09-12T01:26:47.479853+0000","last_unstale":"2024-09-12T01:28:17.506094+0000","last_undegraded":"2024-09-12T01:28:17.506094+0000","last_fullsized":"2024-09-12T01:28:17.506094+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:58:10.578637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"96'3","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.507111+0000","last_change":"2024-09-12T01:26:44.440641+0000","last_active":"2024-09-12T01:28:17.507111+0000","last_peered":"2024-09-12T01:28:17.507111+0000","last_clean":"2024-09-12T01:28:17.507111+0000","last_became_active":"2024-09-12T01:26:44.440315+0000","last_became_peered":"2024-09-12T01:26:44.440315+0000","last_unstale":"2024-09-12T01:28:17.507111+0000","last_undegraded":"2024-09-12T01:28:17.507111+0000","last_fullsized":"2024-09-12T01:28:17.507111+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:21:52.919820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":35,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.509064+0000","last_change":"2024-09-12T01:28:17.509064+0000","last_active":"2024-09-12T01:28:17.509064+0000","last_peered":"2024-09-12T01:28:17.509064+0000","last_clean":"2024-09-12T01:28:17.509064+0000","last_became_active":"2024-09-12T01:28:17.508534+0000","last_became_peered":"2024-09-12T01:28:17.508534+0000","last_unstale":"2024-09-12T01:28:17.509064+0000","last_undegraded":"2024-09-12T01:28:17.509064+0000","last_fullsized":"2024-09-12T01:28:17.509064+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:32:29.004296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.417283+0000","last_change":"2024-09-12T01:26:47.476383+0000","last_active":"2024-09-12T01:28:17.417283+0000","last_peered":"2024-09-12T01:28:17.417283+0000","last_clean":"2024-09-12T01:28:17.417283+0000","last_became_active":"2024-09-12T01:26:47.475759+0000","last_became_peered":"2024-09-12T01:26:47.475759+0000","last_unstale":"2024-09-12T01:28:17.417283+0000","last_undegraded":"2024-09-12T01:28:17.417283+0000","last_fullsized":"2024-09-12T01:28:17.417283+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:14:38.898332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"96'3","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.505920+0000","last_change":"2024-09-12T01:26:44.445418+0000","last_active":"2024-09-12T01:28:17.505920+0000","last_peered":"2024-09-12T01:28:17.505920+0000","last_clean":"2024-09-12T01:28:17.505920+0000","last_became_active":"2024-09-12T01:26:44.444396+0000","last_became_peered":"2024-09-12T01:26:44.444396+0000","last_unstale":"2024-09-12T01:28:17.505920+0000","last_undegraded":"2024-09-12T01:28:17.505920+0000","last_fullsized":"2024-09-12T01:28:17.505920+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:16:16.595947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"96'7","reported_seq":56,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.507070+0000","last_change":"2024-09-12T01:27:16.620062+0000","last_active":"2024-09-12T01:28:17.507070+0000","last_peered":"2024-09-12T01:28:17.507070+0000","last_clean":"2024-09-12T01:28:17.507070+0000","last_became_active":"2024-09-12T01:27:16.619031+0000","last_became_peered":"2024-09-12T01:27:16.619031+0000","last_unstale":"2024-09-12T01:28:17.507070+0000","last_undegraded":"2024-09-12T01:28:17.507070+0000","last_fullsized":"2024-09-12T01:28:17.507070+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:04:04.329211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506596+0000","last_change":"2024-09-12T01:26:47.478706+0000","last_active":"2024-09-12T01:28:17.506596+0000","last_peered":"2024-09-12T01:28:17.506596+0000","last_clean":"2024-09-12T01:28:17.506596+0000","last_became_active":"2024-09-12T01:26:47.478339+0000","last_became_peered":"2024-09-12T01:26:47.478339+0000","last_unstale":"2024-09-12T01:28:17.506596+0000","last_undegraded":"2024-09-12T01:28:17.506596+0000","last_fullsized":"2024-09-12T01:28:17.506596+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:20:37.180485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.420091+0000","last_change":"2024-09-12T01:26:57.533268+0000","last_active":"2024-09-12T01:28:17.420091+0000","last_peered":"2024-09-12T01:28:17.420091+0000","last_clean":"2024-09-12T01:28:17.420091+0000","last_became_active":"2024-09-12T01:26:57.532862+0000","last_became_peered":"2024-09-12T01:26:57.532862+0000","last_unstale":"2024-09-12T01:28:17.420091+0000","last_undegraded":"2024-09-12T01:28:17.420091+0000","last_fullsized":"2024-09-12T01:28:17.420091+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:53.659734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"96'5","reported_seq":110,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:43.421884+0000","last_change":"2024-09-12T01:26:44.443113+0000","last_active":"2024-09-12T01:28:43.421884+0000","last_peered":"2024-09-12T01:28:43.421884+0000","last_clean":"2024-09-12T01:28:43.421884+0000","last_became_active":"2024-09-12T01:26:44.442642+0000","last_became_peered":"2024-09-12T01:26:44.442642+0000","last_unstale":"2024-09-12T01:28:43.421884+0000","last_undegraded":"2024-09-12T01:28:43.421884+0000","last_fullsized":"2024-09-12T01:28:43.421884+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:16:20.143475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":62,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.507504+0000","last_change":"2024-09-12T01:26:47.476115+0000","last_active":"2024-09-12T01:28:17.507504+0000","last_peered":"2024-09-12T01:28:17.507504+0000","last_clean":"2024-09-12T01:28:17.507504+0000","last_became_active":"2024-09-12T01:26:47.475815+0000","last_became_peered":"2024-09-12T01:26:47.475815+0000","last_unstale":"2024-09-12T01:28:17.507504+0000","last_undegraded":"2024-09-12T01:28:17.507504+0000","last_fullsized":"2024-09-12T01:28:17.507504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:50.775693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.420141+0000","last_change":"2024-09-12T01:26:57.518831+0000","last_active":"2024-09-12T01:28:17.420141+0000","last_peered":"2024-09-12T01:28:17.420141+0000","last_clean":"2024-09-12T01:28:17.420141+0000","last_became_active":"2024-09-12T01:26:57.518439+0000","last_became_peered":"2024-09-12T01:26:57.518439+0000","last_unstale":"2024-09-12T01:28:17.420141+0000","last_undegraded":"2024-09-12T01:28:17.420141+0000","last_fullsized":"2024-09-12T01:28:17.420141+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:49:14.845248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.418514+0000","last_change":"2024-09-12T01:27:16.832809+0000","last_active":"2024-09-12T01:28:17.418514+0000","last_peered":"2024-09-12T01:28:17.418514+0000","last_clean":"2024-09-12T01:28:17.418514+0000","last_became_active":"2024-09-12T01:27:16.832243+0000","last_became_peered":"2024-09-12T01:27:16.832243+0000","last_unstale":"2024-09-12T01:28:17.418514+0000","last_undegraded":"2024-09-12T01:28:17.418514+0000","last_fullsized":"2024-09-12T01:28:17.418514+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:55:00.002884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.420355+0000","last_change":"2024-09-12T01:26:47.475959+0000","last_active":"2024-09-12T01:28:17.420355+0000","last_peered":"2024-09-12T01:28:17.420355+0000","last_clean":"2024-09-12T01:28:17.420355+0000","last_became_active":"2024-09-12T01:26:47.475660+0000","last_became_peered":"2024-09-12T01:26:47.475660+0000","last_unstale":"2024-09-12T01:28:17.420355+0000","last_undegraded":"2024-09-12T01:28:17.420355+0000","last_fullsized":"2024-09-12T01:28:17.420355+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:42.556069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.419882+0000","last_change":"2024-09-12T01:26:47.474380+0000","last_active":"2024-09-12T01:28:17.419882+0000","last_peered":"2024-09-12T01:28:17.419882+0000","last_clean":"2024-09-12T01:28:17.419882+0000","last_became_active":"2024-09-12T01:26:47.474050+0000","last_became_peered":"2024-09-12T01:26:47.474050+0000","last_unstale":"2024-09-12T01:28:17.419882+0000","last_undegraded":"2024-09-12T01:28:17.419882+0000","last_fullsized":"2024-09-12T01:28:17.419882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:19:05.032866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.415001+0000","last_change":"2024-09-12T01:26:47.477056+0000","last_active":"2024-09-12T01:28:17.415001+0000","last_peered":"2024-09-12T01:28:17.415001+0000","last_clean":"2024-09-12T01:28:17.415001+0000","last_became_active":"2024-09-12T01:26:47.476641+0000","last_became_peered":"2024-09-12T01:26:47.476641+0000","last_unstale":"2024-09-12T01:28:17.415001+0000","last_undegraded":"2024-09-12T01:28:17.415001+0000","last_fullsized":"2024-09-12T01:28:17.415001+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:52.924471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506547+0000","last_change":"2024-09-12T01:27:16.832763+0000","last_active":"2024-09-12T01:28:17.506547+0000","last_peered":"2024-09-12T01:28:17.506547+0000","last_clean":"2024-09-12T01:28:17.506547+0000","last_became_active":"2024-09-12T01:27:16.832396+0000","last_became_peered":"2024-09-12T01:27:16.832396+0000","last_unstale":"2024-09-12T01:28:17.506547+0000","last_undegraded":"2024-09-12T01:28:17.506547+0000","last_fullsized":"2024-09-12T01:28:17.506547+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:59.444289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.419494+0000","last_change":"2024-09-12T01:26:47.472125+0000","last_active":"2024-09-12T01:28:17.419494+0000","last_peered":"2024-09-12T01:28:17.419494+0000","last_clean":"2024-09-12T01:28:17.419494+0000","last_became_active":"2024-09-12T01:26:47.471758+0000","last_became_peered":"2024-09-12T01:26:47.471758+0000","last_unstale":"2024-09-12T01:28:17.419494+0000","last_undegraded":"2024-09-12T01:28:17.419494+0000","last_fullsized":"2024-09-12T01:28:17.419494+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:38.949566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.420381+0000","last_change":"2024-09-12T01:26:47.473584+0000","last_active":"2024-09-12T01:28:17.420381+0000","last_peered":"2024-09-12T01:28:17.420381+0000","last_clean":"2024-09-12T01:28:17.420381+0000","last_became_active":"2024-09-12T01:26:47.470089+0000","last_became_peered":"2024-09-12T01:26:47.470089+0000","last_unstale":"2024-09-12T01:28:17.420381+0000","last_undegraded":"2024-09-12T01:28:17.420381+0000","last_fullsized":"2024-09-12T01:28:17.420381+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:44:58.878810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.418101+0000","last_change":"2024-09-12T01:26:47.486547+0000","last_active":"2024-09-12T01:28:17.418101+0000","last_peered":"2024-09-12T01:28:17.418101+0000","last_clean":"2024-09-12T01:28:17.418101+0000","last_became_active":"2024-09-12T01:26:47.486105+0000","last_became_peered":"2024-09-12T01:26:47.486105+0000","last_unstale":"2024-09-12T01:28:17.418101+0000","last_undegraded":"2024-09-12T01:28:17.418101+0000","last_fullsized":"2024-09-12T01:28:17.418101+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:21:18.820930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506197+0000","last_change":"2024-09-12T01:26:47.480459+0000","last_active":"2024-09-12T01:28:17.506197+0000","last_peered":"2024-09-12T01:28:17.506197+0000","last_clean":"2024-09-12T01:28:17.506197+0000","last_became_active":"2024-09-12T01:26:47.478917+0000","last_became_peered":"2024-09-12T01:26:47.478917+0000","last_unstale":"2024-09-12T01:28:17.506197+0000","last_undegraded":"2024-09-12T01:28:17.506197+0000","last_fullsized":"2024-09-12T01:28:17.506197+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:20:54.470888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.417534+0000","last_change":"2024-09-12T01:26:47.480744+0000","last_active":"2024-09-12T01:28:17.417534+0000","last_peered":"2024-09-12T01:28:17.417534+0000","last_clean":"2024-09-12T01:28:17.417534+0000","last_became_active":"2024-09-12T01:26:47.480354+0000","last_became_peered":"2024-09-12T01:26:47.480354+0000","last_unstale":"2024-09-12T01:28:17.417534+0000","last_undegraded":"2024-09-12T01:28:17.417534+0000","last_fullsized":"2024-09-12T01:28:17.417534+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:02:04.670945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506875+0000","last_change":"2024-09-12T01:26:47.476431+0000","last_active":"2024-09-12T01:28:17.506875+0000","last_peered":"2024-09-12T01:28:17.506875+0000","last_clean":"2024-09-12T01:28:17.506875+0000","last_became_active":"2024-09-12T01:26:47.476110+0000","last_became_peered":"2024-09-12T01:26:47.476110+0000","last_unstale":"2024-09-12T01:28:17.506875+0000","last_undegraded":"2024-09-12T01:28:17.506875+0000","last_fullsized":"2024-09-12T01:28:17.506875+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:34:24.086677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506287+0000","last_change":"2024-09-12T01:26:47.484568+0000","last_active":"2024-09-12T01:28:17.506287+0000","last_peered":"2024-09-12T01:28:17.506287+0000","last_clean":"2024-09-12T01:28:17.506287+0000","last_became_active":"2024-09-12T01:26:47.484165+0000","last_became_peered":"2024-09-12T01:26:47.484165+0000","last_unstale":"2024-09-12T01:28:17.506287+0000","last_undegraded":"2024-09-12T01:28:17.506287+0000","last_fullsized":"2024-09-12T01:28:17.506287+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:26:51.324754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.419113+0000","last_change":"2024-09-12T01:26:47.482608+0000","last_active":"2024-09-12T01:28:17.419113+0000","last_peered":"2024-09-12T01:28:17.419113+0000","last_clean":"2024-09-12T01:28:17.419113+0000","last_became_active":"2024-09-12T01:26:47.482299+0000","last_became_peered":"2024-09-12T01:26:47.482299+0000","last_unstale":"2024-09-12T01:28:17.419113+0000","last_undegraded":"2024-09-12T01:28:17.419113+0000","last_fullsized":"2024-09-12T01:28:17.419113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:48:39.516539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.420288+0000","last_change":"2024-09-12T01:26:47.472976+0000","last_active":"2024-09-12T01:28:17.420288+0000","last_peered":"2024-09-12T01:28:17.420288+0000","last_clean":"2024-09-12T01:28:17.420288+0000","last_became_active":"2024-09-12T01:26:47.472495+0000","last_became_peered":"2024-09-12T01:26:47.472495+0000","last_unstale":"2024-09-12T01:28:17.420288+0000","last_undegraded":"2024-09-12T01:28:17.420288+0000","last_fullsized":"2024-09-12T01:28:17.420288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:57.556675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.507184+0000","last_change":"2024-09-12T01:26:47.476004+0000","last_active":"2024-09-12T01:28:17.507184+0000","last_peered":"2024-09-12T01:28:17.507184+0000","last_clean":"2024-09-12T01:28:17.507184+0000","last_became_active":"2024-09-12T01:26:47.475440+0000","last_became_peered":"2024-09-12T01:26:47.475440+0000","last_unstale":"2024-09-12T01:28:17.507184+0000","last_undegraded":"2024-09-12T01:28:17.507184+0000","last_fullsized":"2024-09-12T01:28:17.507184+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:36:47.650577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.419430+0000","last_change":"2024-09-12T01:26:47.483848+0000","last_active":"2024-09-12T01:28:17.419430+0000","last_peered":"2024-09-12T01:28:17.419430+0000","last_clean":"2024-09-12T01:28:17.419430+0000","last_became_active":"2024-09-12T01:26:47.483446+0000","last_became_peered":"2024-09-12T01:26:47.483446+0000","last_unstale":"2024-09-12T01:28:17.419430+0000","last_undegraded":"2024-09-12T01:28:17.419430+0000","last_fullsized":"2024-09-12T01:28:17.419430+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:47.844458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506102+0000","last_change":"2024-09-12T01:26:47.473801+0000","last_active":"2024-09-12T01:28:17.506102+0000","last_peered":"2024-09-12T01:28:17.506102+0000","last_clean":"2024-09-12T01:28:17.506102+0000","last_became_active":"2024-09-12T01:26:47.473268+0000","last_became_peered":"2024-09-12T01:26:47.473268+0000","last_unstale":"2024-09-12T01:28:17.506102+0000","last_undegraded":"2024-09-12T01:28:17.506102+0000","last_fullsized":"2024-09-12T01:28:17.506102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:13:30.595643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.414673+0000","last_change":"2024-09-12T01:26:47.475778+0000","last_active":"2024-09-12T01:28:17.414673+0000","last_peered":"2024-09-12T01:28:17.414673+0000","last_clean":"2024-09-12T01:28:17.414673+0000","last_became_active":"2024-09-12T01:26:47.475477+0000","last_became_peered":"2024-09-12T01:26:47.475477+0000","last_unstale":"2024-09-12T01:28:17.414673+0000","last_undegraded":"2024-09-12T01:28:17.414673+0000","last_fullsized":"2024-09-12T01:28:17.414673+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:56.555424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.414671+0000","last_change":"2024-09-12T01:26:47.474736+0000","last_active":"2024-09-12T01:28:17.414671+0000","last_peered":"2024-09-12T01:28:17.414671+0000","last_clean":"2024-09-12T01:28:17.414671+0000","last_became_active":"2024-09-12T01:26:47.474378+0000","last_became_peered":"2024-09-12T01:26:47.474378+0000","last_unstale":"2024-09-12T01:28:17.414671+0000","last_undegraded":"2024-09-12T01:28:17.414671+0000","last_fullsized":"2024-09-12T01:28:17.414671+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:02:37.818552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506260+0000","last_change":"2024-09-12T01:26:47.479586+0000","last_active":"2024-09-12T01:28:17.506260+0000","last_peered":"2024-09-12T01:28:17.506260+0000","last_clean":"2024-09-12T01:28:17.506260+0000","last_became_active":"2024-09-12T01:26:47.473216+0000","last_became_peered":"2024-09-12T01:26:47.473216+0000","last_unstale":"2024-09-12T01:28:17.506260+0000","last_undegraded":"2024-09-12T01:28:17.506260+0000","last_fullsized":"2024-09-12T01:28:17.506260+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:05:28.994081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.507151+0000","last_change":"2024-09-12T01:26:47.475634+0000","last_active":"2024-09-12T01:28:17.507151+0000","last_peered":"2024-09-12T01:28:17.507151+0000","last_clean":"2024-09-12T01:28:17.507151+0000","last_became_active":"2024-09-12T01:26:47.475319+0000","last_became_peered":"2024-09-12T01:26:47.475319+0000","last_unstale":"2024-09-12T01:28:17.507151+0000","last_undegraded":"2024-09-12T01:28:17.507151+0000","last_fullsized":"2024-09-12T01:28:17.507151+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:40.095276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.416094+0000","last_change":"2024-09-12T01:26:47.475721+0000","last_active":"2024-09-12T01:28:17.416094+0000","last_peered":"2024-09-12T01:28:17.416094+0000","last_clean":"2024-09-12T01:28:17.416094+0000","last_became_active":"2024-09-12T01:26:47.475149+0000","last_became_peered":"2024-09-12T01:26:47.475149+0000","last_unstale":"2024-09-12T01:28:17.416094+0000","last_undegraded":"2024-09-12T01:28:17.416094+0000","last_fullsized":"2024-09-12T01:28:17.416094+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:27:19.946756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.507880+0000","last_change":"2024-09-12T01:26:47.482667+0000","last_active":"2024-09-12T01:28:17.507880+0000","last_peered":"2024-09-12T01:28:17.507880+0000","last_clean":"2024-09-12T01:28:17.507880+0000","last_became_active":"2024-09-12T01:26:47.482258+0000","last_became_peered":"2024-09-12T01:26:47.482258+0000","last_unstale":"2024-09-12T01:28:17.507880+0000","last_undegraded":"2024-09-12T01:28:17.507880+0000","last_fullsized":"2024-09-12T01:28:17.507880+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:10.449000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506527+0000","last_change":"2024-09-12T01:26:47.475406+0000","last_active":"2024-09-12T01:28:17.506527+0000","last_peered":"2024-09-12T01:28:17.506527+0000","last_clean":"2024-09-12T01:28:17.506527+0000","last_became_active":"2024-09-12T01:26:47.474846+0000","last_became_peered":"2024-09-12T01:26:47.474846+0000","last_unstale":"2024-09-12T01:28:17.506527+0000","last_undegraded":"2024-09-12T01:28:17.506527+0000","last_fullsized":"2024-09-12T01:28:17.506527+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:54:02.527691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.417011+0000","last_change":"2024-09-12T01:26:47.477383+0000","last_active":"2024-09-12T01:28:17.417011+0000","last_peered":"2024-09-12T01:28:17.417011+0000","last_clean":"2024-09-12T01:28:17.417011+0000","last_became_active":"2024-09-12T01:26:47.476977+0000","last_became_peered":"2024-09-12T01:26:47.476977+0000","last_unstale":"2024-09-12T01:28:17.417011+0000","last_undegraded":"2024-09-12T01:28:17.417011+0000","last_fullsized":"2024-09-12T01:28:17.417011+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:51:45.310556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.415472+0000","last_change":"2024-09-12T01:26:47.473686+0000","last_active":"2024-09-12T01:28:17.415472+0000","last_peered":"2024-09-12T01:28:17.415472+0000","last_clean":"2024-09-12T01:28:17.415472+0000","last_became_active":"2024-09-12T01:26:47.473363+0000","last_became_peered":"2024-09-12T01:26:47.473363+0000","last_unstale":"2024-09-12T01:28:17.415472+0000","last_undegraded":"2024-09-12T01:28:17.415472+0000","last_fullsized":"2024-09-12T01:28:17.415472+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:08:23.238434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506964+0000","last_change":"2024-09-12T01:26:47.479310+0000","last_active":"2024-09-12T01:28:17.506964+0000","last_peered":"2024-09-12T01:28:17.506964+0000","last_clean":"2024-09-12T01:28:17.506964+0000","last_became_active":"2024-09-12T01:26:47.477416+0000","last_became_peered":"2024-09-12T01:26:47.477416+0000","last_unstale":"2024-09-12T01:28:17.506964+0000","last_undegraded":"2024-09-12T01:28:17.506964+0000","last_fullsized":"2024-09-12T01:28:17.506964+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:01:55.023455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.507634+0000","last_change":"2024-09-12T01:26:47.478455+0000","last_active":"2024-09-12T01:28:17.507634+0000","last_peered":"2024-09-12T01:28:17.507634+0000","last_clean":"2024-09-12T01:28:17.507634+0000","last_became_active":"2024-09-12T01:26:47.478093+0000","last_became_peered":"2024-09-12T01:26:47.478093+0000","last_unstale":"2024-09-12T01:28:17.507634+0000","last_undegraded":"2024-09-12T01:28:17.507634+0000","last_fullsized":"2024-09-12T01:28:17.507634+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:04:41.626677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.507789+0000","last_change":"2024-09-12T01:26:47.479385+0000","last_active":"2024-09-12T01:28:17.507789+0000","last_peered":"2024-09-12T01:28:17.507789+0000","last_clean":"2024-09-12T01:28:17.507789+0000","last_became_active":"2024-09-12T01:26:47.479025+0000","last_became_peered":"2024-09-12T01:26:47.479025+0000","last_unstale":"2024-09-12T01:28:17.507789+0000","last_undegraded":"2024-09-12T01:28:17.507789+0000","last_fullsized":"2024-09-12T01:28:17.507789+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:11.927111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.414794+0000","last_change":"2024-09-12T01:26:47.461567+0000","last_active":"2024-09-12T01:28:17.414794+0000","last_peered":"2024-09-12T01:28:17.414794+0000","last_clean":"2024-09-12T01:28:17.414794+0000","last_became_active":"2024-09-12T01:26:47.461230+0000","last_became_peered":"2024-09-12T01:26:47.461230+0000","last_unstale":"2024-09-12T01:28:17.414794+0000","last_undegraded":"2024-09-12T01:28:17.414794+0000","last_fullsized":"2024-09-12T01:28:17.414794+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:09:24.519448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.415210+0000","last_change":"2024-09-12T01:26:47.472323+0000","last_active":"2024-09-12T01:28:17.415210+0000","last_peered":"2024-09-12T01:28:17.415210+0000","last_clean":"2024-09-12T01:28:17.415210+0000","last_became_active":"2024-09-12T01:26:47.469837+0000","last_became_peered":"2024-09-12T01:26:47.469837+0000","last_unstale":"2024-09-12T01:28:17.415210+0000","last_undegraded":"2024-09-12T01:28:17.415210+0000","last_fullsized":"2024-09-12T01:28:17.415210+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:46.202656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.419215+0000","last_change":"2024-09-12T01:26:47.479707+0000","last_active":"2024-09-12T01:28:17.419215+0000","last_peered":"2024-09-12T01:28:17.419215+0000","last_clean":"2024-09-12T01:28:17.419215+0000","last_became_active":"2024-09-12T01:26:47.479368+0000","last_became_peered":"2024-09-12T01:26:47.479368+0000","last_unstale":"2024-09-12T01:28:17.419215+0000","last_undegraded":"2024-09-12T01:28:17.419215+0000","last_fullsized":"2024-09-12T01:28:17.419215+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:04:30.129381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70603,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":648,"num_read_kb":405,"num_write":246,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":211809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":213,"ondisk_log_size":213,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416422,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32296,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712856,"statfs":{"total":95995035648,"available":95961964544,"internally_reserved":0,"allocated":1134592,"data_stored":564248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645355,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37144,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708008,"statfs":{"total":95995035648,"available":95957000192,"internally_reserved":0,"allocated":1835008,"data_stored":1218162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808880,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28160,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716992,"statfs":{"total":95995035648,"available":95966199808,"internally_reserved":0,"allocated":1159168,"data_stored":563785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070517,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1155072,"data_stored":564163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332154,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37064,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708088,"statfs":{"total":95995035648,"available":95957082112,"internally_reserved":0,"allocated":1753088,"data_stored":1161055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495682,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1134592,"data_stored":559682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757319,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37072,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708080,"statfs":{"total":95995035648,"available":95957073920,"internally_reserved":0,"allocated":1761280,"data_stored":1155851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920845,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28220,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716932,"statfs":{"total":95995035648,"available":95966138368,"internally_reserved":0,"allocated":1216512,"data_stored":626758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182482,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36468,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708684,"statfs":{"total":95995035648,"available":95957692416,"internally_reserved":0,"allocated":1146880,"data_stored":558997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36476,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708676,"statfs":{"total":95995035648,"available":95957684224,"internally_reserved":0,"allocated":1150976,"data_stored":560932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36528,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708624,"statfs":{"total":95995035648,"available":95957630976,"internally_reserved":0,"allocated":1204224,"data_stored":623714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36484,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708668,"statfs":{"total":95995035648,"available":95957676032,"internally_reserved":0,"allocated":1159168,"data_stored":561138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":64095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T01:28:49.574 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T01:28:49.795 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:49 smithi001 ceph-mon[30387]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:28:49.796 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:49 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:28:49.796 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:49 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:28:49.796 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:49 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:28:49.796 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:49 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:28:49.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:49 smithi053 ceph-mon[35158]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:28:49.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:49 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:28:49.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:49 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:28:49.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:49 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:28:49.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:49 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:28:50.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:49 smithi104 ceph-mon[35219]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:28:50.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:49 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:28:50.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:49 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:28:50.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:49 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:28:50.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:49 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:28:50.374 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:28:50.860 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:50 smithi001 ceph-mon[30387]: pgmap v375: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T01:28:50.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:50.974+0000 7f3323e00640 1 -- 172.21.15.1:0/63892550 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3324102360 msgr2=0x7f332410e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:50.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:50.974+0000 7f3323e00640 1 --2- 172.21.15.1:0/63892550 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3324102360 0x7f332410e890 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f331c009930 tx=0x7f331c02a420 comp rx=0 tx=0).stop 2024-09-12T01:28:50.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:50.974+0000 7f3323e00640 1 -- 172.21.15.1:0/63892550 shutdown_connections 2024-09-12T01:28:50.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:50.974+0000 7f3323e00640 1 --2- 172.21.15.1:0/63892550 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3324102360 0x7f332410e890 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:50.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:50.974+0000 7f3323e00640 1 --2- 172.21.15.1:0/63892550 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f33241019c0 0x7f3324101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:50.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:50.974+0000 7f3323e00640 1 --2- 172.21.15.1:0/63892550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33241079c0 0x7f3324107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:50.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:50.974+0000 7f3323e00640 1 -- 172.21.15.1:0/63892550 >> 172.21.15.1:0/63892550 conn(0x7f33240fd740 msgr2=0x7f33240ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:50.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:50.974+0000 7f3323e00640 1 -- 172.21.15.1:0/63892550 shutdown_connections 2024-09-12T01:28:50.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:50.974+0000 7f3323e00640 1 -- 172.21.15.1:0/63892550 wait complete. 2024-09-12T01:28:50.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:50.976+0000 7f3323e00640 1 Processor -- start 2024-09-12T01:28:50.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:50.976+0000 7f3323e00640 1 -- start start 2024-09-12T01:28:50.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:50.976+0000 7f3323e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33241019c0 0x7f332419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:50.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:50.976+0000 7f3323e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3324102360 0x7f332419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:50.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:50.976+0000 7f3323e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f33241079c0 0x7f332419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:50.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:50.976+0000 7f3323e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3324111e10 con 0x7f33241019c0 2024-09-12T01:28:50.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:50.976+0000 7f3323e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3324111c90 con 0x7f3324102360 2024-09-12T01:28:50.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:50.976+0000 7f3323e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3324111f90 con 0x7f33241079c0 2024-09-12T01:28:50.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:50.977+0000 7f3322a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33241019c0 0x7f332419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:50.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:50.977+0000 7f3322a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33241019c0 0x7f332419a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53132/0 (socket says 172.21.15.1:53132) 2024-09-12T01:28:50.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:50.977+0000 7f3322a00640 1 -- 172.21.15.1:0/633465453 learned_addr learned my addr 172.21.15.1:0/633465453 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:28:50.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:50.977+0000 7f331a000640 1 --2- 172.21.15.1:0/633465453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3324102360 0x7f332419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:50.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:50.977+0000 7f3323400640 1 --2- 172.21.15.1:0/633465453 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f33241079c0 0x7f332419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:50.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:50.977+0000 7f3322a00640 1 -- 172.21.15.1:0/633465453 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f33241079c0 msgr2=0x7f332419ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:50.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:50.977+0000 7f3322a00640 1 --2- 172.21.15.1:0/633465453 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f33241079c0 0x7f332419ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:50.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:50.977+0000 7f3322a00640 1 -- 172.21.15.1:0/633465453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3324102360 msgr2=0x7f332419a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:50.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:50.977+0000 7f3322a00640 1 --2- 172.21.15.1:0/633465453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3324102360 0x7f332419a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:50.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:50.977+0000 7f3322a00640 1 -- 172.21.15.1:0/633465453 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f332419f400 con 0x7f33241019c0 2024-09-12T01:28:50.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:50.977+0000 7f331a000640 1 --2- 172.21.15.1:0/633465453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3324102360 0x7f332419a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:28:50.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:50.977+0000 7f3322a00640 1 --2- 172.21.15.1:0/633465453 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33241019c0 0x7f332419a450 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3314009d50 tx=0x7f331400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:50.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:50.978+0000 7f331be00640 1 -- 172.21.15.1:0/633465453 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3314003b70 con 0x7f33241019c0 2024-09-12T01:28:50.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:50.978+0000 7f331be00640 1 -- 172.21.15.1:0/633465453 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3314003d10 con 0x7f33241019c0 2024-09-12T01:28:50.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:50.978+0000 7f3323e00640 1 -- 172.21.15.1:0/633465453 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f332419f6f0 con 0x7f33241019c0 2024-09-12T01:28:50.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:50.978+0000 7f331be00640 1 -- 172.21.15.1:0/633465453 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3314010a20 con 0x7f33241019c0 2024-09-12T01:28:50.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:50.978+0000 7f3323e00640 1 -- 172.21.15.1:0/633465453 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33241a6fd0 con 0x7f33241019c0 2024-09-12T01:28:50.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:50.979+0000 7f3323e00640 1 -- 172.21.15.1:0/633465453 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3324101e20 con 0x7f33241019c0 2024-09-12T01:28:50.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:50.981+0000 7f331be00640 1 -- 172.21.15.1:0/633465453 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f3314003ec0 con 0x7f33241019c0 2024-09-12T01:28:50.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:50.982+0000 7f331be00640 1 --2- 172.21.15.1:0/633465453 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f33080a2c90 0x7f33080a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:50.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:50.982+0000 7f331be00640 1 -- 172.21.15.1:0/633465453 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f33140cfb60 con 0x7f33241019c0 2024-09-12T01:28:50.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:50.982+0000 7f331a000640 1 --2- 172.21.15.1:0/633465453 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f33080a2c90 0x7f33080a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:50.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:50.983+0000 7f331a000640 1 --2- 172.21.15.1:0/633465453 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f33080a2c90 0x7f33080a5150 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f332410e4f0 tx=0x7f330c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:50.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:50.986+0000 7f331be00640 1 -- 172.21.15.1:0/633465453 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f331409c570 con 0x7f33241019c0 2024-09-12T01:28:50.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:50 smithi053 ceph-mon[35158]: pgmap v375: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T01:28:51.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:50 smithi104 ceph-mon[35219]: pgmap v375: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T01:28:51.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:51.177+0000 7f3323e00640 1 -- 172.21.15.1:0/633465453 --> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f332410dbb0 con 0x7f33080a2c90 2024-09-12T01:28:51.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:51.186+0000 7f331be00640 1 -- 172.21.15.1:0/633465453 <== mgr.14150 v2:172.21.15.1:6800/1928662456 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340428 (secure 0 0 0) 0x7f332410dbb0 con 0x7f33080a2c90 2024-09-12T01:28:51.189 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:28:51.191 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-12T01:28:51.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:51.190+0000 7f3323e00640 1 -- 172.21.15.1:0/633465453 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f33080a2c90 msgr2=0x7f33080a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:51.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:51.191+0000 7f3323e00640 1 --2- 172.21.15.1:0/633465453 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f33080a2c90 0x7f33080a5150 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f332410e4f0 tx=0x7f330c00b410 comp rx=0 tx=0).stop 2024-09-12T01:28:51.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:51.191+0000 7f3323e00640 1 -- 172.21.15.1:0/633465453 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33241019c0 msgr2=0x7f332419a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:51.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:51.191+0000 7f3323e00640 1 --2- 172.21.15.1:0/633465453 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33241019c0 0x7f332419a450 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3314009d50 tx=0x7f331400c6a0 comp rx=0 tx=0).stop 2024-09-12T01:28:51.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:51.191+0000 7f3323e00640 1 -- 172.21.15.1:0/633465453 shutdown_connections 2024-09-12T01:28:51.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:51.191+0000 7f3323e00640 1 --2- 172.21.15.1:0/633465453 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f33080a2c90 0x7f33080a5150 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:51.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:51.191+0000 7f3323e00640 1 --2- 172.21.15.1:0/633465453 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f33241079c0 0x7f332419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:51.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:51.191+0000 7f3323e00640 1 --2- 172.21.15.1:0/633465453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3324102360 0x7f332419a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:51.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:51.191+0000 7f3323e00640 1 --2- 172.21.15.1:0/633465453 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f33241019c0 0x7f332419a450 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:51.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:51.191+0000 7f3323e00640 1 -- 172.21.15.1:0/633465453 >> 172.21.15.1:0/633465453 conn(0x7f33240fd740 msgr2=0x7f332410d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:51.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:51.191+0000 7f3323e00640 1 -- 172.21.15.1:0/633465453 shutdown_connections 2024-09-12T01:28:51.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:51.191+0000 7f3323e00640 1 -- 172.21.15.1:0/633465453 wait complete. 2024-09-12T01:28:51.984 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":376,"stamp":"2024-09-12T01:28:50.705574+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661205,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":799,"num_read_kb":519,"num_write":429,"num_write_kb":3081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1182348,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":352,"ondisk_log_size":352,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":402208,"kb_used_data":15440,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124539616,"statfs":{"total":1151940427776,"available":1151528566784,"internally_reserved":0,"allocated":15810560,"data_stored":8718485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":395752762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003293"},"pg_stats":[{"pgid":"2.1f","version":"96'13","reported_seq":123,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:49.429001+0000","last_change":"2024-09-12T01:26:44.440383+0000","last_active":"2024-09-12T01:28:49.429001+0000","last_peered":"2024-09-12T01:28:49.429001+0000","last_clean":"2024-09-12T01:28:49.429001+0000","last_became_active":"2024-09-12T01:26:44.440048+0000","last_became_peered":"2024-09-12T01:26:44.440048+0000","last_unstale":"2024-09-12T01:28:49.429001+0000","last_undegraded":"2024-09-12T01:28:49.429001+0000","last_fullsized":"2024-09-12T01:28:49.429001+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:05:26.956556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":66,"num_read_kb":66,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":39,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.417995+0000","last_change":"2024-09-12T01:26:47.483952+0000","last_active":"2024-09-12T01:28:17.417995+0000","last_peered":"2024-09-12T01:28:17.417995+0000","last_clean":"2024-09-12T01:28:17.417995+0000","last_became_active":"2024-09-12T01:26:47.483679+0000","last_became_peered":"2024-09-12T01:26:47.483679+0000","last_unstale":"2024-09-12T01:28:17.417995+0000","last_undegraded":"2024-09-12T01:28:17.417995+0000","last_fullsized":"2024-09-12T01:28:17.417995+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:46:53.997935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.419822+0000","last_change":"2024-09-12T01:26:57.529904+0000","last_active":"2024-09-12T01:28:17.419822+0000","last_peered":"2024-09-12T01:28:17.419822+0000","last_clean":"2024-09-12T01:28:17.419822+0000","last_became_active":"2024-09-12T01:26:57.529332+0000","last_became_peered":"2024-09-12T01:26:57.529332+0000","last_unstale":"2024-09-12T01:28:17.419822+0000","last_undegraded":"2024-09-12T01:28:17.419822+0000","last_fullsized":"2024-09-12T01:28:17.419822+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:54.733150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"96'2","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.418199+0000","last_change":"2024-09-12T01:26:44.448605+0000","last_active":"2024-09-12T01:28:17.418199+0000","last_peered":"2024-09-12T01:28:17.418199+0000","last_clean":"2024-09-12T01:28:17.418199+0000","last_became_active":"2024-09-12T01:26:44.447035+0000","last_became_peered":"2024-09-12T01:26:44.447035+0000","last_unstale":"2024-09-12T01:28:17.418199+0000","last_undegraded":"2024-09-12T01:28:17.418199+0000","last_fullsized":"2024-09-12T01:28:17.418199+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:29:44.222912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.415243+0000","last_change":"2024-09-12T01:26:47.475058+0000","last_active":"2024-09-12T01:28:17.415243+0000","last_peered":"2024-09-12T01:28:17.415243+0000","last_clean":"2024-09-12T01:28:17.415243+0000","last_became_active":"2024-09-12T01:26:47.474057+0000","last_became_peered":"2024-09-12T01:26:47.474057+0000","last_unstale":"2024-09-12T01:28:17.415243+0000","last_undegraded":"2024-09-12T01:28:17.415243+0000","last_fullsized":"2024-09-12T01:28:17.415243+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:58:20.039831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.505782+0000","last_change":"2024-09-12T01:26:57.532139+0000","last_active":"2024-09-12T01:28:17.505782+0000","last_peered":"2024-09-12T01:28:17.505782+0000","last_clean":"2024-09-12T01:28:17.505782+0000","last_became_active":"2024-09-12T01:26:57.531718+0000","last_became_peered":"2024-09-12T01:26:57.531718+0000","last_unstale":"2024-09-12T01:28:17.505782+0000","last_undegraded":"2024-09-12T01:28:17.505782+0000","last_fullsized":"2024-09-12T01:28:17.505782+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:10.414782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"96'12","reported_seq":66,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.508075+0000","last_change":"2024-09-12T01:27:16.631523+0000","last_active":"2024-09-12T01:28:17.508075+0000","last_peered":"2024-09-12T01:28:17.508075+0000","last_clean":"2024-09-12T01:28:17.508075+0000","last_became_active":"2024-09-12T01:27:16.622591+0000","last_became_peered":"2024-09-12T01:27:16.622591+0000","last_unstale":"2024-09-12T01:28:17.508075+0000","last_undegraded":"2024-09-12T01:28:17.508075+0000","last_fullsized":"2024-09-12T01:28:17.508075+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:14:12.793775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506754+0000","last_change":"2024-09-12T01:26:47.477891+0000","last_active":"2024-09-12T01:28:17.506754+0000","last_peered":"2024-09-12T01:28:17.506754+0000","last_clean":"2024-09-12T01:28:17.506754+0000","last_became_active":"2024-09-12T01:26:47.477107+0000","last_became_peered":"2024-09-12T01:26:47.477107+0000","last_unstale":"2024-09-12T01:28:17.506754+0000","last_undegraded":"2024-09-12T01:28:17.506754+0000","last_fullsized":"2024-09-12T01:28:17.506754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:35:17.262896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":35,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.507397+0000","last_change":"2024-09-12T01:27:16.832962+0000","last_active":"2024-09-12T01:28:17.507397+0000","last_peered":"2024-09-12T01:28:17.507397+0000","last_clean":"2024-09-12T01:28:17.507397+0000","last_became_active":"2024-09-12T01:27:16.832463+0000","last_became_peered":"2024-09-12T01:27:16.832463+0000","last_unstale":"2024-09-12T01:28:17.507397+0000","last_undegraded":"2024-09-12T01:28:17.507397+0000","last_fullsized":"2024-09-12T01:28:17.507397+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:10:24.321133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"96'5","reported_seq":49,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.508051+0000","last_change":"2024-09-12T01:26:44.445375+0000","last_active":"2024-09-12T01:28:17.508051+0000","last_peered":"2024-09-12T01:28:17.508051+0000","last_clean":"2024-09-12T01:28:17.508051+0000","last_became_active":"2024-09-12T01:26:44.444820+0000","last_became_peered":"2024-09-12T01:26:44.444820+0000","last_unstale":"2024-09-12T01:28:17.508051+0000","last_undegraded":"2024-09-12T01:28:17.508051+0000","last_fullsized":"2024-09-12T01:28:17.508051+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:05:40.790222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506747+0000","last_change":"2024-09-12T01:26:47.480650+0000","last_active":"2024-09-12T01:28:17.506747+0000","last_peered":"2024-09-12T01:28:17.506747+0000","last_clean":"2024-09-12T01:28:17.506747+0000","last_became_active":"2024-09-12T01:26:47.480266+0000","last_became_peered":"2024-09-12T01:26:47.480266+0000","last_unstale":"2024-09-12T01:28:17.506747+0000","last_undegraded":"2024-09-12T01:28:17.506747+0000","last_fullsized":"2024-09-12T01:28:17.506747+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:00.218397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506101+0000","last_change":"2024-09-12T01:26:57.532582+0000","last_active":"2024-09-12T01:28:17.506101+0000","last_peered":"2024-09-12T01:28:17.506101+0000","last_clean":"2024-09-12T01:28:17.506101+0000","last_became_active":"2024-09-12T01:26:57.531966+0000","last_became_peered":"2024-09-12T01:26:57.531966+0000","last_unstale":"2024-09-12T01:28:17.506101+0000","last_undegraded":"2024-09-12T01:28:17.506101+0000","last_fullsized":"2024-09-12T01:28:17.506101+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:01:13.886678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.415608+0000","last_change":"2024-09-12T01:26:47.486567+0000","last_active":"2024-09-12T01:28:17.415608+0000","last_peered":"2024-09-12T01:28:17.415608+0000","last_clean":"2024-09-12T01:28:17.415608+0000","last_became_active":"2024-09-12T01:26:47.486219+0000","last_became_peered":"2024-09-12T01:26:47.486219+0000","last_unstale":"2024-09-12T01:28:17.415608+0000","last_undegraded":"2024-09-12T01:28:17.415608+0000","last_fullsized":"2024-09-12T01:28:17.415608+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:48:53.889111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"96'4","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.419178+0000","last_change":"2024-09-12T01:27:16.624095+0000","last_active":"2024-09-12T01:28:17.419178+0000","last_peered":"2024-09-12T01:28:17.419178+0000","last_clean":"2024-09-12T01:28:17.419178+0000","last_became_active":"2024-09-12T01:27:16.620392+0000","last_became_peered":"2024-09-12T01:27:16.620392+0000","last_unstale":"2024-09-12T01:28:17.419178+0000","last_undegraded":"2024-09-12T01:28:17.419178+0000","last_fullsized":"2024-09-12T01:28:17.419178+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:56:05.516283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506581+0000","last_change":"2024-09-12T01:26:57.729865+0000","last_active":"2024-09-12T01:28:17.506581+0000","last_peered":"2024-09-12T01:28:17.506581+0000","last_clean":"2024-09-12T01:28:17.506581+0000","last_became_active":"2024-09-12T01:26:57.728819+0000","last_became_peered":"2024-09-12T01:26:57.728819+0000","last_unstale":"2024-09-12T01:28:17.506581+0000","last_undegraded":"2024-09-12T01:28:17.506581+0000","last_fullsized":"2024-09-12T01:28:17.506581+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:11.337683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.417310+0000","last_change":"2024-09-12T01:26:47.485503+0000","last_active":"2024-09-12T01:28:17.417310+0000","last_peered":"2024-09-12T01:28:17.417310+0000","last_clean":"2024-09-12T01:28:17.417310+0000","last_became_active":"2024-09-12T01:26:47.485187+0000","last_became_peered":"2024-09-12T01:26:47.485187+0000","last_unstale":"2024-09-12T01:28:17.417310+0000","last_undegraded":"2024-09-12T01:28:17.417310+0000","last_fullsized":"2024-09-12T01:28:17.417310+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:24:52.544761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"96'3","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.419977+0000","last_change":"2024-09-12T01:26:44.447010+0000","last_active":"2024-09-12T01:28:17.419977+0000","last_peered":"2024-09-12T01:28:17.419977+0000","last_clean":"2024-09-12T01:28:17.419977+0000","last_became_active":"2024-09-12T01:26:44.446643+0000","last_became_peered":"2024-09-12T01:26:44.446643+0000","last_unstale":"2024-09-12T01:28:17.419977+0000","last_undegraded":"2024-09-12T01:28:17.419977+0000","last_fullsized":"2024-09-12T01:28:17.419977+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:56:33.298732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"96'1","reported_seq":29,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506880+0000","last_change":"2024-09-12T01:26:57.524687+0000","last_active":"2024-09-12T01:28:17.506880+0000","last_peered":"2024-09-12T01:28:17.506880+0000","last_clean":"2024-09-12T01:28:17.506880+0000","last_became_active":"2024-09-12T01:26:57.524201+0000","last_became_peered":"2024-09-12T01:26:57.524201+0000","last_unstale":"2024-09-12T01:28:17.506880+0000","last_undegraded":"2024-09-12T01:28:17.506880+0000","last_fullsized":"2024-09-12T01:28:17.506880+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:51:49.147460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.418610+0000","last_change":"2024-09-12T01:26:47.478432+0000","last_active":"2024-09-12T01:28:17.418610+0000","last_peered":"2024-09-12T01:28:17.418610+0000","last_clean":"2024-09-12T01:28:17.418610+0000","last_became_active":"2024-09-12T01:26:47.478109+0000","last_became_peered":"2024-09-12T01:26:47.478109+0000","last_unstale":"2024-09-12T01:28:17.418610+0000","last_undegraded":"2024-09-12T01:28:17.418610+0000","last_fullsized":"2024-09-12T01:28:17.418610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:30:19.917875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"96'7","reported_seq":51,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.417657+0000","last_change":"2024-09-12T01:26:44.441961+0000","last_active":"2024-09-12T01:28:17.417657+0000","last_peered":"2024-09-12T01:28:17.417657+0000","last_clean":"2024-09-12T01:28:17.417657+0000","last_became_active":"2024-09-12T01:26:44.441644+0000","last_became_peered":"2024-09-12T01:26:44.441644+0000","last_unstale":"2024-09-12T01:28:17.417657+0000","last_undegraded":"2024-09-12T01:28:17.417657+0000","last_fullsized":"2024-09-12T01:28:17.417657+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:40:48.561119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506845+0000","last_change":"2024-09-12T01:26:57.531164+0000","last_active":"2024-09-12T01:28:17.506845+0000","last_peered":"2024-09-12T01:28:17.506845+0000","last_clean":"2024-09-12T01:28:17.506845+0000","last_became_active":"2024-09-12T01:26:57.530513+0000","last_became_peered":"2024-09-12T01:26:57.530513+0000","last_unstale":"2024-09-12T01:28:17.506845+0000","last_undegraded":"2024-09-12T01:28:17.506845+0000","last_fullsized":"2024-09-12T01:28:17.506845+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:08:55.385620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.507365+0000","last_change":"2024-09-12T01:26:47.475562+0000","last_active":"2024-09-12T01:28:17.507365+0000","last_peered":"2024-09-12T01:28:17.507365+0000","last_clean":"2024-09-12T01:28:17.507365+0000","last_became_active":"2024-09-12T01:26:47.475167+0000","last_became_peered":"2024-09-12T01:26:47.475167+0000","last_unstale":"2024-09-12T01:28:17.507365+0000","last_undegraded":"2024-09-12T01:28:17.507365+0000","last_fullsized":"2024-09-12T01:28:17.507365+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:39:34.706811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"96'2","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506301+0000","last_change":"2024-09-12T01:26:44.447264+0000","last_active":"2024-09-12T01:28:17.506301+0000","last_peered":"2024-09-12T01:28:17.506301+0000","last_clean":"2024-09-12T01:28:17.506301+0000","last_became_active":"2024-09-12T01:26:44.446885+0000","last_became_peered":"2024-09-12T01:26:44.446885+0000","last_unstale":"2024-09-12T01:28:17.506301+0000","last_undegraded":"2024-09-12T01:28:17.506301+0000","last_fullsized":"2024-09-12T01:28:17.506301+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:56:27.229949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506780+0000","last_change":"2024-09-12T01:26:57.729792+0000","last_active":"2024-09-12T01:28:17.506780+0000","last_peered":"2024-09-12T01:28:17.506780+0000","last_clean":"2024-09-12T01:28:17.506780+0000","last_became_active":"2024-09-12T01:26:57.729252+0000","last_became_peered":"2024-09-12T01:26:57.729252+0000","last_unstale":"2024-09-12T01:28:17.506780+0000","last_undegraded":"2024-09-12T01:28:17.506780+0000","last_fullsized":"2024-09-12T01:28:17.506780+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:34:57.106270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.415242+0000","last_change":"2024-09-12T01:26:47.479179+0000","last_active":"2024-09-12T01:28:17.415242+0000","last_peered":"2024-09-12T01:28:17.415242+0000","last_clean":"2024-09-12T01:28:17.415242+0000","last_became_active":"2024-09-12T01:26:47.478783+0000","last_became_peered":"2024-09-12T01:26:47.478783+0000","last_unstale":"2024-09-12T01:28:17.415242+0000","last_undegraded":"2024-09-12T01:28:17.415242+0000","last_fullsized":"2024-09-12T01:28:17.415242+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:23:22.548509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"96'7","reported_seq":51,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.507766+0000","last_change":"2024-09-12T01:26:44.471854+0000","last_active":"2024-09-12T01:28:17.507766+0000","last_peered":"2024-09-12T01:28:17.507766+0000","last_clean":"2024-09-12T01:28:17.507766+0000","last_became_active":"2024-09-12T01:26:44.471511+0000","last_became_peered":"2024-09-12T01:26:44.471511+0000","last_unstale":"2024-09-12T01:28:17.507766+0000","last_undegraded":"2024-09-12T01:28:17.507766+0000","last_fullsized":"2024-09-12T01:28:17.507766+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:00:26.639984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.419824+0000","last_change":"2024-09-12T01:26:57.520467+0000","last_active":"2024-09-12T01:28:17.419824+0000","last_peered":"2024-09-12T01:28:17.419824+0000","last_clean":"2024-09-12T01:28:17.419824+0000","last_became_active":"2024-09-12T01:26:57.520048+0000","last_became_peered":"2024-09-12T01:26:57.520048+0000","last_unstale":"2024-09-12T01:28:17.419824+0000","last_undegraded":"2024-09-12T01:28:17.419824+0000","last_fullsized":"2024-09-12T01:28:17.419824+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:42:05.303804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.418170+0000","last_change":"2024-09-12T01:26:47.476484+0000","last_active":"2024-09-12T01:28:17.418170+0000","last_peered":"2024-09-12T01:28:17.418170+0000","last_clean":"2024-09-12T01:28:17.418170+0000","last_became_active":"2024-09-12T01:26:47.475610+0000","last_became_peered":"2024-09-12T01:26:47.475610+0000","last_unstale":"2024-09-12T01:28:17.418170+0000","last_undegraded":"2024-09-12T01:28:17.418170+0000","last_fullsized":"2024-09-12T01:28:17.418170+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:44.637986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"96'14","reported_seq":57,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.507202+0000","last_change":"2024-09-12T01:26:44.442988+0000","last_active":"2024-09-12T01:28:17.507202+0000","last_peered":"2024-09-12T01:28:17.507202+0000","last_clean":"2024-09-12T01:28:17.507202+0000","last_became_active":"2024-09-12T01:26:44.442612+0000","last_became_peered":"2024-09-12T01:26:44.442612+0000","last_unstale":"2024-09-12T01:28:17.507202+0000","last_undegraded":"2024-09-12T01:28:17.507202+0000","last_fullsized":"2024-09-12T01:28:17.507202+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:38:25.486470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.416883+0000","last_change":"2024-09-12T01:26:57.729762+0000","last_active":"2024-09-12T01:28:17.416883+0000","last_peered":"2024-09-12T01:28:17.416883+0000","last_clean":"2024-09-12T01:28:17.416883+0000","last_became_active":"2024-09-12T01:26:57.728944+0000","last_became_peered":"2024-09-12T01:26:57.728944+0000","last_unstale":"2024-09-12T01:28:17.416883+0000","last_undegraded":"2024-09-12T01:28:17.416883+0000","last_fullsized":"2024-09-12T01:28:17.416883+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:33:58.030332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506573+0000","last_change":"2024-09-12T01:26:47.482512+0000","last_active":"2024-09-12T01:28:17.506573+0000","last_peered":"2024-09-12T01:28:17.506573+0000","last_clean":"2024-09-12T01:28:17.506573+0000","last_became_active":"2024-09-12T01:26:47.481344+0000","last_became_peered":"2024-09-12T01:26:47.481344+0000","last_unstale":"2024-09-12T01:28:17.506573+0000","last_undegraded":"2024-09-12T01:28:17.506573+0000","last_fullsized":"2024-09-12T01:28:17.506573+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:52:22.920248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"96'4","reported_seq":48,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.508081+0000","last_change":"2024-09-12T01:26:44.447654+0000","last_active":"2024-09-12T01:28:17.508081+0000","last_peered":"2024-09-12T01:28:17.508081+0000","last_clean":"2024-09-12T01:28:17.508081+0000","last_became_active":"2024-09-12T01:26:44.447148+0000","last_became_peered":"2024-09-12T01:26:44.447148+0000","last_unstale":"2024-09-12T01:28:17.508081+0000","last_undegraded":"2024-09-12T01:28:17.508081+0000","last_fullsized":"2024-09-12T01:28:17.508081+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:40:02.928329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.419100+0000","last_change":"2024-09-12T01:26:57.530191+0000","last_active":"2024-09-12T01:28:17.419100+0000","last_peered":"2024-09-12T01:28:17.419100+0000","last_clean":"2024-09-12T01:28:17.419100+0000","last_became_active":"2024-09-12T01:26:57.529752+0000","last_became_peered":"2024-09-12T01:26:57.529752+0000","last_unstale":"2024-09-12T01:28:17.419100+0000","last_undegraded":"2024-09-12T01:28:17.419100+0000","last_fullsized":"2024-09-12T01:28:17.419100+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:00:16.639297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.507519+0000","last_change":"2024-09-12T01:26:47.478991+0000","last_active":"2024-09-12T01:28:17.507519+0000","last_peered":"2024-09-12T01:28:17.507519+0000","last_clean":"2024-09-12T01:28:17.507519+0000","last_became_active":"2024-09-12T01:26:47.478228+0000","last_became_peered":"2024-09-12T01:26:47.478228+0000","last_unstale":"2024-09-12T01:28:17.507519+0000","last_undegraded":"2024-09-12T01:28:17.507519+0000","last_fullsized":"2024-09-12T01:28:17.507519+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:45.236546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"96'1","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.507039+0000","last_change":"2024-09-12T01:26:44.441143+0000","last_active":"2024-09-12T01:28:17.507039+0000","last_peered":"2024-09-12T01:28:17.507039+0000","last_clean":"2024-09-12T01:28:17.507039+0000","last_became_active":"2024-09-12T01:26:44.439361+0000","last_became_peered":"2024-09-12T01:26:44.439361+0000","last_unstale":"2024-09-12T01:28:17.507039+0000","last_undegraded":"2024-09-12T01:28:17.507039+0000","last_fullsized":"2024-09-12T01:28:17.507039+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:22:34.257977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.507368+0000","last_change":"2024-09-12T01:26:57.729010+0000","last_active":"2024-09-12T01:28:17.507368+0000","last_peered":"2024-09-12T01:28:17.507368+0000","last_clean":"2024-09-12T01:28:17.507368+0000","last_became_active":"2024-09-12T01:26:57.728576+0000","last_became_peered":"2024-09-12T01:26:57.728576+0000","last_unstale":"2024-09-12T01:28:17.507368+0000","last_undegraded":"2024-09-12T01:28:17.507368+0000","last_fullsized":"2024-09-12T01:28:17.507368+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:48:48.798028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.507275+0000","last_change":"2024-09-12T01:27:16.832723+0000","last_active":"2024-09-12T01:28:17.507275+0000","last_peered":"2024-09-12T01:28:17.507275+0000","last_clean":"2024-09-12T01:28:17.507275+0000","last_became_active":"2024-09-12T01:27:16.832241+0000","last_became_peered":"2024-09-12T01:27:16.832241+0000","last_unstale":"2024-09-12T01:28:17.507275+0000","last_undegraded":"2024-09-12T01:28:17.507275+0000","last_fullsized":"2024-09-12T01:28:17.507275+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:26:09.610468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"96'6","reported_seq":115,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:46.425493+0000","last_change":"2024-09-12T01:26:44.440802+0000","last_active":"2024-09-12T01:28:46.425493+0000","last_peered":"2024-09-12T01:28:46.425493+0000","last_clean":"2024-09-12T01:28:46.425493+0000","last_became_active":"2024-09-12T01:26:44.439686+0000","last_became_peered":"2024-09-12T01:26:44.439686+0000","last_unstale":"2024-09-12T01:28:46.425493+0000","last_undegraded":"2024-09-12T01:28:46.425493+0000","last_fullsized":"2024-09-12T01:28:46.425493+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:56.608975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":65,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506438+0000","last_change":"2024-09-12T01:26:57.537046+0000","last_active":"2024-09-12T01:28:17.506438+0000","last_peered":"2024-09-12T01:28:17.506438+0000","last_clean":"2024-09-12T01:28:17.506438+0000","last_became_active":"2024-09-12T01:26:57.536589+0000","last_became_peered":"2024-09-12T01:26:57.536589+0000","last_unstale":"2024-09-12T01:28:17.506438+0000","last_undegraded":"2024-09-12T01:28:17.506438+0000","last_fullsized":"2024-09-12T01:28:17.506438+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:24:54.232963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506348+0000","last_change":"2024-09-12T01:26:47.476288+0000","last_active":"2024-09-12T01:28:17.506348+0000","last_peered":"2024-09-12T01:28:17.506348+0000","last_clean":"2024-09-12T01:28:17.506348+0000","last_became_active":"2024-09-12T01:26:47.475836+0000","last_became_peered":"2024-09-12T01:26:47.475836+0000","last_unstale":"2024-09-12T01:28:17.506348+0000","last_undegraded":"2024-09-12T01:28:17.506348+0000","last_fullsized":"2024-09-12T01:28:17.506348+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:39:43.017253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"96'5","reported_seq":110,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:44.422289+0000","last_change":"2024-09-12T01:26:44.446539+0000","last_active":"2024-09-12T01:28:44.422289+0000","last_peered":"2024-09-12T01:28:44.422289+0000","last_clean":"2024-09-12T01:28:44.422289+0000","last_became_active":"2024-09-12T01:26:44.446219+0000","last_became_peered":"2024-09-12T01:26:44.446219+0000","last_unstale":"2024-09-12T01:28:44.422289+0000","last_undegraded":"2024-09-12T01:28:44.422289+0000","last_fullsized":"2024-09-12T01:28:44.422289+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:50:32.584078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":62,"num_read_kb":62,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506648+0000","last_change":"2024-09-12T01:26:57.729445+0000","last_active":"2024-09-12T01:28:17.506648+0000","last_peered":"2024-09-12T01:28:17.506648+0000","last_clean":"2024-09-12T01:28:17.506648+0000","last_became_active":"2024-09-12T01:26:57.728948+0000","last_became_peered":"2024-09-12T01:26:57.728948+0000","last_unstale":"2024-09-12T01:28:17.506648+0000","last_undegraded":"2024-09-12T01:28:17.506648+0000","last_fullsized":"2024-09-12T01:28:17.506648+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:33:27.017565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.505831+0000","last_change":"2024-09-12T01:26:47.481965+0000","last_active":"2024-09-12T01:28:17.505831+0000","last_peered":"2024-09-12T01:28:17.505831+0000","last_clean":"2024-09-12T01:28:17.505831+0000","last_became_active":"2024-09-12T01:26:47.481600+0000","last_became_peered":"2024-09-12T01:26:47.481600+0000","last_unstale":"2024-09-12T01:28:17.505831+0000","last_undegraded":"2024-09-12T01:28:17.505831+0000","last_fullsized":"2024-09-12T01:28:17.505831+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:20.078494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"96'4","reported_seq":19,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.418328+0000","last_change":"2024-09-12T01:27:16.832522+0000","last_active":"2024-09-12T01:28:17.418328+0000","last_peered":"2024-09-12T01:28:17.418328+0000","last_clean":"2024-09-12T01:28:17.418328+0000","last_became_active":"2024-09-12T01:27:16.832090+0000","last_became_peered":"2024-09-12T01:27:16.832090+0000","last_unstale":"2024-09-12T01:28:17.418328+0000","last_undegraded":"2024-09-12T01:28:17.418328+0000","last_fullsized":"2024-09-12T01:28:17.418328+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:15:48.061559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506492+0000","last_change":"2024-09-12T01:26:57.729622+0000","last_active":"2024-09-12T01:28:17.506492+0000","last_peered":"2024-09-12T01:28:17.506492+0000","last_clean":"2024-09-12T01:28:17.506492+0000","last_became_active":"2024-09-12T01:26:57.729175+0000","last_became_peered":"2024-09-12T01:26:57.729175+0000","last_unstale":"2024-09-12T01:28:17.506492+0000","last_undegraded":"2024-09-12T01:28:17.506492+0000","last_fullsized":"2024-09-12T01:28:17.506492+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:41.691685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.417373+0000","last_change":"2024-09-12T01:26:47.479155+0000","last_active":"2024-09-12T01:28:17.417373+0000","last_peered":"2024-09-12T01:28:17.417373+0000","last_clean":"2024-09-12T01:28:17.417373+0000","last_became_active":"2024-09-12T01:26:47.478781+0000","last_became_peered":"2024-09-12T01:26:47.478781+0000","last_unstale":"2024-09-12T01:28:17.417373+0000","last_undegraded":"2024-09-12T01:28:17.417373+0000","last_fullsized":"2024-09-12T01:28:17.417373+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:40:49.730016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"96'6","reported_seq":53,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.507683+0000","last_change":"2024-09-12T01:26:44.437048+0000","last_active":"2024-09-12T01:28:17.507683+0000","last_peered":"2024-09-12T01:28:17.507683+0000","last_clean":"2024-09-12T01:28:17.507683+0000","last_became_active":"2024-09-12T01:26:44.436702+0000","last_became_peered":"2024-09-12T01:26:44.436702+0000","last_unstale":"2024-09-12T01:28:17.507683+0000","last_undegraded":"2024-09-12T01:28:17.507683+0000","last_fullsized":"2024-09-12T01:28:17.507683+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:15:18.857093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":363,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.415480+0000","last_change":"2024-09-12T01:25:28.683724+0000","last_active":"2024-09-12T01:28:17.415480+0000","last_peered":"2024-09-12T01:28:17.415480+0000","last_clean":"2024-09-12T01:28:17.415480+0000","last_became_active":"2024-09-12T01:25:28.683155+0000","last_became_peered":"2024-09-12T01:25:28.683155+0000","last_unstale":"2024-09-12T01:28:17.415480+0000","last_undegraded":"2024-09-12T01:28:17.415480+0000","last_fullsized":"2024-09-12T01:28:17.415480+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:21:41.529611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:21:41.529611+0000","last_clean_scrub_stamp":"2024-09-12T01:21:41.529611+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:13:02.724915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"96'1","reported_seq":36,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.415877+0000","last_change":"2024-09-12T01:27:16.833028+0000","last_active":"2024-09-12T01:28:17.415877+0000","last_peered":"2024-09-12T01:28:17.415877+0000","last_clean":"2024-09-12T01:28:17.415877+0000","last_became_active":"2024-09-12T01:27:16.832416+0000","last_became_peered":"2024-09-12T01:27:16.832416+0000","last_unstale":"2024-09-12T01:28:17.415877+0000","last_undegraded":"2024-09-12T01:28:17.415877+0000","last_fullsized":"2024-09-12T01:28:17.415877+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:49:43.008107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.416164+0000","last_change":"2024-09-12T01:26:47.475467+0000","last_active":"2024-09-12T01:28:17.416164+0000","last_peered":"2024-09-12T01:28:17.416164+0000","last_clean":"2024-09-12T01:28:17.416164+0000","last_became_active":"2024-09-12T01:26:47.474724+0000","last_became_peered":"2024-09-12T01:26:47.474724+0000","last_unstale":"2024-09-12T01:28:17.416164+0000","last_undegraded":"2024-09-12T01:28:17.416164+0000","last_fullsized":"2024-09-12T01:28:17.416164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:56:54.651180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"96'6","reported_seq":119,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:49.444583+0000","last_change":"2024-09-12T01:26:44.472048+0000","last_active":"2024-09-12T01:28:49.444583+0000","last_peered":"2024-09-12T01:28:49.444583+0000","last_clean":"2024-09-12T01:28:49.444583+0000","last_became_active":"2024-09-12T01:26:44.471697+0000","last_became_peered":"2024-09-12T01:26:44.471697+0000","last_unstale":"2024-09-12T01:28:49.444583+0000","last_undegraded":"2024-09-12T01:28:49.444583+0000","last_fullsized":"2024-09-12T01:28:49.444583+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:44:13.071983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":69,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506525+0000","last_change":"2024-09-12T01:26:57.528837+0000","last_active":"2024-09-12T01:28:17.506525+0000","last_peered":"2024-09-12T01:28:17.506525+0000","last_clean":"2024-09-12T01:28:17.506525+0000","last_became_active":"2024-09-12T01:26:57.528285+0000","last_became_peered":"2024-09-12T01:26:57.528285+0000","last_unstale":"2024-09-12T01:28:17.506525+0000","last_undegraded":"2024-09-12T01:28:17.506525+0000","last_fullsized":"2024-09-12T01:28:17.506525+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:12:13.534002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.415874+0000","last_change":"2024-09-12T01:26:47.483456+0000","last_active":"2024-09-12T01:28:17.415874+0000","last_peered":"2024-09-12T01:28:17.415874+0000","last_clean":"2024-09-12T01:28:17.415874+0000","last_became_active":"2024-09-12T01:26:47.482189+0000","last_became_peered":"2024-09-12T01:26:47.482189+0000","last_unstale":"2024-09-12T01:28:17.415874+0000","last_undegraded":"2024-09-12T01:28:17.415874+0000","last_fullsized":"2024-09-12T01:28:17.415874+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:18:55.466266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"96'3","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506871+0000","last_change":"2024-09-12T01:26:44.443507+0000","last_active":"2024-09-12T01:28:17.506871+0000","last_peered":"2024-09-12T01:28:17.506871+0000","last_clean":"2024-09-12T01:28:17.506871+0000","last_became_active":"2024-09-12T01:26:44.443147+0000","last_became_peered":"2024-09-12T01:26:44.443147+0000","last_unstale":"2024-09-12T01:28:17.506871+0000","last_undegraded":"2024-09-12T01:28:17.506871+0000","last_fullsized":"2024-09-12T01:28:17.506871+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:38:40.433581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.415124+0000","last_change":"2024-09-12T01:26:57.523029+0000","last_active":"2024-09-12T01:28:17.415124+0000","last_peered":"2024-09-12T01:28:17.415124+0000","last_clean":"2024-09-12T01:28:17.415124+0000","last_became_active":"2024-09-12T01:26:57.522565+0000","last_became_peered":"2024-09-12T01:26:57.522565+0000","last_unstale":"2024-09-12T01:28:17.415124+0000","last_undegraded":"2024-09-12T01:28:17.415124+0000","last_fullsized":"2024-09-12T01:28:17.415124+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:36:27.432126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.416019+0000","last_change":"2024-09-12T01:26:47.471958+0000","last_active":"2024-09-12T01:28:17.416019+0000","last_peered":"2024-09-12T01:28:17.416019+0000","last_clean":"2024-09-12T01:28:17.416019+0000","last_became_active":"2024-09-12T01:26:47.471621+0000","last_became_peered":"2024-09-12T01:26:47.471621+0000","last_unstale":"2024-09-12T01:28:17.416019+0000","last_undegraded":"2024-09-12T01:28:17.416019+0000","last_fullsized":"2024-09-12T01:28:17.416019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:46:38.004188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"96'5","reported_seq":48,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506660+0000","last_change":"2024-09-12T01:26:44.442852+0000","last_active":"2024-09-12T01:28:17.506660+0000","last_peered":"2024-09-12T01:28:17.506660+0000","last_clean":"2024-09-12T01:28:17.506660+0000","last_became_active":"2024-09-12T01:26:44.442487+0000","last_became_peered":"2024-09-12T01:26:44.442487+0000","last_unstale":"2024-09-12T01:28:17.506660+0000","last_undegraded":"2024-09-12T01:28:17.506660+0000","last_fullsized":"2024-09-12T01:28:17.506660+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:04:32.688925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506049+0000","last_change":"2024-09-12T01:26:57.521155+0000","last_active":"2024-09-12T01:28:17.506049+0000","last_peered":"2024-09-12T01:28:17.506049+0000","last_clean":"2024-09-12T01:28:17.506049+0000","last_became_active":"2024-09-12T01:26:57.520721+0000","last_became_peered":"2024-09-12T01:26:57.520721+0000","last_unstale":"2024-09-12T01:28:17.506049+0000","last_undegraded":"2024-09-12T01:28:17.506049+0000","last_fullsized":"2024-09-12T01:28:17.506049+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:58:52.004697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.419297+0000","last_change":"2024-09-12T01:26:47.482564+0000","last_active":"2024-09-12T01:28:17.419297+0000","last_peered":"2024-09-12T01:28:17.419297+0000","last_clean":"2024-09-12T01:28:17.419297+0000","last_became_active":"2024-09-12T01:26:47.482215+0000","last_became_peered":"2024-09-12T01:26:47.482215+0000","last_unstale":"2024-09-12T01:28:17.419297+0000","last_undegraded":"2024-09-12T01:28:17.419297+0000","last_fullsized":"2024-09-12T01:28:17.419297+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:27:40.882473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"96'2","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.508036+0000","last_change":"2024-09-12T01:26:44.442774+0000","last_active":"2024-09-12T01:28:17.508036+0000","last_peered":"2024-09-12T01:28:17.508036+0000","last_clean":"2024-09-12T01:28:17.508036+0000","last_became_active":"2024-09-12T01:26:44.442389+0000","last_became_peered":"2024-09-12T01:26:44.442389+0000","last_unstale":"2024-09-12T01:28:17.508036+0000","last_undegraded":"2024-09-12T01:28:17.508036+0000","last_fullsized":"2024-09-12T01:28:17.508036+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:22.628392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506306+0000","last_change":"2024-09-12T01:26:57.526182+0000","last_active":"2024-09-12T01:28:17.506306+0000","last_peered":"2024-09-12T01:28:17.506306+0000","last_clean":"2024-09-12T01:28:17.506306+0000","last_became_active":"2024-09-12T01:26:57.525869+0000","last_became_peered":"2024-09-12T01:26:57.525869+0000","last_unstale":"2024-09-12T01:28:17.506306+0000","last_undegraded":"2024-09-12T01:28:17.506306+0000","last_fullsized":"2024-09-12T01:28:17.506306+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:40:09.994361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506628+0000","last_change":"2024-09-12T01:26:47.475872+0000","last_active":"2024-09-12T01:28:17.506628+0000","last_peered":"2024-09-12T01:28:17.506628+0000","last_clean":"2024-09-12T01:28:17.506628+0000","last_became_active":"2024-09-12T01:26:47.475536+0000","last_became_peered":"2024-09-12T01:26:47.475536+0000","last_unstale":"2024-09-12T01:28:17.506628+0000","last_undegraded":"2024-09-12T01:28:17.506628+0000","last_fullsized":"2024-09-12T01:28:17.506628+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:03:37.519530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"96'17","reported_seq":61,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506976+0000","last_change":"2024-09-12T01:26:44.448843+0000","last_active":"2024-09-12T01:28:17.506976+0000","last_peered":"2024-09-12T01:28:17.506976+0000","last_clean":"2024-09-12T01:28:17.506976+0000","last_became_active":"2024-09-12T01:26:44.448451+0000","last_became_peered":"2024-09-12T01:26:44.448451+0000","last_unstale":"2024-09-12T01:28:17.506976+0000","last_undegraded":"2024-09-12T01:28:17.506976+0000","last_fullsized":"2024-09-12T01:28:17.506976+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:59:21.422174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506602+0000","last_change":"2024-09-12T01:26:57.541164+0000","last_active":"2024-09-12T01:28:17.506602+0000","last_peered":"2024-09-12T01:28:17.506602+0000","last_clean":"2024-09-12T01:28:17.506602+0000","last_became_active":"2024-09-12T01:26:57.540746+0000","last_became_peered":"2024-09-12T01:26:57.540746+0000","last_unstale":"2024-09-12T01:28:17.506602+0000","last_undegraded":"2024-09-12T01:28:17.506602+0000","last_fullsized":"2024-09-12T01:28:17.506602+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:58:26.870141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.415354+0000","last_change":"2024-09-12T01:26:47.473261+0000","last_active":"2024-09-12T01:28:17.415354+0000","last_peered":"2024-09-12T01:28:17.415354+0000","last_clean":"2024-09-12T01:28:17.415354+0000","last_became_active":"2024-09-12T01:26:47.472902+0000","last_became_peered":"2024-09-12T01:26:47.472902+0000","last_unstale":"2024-09-12T01:28:17.415354+0000","last_undegraded":"2024-09-12T01:28:17.415354+0000","last_fullsized":"2024-09-12T01:28:17.415354+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:58:41.080240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"96'6","reported_seq":116,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:47.427190+0000","last_change":"2024-09-12T01:26:44.440503+0000","last_active":"2024-09-12T01:28:47.427190+0000","last_peered":"2024-09-12T01:28:47.427190+0000","last_clean":"2024-09-12T01:28:47.427190+0000","last_became_active":"2024-09-12T01:26:44.440128+0000","last_became_peered":"2024-09-12T01:26:44.440128+0000","last_unstale":"2024-09-12T01:28:47.427190+0000","last_undegraded":"2024-09-12T01:28:47.427190+0000","last_fullsized":"2024-09-12T01:28:47.427190+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:39:35.373682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":68,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.507000+0000","last_change":"2024-09-12T01:26:57.533883+0000","last_active":"2024-09-12T01:28:17.507000+0000","last_peered":"2024-09-12T01:28:17.507000+0000","last_clean":"2024-09-12T01:28:17.507000+0000","last_became_active":"2024-09-12T01:26:57.533329+0000","last_became_peered":"2024-09-12T01:26:57.533329+0000","last_unstale":"2024-09-12T01:28:17.507000+0000","last_undegraded":"2024-09-12T01:28:17.507000+0000","last_fullsized":"2024-09-12T01:28:17.507000+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:17.655219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.415790+0000","last_change":"2024-09-12T01:26:47.480276+0000","last_active":"2024-09-12T01:28:17.415790+0000","last_peered":"2024-09-12T01:28:17.415790+0000","last_clean":"2024-09-12T01:28:17.415790+0000","last_became_active":"2024-09-12T01:26:47.479965+0000","last_became_peered":"2024-09-12T01:26:47.479965+0000","last_unstale":"2024-09-12T01:28:17.415790+0000","last_undegraded":"2024-09-12T01:28:17.415790+0000","last_fullsized":"2024-09-12T01:28:17.415790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:24:10.339000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"96'5","reported_seq":49,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506650+0000","last_change":"2024-09-12T01:26:44.442346+0000","last_active":"2024-09-12T01:28:17.506650+0000","last_peered":"2024-09-12T01:28:17.506650+0000","last_clean":"2024-09-12T01:28:17.506650+0000","last_became_active":"2024-09-12T01:26:44.442044+0000","last_became_peered":"2024-09-12T01:26:44.442044+0000","last_unstale":"2024-09-12T01:28:17.506650+0000","last_undegraded":"2024-09-12T01:28:17.506650+0000","last_fullsized":"2024-09-12T01:28:17.506650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:47:00.193554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.419065+0000","last_change":"2024-09-12T01:26:57.531650+0000","last_active":"2024-09-12T01:28:17.419065+0000","last_peered":"2024-09-12T01:28:17.419065+0000","last_clean":"2024-09-12T01:28:17.419065+0000","last_became_active":"2024-09-12T01:26:57.531184+0000","last_became_peered":"2024-09-12T01:26:57.531184+0000","last_unstale":"2024-09-12T01:28:17.419065+0000","last_undegraded":"2024-09-12T01:28:17.419065+0000","last_fullsized":"2024-09-12T01:28:17.419065+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:42:56.191490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506152+0000","last_change":"2024-09-12T01:26:47.480344+0000","last_active":"2024-09-12T01:28:17.506152+0000","last_peered":"2024-09-12T01:28:17.506152+0000","last_clean":"2024-09-12T01:28:17.506152+0000","last_became_active":"2024-09-12T01:26:47.479966+0000","last_became_peered":"2024-09-12T01:26:47.479966+0000","last_unstale":"2024-09-12T01:28:17.506152+0000","last_undegraded":"2024-09-12T01:28:17.506152+0000","last_fullsized":"2024-09-12T01:28:17.506152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:44:31.968977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"96'9","reported_seq":55,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.420176+0000","last_change":"2024-09-12T01:26:44.447708+0000","last_active":"2024-09-12T01:28:17.420176+0000","last_peered":"2024-09-12T01:28:17.420176+0000","last_clean":"2024-09-12T01:28:17.420176+0000","last_became_active":"2024-09-12T01:26:44.446974+0000","last_became_peered":"2024-09-12T01:26:44.446974+0000","last_unstale":"2024-09-12T01:28:17.420176+0000","last_undegraded":"2024-09-12T01:28:17.420176+0000","last_fullsized":"2024-09-12T01:28:17.420176+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:01:10.440044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506451+0000","last_change":"2024-09-12T01:26:57.533128+0000","last_active":"2024-09-12T01:28:17.506451+0000","last_peered":"2024-09-12T01:28:17.506451+0000","last_clean":"2024-09-12T01:28:17.506451+0000","last_became_active":"2024-09-12T01:26:57.532731+0000","last_became_peered":"2024-09-12T01:26:57.532731+0000","last_unstale":"2024-09-12T01:28:17.506451+0000","last_undegraded":"2024-09-12T01:28:17.506451+0000","last_fullsized":"2024-09-12T01:28:17.506451+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:09:06.118313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.507882+0000","last_change":"2024-09-12T01:26:47.471545+0000","last_active":"2024-09-12T01:28:17.507882+0000","last_peered":"2024-09-12T01:28:17.507882+0000","last_clean":"2024-09-12T01:28:17.507882+0000","last_became_active":"2024-09-12T01:26:47.471234+0000","last_became_peered":"2024-09-12T01:26:47.471234+0000","last_unstale":"2024-09-12T01:28:17.507882+0000","last_undegraded":"2024-09-12T01:28:17.507882+0000","last_fullsized":"2024-09-12T01:28:17.507882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:24:00.844203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"96'34","reported_seq":225,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:46.425561+0000","last_change":"2024-09-12T01:27:16.629235+0000","last_active":"2024-09-12T01:28:46.425561+0000","last_peered":"2024-09-12T01:28:46.425561+0000","last_clean":"2024-09-12T01:28:46.425561+0000","last_became_active":"2024-09-12T01:27:16.621719+0000","last_became_peered":"2024-09-12T01:27:16.621719+0000","last_unstale":"2024-09-12T01:28:46.425561+0000","last_undegraded":"2024-09-12T01:28:46.425561+0000","last_fullsized":"2024-09-12T01:28:46.425561+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:45:43.011452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63002,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":132,"num_read_kb":127,"num_write":36,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506245+0000","last_change":"2024-09-12T01:26:57.521950+0000","last_active":"2024-09-12T01:28:17.506245+0000","last_peered":"2024-09-12T01:28:17.506245+0000","last_clean":"2024-09-12T01:28:17.506245+0000","last_became_active":"2024-09-12T01:26:57.521574+0000","last_became_peered":"2024-09-12T01:26:57.521574+0000","last_unstale":"2024-09-12T01:28:17.506245+0000","last_undegraded":"2024-09-12T01:28:17.506245+0000","last_fullsized":"2024-09-12T01:28:17.506245+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:23.086233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506860+0000","last_change":"2024-09-12T01:26:47.478212+0000","last_active":"2024-09-12T01:28:17.506860+0000","last_peered":"2024-09-12T01:28:17.506860+0000","last_clean":"2024-09-12T01:28:17.506860+0000","last_became_active":"2024-09-12T01:26:47.477553+0000","last_became_peered":"2024-09-12T01:26:47.477553+0000","last_unstale":"2024-09-12T01:28:17.506860+0000","last_undegraded":"2024-09-12T01:28:17.506860+0000","last_fullsized":"2024-09-12T01:28:17.506860+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:48:22.390040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"96'5","reported_seq":49,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506863+0000","last_change":"2024-09-12T01:26:44.441558+0000","last_active":"2024-09-12T01:28:17.506863+0000","last_peered":"2024-09-12T01:28:17.506863+0000","last_clean":"2024-09-12T01:28:17.506863+0000","last_became_active":"2024-09-12T01:26:44.441099+0000","last_became_peered":"2024-09-12T01:26:44.441099+0000","last_unstale":"2024-09-12T01:28:17.506863+0000","last_undegraded":"2024-09-12T01:28:17.506863+0000","last_fullsized":"2024-09-12T01:28:17.506863+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:07:40.566150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.507261+0000","last_change":"2024-09-12T01:26:57.524838+0000","last_active":"2024-09-12T01:28:17.507261+0000","last_peered":"2024-09-12T01:28:17.507261+0000","last_clean":"2024-09-12T01:28:17.507261+0000","last_became_active":"2024-09-12T01:26:57.524182+0000","last_became_peered":"2024-09-12T01:26:57.524182+0000","last_unstale":"2024-09-12T01:28:17.507261+0000","last_undegraded":"2024-09-12T01:28:17.507261+0000","last_fullsized":"2024-09-12T01:28:17.507261+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:18:55.493116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.415331+0000","last_change":"2024-09-12T01:26:47.478562+0000","last_active":"2024-09-12T01:28:17.415331+0000","last_peered":"2024-09-12T01:28:17.415331+0000","last_clean":"2024-09-12T01:28:17.415331+0000","last_became_active":"2024-09-12T01:26:47.478223+0000","last_became_peered":"2024-09-12T01:26:47.478223+0000","last_unstale":"2024-09-12T01:28:17.415331+0000","last_undegraded":"2024-09-12T01:28:17.415331+0000","last_fullsized":"2024-09-12T01:28:17.415331+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:32:20.472870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"96'4","reported_seq":48,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.415988+0000","last_change":"2024-09-12T01:26:44.443434+0000","last_active":"2024-09-12T01:28:17.415988+0000","last_peered":"2024-09-12T01:28:17.415988+0000","last_clean":"2024-09-12T01:28:17.415988+0000","last_became_active":"2024-09-12T01:26:44.443097+0000","last_became_peered":"2024-09-12T01:26:44.443097+0000","last_unstale":"2024-09-12T01:28:17.415988+0000","last_undegraded":"2024-09-12T01:28:17.415988+0000","last_fullsized":"2024-09-12T01:28:17.415988+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:48:56.626378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.505831+0000","last_change":"2024-09-12T01:26:57.532507+0000","last_active":"2024-09-12T01:28:17.505831+0000","last_peered":"2024-09-12T01:28:17.505831+0000","last_clean":"2024-09-12T01:28:17.505831+0000","last_became_active":"2024-09-12T01:26:57.529757+0000","last_became_peered":"2024-09-12T01:26:57.529757+0000","last_unstale":"2024-09-12T01:28:17.505831+0000","last_undegraded":"2024-09-12T01:28:17.505831+0000","last_fullsized":"2024-09-12T01:28:17.505831+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:35.589141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506805+0000","last_change":"2024-09-12T01:26:47.474632+0000","last_active":"2024-09-12T01:28:17.506805+0000","last_peered":"2024-09-12T01:28:17.506805+0000","last_clean":"2024-09-12T01:28:17.506805+0000","last_became_active":"2024-09-12T01:26:47.473791+0000","last_became_peered":"2024-09-12T01:26:47.473791+0000","last_unstale":"2024-09-12T01:28:17.506805+0000","last_undegraded":"2024-09-12T01:28:17.506805+0000","last_fullsized":"2024-09-12T01:28:17.506805+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:58:45.241657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"96'4","reported_seq":171,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:46.425565+0000","last_change":"2024-09-12T01:26:44.445045+0000","last_active":"2024-09-12T01:28:46.425565+0000","last_peered":"2024-09-12T01:28:46.425565+0000","last_clean":"2024-09-12T01:28:46.425565+0000","last_became_active":"2024-09-12T01:26:44.444668+0000","last_became_peered":"2024-09-12T01:26:44.444668+0000","last_unstale":"2024-09-12T01:28:46.425565+0000","last_undegraded":"2024-09-12T01:28:46.425565+0000","last_fullsized":"2024-09-12T01:28:46.425565+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:05:14.112438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":124,"num_read_kb":124,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":35,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.508728+0000","last_change":"2024-09-12T01:28:17.508728+0000","last_active":"2024-09-12T01:28:17.508728+0000","last_peered":"2024-09-12T01:28:17.508728+0000","last_clean":"2024-09-12T01:28:17.508728+0000","last_became_active":"2024-09-12T01:28:17.508333+0000","last_became_peered":"2024-09-12T01:28:17.508333+0000","last_unstale":"2024-09-12T01:28:17.508728+0000","last_undegraded":"2024-09-12T01:28:17.508728+0000","last_fullsized":"2024-09-12T01:28:17.508728+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:26:00.670909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8],"acting":[3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.415143+0000","last_change":"2024-09-12T01:26:57.534204+0000","last_active":"2024-09-12T01:28:17.415143+0000","last_peered":"2024-09-12T01:28:17.415143+0000","last_clean":"2024-09-12T01:28:17.415143+0000","last_became_active":"2024-09-12T01:26:57.532094+0000","last_became_peered":"2024-09-12T01:26:57.532094+0000","last_unstale":"2024-09-12T01:28:17.415143+0000","last_undegraded":"2024-09-12T01:28:17.415143+0000","last_fullsized":"2024-09-12T01:28:17.415143+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:43:17.714896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506094+0000","last_change":"2024-09-12T01:26:47.480169+0000","last_active":"2024-09-12T01:28:17.506094+0000","last_peered":"2024-09-12T01:28:17.506094+0000","last_clean":"2024-09-12T01:28:17.506094+0000","last_became_active":"2024-09-12T01:26:47.479853+0000","last_became_peered":"2024-09-12T01:26:47.479853+0000","last_unstale":"2024-09-12T01:28:17.506094+0000","last_undegraded":"2024-09-12T01:28:17.506094+0000","last_fullsized":"2024-09-12T01:28:17.506094+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:58:10.578637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"96'3","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.507111+0000","last_change":"2024-09-12T01:26:44.440641+0000","last_active":"2024-09-12T01:28:17.507111+0000","last_peered":"2024-09-12T01:28:17.507111+0000","last_clean":"2024-09-12T01:28:17.507111+0000","last_became_active":"2024-09-12T01:26:44.440315+0000","last_became_peered":"2024-09-12T01:26:44.440315+0000","last_unstale":"2024-09-12T01:28:17.507111+0000","last_undegraded":"2024-09-12T01:28:17.507111+0000","last_fullsized":"2024-09-12T01:28:17.507111+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:21:52.919820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":35,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.509064+0000","last_change":"2024-09-12T01:28:17.509064+0000","last_active":"2024-09-12T01:28:17.509064+0000","last_peered":"2024-09-12T01:28:17.509064+0000","last_clean":"2024-09-12T01:28:17.509064+0000","last_became_active":"2024-09-12T01:28:17.508534+0000","last_became_peered":"2024-09-12T01:28:17.508534+0000","last_unstale":"2024-09-12T01:28:17.509064+0000","last_undegraded":"2024-09-12T01:28:17.509064+0000","last_fullsized":"2024-09-12T01:28:17.509064+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:32:29.004296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.417283+0000","last_change":"2024-09-12T01:26:47.476383+0000","last_active":"2024-09-12T01:28:17.417283+0000","last_peered":"2024-09-12T01:28:17.417283+0000","last_clean":"2024-09-12T01:28:17.417283+0000","last_became_active":"2024-09-12T01:26:47.475759+0000","last_became_peered":"2024-09-12T01:26:47.475759+0000","last_unstale":"2024-09-12T01:28:17.417283+0000","last_undegraded":"2024-09-12T01:28:17.417283+0000","last_fullsized":"2024-09-12T01:28:17.417283+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:14:38.898332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"96'3","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.505920+0000","last_change":"2024-09-12T01:26:44.445418+0000","last_active":"2024-09-12T01:28:17.505920+0000","last_peered":"2024-09-12T01:28:17.505920+0000","last_clean":"2024-09-12T01:28:17.505920+0000","last_became_active":"2024-09-12T01:26:44.444396+0000","last_became_peered":"2024-09-12T01:26:44.444396+0000","last_unstale":"2024-09-12T01:28:17.505920+0000","last_undegraded":"2024-09-12T01:28:17.505920+0000","last_fullsized":"2024-09-12T01:28:17.505920+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:16:16.595947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"96'7","reported_seq":56,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.507070+0000","last_change":"2024-09-12T01:27:16.620062+0000","last_active":"2024-09-12T01:28:17.507070+0000","last_peered":"2024-09-12T01:28:17.507070+0000","last_clean":"2024-09-12T01:28:17.507070+0000","last_became_active":"2024-09-12T01:27:16.619031+0000","last_became_peered":"2024-09-12T01:27:16.619031+0000","last_unstale":"2024-09-12T01:28:17.507070+0000","last_undegraded":"2024-09-12T01:28:17.507070+0000","last_fullsized":"2024-09-12T01:28:17.507070+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:04:04.329211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506596+0000","last_change":"2024-09-12T01:26:47.478706+0000","last_active":"2024-09-12T01:28:17.506596+0000","last_peered":"2024-09-12T01:28:17.506596+0000","last_clean":"2024-09-12T01:28:17.506596+0000","last_became_active":"2024-09-12T01:26:47.478339+0000","last_became_peered":"2024-09-12T01:26:47.478339+0000","last_unstale":"2024-09-12T01:28:17.506596+0000","last_undegraded":"2024-09-12T01:28:17.506596+0000","last_fullsized":"2024-09-12T01:28:17.506596+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:20:37.180485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.420091+0000","last_change":"2024-09-12T01:26:57.533268+0000","last_active":"2024-09-12T01:28:17.420091+0000","last_peered":"2024-09-12T01:28:17.420091+0000","last_clean":"2024-09-12T01:28:17.420091+0000","last_became_active":"2024-09-12T01:26:57.532862+0000","last_became_peered":"2024-09-12T01:26:57.532862+0000","last_unstale":"2024-09-12T01:28:17.420091+0000","last_undegraded":"2024-09-12T01:28:17.420091+0000","last_fullsized":"2024-09-12T01:28:17.420091+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:53.659734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"96'5","reported_seq":115,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:48.428905+0000","last_change":"2024-09-12T01:26:44.443113+0000","last_active":"2024-09-12T01:28:48.428905+0000","last_peered":"2024-09-12T01:28:48.428905+0000","last_clean":"2024-09-12T01:28:48.428905+0000","last_became_active":"2024-09-12T01:26:44.442642+0000","last_became_peered":"2024-09-12T01:26:44.442642+0000","last_unstale":"2024-09-12T01:28:48.428905+0000","last_undegraded":"2024-09-12T01:28:48.428905+0000","last_fullsized":"2024-09-12T01:28:48.428905+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:43.420392+0000","last_clean_scrub_stamp":"2024-09-12T01:26:43.420392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:16:20.143475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":67,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.507504+0000","last_change":"2024-09-12T01:26:47.476115+0000","last_active":"2024-09-12T01:28:17.507504+0000","last_peered":"2024-09-12T01:28:17.507504+0000","last_clean":"2024-09-12T01:28:17.507504+0000","last_became_active":"2024-09-12T01:26:47.475815+0000","last_became_peered":"2024-09-12T01:26:47.475815+0000","last_unstale":"2024-09-12T01:28:17.507504+0000","last_undegraded":"2024-09-12T01:28:17.507504+0000","last_fullsized":"2024-09-12T01:28:17.507504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:50.775693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.420141+0000","last_change":"2024-09-12T01:26:57.518831+0000","last_active":"2024-09-12T01:28:17.420141+0000","last_peered":"2024-09-12T01:28:17.420141+0000","last_clean":"2024-09-12T01:28:17.420141+0000","last_became_active":"2024-09-12T01:26:57.518439+0000","last_became_peered":"2024-09-12T01:26:57.518439+0000","last_unstale":"2024-09-12T01:28:17.420141+0000","last_undegraded":"2024-09-12T01:28:17.420141+0000","last_fullsized":"2024-09-12T01:28:17.420141+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:56.502907+0000","last_clean_scrub_stamp":"2024-09-12T01:26:56.502907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:49:14.845248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.418514+0000","last_change":"2024-09-12T01:27:16.832809+0000","last_active":"2024-09-12T01:28:17.418514+0000","last_peered":"2024-09-12T01:28:17.418514+0000","last_clean":"2024-09-12T01:28:17.418514+0000","last_became_active":"2024-09-12T01:27:16.832243+0000","last_became_peered":"2024-09-12T01:27:16.832243+0000","last_unstale":"2024-09-12T01:28:17.418514+0000","last_undegraded":"2024-09-12T01:28:17.418514+0000","last_fullsized":"2024-09-12T01:28:17.418514+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:55:00.002884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.420355+0000","last_change":"2024-09-12T01:26:47.475959+0000","last_active":"2024-09-12T01:28:17.420355+0000","last_peered":"2024-09-12T01:28:17.420355+0000","last_clean":"2024-09-12T01:28:17.420355+0000","last_became_active":"2024-09-12T01:26:47.475660+0000","last_became_peered":"2024-09-12T01:26:47.475660+0000","last_unstale":"2024-09-12T01:28:17.420355+0000","last_undegraded":"2024-09-12T01:28:17.420355+0000","last_fullsized":"2024-09-12T01:28:17.420355+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:42.556069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.419882+0000","last_change":"2024-09-12T01:26:47.474380+0000","last_active":"2024-09-12T01:28:17.419882+0000","last_peered":"2024-09-12T01:28:17.419882+0000","last_clean":"2024-09-12T01:28:17.419882+0000","last_became_active":"2024-09-12T01:26:47.474050+0000","last_became_peered":"2024-09-12T01:26:47.474050+0000","last_unstale":"2024-09-12T01:28:17.419882+0000","last_undegraded":"2024-09-12T01:28:17.419882+0000","last_fullsized":"2024-09-12T01:28:17.419882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:19:05.032866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.415001+0000","last_change":"2024-09-12T01:26:47.477056+0000","last_active":"2024-09-12T01:28:17.415001+0000","last_peered":"2024-09-12T01:28:17.415001+0000","last_clean":"2024-09-12T01:28:17.415001+0000","last_became_active":"2024-09-12T01:26:47.476641+0000","last_became_peered":"2024-09-12T01:26:47.476641+0000","last_unstale":"2024-09-12T01:28:17.415001+0000","last_undegraded":"2024-09-12T01:28:17.415001+0000","last_fullsized":"2024-09-12T01:28:17.415001+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:52.924471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506547+0000","last_change":"2024-09-12T01:27:16.832763+0000","last_active":"2024-09-12T01:28:17.506547+0000","last_peered":"2024-09-12T01:28:17.506547+0000","last_clean":"2024-09-12T01:28:17.506547+0000","last_became_active":"2024-09-12T01:27:16.832396+0000","last_became_peered":"2024-09-12T01:27:16.832396+0000","last_unstale":"2024-09-12T01:28:17.506547+0000","last_undegraded":"2024-09-12T01:28:17.506547+0000","last_fullsized":"2024-09-12T01:28:17.506547+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:59.444289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.419494+0000","last_change":"2024-09-12T01:26:47.472125+0000","last_active":"2024-09-12T01:28:17.419494+0000","last_peered":"2024-09-12T01:28:17.419494+0000","last_clean":"2024-09-12T01:28:17.419494+0000","last_became_active":"2024-09-12T01:26:47.471758+0000","last_became_peered":"2024-09-12T01:26:47.471758+0000","last_unstale":"2024-09-12T01:28:17.419494+0000","last_undegraded":"2024-09-12T01:28:17.419494+0000","last_fullsized":"2024-09-12T01:28:17.419494+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:38.949566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.420381+0000","last_change":"2024-09-12T01:26:47.473584+0000","last_active":"2024-09-12T01:28:17.420381+0000","last_peered":"2024-09-12T01:28:17.420381+0000","last_clean":"2024-09-12T01:28:17.420381+0000","last_became_active":"2024-09-12T01:26:47.470089+0000","last_became_peered":"2024-09-12T01:26:47.470089+0000","last_unstale":"2024-09-12T01:28:17.420381+0000","last_undegraded":"2024-09-12T01:28:17.420381+0000","last_fullsized":"2024-09-12T01:28:17.420381+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:44:58.878810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.418101+0000","last_change":"2024-09-12T01:26:47.486547+0000","last_active":"2024-09-12T01:28:17.418101+0000","last_peered":"2024-09-12T01:28:17.418101+0000","last_clean":"2024-09-12T01:28:17.418101+0000","last_became_active":"2024-09-12T01:26:47.486105+0000","last_became_peered":"2024-09-12T01:26:47.486105+0000","last_unstale":"2024-09-12T01:28:17.418101+0000","last_undegraded":"2024-09-12T01:28:17.418101+0000","last_fullsized":"2024-09-12T01:28:17.418101+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:21:18.820930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506197+0000","last_change":"2024-09-12T01:26:47.480459+0000","last_active":"2024-09-12T01:28:17.506197+0000","last_peered":"2024-09-12T01:28:17.506197+0000","last_clean":"2024-09-12T01:28:17.506197+0000","last_became_active":"2024-09-12T01:26:47.478917+0000","last_became_peered":"2024-09-12T01:26:47.478917+0000","last_unstale":"2024-09-12T01:28:17.506197+0000","last_undegraded":"2024-09-12T01:28:17.506197+0000","last_fullsized":"2024-09-12T01:28:17.506197+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:20:54.470888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.417534+0000","last_change":"2024-09-12T01:26:47.480744+0000","last_active":"2024-09-12T01:28:17.417534+0000","last_peered":"2024-09-12T01:28:17.417534+0000","last_clean":"2024-09-12T01:28:17.417534+0000","last_became_active":"2024-09-12T01:26:47.480354+0000","last_became_peered":"2024-09-12T01:26:47.480354+0000","last_unstale":"2024-09-12T01:28:17.417534+0000","last_undegraded":"2024-09-12T01:28:17.417534+0000","last_fullsized":"2024-09-12T01:28:17.417534+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:02:04.670945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506875+0000","last_change":"2024-09-12T01:26:47.476431+0000","last_active":"2024-09-12T01:28:17.506875+0000","last_peered":"2024-09-12T01:28:17.506875+0000","last_clean":"2024-09-12T01:28:17.506875+0000","last_became_active":"2024-09-12T01:26:47.476110+0000","last_became_peered":"2024-09-12T01:26:47.476110+0000","last_unstale":"2024-09-12T01:28:17.506875+0000","last_undegraded":"2024-09-12T01:28:17.506875+0000","last_fullsized":"2024-09-12T01:28:17.506875+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:34:24.086677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506287+0000","last_change":"2024-09-12T01:26:47.484568+0000","last_active":"2024-09-12T01:28:17.506287+0000","last_peered":"2024-09-12T01:28:17.506287+0000","last_clean":"2024-09-12T01:28:17.506287+0000","last_became_active":"2024-09-12T01:26:47.484165+0000","last_became_peered":"2024-09-12T01:26:47.484165+0000","last_unstale":"2024-09-12T01:28:17.506287+0000","last_undegraded":"2024-09-12T01:28:17.506287+0000","last_fullsized":"2024-09-12T01:28:17.506287+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:26:51.324754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.419113+0000","last_change":"2024-09-12T01:26:47.482608+0000","last_active":"2024-09-12T01:28:17.419113+0000","last_peered":"2024-09-12T01:28:17.419113+0000","last_clean":"2024-09-12T01:28:17.419113+0000","last_became_active":"2024-09-12T01:26:47.482299+0000","last_became_peered":"2024-09-12T01:26:47.482299+0000","last_unstale":"2024-09-12T01:28:17.419113+0000","last_undegraded":"2024-09-12T01:28:17.419113+0000","last_fullsized":"2024-09-12T01:28:17.419113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:48:39.516539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.420288+0000","last_change":"2024-09-12T01:26:47.472976+0000","last_active":"2024-09-12T01:28:17.420288+0000","last_peered":"2024-09-12T01:28:17.420288+0000","last_clean":"2024-09-12T01:28:17.420288+0000","last_became_active":"2024-09-12T01:26:47.472495+0000","last_became_peered":"2024-09-12T01:26:47.472495+0000","last_unstale":"2024-09-12T01:28:17.420288+0000","last_undegraded":"2024-09-12T01:28:17.420288+0000","last_fullsized":"2024-09-12T01:28:17.420288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:57.556675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.507184+0000","last_change":"2024-09-12T01:26:47.476004+0000","last_active":"2024-09-12T01:28:17.507184+0000","last_peered":"2024-09-12T01:28:17.507184+0000","last_clean":"2024-09-12T01:28:17.507184+0000","last_became_active":"2024-09-12T01:26:47.475440+0000","last_became_peered":"2024-09-12T01:26:47.475440+0000","last_unstale":"2024-09-12T01:28:17.507184+0000","last_undegraded":"2024-09-12T01:28:17.507184+0000","last_fullsized":"2024-09-12T01:28:17.507184+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:36:47.650577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.419430+0000","last_change":"2024-09-12T01:26:47.483848+0000","last_active":"2024-09-12T01:28:17.419430+0000","last_peered":"2024-09-12T01:28:17.419430+0000","last_clean":"2024-09-12T01:28:17.419430+0000","last_became_active":"2024-09-12T01:26:47.483446+0000","last_became_peered":"2024-09-12T01:26:47.483446+0000","last_unstale":"2024-09-12T01:28:17.419430+0000","last_undegraded":"2024-09-12T01:28:17.419430+0000","last_fullsized":"2024-09-12T01:28:17.419430+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:47.844458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506102+0000","last_change":"2024-09-12T01:26:47.473801+0000","last_active":"2024-09-12T01:28:17.506102+0000","last_peered":"2024-09-12T01:28:17.506102+0000","last_clean":"2024-09-12T01:28:17.506102+0000","last_became_active":"2024-09-12T01:26:47.473268+0000","last_became_peered":"2024-09-12T01:26:47.473268+0000","last_unstale":"2024-09-12T01:28:17.506102+0000","last_undegraded":"2024-09-12T01:28:17.506102+0000","last_fullsized":"2024-09-12T01:28:17.506102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:13:30.595643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.414673+0000","last_change":"2024-09-12T01:26:47.475778+0000","last_active":"2024-09-12T01:28:17.414673+0000","last_peered":"2024-09-12T01:28:17.414673+0000","last_clean":"2024-09-12T01:28:17.414673+0000","last_became_active":"2024-09-12T01:26:47.475477+0000","last_became_peered":"2024-09-12T01:26:47.475477+0000","last_unstale":"2024-09-12T01:28:17.414673+0000","last_undegraded":"2024-09-12T01:28:17.414673+0000","last_fullsized":"2024-09-12T01:28:17.414673+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:56.555424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.414671+0000","last_change":"2024-09-12T01:26:47.474736+0000","last_active":"2024-09-12T01:28:17.414671+0000","last_peered":"2024-09-12T01:28:17.414671+0000","last_clean":"2024-09-12T01:28:17.414671+0000","last_became_active":"2024-09-12T01:26:47.474378+0000","last_became_peered":"2024-09-12T01:26:47.474378+0000","last_unstale":"2024-09-12T01:28:17.414671+0000","last_undegraded":"2024-09-12T01:28:17.414671+0000","last_fullsized":"2024-09-12T01:28:17.414671+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:02:37.818552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506260+0000","last_change":"2024-09-12T01:26:47.479586+0000","last_active":"2024-09-12T01:28:17.506260+0000","last_peered":"2024-09-12T01:28:17.506260+0000","last_clean":"2024-09-12T01:28:17.506260+0000","last_became_active":"2024-09-12T01:26:47.473216+0000","last_became_peered":"2024-09-12T01:26:47.473216+0000","last_unstale":"2024-09-12T01:28:17.506260+0000","last_undegraded":"2024-09-12T01:28:17.506260+0000","last_fullsized":"2024-09-12T01:28:17.506260+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:05:28.994081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.507151+0000","last_change":"2024-09-12T01:26:47.475634+0000","last_active":"2024-09-12T01:28:17.507151+0000","last_peered":"2024-09-12T01:28:17.507151+0000","last_clean":"2024-09-12T01:28:17.507151+0000","last_became_active":"2024-09-12T01:26:47.475319+0000","last_became_peered":"2024-09-12T01:26:47.475319+0000","last_unstale":"2024-09-12T01:28:17.507151+0000","last_undegraded":"2024-09-12T01:28:17.507151+0000","last_fullsized":"2024-09-12T01:28:17.507151+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:40.095276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.416094+0000","last_change":"2024-09-12T01:26:47.475721+0000","last_active":"2024-09-12T01:28:17.416094+0000","last_peered":"2024-09-12T01:28:17.416094+0000","last_clean":"2024-09-12T01:28:17.416094+0000","last_became_active":"2024-09-12T01:26:47.475149+0000","last_became_peered":"2024-09-12T01:26:47.475149+0000","last_unstale":"2024-09-12T01:28:17.416094+0000","last_undegraded":"2024-09-12T01:28:17.416094+0000","last_fullsized":"2024-09-12T01:28:17.416094+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:27:19.946756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.507880+0000","last_change":"2024-09-12T01:26:47.482667+0000","last_active":"2024-09-12T01:28:17.507880+0000","last_peered":"2024-09-12T01:28:17.507880+0000","last_clean":"2024-09-12T01:28:17.507880+0000","last_became_active":"2024-09-12T01:26:47.482258+0000","last_became_peered":"2024-09-12T01:26:47.482258+0000","last_unstale":"2024-09-12T01:28:17.507880+0000","last_undegraded":"2024-09-12T01:28:17.507880+0000","last_fullsized":"2024-09-12T01:28:17.507880+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:10.449000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506527+0000","last_change":"2024-09-12T01:26:47.475406+0000","last_active":"2024-09-12T01:28:17.506527+0000","last_peered":"2024-09-12T01:28:17.506527+0000","last_clean":"2024-09-12T01:28:17.506527+0000","last_became_active":"2024-09-12T01:26:47.474846+0000","last_became_peered":"2024-09-12T01:26:47.474846+0000","last_unstale":"2024-09-12T01:28:17.506527+0000","last_undegraded":"2024-09-12T01:28:17.506527+0000","last_fullsized":"2024-09-12T01:28:17.506527+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:54:02.527691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.417011+0000","last_change":"2024-09-12T01:26:47.477383+0000","last_active":"2024-09-12T01:28:17.417011+0000","last_peered":"2024-09-12T01:28:17.417011+0000","last_clean":"2024-09-12T01:28:17.417011+0000","last_became_active":"2024-09-12T01:26:47.476977+0000","last_became_peered":"2024-09-12T01:26:47.476977+0000","last_unstale":"2024-09-12T01:28:17.417011+0000","last_undegraded":"2024-09-12T01:28:17.417011+0000","last_fullsized":"2024-09-12T01:28:17.417011+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:51:45.310556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.415472+0000","last_change":"2024-09-12T01:26:47.473686+0000","last_active":"2024-09-12T01:28:17.415472+0000","last_peered":"2024-09-12T01:28:17.415472+0000","last_clean":"2024-09-12T01:28:17.415472+0000","last_became_active":"2024-09-12T01:26:47.473363+0000","last_became_peered":"2024-09-12T01:26:47.473363+0000","last_unstale":"2024-09-12T01:28:17.415472+0000","last_undegraded":"2024-09-12T01:28:17.415472+0000","last_fullsized":"2024-09-12T01:28:17.415472+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:08:23.238434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.506964+0000","last_change":"2024-09-12T01:26:47.479310+0000","last_active":"2024-09-12T01:28:17.506964+0000","last_peered":"2024-09-12T01:28:17.506964+0000","last_clean":"2024-09-12T01:28:17.506964+0000","last_became_active":"2024-09-12T01:26:47.477416+0000","last_became_peered":"2024-09-12T01:26:47.477416+0000","last_unstale":"2024-09-12T01:28:17.506964+0000","last_undegraded":"2024-09-12T01:28:17.506964+0000","last_fullsized":"2024-09-12T01:28:17.506964+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:01:55.023455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.507634+0000","last_change":"2024-09-12T01:26:47.478455+0000","last_active":"2024-09-12T01:28:17.507634+0000","last_peered":"2024-09-12T01:28:17.507634+0000","last_clean":"2024-09-12T01:28:17.507634+0000","last_became_active":"2024-09-12T01:26:47.478093+0000","last_became_peered":"2024-09-12T01:26:47.478093+0000","last_unstale":"2024-09-12T01:28:17.507634+0000","last_undegraded":"2024-09-12T01:28:17.507634+0000","last_fullsized":"2024-09-12T01:28:17.507634+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:04:41.626677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.507831+0000","last_change":"2024-09-12T01:26:47.479385+0000","last_active":"2024-09-12T01:28:17.507831+0000","last_peered":"2024-09-12T01:28:17.507831+0000","last_clean":"2024-09-12T01:28:17.507831+0000","last_became_active":"2024-09-12T01:26:47.479025+0000","last_became_peered":"2024-09-12T01:26:47.479025+0000","last_unstale":"2024-09-12T01:28:17.507831+0000","last_undegraded":"2024-09-12T01:28:17.507831+0000","last_fullsized":"2024-09-12T01:28:17.507831+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:11.927111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.414794+0000","last_change":"2024-09-12T01:26:47.461567+0000","last_active":"2024-09-12T01:28:17.414794+0000","last_peered":"2024-09-12T01:28:17.414794+0000","last_clean":"2024-09-12T01:28:17.414794+0000","last_became_active":"2024-09-12T01:26:47.461230+0000","last_became_peered":"2024-09-12T01:26:47.461230+0000","last_unstale":"2024-09-12T01:28:17.414794+0000","last_undegraded":"2024-09-12T01:28:17.414794+0000","last_fullsized":"2024-09-12T01:28:17.414794+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:09:24.519448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.415210+0000","last_change":"2024-09-12T01:26:47.472323+0000","last_active":"2024-09-12T01:28:17.415210+0000","last_peered":"2024-09-12T01:28:17.415210+0000","last_clean":"2024-09-12T01:28:17.415210+0000","last_became_active":"2024-09-12T01:26:47.469837+0000","last_became_peered":"2024-09-12T01:26:47.469837+0000","last_unstale":"2024-09-12T01:28:17.415210+0000","last_undegraded":"2024-09-12T01:28:17.415210+0000","last_fullsized":"2024-09-12T01:28:17.415210+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:46.202656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T01:28:17.419215+0000","last_change":"2024-09-12T01:26:47.479707+0000","last_active":"2024-09-12T01:28:17.419215+0000","last_peered":"2024-09-12T01:28:17.419215+0000","last_clean":"2024-09-12T01:28:17.419215+0000","last_became_active":"2024-09-12T01:26:47.479368+0000","last_became_peered":"2024-09-12T01:26:47.479368+0000","last_unstale":"2024-09-12T01:28:17.419215+0000","last_undegraded":"2024-09-12T01:28:17.419215+0000","last_fullsized":"2024-09-12T01:28:17.419215+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:26:46.440911+0000","last_clean_scrub_stamp":"2024-09-12T01:26:46.440911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:04:30.129381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70603,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":673,"num_read_kb":410,"num_write":246,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":211809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":213,"ondisk_log_size":213,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416422,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32296,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712856,"statfs":{"total":95995035648,"available":95961964544,"internally_reserved":0,"allocated":1134592,"data_stored":564248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645355,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37144,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708008,"statfs":{"total":95995035648,"available":95957000192,"internally_reserved":0,"allocated":1835008,"data_stored":1218162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808881,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28160,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716992,"statfs":{"total":95995035648,"available":95966199808,"internally_reserved":0,"allocated":1159168,"data_stored":563785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070518,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1155072,"data_stored":564163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332155,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37064,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708088,"statfs":{"total":95995035648,"available":95957082112,"internally_reserved":0,"allocated":1753088,"data_stored":1161055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495682,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1134592,"data_stored":559682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757320,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37072,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708080,"statfs":{"total":95995035648,"available":95957073920,"internally_reserved":0,"allocated":1761280,"data_stored":1155851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920846,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28220,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716932,"statfs":{"total":95995035648,"available":95966138368,"internally_reserved":0,"allocated":1216512,"data_stored":626758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182483,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36468,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708684,"statfs":{"total":95995035648,"available":95957692416,"internally_reserved":0,"allocated":1146880,"data_stored":558997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411418,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36476,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708676,"statfs":{"total":95995035648,"available":95957684224,"internally_reserved":0,"allocated":1150976,"data_stored":560932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574943,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36528,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708624,"statfs":{"total":95995035648,"available":95957630976,"internally_reserved":0,"allocated":1204224,"data_stored":623714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738468,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36484,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708668,"statfs":{"total":95995035648,"available":95957676032,"internally_reserved":0,"allocated":1159168,"data_stored":561138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":64095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T01:28:51.989 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-12T01:28:51.990 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-12T01:28:51.990 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-12T01:28:51.990 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-12T01:28:52.401 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:28:52.902 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:52 smithi001 ceph-mon[30387]: pgmap v376: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T01:28:52.902 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:52 smithi001 ceph-mon[30387]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:28:52.902 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:52 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:28:52.902 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:52 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:28:52.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:52 smithi053 ceph-mon[35158]: pgmap v376: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T01:28:52.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:52 smithi053 ceph-mon[35158]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:28:52.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:52 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:28:52.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:52 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:28:53.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.016+0000 7f7199400640 1 -- 172.21.15.1:0/2204600521 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7194103ff0 msgr2=0x7f71941043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:53.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.016+0000 7f7199400640 1 --2- 172.21.15.1:0/2204600521 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7194103ff0 0x7f71941043d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7184009900 tx=0x7f718402a3f0 comp rx=0 tx=0).stop 2024-09-12T01:28:53.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.016+0000 7f7199400640 1 -- 172.21.15.1:0/2204600521 shutdown_connections 2024-09-12T01:28:53.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.016+0000 7f7199400640 1 --2- 172.21.15.1:0/2204600521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7194109120 0x7f7194110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:53.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.016+0000 7f7199400640 1 --2- 172.21.15.1:0/2204600521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71941049a0 0x7f71941089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:53.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.016+0000 7f7199400640 1 --2- 172.21.15.1:0/2204600521 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7194103ff0 0x7f71941043d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:53.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.016+0000 7f7199400640 1 -- 172.21.15.1:0/2204600521 >> 172.21.15.1:0/2204600521 conn(0x7f71940ffa90 msgr2=0x7f7194101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:53.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.016+0000 7f7199400640 1 -- 172.21.15.1:0/2204600521 shutdown_connections 2024-09-12T01:28:53.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.016+0000 7f7199400640 1 -- 172.21.15.1:0/2204600521 wait complete. 2024-09-12T01:28:53.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.017+0000 7f7199400640 1 Processor -- start 2024-09-12T01:28:53.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.017+0000 7f7199400640 1 -- start start 2024-09-12T01:28:53.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.017+0000 7f7199400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7194103ff0 0x7f719419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:53.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.017+0000 7f7199400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71941049a0 0x7f719419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:53.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.017+0000 7f7199400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7194109120 0x7f719419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:53.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.017+0000 7f7199400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7194114170 con 0x7f71941049a0 2024-09-12T01:28:53.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.018+0000 7f7199400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7194113ff0 con 0x7f7194109120 2024-09-12T01:28:53.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.018+0000 7f7199400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f71941142f0 con 0x7f7194103ff0 2024-09-12T01:28:53.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.018+0000 7f718be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71941049a0 0x7f719419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:53.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.018+0000 7f7193200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7194109120 0x7f719419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:53.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.018+0000 7f7193200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7194109120 0x7f719419ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:53414/0 (socket says 172.21.15.1:53414) 2024-09-12T01:28:53.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.018+0000 7f7193200640 1 -- 172.21.15.1:0/1528702633 learned_addr learned my addr 172.21.15.1:0/1528702633 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:28:53.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.018+0000 7f7192800640 1 --2- 172.21.15.1:0/1528702633 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7194103ff0 0x7f719419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:53.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.018+0000 7f7193200640 1 -- 172.21.15.1:0/1528702633 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7194103ff0 msgr2=0x7f719419a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:53.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.018+0000 7f7193200640 1 --2- 172.21.15.1:0/1528702633 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7194103ff0 0x7f719419a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:53.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.018+0000 7f7193200640 1 -- 172.21.15.1:0/1528702633 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71941049a0 msgr2=0x7f719419abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:53.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.018+0000 7f7193200640 1 --2- 172.21.15.1:0/1528702633 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71941049a0 0x7f719419abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:53.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.018+0000 7f7193200640 1 -- 172.21.15.1:0/1528702633 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f719419f620 con 0x7f7194109120 2024-09-12T01:28:53.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.018+0000 7f718be00640 1 --2- 172.21.15.1:0/1528702633 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71941049a0 0x7f719419abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:28:53.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.019+0000 7f7193200640 1 --2- 172.21.15.1:0/1528702633 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7194109120 0x7f719419ef40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f718c00eb10 tx=0x7f718c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:53.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.019+0000 7f718b400640 1 -- 172.21.15.1:0/1528702633 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718c004b20 con 0x7f7194109120 2024-09-12T01:28:53.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.019+0000 7f718b400640 1 -- 172.21.15.1:0/1528702633 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f718c004cc0 con 0x7f7194109120 2024-09-12T01:28:53.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.019+0000 7f718b400640 1 -- 172.21.15.1:0/1528702633 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718c010760 con 0x7f7194109120 2024-09-12T01:28:53.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.019+0000 7f7199400640 1 -- 172.21.15.1:0/1528702633 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f719419f910 con 0x7f7194109120 2024-09-12T01:28:53.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.019+0000 7f7199400640 1 -- 172.21.15.1:0/1528702633 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71941a71f0 con 0x7f7194109120 2024-09-12T01:28:53.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.020+0000 7f7199400640 1 -- 172.21.15.1:0/1528702633 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71941043d0 con 0x7f7194109120 2024-09-12T01:28:53.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.023+0000 7f718b400640 1 -- 172.21.15.1:0/1528702633 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f718c005000 con 0x7f7194109120 2024-09-12T01:28:53.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.025+0000 7f718b400640 1 --2- 172.21.15.1:0/1528702633 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f71780a2c90 0x7f71780a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:53.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.025+0000 7f718b400640 1 -- 172.21.15.1:0/1528702633 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f718c0cfc50 con 0x7f7194109120 2024-09-12T01:28:53.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.025+0000 7f7192800640 1 --2- 172.21.15.1:0/1528702633 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f71780a2c90 0x7f71780a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:53.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.026+0000 7f7192800640 1 --2- 172.21.15.1:0/1528702633 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f71780a2c90 0x7f71780a5150 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7184000c00 tx=0x7f7184005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:53.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.026+0000 7f718b400640 1 -- 172.21.15.1:0/1528702633 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f718c0541b0 con 0x7f7194109120 2024-09-12T01:28:53.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:52 smithi104 ceph-mon[35219]: pgmap v376: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T01:28:53.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:52 smithi104 ceph-mon[35219]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:28:53.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:52 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:28:53.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:52 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:28:53.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.249+0000 7f7199400640 1 -- 172.21.15.1:0/1528702633 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f719410c360 con 0x7f7194109120 2024-09-12T01:28:53.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.249+0000 7f718b400640 1 -- 172.21.15.1:0/1528702633 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f718c09c660 con 0x7f7194109120 2024-09-12T01:28:53.249 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:28:53.249 INFO:teuthology.orchestra.run.smithi001.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-12T01:28:53.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.251+0000 7f7199400640 1 -- 172.21.15.1:0/1528702633 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f71780a2c90 msgr2=0x7f71780a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:53.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.251+0000 7f7199400640 1 --2- 172.21.15.1:0/1528702633 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f71780a2c90 0x7f71780a5150 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7184000c00 tx=0x7f7184005b50 comp rx=0 tx=0).stop 2024-09-12T01:28:53.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.251+0000 7f7199400640 1 -- 172.21.15.1:0/1528702633 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7194109120 msgr2=0x7f719419ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:53.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.251+0000 7f7199400640 1 --2- 172.21.15.1:0/1528702633 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7194109120 0x7f719419ef40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f718c00eb10 tx=0x7f718c00c2d0 comp rx=0 tx=0).stop 2024-09-12T01:28:53.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.251+0000 7f7199400640 1 -- 172.21.15.1:0/1528702633 shutdown_connections 2024-09-12T01:28:53.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.251+0000 7f7199400640 1 --2- 172.21.15.1:0/1528702633 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7194109120 0x7f719419ef40 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:53.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.251+0000 7f7199400640 1 --2- 172.21.15.1:0/1528702633 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71941049a0 0x7f719419abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:53.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.251+0000 7f7199400640 1 --2- 172.21.15.1:0/1528702633 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f71780a2c90 0x7f71780a5150 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:53.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.251+0000 7f7199400640 1 --2- 172.21.15.1:0/1528702633 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7194103ff0 0x7f719419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:53.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.251+0000 7f7199400640 1 -- 172.21.15.1:0/1528702633 >> 172.21.15.1:0/1528702633 conn(0x7f71940ffa90 msgr2=0x7f7194100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:53.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.252+0000 7f7199400640 1 -- 172.21.15.1:0/1528702633 shutdown_connections 2024-09-12T01:28:53.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:53.252+0000 7f7199400640 1 -- 172.21.15.1:0/1528702633 wait complete. 2024-09-12T01:28:53.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:53 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:28:53.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:53 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:28:53.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:53 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:28:53.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:53 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/1528702633' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-12T01:28:53.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:53 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:28:53.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:53 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:28:53.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:53 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:28:53.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:53 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/1528702633' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-12T01:28:54.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:53 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:28:54.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:53 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:28:54.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:53 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:28:54.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:53 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/1528702633' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-12T01:28:54.113 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-12T01:28:54.113 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:28:54.493 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:28:54.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:54 smithi001 ceph-mon[30387]: pgmap v377: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T01:28:54.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:54 smithi053 ceph-mon[35158]: pgmap v377: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T01:28:55.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:54 smithi104 ceph-mon[35219]: pgmap v377: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T01:28:55.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.091+0000 7f5b13200640 1 -- 172.21.15.1:0/3072945795 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b14103e30 msgr2=0x7f5b14104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:55.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.091+0000 7f5b13200640 1 --2- 172.21.15.1:0/3072945795 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b14103e30 0x7f5b14104210 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5b04009900 tx=0x7f5b0402a410 comp rx=0 tx=0).stop 2024-09-12T01:28:55.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.091+0000 7f5b13200640 1 -- 172.21.15.1:0/3072945795 shutdown_connections 2024-09-12T01:28:55.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.091+0000 7f5b13200640 1 --2- 172.21.15.1:0/3072945795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b14108f60 0x7f5b14110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:55.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.091+0000 7f5b13200640 1 --2- 172.21.15.1:0/3072945795 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b141047e0 0x7f5b14108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:55.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.091+0000 7f5b13200640 1 --2- 172.21.15.1:0/3072945795 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b14103e30 0x7f5b14104210 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:55.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.091+0000 7f5b13200640 1 -- 172.21.15.1:0/3072945795 >> 172.21.15.1:0/3072945795 conn(0x7f5b140ff8d0 msgr2=0x7f5b14101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:55.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.091+0000 7f5b13200640 1 -- 172.21.15.1:0/3072945795 shutdown_connections 2024-09-12T01:28:55.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.091+0000 7f5b13200640 1 -- 172.21.15.1:0/3072945795 wait complete. 2024-09-12T01:28:55.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.092+0000 7f5b13200640 1 Processor -- start 2024-09-12T01:28:55.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.093+0000 7f5b13200640 1 -- start start 2024-09-12T01:28:55.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.093+0000 7f5b13200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b14103e30 0x7f5b1419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:55.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.093+0000 7f5b13200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b141047e0 0x7f5b1419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:55.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.093+0000 7f5b13200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b14108f60 0x7f5b1419ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:55.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.093+0000 7f5b13200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5b14113fc0 con 0x7f5b14103e30 2024-09-12T01:28:55.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.093+0000 7f5b13200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5b14113e40 con 0x7f5b141047e0 2024-09-12T01:28:55.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.093+0000 7f5b13200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5b14114140 con 0x7f5b14108f60 2024-09-12T01:28:55.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.093+0000 7f5b11e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b14103e30 0x7f5b1419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:55.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.093+0000 7f5b11e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b14103e30 0x7f5b1419a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53174/0 (socket says 172.21.15.1:53174) 2024-09-12T01:28:55.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.093+0000 7f5b11e00640 1 -- 172.21.15.1:0/16400087 learned_addr learned my addr 172.21.15.1:0/16400087 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:28:55.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.093+0000 7f5b11400640 1 --2- 172.21.15.1:0/16400087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b141047e0 0x7f5b1419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:55.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.094+0000 7f5b12800640 1 --2- 172.21.15.1:0/16400087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b14108f60 0x7f5b1419ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:55.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.094+0000 7f5b11e00640 1 -- 172.21.15.1:0/16400087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b14108f60 msgr2=0x7f5b1419ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:55.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.094+0000 7f5b11e00640 1 --2- 172.21.15.1:0/16400087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b14108f60 0x7f5b1419ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:55.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.094+0000 7f5b11e00640 1 -- 172.21.15.1:0/16400087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b141047e0 msgr2=0x7f5b1419a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:55.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.094+0000 7f5b11e00640 1 --2- 172.21.15.1:0/16400087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b141047e0 0x7f5b1419a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:55.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.094+0000 7f5b11e00640 1 -- 172.21.15.1:0/16400087 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b1419f430 con 0x7f5b14103e30 2024-09-12T01:28:55.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.094+0000 7f5b11e00640 1 --2- 172.21.15.1:0/16400087 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b14103e30 0x7f5b1419a480 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5b04006010 tx=0x7f5b04004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:55.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.095+0000 7f5b0aa00640 1 -- 172.21.15.1:0/16400087 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0403f030 con 0x7f5b14103e30 2024-09-12T01:28:55.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.095+0000 7f5b0aa00640 1 -- 172.21.15.1:0/16400087 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b04004440 con 0x7f5b14103e30 2024-09-12T01:28:55.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.095+0000 7f5b13200640 1 -- 172.21.15.1:0/16400087 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b1419f6c0 con 0x7f5b14103e30 2024-09-12T01:28:55.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.095+0000 7f5b0aa00640 1 -- 172.21.15.1:0/16400087 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b04035070 con 0x7f5b14103e30 2024-09-12T01:28:55.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.095+0000 7f5b13200640 1 -- 172.21.15.1:0/16400087 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b141a7000 con 0x7f5b14103e30 2024-09-12T01:28:55.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.096+0000 7f5b13200640 1 -- 172.21.15.1:0/16400087 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b14105310 con 0x7f5b14103e30 2024-09-12T01:28:55.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.098+0000 7f5b0aa00640 1 -- 172.21.15.1:0/16400087 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f5b04042030 con 0x7f5b14103e30 2024-09-12T01:28:55.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.099+0000 7f5b0aa00640 1 --2- 172.21.15.1:0/16400087 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f5af80a2c10 0x7f5af80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:55.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.099+0000 7f5b0aa00640 1 -- 172.21.15.1:0/16400087 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f5b040ef900 con 0x7f5b14103e30 2024-09-12T01:28:55.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.099+0000 7f5b11400640 1 --2- 172.21.15.1:0/16400087 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f5af80a2c10 0x7f5af80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:55.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.100+0000 7f5b11400640 1 --2- 172.21.15.1:0/16400087 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f5af80a2c10 0x7f5af80a50d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5b140ffcf0 tx=0x7f5b00005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:55.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.103+0000 7f5b0aa00640 1 -- 172.21.15.1:0/16400087 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5b040bbcc0 con 0x7f5b14103e30 2024-09-12T01:28:55.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.325+0000 7f5b13200640 1 -- 172.21.15.1:0/16400087 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b1410c140 con 0x7f5b14103e30 2024-09-12T01:28:55.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.326+0000 7f5b0aa00640 1 -- 172.21.15.1:0/16400087 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11962 (secure 0 0 0) 0x7f5b040bffc0 con 0x7f5b14103e30 2024-09-12T01:28:55.326 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:28:55.326 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":28,"btime":"2024-09-12T01:27:46:332106+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14430,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/572192306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":572192306},{"type":"v1","addr":"172.21.15.1:6835","nonce":572192306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14442,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/1050734277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":1050734277},{"type":"v1","addr":"172.21.15.1:6837","nonce":1050734277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:26:48.555447+0000","modified":"2024-09-12T01:27:46.206001+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24545,"mds_1":24511,"mds_2":24506,"mds_3":24493,"mds_4":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14481":{"gid":14481,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/3651849993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":3651849993},{"type":"v1","addr":"172.21.15.1:6839","nonce":3651849993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14493":{"gid":14493,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2464781973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2464781973},{"type":"v1","addr":"172.21.15.1:6841","nonce":2464781973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2071825633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2071825633},{"type":"v1","addr":"172.21.15.104:6833","nonce":2071825633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/2930310711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2930310711},{"type":"v1","addr":"172.21.15.53:6833","nonce":2930310711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2032381725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2032381725},{"type":"v1","addr":"172.21.15.53:6835","nonce":2032381725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.104:6835/1547038540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1547038540},{"type":"v1","addr":"172.21.15.104:6835","nonce":1547038540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":23,"addr":"172.21.15.104:6837/464720247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":464720247},{"type":"v1","addr":"172.21.15.104:6837","nonce":464720247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":22,"addr":"172.21.15.53:6837/2848743754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2848743754},{"type":"v1","addr":"172.21.15.53:6837","nonce":2848743754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":21,"addr":"172.21.15.104:6839/1459976041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1459976041},{"type":"v1","addr":"172.21.15.104:6839","nonce":1459976041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.53:6839/4019389301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":4019389301},{"type":"v1","addr":"172.21.15.53:6839","nonce":4019389301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24487,24493,24506,24545,24511]},"id":1}]} 2024-09-12T01:28:55.327 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 28 2024-09-12T01:28:55.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.329+0000 7f5b13200640 1 -- 172.21.15.1:0/16400087 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f5af80a2c10 msgr2=0x7f5af80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:55.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.329+0000 7f5b13200640 1 --2- 172.21.15.1:0/16400087 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f5af80a2c10 0x7f5af80a50d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5b140ffcf0 tx=0x7f5b00005f90 comp rx=0 tx=0).stop 2024-09-12T01:28:55.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.329+0000 7f5b13200640 1 -- 172.21.15.1:0/16400087 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b14103e30 msgr2=0x7f5b1419a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:55.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.329+0000 7f5b13200640 1 --2- 172.21.15.1:0/16400087 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b14103e30 0x7f5b1419a480 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5b04006010 tx=0x7f5b04004040 comp rx=0 tx=0).stop 2024-09-12T01:28:55.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.329+0000 7f5b13200640 1 -- 172.21.15.1:0/16400087 shutdown_connections 2024-09-12T01:28:55.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.329+0000 7f5b13200640 1 --2- 172.21.15.1:0/16400087 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f5af80a2c10 0x7f5af80a50d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:55.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.329+0000 7f5b13200640 1 --2- 172.21.15.1:0/16400087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b14108f60 0x7f5b1419ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:55.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.329+0000 7f5b13200640 1 --2- 172.21.15.1:0/16400087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b141047e0 0x7f5b1419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:55.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.329+0000 7f5b13200640 1 --2- 172.21.15.1:0/16400087 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b14103e30 0x7f5b1419a480 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:55.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.329+0000 7f5b13200640 1 -- 172.21.15.1:0/16400087 >> 172.21.15.1:0/16400087 conn(0x7f5b140ff8d0 msgr2=0x7f5b14100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:55.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.329+0000 7f5b13200640 1 -- 172.21.15.1:0/16400087 shutdown_connections 2024-09-12T01:28:55.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:55.330+0000 7f5b13200640 1 -- 172.21.15.1:0/16400087 wait complete. 2024-09-12T01:28:55.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:55 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/16400087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:28:55.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:55 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/16400087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:28:56.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:55 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/16400087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:28:56.183 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:28:56.566 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:28:56.876 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:56 smithi001 ceph-mon[30387]: pgmap v378: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T01:28:56.976 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:56 smithi104 ceph-mon[35219]: pgmap v378: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T01:28:56.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:56 smithi053 ceph-mon[35158]: pgmap v378: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T01:28:57.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.133+0000 7f48e1c00640 1 -- 172.21.15.1:0/2513907707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f48dc103e30 msgr2=0x7f48dc104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:57.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.133+0000 7f48e1c00640 1 --2- 172.21.15.1:0/2513907707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f48dc103e30 0x7f48dc104210 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f48cc009900 tx=0x7f48cc02a3f0 comp rx=0 tx=0).stop 2024-09-12T01:28:57.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.133+0000 7f48e1c00640 1 -- 172.21.15.1:0/2513907707 shutdown_connections 2024-09-12T01:28:57.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.133+0000 7f48e1c00640 1 --2- 172.21.15.1:0/2513907707 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48dc108f60 0x7f48dc110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:57.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.133+0000 7f48e1c00640 1 --2- 172.21.15.1:0/2513907707 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48dc1047e0 0x7f48dc108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:57.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.133+0000 7f48e1c00640 1 --2- 172.21.15.1:0/2513907707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f48dc103e30 0x7f48dc104210 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:57.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.133+0000 7f48e1c00640 1 -- 172.21.15.1:0/2513907707 >> 172.21.15.1:0/2513907707 conn(0x7f48dc0ff910 msgr2=0x7f48dc101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:57.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.133+0000 7f48e1c00640 1 -- 172.21.15.1:0/2513907707 shutdown_connections 2024-09-12T01:28:57.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.133+0000 7f48e1c00640 1 -- 172.21.15.1:0/2513907707 wait complete. 2024-09-12T01:28:57.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.134+0000 7f48e1c00640 1 Processor -- start 2024-09-12T01:28:57.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.134+0000 7f48e1c00640 1 -- start start 2024-09-12T01:28:57.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.135+0000 7f48e1c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48dc103e30 0x7f48dc19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:57.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.135+0000 7f48e1c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48dc1047e0 0x7f48dc19a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:57.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.135+0000 7f48e1c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f48dc108f60 0x7f48dc19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:57.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.135+0000 7f48e1c00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f48dc114010 con 0x7f48dc108f60 2024-09-12T01:28:57.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.135+0000 7f48e1c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f48dc113e90 con 0x7f48dc1047e0 2024-09-12T01:28:57.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.135+0000 7f48e1c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f48dc114190 con 0x7f48dc103e30 2024-09-12T01:28:57.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.135+0000 7f48dbe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48dc103e30 0x7f48dc19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:57.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.135+0000 7f48e1200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f48dc108f60 0x7f48dc19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:57.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.135+0000 7f48dbe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48dc103e30 0x7f48dc19a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:32904/0 (socket says 172.21.15.1:32904) 2024-09-12T01:28:57.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.135+0000 7f48dbe00640 1 -- 172.21.15.1:0/1250334466 learned_addr learned my addr 172.21.15.1:0/1250334466 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:28:57.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.135+0000 7f48d3e00640 1 --2- 172.21.15.1:0/1250334466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48dc1047e0 0x7f48dc19a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:57.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.135+0000 7f48dbe00640 1 -- 172.21.15.1:0/1250334466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48dc1047e0 msgr2=0x7f48dc19a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:57.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.136+0000 7f48dbe00640 1 --2- 172.21.15.1:0/1250334466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48dc1047e0 0x7f48dc19a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:57.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.136+0000 7f48dbe00640 1 -- 172.21.15.1:0/1250334466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f48dc108f60 msgr2=0x7f48dc19ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:57.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.136+0000 7f48dbe00640 1 --2- 172.21.15.1:0/1250334466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f48dc108f60 0x7f48dc19ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:57.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.136+0000 7f48dbe00640 1 -- 172.21.15.1:0/1250334466 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48dc19f400 con 0x7f48dc103e30 2024-09-12T01:28:57.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.136+0000 7f48e1200640 1 --2- 172.21.15.1:0/1250334466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f48dc108f60 0x7f48dc19ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:28:57.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.136+0000 7f48dbe00640 1 --2- 172.21.15.1:0/1250334466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48dc103e30 0x7f48dc19a450 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f48cc004770 tx=0x7f48cc003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:57.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.136+0000 7f48d9600640 1 -- 172.21.15.1:0/1250334466 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48cc03f070 con 0x7f48dc103e30 2024-09-12T01:28:57.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.136+0000 7f48d9600640 1 -- 172.21.15.1:0/1250334466 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48cc031ca0 con 0x7f48dc103e30 2024-09-12T01:28:57.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.136+0000 7f48e1c00640 1 -- 172.21.15.1:0/1250334466 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48dc19f690 con 0x7f48dc103e30 2024-09-12T01:28:57.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.136+0000 7f48d9600640 1 -- 172.21.15.1:0/1250334466 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48cc035030 con 0x7f48dc103e30 2024-09-12T01:28:57.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.136+0000 7f48e1c00640 1 -- 172.21.15.1:0/1250334466 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48dc1a6fd0 con 0x7f48dc103e30 2024-09-12T01:28:57.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.137+0000 7f48e1c00640 1 -- 172.21.15.1:0/1250334466 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48dc105310 con 0x7f48dc103e30 2024-09-12T01:28:57.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.139+0000 7f48d9600640 1 -- 172.21.15.1:0/1250334466 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f48cc042070 con 0x7f48dc103e30 2024-09-12T01:28:57.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.140+0000 7f48d9600640 1 --2- 172.21.15.1:0/1250334466 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f48c00a2c10 0x7f48c00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:57.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.141+0000 7f48d9600640 1 -- 172.21.15.1:0/1250334466 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f48cc0efed0 con 0x7f48dc103e30 2024-09-12T01:28:57.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.141+0000 7f48d3e00640 1 --2- 172.21.15.1:0/1250334466 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f48c00a2c10 0x7f48c00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:57.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.141+0000 7f48d3e00640 1 --2- 172.21.15.1:0/1250334466 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f48c00a2c10 0x7f48c00a50d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f48dc0ffd30 tx=0x7f48c4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:57.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.143+0000 7f48d9600640 1 -- 172.21.15.1:0/1250334466 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f48cc0bc350 con 0x7f48dc103e30 2024-09-12T01:28:57.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.367+0000 7f48e1c00640 1 -- 172.21.15.1:0/1250334466 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48dc104210 con 0x7f48dc103e30 2024-09-12T01:28:57.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.368+0000 7f48d9600640 1 -- 172.21.15.1:0/1250334466 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11962 (secure 0 0 0) 0x7f48cc0dae20 con 0x7f48dc103e30 2024-09-12T01:28:57.368 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:28:57.368 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":28,"btime":"2024-09-12T01:27:46:332106+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14430,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/572192306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":572192306},{"type":"v1","addr":"172.21.15.1:6835","nonce":572192306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14442,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/1050734277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":1050734277},{"type":"v1","addr":"172.21.15.1:6837","nonce":1050734277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:26:48.555447+0000","modified":"2024-09-12T01:27:46.206001+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24545,"mds_1":24511,"mds_2":24506,"mds_3":24493,"mds_4":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14481":{"gid":14481,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/3651849993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":3651849993},{"type":"v1","addr":"172.21.15.1:6839","nonce":3651849993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14493":{"gid":14493,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2464781973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2464781973},{"type":"v1","addr":"172.21.15.1:6841","nonce":2464781973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2071825633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2071825633},{"type":"v1","addr":"172.21.15.104:6833","nonce":2071825633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/2930310711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2930310711},{"type":"v1","addr":"172.21.15.53:6833","nonce":2930310711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2032381725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2032381725},{"type":"v1","addr":"172.21.15.53:6835","nonce":2032381725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.104:6835/1547038540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1547038540},{"type":"v1","addr":"172.21.15.104:6835","nonce":1547038540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":23,"addr":"172.21.15.104:6837/464720247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":464720247},{"type":"v1","addr":"172.21.15.104:6837","nonce":464720247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":22,"addr":"172.21.15.53:6837/2848743754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2848743754},{"type":"v1","addr":"172.21.15.53:6837","nonce":2848743754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":21,"addr":"172.21.15.104:6839/1459976041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1459976041},{"type":"v1","addr":"172.21.15.104:6839","nonce":1459976041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.53:6839/4019389301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":4019389301},{"type":"v1","addr":"172.21.15.53:6839","nonce":4019389301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24487,24493,24506,24545,24511]},"id":1}]} 2024-09-12T01:28:57.368 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 28 2024-09-12T01:28:57.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.370+0000 7f48e1c00640 1 -- 172.21.15.1:0/1250334466 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f48c00a2c10 msgr2=0x7f48c00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:57.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.370+0000 7f48e1c00640 1 --2- 172.21.15.1:0/1250334466 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f48c00a2c10 0x7f48c00a50d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f48dc0ffd30 tx=0x7f48c4005f90 comp rx=0 tx=0).stop 2024-09-12T01:28:57.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.370+0000 7f48e1c00640 1 -- 172.21.15.1:0/1250334466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48dc103e30 msgr2=0x7f48dc19a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:57.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.370+0000 7f48e1c00640 1 --2- 172.21.15.1:0/1250334466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48dc103e30 0x7f48dc19a450 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f48cc004770 tx=0x7f48cc003c10 comp rx=0 tx=0).stop 2024-09-12T01:28:57.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.370+0000 7f48e1c00640 1 -- 172.21.15.1:0/1250334466 shutdown_connections 2024-09-12T01:28:57.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.370+0000 7f48e1c00640 1 --2- 172.21.15.1:0/1250334466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f48dc108f60 0x7f48dc19ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:57.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.370+0000 7f48e1c00640 1 --2- 172.21.15.1:0/1250334466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48dc1047e0 0x7f48dc19a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:57.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.370+0000 7f48e1c00640 1 --2- 172.21.15.1:0/1250334466 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f48c00a2c10 0x7f48c00a50d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:57.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.370+0000 7f48e1c00640 1 --2- 172.21.15.1:0/1250334466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48dc103e30 0x7f48dc19a450 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:57.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.370+0000 7f48e1c00640 1 -- 172.21.15.1:0/1250334466 >> 172.21.15.1:0/1250334466 conn(0x7f48dc0ff910 msgr2=0x7f48dc100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:57.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.371+0000 7f48e1c00640 1 -- 172.21.15.1:0/1250334466 shutdown_connections 2024-09-12T01:28:57.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:57.371+0000 7f48e1c00640 1 -- 172.21.15.1:0/1250334466 wait complete. 2024-09-12T01:28:57.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:57 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/1250334466' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:28:57.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:57 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/1250334466' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:28:58.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:57 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/1250334466' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:28:58.389 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T01:28:58.783 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:28:58.924 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:28:58 smithi001 ceph-mon[30387]: pgmap v379: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-12T01:28:58.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:28:58 smithi053 ceph-mon[35158]: pgmap v379: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-12T01:28:59.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:28:58 smithi104 ceph-mon[35219]: pgmap v379: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-12T01:28:59.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.516+0000 7f97b6200640 1 -- 172.21.15.1:0/3680593734 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97b0103ff0 msgr2=0x7f97b01043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:59.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.516+0000 7f97b6200640 1 --2- 172.21.15.1:0/3680593734 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97b0103ff0 0x7f97b01043d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f97a0009900 tx=0x7f97a002a410 comp rx=0 tx=0).stop 2024-09-12T01:28:59.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.516+0000 7f97b6200640 1 -- 172.21.15.1:0/3680593734 shutdown_connections 2024-09-12T01:28:59.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.516+0000 7f97b6200640 1 --2- 172.21.15.1:0/3680593734 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f97b0109120 0x7f97b0110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:59.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.516+0000 7f97b6200640 1 --2- 172.21.15.1:0/3680593734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97b01049a0 0x7f97b01089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:59.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.516+0000 7f97b6200640 1 --2- 172.21.15.1:0/3680593734 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97b0103ff0 0x7f97b01043d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:59.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.516+0000 7f97b6200640 1 -- 172.21.15.1:0/3680593734 >> 172.21.15.1:0/3680593734 conn(0x7f97b00ffa90 msgr2=0x7f97b0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:59.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.516+0000 7f97b6200640 1 -- 172.21.15.1:0/3680593734 shutdown_connections 2024-09-12T01:28:59.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.516+0000 7f97b6200640 1 -- 172.21.15.1:0/3680593734 wait complete. 2024-09-12T01:28:59.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.517+0000 7f97b6200640 1 Processor -- start 2024-09-12T01:28:59.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.517+0000 7f97b6200640 1 -- start start 2024-09-12T01:28:59.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.517+0000 7f97b6200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97b0103ff0 0x7f97b019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:59.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.517+0000 7f97b6200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f97b01049a0 0x7f97b019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:59.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.517+0000 7f97b6200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97b0109120 0x7f97b019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:59.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.517+0000 7f97b6200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f97b01141a0 con 0x7f97b0109120 2024-09-12T01:28:59.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.517+0000 7f97b6200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f97b0114020 con 0x7f97b0103ff0 2024-09-12T01:28:59.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.517+0000 7f97b6200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f97b0114320 con 0x7f97b01049a0 2024-09-12T01:28:59.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.518+0000 7f97af400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97b0103ff0 0x7f97b019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:59.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.518+0000 7f97a7e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f97b01049a0 0x7f97b019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:59.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.518+0000 7f97af400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97b0103ff0 0x7f97b019a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:34110/0 (socket says 172.21.15.1:34110) 2024-09-12T01:28:59.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.518+0000 7f97af400640 1 -- 172.21.15.1:0/484620779 learned_addr learned my addr 172.21.15.1:0/484620779 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:28:59.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.518+0000 7f97afe00640 1 --2- 172.21.15.1:0/484620779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97b0109120 0x7f97b019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:59.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.518+0000 7f97a7e00640 1 -- 172.21.15.1:0/484620779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97b0103ff0 msgr2=0x7f97b019a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:59.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.518+0000 7f97a7e00640 1 --2- 172.21.15.1:0/484620779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97b0103ff0 0x7f97b019a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:59.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.518+0000 7f97a7e00640 1 -- 172.21.15.1:0/484620779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97b0109120 msgr2=0x7f97b019eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:59.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.518+0000 7f97a7e00640 1 --2- 172.21.15.1:0/484620779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97b0109120 0x7f97b019eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:59.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.518+0000 7f97a7e00640 1 -- 172.21.15.1:0/484620779 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97b019f5d0 con 0x7f97b01049a0 2024-09-12T01:28:59.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.518+0000 7f97a7e00640 1 --2- 172.21.15.1:0/484620779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f97b01049a0 0x7f97b019ab60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f979800e9f0 tx=0x7f979800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:59.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.519+0000 7f97acc00640 1 -- 172.21.15.1:0/484620779 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979800ccc0 con 0x7f97b01049a0 2024-09-12T01:28:59.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.519+0000 7f97acc00640 1 -- 172.21.15.1:0/484620779 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9798010910 con 0x7f97b01049a0 2024-09-12T01:28:59.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.519+0000 7f97acc00640 1 -- 172.21.15.1:0/484620779 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9798018960 con 0x7f97b01049a0 2024-09-12T01:28:59.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.519+0000 7f97b6200640 1 -- 172.21.15.1:0/484620779 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97b019f8c0 con 0x7f97b01049a0 2024-09-12T01:28:59.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.519+0000 7f97b6200640 1 -- 172.21.15.1:0/484620779 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97b01a71a0 con 0x7f97b01049a0 2024-09-12T01:28:59.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.520+0000 7f97b6200640 1 -- 172.21.15.1:0/484620779 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97b01043d0 con 0x7f97b01049a0 2024-09-12T01:28:59.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.522+0000 7f97acc00640 1 -- 172.21.15.1:0/484620779 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f9798010ac0 con 0x7f97b01049a0 2024-09-12T01:28:59.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.523+0000 7f97acc00640 1 --2- 172.21.15.1:0/484620779 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f97940a2c90 0x7f97940a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:28:59.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.523+0000 7f97acc00640 1 -- 172.21.15.1:0/484620779 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f97980cfd60 con 0x7f97b01049a0 2024-09-12T01:28:59.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.524+0000 7f97af400640 1 --2- 172.21.15.1:0/484620779 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f97940a2c90 0x7f97940a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:28:59.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.524+0000 7f97af400640 1 --2- 172.21.15.1:0/484620779 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f97940a2c90 0x7f97940a5150 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f97a00096c0 tx=0x7f97a00046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:28:59.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.525+0000 7f97acc00640 1 -- 172.21.15.1:0/484620779 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f979809c120 con 0x7f97b01049a0 2024-09-12T01:28:59.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.711+0000 7f97b6200640 1 -- 172.21.15.1:0/484620779 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f97b002d7c0 con 0x7f97b01049a0 2024-09-12T01:28:59.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.713+0000 7f97acc00640 1 -- 172.21.15.1:0/484620779 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+24751 (secure 0 0 0) 0x7f97980a0420 con 0x7f97b01049a0 2024-09-12T01:28:59.713 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:28:59.713 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":98,"fsid":"c0e4f180-70a4-11ef-bcea-c7b262605968","created":"2024-09-12T01:18:19.337406+0000","modified":"2024-09-12T01:28:17.407774+0000","last_up_change":"2024-09-12T01:25:50.601477+0000","last_in_change":"2024-09-12T01:25:27.674600+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:21:41.204242+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:26:43.066475+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:26:46.112771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T01:26:56.040130+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b9a396ae-bf97-462f-9ef9-b10ef6b9d337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6803","nonce":1125335136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6805","nonce":1125335136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6809","nonce":1125335136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6807","nonce":1125335136}]},"public_addr":"172.21.15.1:6803/1125335136","cluster_addr":"172.21.15.1:6805/1125335136","heartbeat_back_addr":"172.21.15.1:6809/1125335136","heartbeat_front_addr":"172.21.15.1:6807/1125335136","state":["exists","up"]},{"osd":1,"uuid":"dfe7a8a3-3700-4789-ac64-88660de9e0d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6801","nonce":3131049165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6803","nonce":3131049165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6807","nonce":3131049165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6805","nonce":3131049165}]},"public_addr":"172.21.15.53:6801/3131049165","cluster_addr":"172.21.15.53:6803/3131049165","heartbeat_back_addr":"172.21.15.53:6807/3131049165","heartbeat_front_addr":"172.21.15.53:6805/3131049165","state":["exists","up"]},{"osd":2,"uuid":"5d9d3603-ba7c-4115-9267-12abcb20d55b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6801","nonce":1133285636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6803","nonce":1133285636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6807","nonce":1133285636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6805","nonce":1133285636}]},"public_addr":"172.21.15.104:6801/1133285636","cluster_addr":"172.21.15.104:6803/1133285636","heartbeat_back_addr":"172.21.15.104:6807/1133285636","heartbeat_front_addr":"172.21.15.104:6805/1133285636","state":["exists","up"]},{"osd":3,"uuid":"1e063c66-41b4-4db1-a964-28e122ce628e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6811","nonce":1247468803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6813","nonce":1247468803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6817","nonce":1247468803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6815","nonce":1247468803}]},"public_addr":"172.21.15.1:6811/1247468803","cluster_addr":"172.21.15.1:6813/1247468803","heartbeat_back_addr":"172.21.15.1:6817/1247468803","heartbeat_front_addr":"172.21.15.1:6815/1247468803","state":["exists","up"]},{"osd":4,"uuid":"fe89ece0-2171-45d5-a06d-e400f77233e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6809","nonce":2673413639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6811","nonce":2673413639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6815","nonce":2673413639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6813","nonce":2673413639}]},"public_addr":"172.21.15.53:6809/2673413639","cluster_addr":"172.21.15.53:6811/2673413639","heartbeat_back_addr":"172.21.15.53:6815/2673413639","heartbeat_front_addr":"172.21.15.53:6813/2673413639","state":["exists","up"]},{"osd":5,"uuid":"daab795c-9942-4d16-aa34-8bba30a4b99b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6809","nonce":3131498790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6811","nonce":3131498790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6815","nonce":3131498790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6813","nonce":3131498790}]},"public_addr":"172.21.15.104:6809/3131498790","cluster_addr":"172.21.15.104:6811/3131498790","heartbeat_back_addr":"172.21.15.104:6815/3131498790","heartbeat_front_addr":"172.21.15.104:6813/3131498790","state":["exists","up"]},{"osd":6,"uuid":"f31d8633-e435-499b-9d46-0aa11a2481cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6819","nonce":3413067918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6821","nonce":3413067918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6825","nonce":3413067918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6823","nonce":3413067918}]},"public_addr":"172.21.15.1:6819/3413067918","cluster_addr":"172.21.15.1:6821/3413067918","heartbeat_back_addr":"172.21.15.1:6825/3413067918","heartbeat_front_addr":"172.21.15.1:6823/3413067918","state":["exists","up"]},{"osd":7,"uuid":"e64da49e-d31e-4801-b6cc-634a233b4761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6817","nonce":2841484136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6819","nonce":2841484136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6823","nonce":2841484136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6821","nonce":2841484136}]},"public_addr":"172.21.15.53:6817/2841484136","cluster_addr":"172.21.15.53:6819/2841484136","heartbeat_back_addr":"172.21.15.53:6823/2841484136","heartbeat_front_addr":"172.21.15.53:6821/2841484136","state":["exists","up"]},{"osd":8,"uuid":"fdfb5a42-dca7-4874-b479-176b4a191e21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6817","nonce":941453547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6819","nonce":941453547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6823","nonce":941453547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6821","nonce":941453547}]},"public_addr":"172.21.15.104:6817/941453547","cluster_addr":"172.21.15.104:6819/941453547","heartbeat_back_addr":"172.21.15.104:6823/941453547","heartbeat_front_addr":"172.21.15.104:6821/941453547","state":["exists","up"]},{"osd":9,"uuid":"7f81537a-cae5-4bb8-aec2-4eb07deb7958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6827","nonce":958287586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6829","nonce":958287586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6833","nonce":958287586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6831","nonce":958287586}]},"public_addr":"172.21.15.1:6827/958287586","cluster_addr":"172.21.15.1:6829/958287586","heartbeat_back_addr":"172.21.15.1:6833/958287586","heartbeat_front_addr":"172.21.15.1:6831/958287586","state":["exists","up"]},{"osd":10,"uuid":"e3c9c37d-46b9-4585-990d-69c447453bd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6825","nonce":3357909498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6827","nonce":3357909498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6831","nonce":3357909498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6829","nonce":3357909498}]},"public_addr":"172.21.15.53:6825/3357909498","cluster_addr":"172.21.15.53:6827/3357909498","heartbeat_back_addr":"172.21.15.53:6831/3357909498","heartbeat_front_addr":"172.21.15.53:6829/3357909498","state":["exists","up"]},{"osd":11,"uuid":"474b5345-f2ae-4e0e-a197-53ab11f58c04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6825","nonce":3381427987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6827","nonce":3381427987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6831","nonce":3381427987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6829","nonce":3381427987}]},"public_addr":"172.21.15.104:6825/3381427987","cluster_addr":"172.21.15.104:6827/3381427987","heartbeat_back_addr":"172.21.15.104:6831/3381427987","heartbeat_front_addr":"172.21.15.104:6829/3381427987","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:20:46.903181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:21:12.735556+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:21:37.868739+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:22:09.141228+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:22:36.438563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:23:02.220948+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:23:33.505881+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:24:00.982176+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:24:26.010444+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:24:56.711121+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:25:23.846495+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:25:48.815427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1373984726":"2024-09-13T01:19:14.580637+0000","172.21.15.1:0/1429963718":"2024-09-13T01:19:14.580637+0000","172.21.15.1:6801/4101281615":"2024-09-13T01:19:14.580637+0000","172.21.15.1:6800/4101281615":"2024-09-13T01:19:14.580637+0000","172.21.15.1:0/2564759586":"2024-09-13T01:19:14.580637+0000","172.21.15.1:6801/2935968453":"2024-09-13T01:18:45.882682+0000","172.21.15.1:6800/2935968453":"2024-09-13T01:18:45.882682+0000","172.21.15.1:0/1459848813":"2024-09-13T01:18:45.882682+0000","172.21.15.1:0/578605922":"2024-09-13T01:18:45.882682+0000","172.21.15.1:0/738035035":"2024-09-13T01:18:45.882682+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:28:59.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.715+0000 7f97b6200640 1 -- 172.21.15.1:0/484620779 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f97940a2c90 msgr2=0x7f97940a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:59.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.715+0000 7f97b6200640 1 --2- 172.21.15.1:0/484620779 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f97940a2c90 0x7f97940a5150 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f97a00096c0 tx=0x7f97a00046c0 comp rx=0 tx=0).stop 2024-09-12T01:28:59.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.715+0000 7f97b6200640 1 -- 172.21.15.1:0/484620779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f97b01049a0 msgr2=0x7f97b019ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:28:59.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.715+0000 7f97b6200640 1 --2- 172.21.15.1:0/484620779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f97b01049a0 0x7f97b019ab60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f979800e9f0 tx=0x7f979800eeb0 comp rx=0 tx=0).stop 2024-09-12T01:28:59.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.716+0000 7f97b6200640 1 -- 172.21.15.1:0/484620779 shutdown_connections 2024-09-12T01:28:59.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.716+0000 7f97b6200640 1 --2- 172.21.15.1:0/484620779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97b0109120 0x7f97b019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:59.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.716+0000 7f97b6200640 1 --2- 172.21.15.1:0/484620779 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f97940a2c90 0x7f97940a5150 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:59.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.716+0000 7f97b6200640 1 --2- 172.21.15.1:0/484620779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f97b01049a0 0x7f97b019ab60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:59.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.716+0000 7f97b6200640 1 --2- 172.21.15.1:0/484620779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97b0103ff0 0x7f97b019a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:28:59.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.716+0000 7f97b6200640 1 -- 172.21.15.1:0/484620779 >> 172.21.15.1:0/484620779 conn(0x7f97b00ffa90 msgr2=0x7f97b0100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:28:59.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.716+0000 7f97b6200640 1 -- 172.21.15.1:0/484620779 shutdown_connections 2024-09-12T01:28:59.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:28:59.716+0000 7f97b6200640 1 -- 172.21.15.1:0/484620779 wait complete. 2024-09-12T01:29:00.531 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:29:00.913 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:29:00.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:00 smithi001 ceph-mon[30387]: pgmap v380: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T01:29:00.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:00 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/484620779' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:29:00.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:00 smithi053 ceph-mon[35158]: pgmap v380: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T01:29:00.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:00 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/484620779' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:29:01.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:00 smithi104 ceph-mon[35219]: pgmap v380: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T01:29:01.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:00 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/484620779' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:29:01.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.494+0000 7ff751e00640 1 -- 172.21.15.1:0/2492995759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff74c103ff0 msgr2=0x7ff74c1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:01.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.494+0000 7ff751e00640 1 --2- 172.21.15.1:0/2492995759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff74c103ff0 0x7ff74c1043d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff73c009900 tx=0x7ff73c02a410 comp rx=0 tx=0).stop 2024-09-12T01:29:01.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.494+0000 7ff751e00640 1 -- 172.21.15.1:0/2492995759 shutdown_connections 2024-09-12T01:29:01.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.494+0000 7ff751e00640 1 --2- 172.21.15.1:0/2492995759 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff74c109120 0x7ff74c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:01.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.494+0000 7ff751e00640 1 --2- 172.21.15.1:0/2492995759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff74c1049a0 0x7ff74c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:01.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.494+0000 7ff751e00640 1 --2- 172.21.15.1:0/2492995759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff74c103ff0 0x7ff74c1043d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:01.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.494+0000 7ff751e00640 1 -- 172.21.15.1:0/2492995759 >> 172.21.15.1:0/2492995759 conn(0x7ff74c0ffa90 msgr2=0x7ff74c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:01.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.494+0000 7ff751e00640 1 -- 172.21.15.1:0/2492995759 shutdown_connections 2024-09-12T01:29:01.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.494+0000 7ff751e00640 1 -- 172.21.15.1:0/2492995759 wait complete. 2024-09-12T01:29:01.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.495+0000 7ff751e00640 1 Processor -- start 2024-09-12T01:29:01.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.496+0000 7ff751e00640 1 -- start start 2024-09-12T01:29:01.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.496+0000 7ff751e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff74c103ff0 0x7ff74c19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:01.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.496+0000 7ff751e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff74c1049a0 0x7ff74c19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:01.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.496+0000 7ff751e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff74c109120 0x7ff74c19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:01.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.496+0000 7ff751e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff74c114150 con 0x7ff74c1049a0 2024-09-12T01:29:01.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.496+0000 7ff751e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff74c113fd0 con 0x7ff74c109120 2024-09-12T01:29:01.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.496+0000 7ff751e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff74c1142d0 con 0x7ff74c103ff0 2024-09-12T01:29:01.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.496+0000 7ff74a000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff74c1049a0 0x7ff74c19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:01.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.496+0000 7ff74a000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff74c1049a0 0x7ff74c19ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58228/0 (socket says 172.21.15.1:58228) 2024-09-12T01:29:01.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.496+0000 7ff74a000640 1 -- 172.21.15.1:0/855531434 learned_addr learned my addr 172.21.15.1:0/855531434 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:01.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.497+0000 7ff74aa00640 1 --2- 172.21.15.1:0/855531434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff74c103ff0 0x7ff74c19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:01.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.497+0000 7ff74b400640 1 --2- 172.21.15.1:0/855531434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff74c109120 0x7ff74c19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:01.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.497+0000 7ff74a000640 1 -- 172.21.15.1:0/855531434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff74c103ff0 msgr2=0x7ff74c19a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:01.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.497+0000 7ff74a000640 1 --2- 172.21.15.1:0/855531434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff74c103ff0 0x7ff74c19a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:01.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.497+0000 7ff74a000640 1 -- 172.21.15.1:0/855531434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff74c109120 msgr2=0x7ff74c19eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:01.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.497+0000 7ff74a000640 1 --2- 172.21.15.1:0/855531434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff74c109120 0x7ff74c19eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:01.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.497+0000 7ff74a000640 1 -- 172.21.15.1:0/855531434 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff74c19f5a0 con 0x7ff74c1049a0 2024-09-12T01:29:01.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.497+0000 7ff74b400640 1 --2- 172.21.15.1:0/855531434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff74c109120 0x7ff74c19eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:29:01.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.497+0000 7ff74a000640 1 --2- 172.21.15.1:0/855531434 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff74c1049a0 0x7ff74c19ab30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff73800e9f0 tx=0x7ff73800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:01.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.498+0000 7ff750a00640 1 -- 172.21.15.1:0/855531434 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73800cd80 con 0x7ff74c1049a0 2024-09-12T01:29:01.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.498+0000 7ff751e00640 1 -- 172.21.15.1:0/855531434 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff74c19f890 con 0x7ff74c1049a0 2024-09-12T01:29:01.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.498+0000 7ff750a00640 1 -- 172.21.15.1:0/855531434 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff738010910 con 0x7ff74c1049a0 2024-09-12T01:29:01.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.498+0000 7ff750a00640 1 -- 172.21.15.1:0/855531434 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7380188f0 con 0x7ff74c1049a0 2024-09-12T01:29:01.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.498+0000 7ff751e00640 1 -- 172.21.15.1:0/855531434 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff74c1a7170 con 0x7ff74c1049a0 2024-09-12T01:29:01.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.499+0000 7ff751e00640 1 -- 172.21.15.1:0/855531434 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff74c1043d0 con 0x7ff74c1049a0 2024-09-12T01:29:01.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.501+0000 7ff750a00640 1 -- 172.21.15.1:0/855531434 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7ff738010ac0 con 0x7ff74c1049a0 2024-09-12T01:29:01.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.503+0000 7ff750a00640 1 --2- 172.21.15.1:0/855531434 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff7300a2c90 0x7ff7300a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:01.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.503+0000 7ff750a00640 1 -- 172.21.15.1:0/855531434 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7ff7380cf8f0 con 0x7ff74c1049a0 2024-09-12T01:29:01.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.503+0000 7ff74aa00640 1 --2- 172.21.15.1:0/855531434 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff7300a2c90 0x7ff7300a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:01.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.504+0000 7ff74aa00640 1 --2- 172.21.15.1:0/855531434 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff7300a2c90 0x7ff7300a5150 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff73c009230 tx=0x7ff73c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:01.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.507+0000 7ff750a00640 1 -- 172.21.15.1:0/855531434 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff73809bcb0 con 0x7ff74c1049a0 2024-09-12T01:29:01.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.731+0000 7ff751e00640 1 -- 172.21.15.1:0/855531434 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff74c19b950 con 0x7ff74c1049a0 2024-09-12T01:29:01.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.732+0000 7ff750a00640 1 -- 172.21.15.1:0/855531434 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11962 (secure 0 0 0) 0x7ff73809ffb0 con 0x7ff74c1049a0 2024-09-12T01:29:01.732 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:29:01.732 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":28,"btime":"2024-09-12T01:27:46:332106+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14430,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/572192306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":572192306},{"type":"v1","addr":"172.21.15.1:6835","nonce":572192306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14442,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/1050734277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":1050734277},{"type":"v1","addr":"172.21.15.1:6837","nonce":1050734277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:26:48.555447+0000","modified":"2024-09-12T01:27:46.206001+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24545,"mds_1":24511,"mds_2":24506,"mds_3":24493,"mds_4":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14481":{"gid":14481,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/3651849993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":3651849993},{"type":"v1","addr":"172.21.15.1:6839","nonce":3651849993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14493":{"gid":14493,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2464781973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2464781973},{"type":"v1","addr":"172.21.15.1:6841","nonce":2464781973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2071825633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2071825633},{"type":"v1","addr":"172.21.15.104:6833","nonce":2071825633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/2930310711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2930310711},{"type":"v1","addr":"172.21.15.53:6833","nonce":2930310711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2032381725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2032381725},{"type":"v1","addr":"172.21.15.53:6835","nonce":2032381725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.104:6835/1547038540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1547038540},{"type":"v1","addr":"172.21.15.104:6835","nonce":1547038540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":23,"addr":"172.21.15.104:6837/464720247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":464720247},{"type":"v1","addr":"172.21.15.104:6837","nonce":464720247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":22,"addr":"172.21.15.53:6837/2848743754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2848743754},{"type":"v1","addr":"172.21.15.53:6837","nonce":2848743754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":21,"addr":"172.21.15.104:6839/1459976041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1459976041},{"type":"v1","addr":"172.21.15.104:6839","nonce":1459976041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.53:6839/4019389301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":4019389301},{"type":"v1","addr":"172.21.15.53:6839","nonce":4019389301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24487,24493,24506,24545,24511]},"id":1}]} 2024-09-12T01:29:01.732 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 28 2024-09-12T01:29:01.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.734+0000 7ff751e00640 1 -- 172.21.15.1:0/855531434 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff7300a2c90 msgr2=0x7ff7300a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:01.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.734+0000 7ff751e00640 1 --2- 172.21.15.1:0/855531434 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff7300a2c90 0x7ff7300a5150 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff73c009230 tx=0x7ff73c0058d0 comp rx=0 tx=0).stop 2024-09-12T01:29:01.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.734+0000 7ff751e00640 1 -- 172.21.15.1:0/855531434 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff74c1049a0 msgr2=0x7ff74c19ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:01.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.734+0000 7ff751e00640 1 --2- 172.21.15.1:0/855531434 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff74c1049a0 0x7ff74c19ab30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff73800e9f0 tx=0x7ff73800eeb0 comp rx=0 tx=0).stop 2024-09-12T01:29:01.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.735+0000 7ff751e00640 1 -- 172.21.15.1:0/855531434 shutdown_connections 2024-09-12T01:29:01.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.735+0000 7ff751e00640 1 --2- 172.21.15.1:0/855531434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff74c109120 0x7ff74c19eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:01.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.735+0000 7ff751e00640 1 --2- 172.21.15.1:0/855531434 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff74c1049a0 0x7ff74c19ab30 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:01.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.735+0000 7ff751e00640 1 --2- 172.21.15.1:0/855531434 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff7300a2c90 0x7ff7300a5150 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:01.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.735+0000 7ff751e00640 1 --2- 172.21.15.1:0/855531434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff74c103ff0 0x7ff74c19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:01.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.735+0000 7ff751e00640 1 -- 172.21.15.1:0/855531434 >> 172.21.15.1:0/855531434 conn(0x7ff74c0ffa90 msgr2=0x7ff74c100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:01.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.735+0000 7ff751e00640 1 -- 172.21.15.1:0/855531434 shutdown_connections 2024-09-12T01:29:01.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:01.735+0000 7ff751e00640 1 -- 172.21.15.1:0/855531434 wait complete. 2024-09-12T01:29:02.565 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-12T01:26:48.555447+0000', 'modified': '2024-09-12T01:27:46.206001+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24545, 'mds_1': 24511, 'mds_2': 24506, 'mds_3': 24493, 'mds_4': 24487}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14481': {'gid': 14481, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.1:6839/3651849993', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6838', 'nonce': 3651849993}, {'type': 'v1', 'addr': '172.21.15.1:6839', 'nonce': 3651849993}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14493': {'gid': 14493, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.1:6841/2464781973', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6840', 'nonce': 2464781973}, {'type': 'v1', 'addr': '172.21.15.1:6841', 'nonce': 2464781973}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24439': {'gid': 24439, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.104:6833/2071825633', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6832', 'nonce': 2071825633}, {'type': 'v1', 'addr': '172.21.15.104:6833', 'nonce': 2071825633}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24446': {'gid': 24446, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.53:6833/2930310711', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6832', 'nonce': 2930310711}, {'type': 'v1', 'addr': '172.21.15.53:6833', 'nonce': 2930310711}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.53:6835/2032381725', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6834', 'nonce': 2032381725}, {'type': 'v1', 'addr': '172.21.15.53:6835', 'nonce': 2032381725}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 23, 'addr': '172.21.15.104:6835/1547038540', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6834', 'nonce': 1547038540}, {'type': 'v1', 'addr': '172.21.15.104:6835', 'nonce': 1547038540}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 23, 'addr': '172.21.15.104:6837/464720247', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6836', 'nonce': 464720247}, {'type': 'v1', 'addr': '172.21.15.104:6837', 'nonce': 464720247}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24506': {'gid': 24506, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.53:6837/2848743754', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6836', 'nonce': 2848743754}, {'type': 'v1', 'addr': '172.21.15.53:6837', 'nonce': 2848743754}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24511': {'gid': 24511, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.104:6839/1459976041', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6838', 'nonce': 1459976041}, {'type': 'v1', 'addr': '172.21.15.104:6839', 'nonce': 1459976041}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24545': {'gid': 24545, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.53:6839/4019389301', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6838', 'nonce': 4019389301}, {'type': 'v1', 'addr': '172.21.15.53:6839', 'nonce': 4019389301}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24545, 'qdb_cluster': [24487, 24493, 24506, 24545, 24511]} 2024-09-12T01:29:02.565 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-12T01:29:02.565 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph tell mds.f status 2024-09-12T01:29:02.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:02 smithi001 ceph-mon[30387]: pgmap v381: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T01:29:02.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:02 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/855531434' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:29:02.943 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:29:02.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:02 smithi053 ceph-mon[35158]: pgmap v381: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T01:29:02.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:02 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/855531434' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:29:03.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:02 smithi104 ceph-mon[35219]: pgmap v381: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T01:29:03.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:02 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/855531434' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:29:03.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.541+0000 7fc0eda00640 1 -- 172.21.15.1:0/546952644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0e8076720 msgr2=0x7fc0e8076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:03.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.541+0000 7fc0eda00640 1 --2- 172.21.15.1:0/546952644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0e8076720 0x7fc0e8076b00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8009900 tx=0x7fc0d802a3f0 comp rx=0 tx=0).stop 2024-09-12T01:29:03.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.541+0000 7fc0eda00640 1 -- 172.21.15.1:0/546952644 shutdown_connections 2024-09-12T01:29:03.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.541+0000 7fc0eda00640 1 --2- 172.21.15.1:0/546952644 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc0e8112a30 0x7fc0e8114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:03.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.541+0000 7fc0eda00640 1 --2- 172.21.15.1:0/546952644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0e8077040 0x7fc0e81124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:03.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.541+0000 7fc0eda00640 1 --2- 172.21.15.1:0/546952644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0e8076720 0x7fc0e8076b00 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:03.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.541+0000 7fc0eda00640 1 -- 172.21.15.1:0/546952644 >> 172.21.15.1:0/546952644 conn(0x7fc0e80ffa90 msgr2=0x7fc0e8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:03.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.541+0000 7fc0eda00640 1 -- 172.21.15.1:0/546952644 shutdown_connections 2024-09-12T01:29:03.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.541+0000 7fc0eda00640 1 -- 172.21.15.1:0/546952644 wait complete. 2024-09-12T01:29:03.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.542+0000 7fc0eda00640 1 Processor -- start 2024-09-12T01:29:03.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.542+0000 7fc0eda00640 1 -- start start 2024-09-12T01:29:03.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.543+0000 7fc0eda00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc0e8076720 0x7fc0e81a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:03.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.543+0000 7fc0eda00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0e8077040 0x7fc0e81a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:03.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.543+0000 7fc0eda00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0e8112a30 0x7fc0e819e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:03.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.543+0000 7fc0eda00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e81185f0 con 0x7fc0e8112a30 2024-09-12T01:29:03.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.543+0000 7fc0eda00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e8118470 con 0x7fc0e8077040 2024-09-12T01:29:03.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.543+0000 7fc0eda00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e8118770 con 0x7fc0e8076720 2024-09-12T01:29:03.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.543+0000 7fc0e7400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0e8112a30 0x7fc0e819e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:03.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.543+0000 7fc0e7400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0e8112a30 0x7fc0e819e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58240/0 (socket says 172.21.15.1:58240) 2024-09-12T01:29:03.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.543+0000 7fc0de000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0e8077040 0x7fc0e81a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:03.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.543+0000 7fc0e6a00640 1 --2- 172.21.15.1:0/4283365322 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc0e8076720 0x7fc0e81a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:03.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.543+0000 7fc0e7400640 1 -- 172.21.15.1:0/4283365322 learned_addr learned my addr 172.21.15.1:0/4283365322 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:03.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.544+0000 7fc0de000640 1 -- 172.21.15.1:0/4283365322 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc0e8076720 msgr2=0x7fc0e81a4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:03.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.544+0000 7fc0de000640 1 --2- 172.21.15.1:0/4283365322 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc0e8076720 0x7fc0e81a4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:03.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.544+0000 7fc0de000640 1 -- 172.21.15.1:0/4283365322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0e8112a30 msgr2=0x7fc0e819e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:03.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.544+0000 7fc0de000640 1 --2- 172.21.15.1:0/4283365322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0e8112a30 0x7fc0e819e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:03.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.544+0000 7fc0de000640 1 -- 172.21.15.1:0/4283365322 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0e819eff0 con 0x7fc0e8077040 2024-09-12T01:29:03.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.544+0000 7fc0de000640 1 --2- 172.21.15.1:0/4283365322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0e8077040 0x7fc0e81a4c40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc0d000e9f0 tx=0x7fc0d000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:03.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.544+0000 7fc0dfe00640 1 -- 172.21.15.1:0/4283365322 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d000cd80 con 0x7fc0e8077040 2024-09-12T01:29:03.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.544+0000 7fc0eda00640 1 -- 172.21.15.1:0/4283365322 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0e819f2e0 con 0x7fc0e8077040 2024-09-12T01:29:03.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.544+0000 7fc0dfe00640 1 -- 172.21.15.1:0/4283365322 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0d00035f0 con 0x7fc0e8077040 2024-09-12T01:29:03.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.544+0000 7fc0dfe00640 1 -- 172.21.15.1:0/4283365322 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d0010670 con 0x7fc0e8077040 2024-09-12T01:29:03.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.544+0000 7fc0eda00640 1 -- 172.21.15.1:0/4283365322 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0e81ab5d0 con 0x7fc0e8077040 2024-09-12T01:29:03.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.548+0000 7fc0f05bc740 1 Processor -- start 2024-09-12T01:29:03.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.549+0000 7fc0f05bc740 1 -- start start 2024-09-12T01:29:03.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.549+0000 7fc0f05bc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d8214b330 0x556d8214b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:03.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.549+0000 7fc0f05bc740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d8214bc50 0x556d82155fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:03.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.549+0000 7fc0f05bc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d82156510 0x556d82158900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:03.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.549+0000 7fc0f05bc740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x556d820b10b0 con 0x556d8214bc50 2024-09-12T01:29:03.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.549+0000 7fc0f05bc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x556d82058ab0 con 0x556d82156510 2024-09-12T01:29:03.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.549+0000 7fc0f05bc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x556d820ad070 con 0x556d8214b330 2024-09-12T01:29:03.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.549+0000 7fc0e6a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d8214bc50 0x556d82155fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:03.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.549+0000 7fc0e7400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d8214b330 0x556d8214b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:03.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.549+0000 7fc0dfe00640 1 -- 172.21.15.1:0/4283365322 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fc0d00037a0 con 0x7fc0e8077040 2024-09-12T01:29:03.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.549+0000 7fc0e6a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d8214bc50 0x556d82155fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58254/0 (socket says 172.21.15.1:58254) 2024-09-12T01:29:03.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.549+0000 7fc0e6a00640 1 -- 172.21.15.1:0/1326934655 learned_addr learned my addr 172.21.15.1:0/1326934655 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:03.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.550+0000 7fc0e7400640 1 -- 172.21.15.1:0/1326934655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d82156510 msgr2=0x556d82158900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:03.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.550+0000 7fc0e7400640 1 --2- 172.21.15.1:0/1326934655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d82156510 0x556d82158900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:03.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.550+0000 7fc0e7400640 1 -- 172.21.15.1:0/1326934655 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d8214bc50 msgr2=0x556d82155fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:03.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.550+0000 7fc0e7400640 1 --2- 172.21.15.1:0/1326934655 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d8214bc50 0x556d82155fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:03.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.550+0000 7fc0e7400640 1 -- 172.21.15.1:0/1326934655 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d820654a0 con 0x556d8214b330 2024-09-12T01:29:03.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.550+0000 7fc0e6a00640 1 --2- 172.21.15.1:0/1326934655 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d8214bc50 0x556d82155fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:29:03.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.550+0000 7fc0e7400640 1 --2- 172.21.15.1:0/1326934655 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d8214b330 0x556d8214b710 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0009f60 tx=0x7fc0e0009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83867858d4766589 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:03.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.551+0000 7fc0d7e00640 1 -- 172.21.15.1:0/1326934655 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e0004b20 con 0x556d8214b330 2024-09-12T01:29:03.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.551+0000 7fc0d7e00640 1 -- 172.21.15.1:0/1326934655 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0e0003e50 con 0x556d8214b330 2024-09-12T01:29:03.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.551+0000 7fc0d7e00640 1 -- 172.21.15.1:0/1326934655 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e000f540 con 0x556d8214b330 2024-09-12T01:29:03.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.551+0000 7fc0dfe00640 1 --2- 172.21.15.1:0/4283365322 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fc0cc0a2c10 0x7fc0cc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:03.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.551+0000 7fc0f05bc740 1 -- 172.21.15.1:0/1326934655 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d8214b330 msgr2=0x556d8214b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:03.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.551+0000 7fc0f05bc740 1 --2- 172.21.15.1:0/1326934655 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d8214b330 0x556d8214b710 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0009f60 tx=0x7fc0e0009f90 comp rx=0 tx=0).stop 2024-09-12T01:29:03.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.551+0000 7fc0dfe00640 1 -- 172.21.15.1:0/4283365322 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fc0d00cf8a0 con 0x7fc0e8077040 2024-09-12T01:29:03.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.552+0000 7fc0f05bc740 1 -- 172.21.15.1:0/1326934655 shutdown_connections 2024-09-12T01:29:03.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.552+0000 7fc0f05bc740 1 --2- 172.21.15.1:0/1326934655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d82156510 0x556d82158900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:03.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.552+0000 7fc0f05bc740 1 --2- 172.21.15.1:0/1326934655 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d8214bc50 0x556d82155fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:03.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.552+0000 7fc0f05bc740 1 --2- 172.21.15.1:0/1326934655 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d8214b330 0x556d8214b710 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:03.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.552+0000 7fc0f05bc740 1 -- 172.21.15.1:0/1326934655 >> 172.21.15.1:0/1326934655 conn(0x556d821478c0 msgr2=0x556d82147ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:03.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.552+0000 7fc0e6a00640 1 --2- 172.21.15.1:0/4283365322 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fc0cc0a2c10 0x7fc0cc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:03.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.552+0000 7fc0f05bc740 1 -- 172.21.15.1:0/1326934655 shutdown_connections 2024-09-12T01:29:03.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.552+0000 7fc0f05bc740 1 -- 172.21.15.1:0/1326934655 wait complete. 2024-09-12T01:29:03.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.552+0000 7fc0f05bc740 1 -- start start 2024-09-12T01:29:03.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.552+0000 7fc0e6a00640 1 --2- 172.21.15.1:0/4283365322 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fc0cc0a2c10 0x7fc0cc0a50d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8031550 tx=0x7fc0d8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:03.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.553+0000 7fc0f05bc740 1 Processor -- start 2024-09-12T01:29:03.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.553+0000 7fc0f05bc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d8214b330 0x556d821535d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:03.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.553+0000 7fc0f05bc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d8214bc50 0x556d82153b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:03.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.553+0000 7fc0f05bc740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d82156510 0x556d82154050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:03.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.553+0000 7fc0f05bc740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x556d82159060 con 0x556d82156510 2024-09-12T01:29:03.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.553+0000 7fc0f05bc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x556d82039920 con 0x556d8214b330 2024-09-12T01:29:03.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.553+0000 7fc0f05bc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x556d821591e0 con 0x556d8214bc50 2024-09-12T01:29:03.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.553+0000 7fc0f05bc740 10 client.0 _ll_register_callbacks cb 0x556d8205d8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:29:03.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.553+0000 7fc0e7400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d82156510 0x556d82154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:03.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.554+0000 7fc0de000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d8214bc50 0x556d82153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:03.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.554+0000 7fc0e7400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d82156510 0x556d82154050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58262/0 (socket says 172.21.15.1:58262) 2024-09-12T01:29:03.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.554+0000 7fc0de000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d8214bc50 0x556d82153b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:59624/0 (socket says 172.21.15.1:59624) 2024-09-12T01:29:03.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.554+0000 7fc0e7400640 1 -- 172.21.15.1:0/4106157394 learned_addr learned my addr 172.21.15.1:0/4106157394 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:03.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.554+0000 7fc0e6a00640 1 --2- 172.21.15.1:0/4106157394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d8214b330 0x556d821535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:03.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.554+0000 7fc0e7400640 1 -- 172.21.15.1:0/4106157394 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d8214bc50 msgr2=0x556d82153b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:03.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.554+0000 7fc0e7400640 1 --2- 172.21.15.1:0/4106157394 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d8214bc50 0x556d82153b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:03.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.554+0000 7fc0e7400640 1 -- 172.21.15.1:0/4106157394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d8214b330 msgr2=0x556d821535d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:03.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.554+0000 7fc0e7400640 1 --2- 172.21.15.1:0/4106157394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d8214b330 0x556d821535d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:03.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.554+0000 7fc0e7400640 1 -- 172.21.15.1:0/4106157394 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d81e222b0 con 0x556d82156510 2024-09-12T01:29:03.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.554+0000 7fc0e7400640 1 --2- 172.21.15.1:0/4106157394 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d82156510 0x556d82154050 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0005f40 tx=0x7fc0e0003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=653c58fb24fbd9a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:03.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.554+0000 7fc0d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-12T01:29:03.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.555+0000 7fc0d4c00640 1 -- 172.21.15.1:0/4106157394 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e001d050 con 0x556d82156510 2024-09-12T01:29:03.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.555+0000 7fc0d4c00640 1 -- 172.21.15.1:0/4106157394 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fc0e0016040 con 0x556d82156510 2024-09-12T01:29:03.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.555+0000 7fc0f05bc740 1 -- 172.21.15.1:0/4106157394 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0e0003e50 con 0x556d82156510 2024-09-12T01:29:03.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.555+0000 7fc0d4c00640 1 -- 172.21.15.1:0/4106157394 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc0e0031050 con 0x556d82156510 2024-09-12T01:29:03.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.555+0000 7fc0f05bc740 10 client.14946 fetch_fsmap learned FSMap version 28 2024-09-12T01:29:03.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.555+0000 7fc0f05bc740 1 -- 172.21.15.1:0/4106157394 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556d82164ea0 con 0x556d82156510 2024-09-12T01:29:03.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.555+0000 7fc0d4c00640 1 -- 172.21.15.1:0/4106157394 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0e0003e50 con 0x556d82156510 2024-09-12T01:29:03.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.555+0000 7fc0d4c00640 1 -- 172.21.15.1:0/4106157394 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fc0e0005040 con 0x556d82156510 2024-09-12T01:29:03.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.556+0000 7fc0f05bc740 10 client.14946 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:29:03.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.556+0000 7fc0f05bc740 10 client.14946 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-12T01:29:03.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.556+0000 7fc0f05bc740 20 client.14946 populate_metadata read hostname 'smithi001' 2024-09-12T01:29:03.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.556+0000 7fc0f05bc740 1 --2- 172.21.15.1:0/4106157394 >> [v2:172.21.15.104:6834/1547038540,v1:172.21.15.104:6835/1547038540] conn(0x556d82167220 0x556d82168690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:03.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.556+0000 7fc0f05bc740 4 client.14946 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:29:03.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.556+0000 7fc0f05bc740 1 -- 172.21.15.1:0/4106157394 --> [v2:172.21.15.104:6834/1547038540,v1:172.21.15.104:6835/1547038540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d820be620 con 0x556d82167220 2024-09-12T01:29:03.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.556+0000 7fc0e6a00640 1 --2- 172.21.15.1:0/4106157394 >> [v2:172.21.15.104:6834/1547038540,v1:172.21.15.104:6835/1547038540] conn(0x556d82167220 0x556d82168690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:03.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.557+0000 7fc0e6a00640 1 --2- 172.21.15.1:0/4106157394 >> [v2:172.21.15.104:6834/1547038540,v1:172.21.15.104:6835/1547038540] conn(0x556d82167220 0x556d82168690 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3c8de7701e74396f server_cookie=92520e624917f84 in_seq=0 out_seq=0 2024-09-12T01:29:03.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.557+0000 7fc0d4c00640 10 client.14946 ms_handle_connect on v2:172.21.15.104:6834/1547038540 2024-09-12T01:29:03.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.558+0000 7fc0d4c00640 1 -- 172.21.15.1:0/4106157394 <== mds.4 v2:172.21.15.104:6834/1547038540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556d820be620 con 0x556d82167220 2024-09-12T01:29:03.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.558+0000 7fc0d4c00640 10 client.14946 handle_command_reply: tid=0 2024-09-12T01:29:03.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.558+0000 7fc0f05bc740 1 client.14946 shutdown 2024-09-12T01:29:03.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.558+0000 7fc0f05bc740 1 -- 172.21.15.1:0/4106157394 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d82156510 msgr2=0x556d82154050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:03.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.558+0000 7fc0f05bc740 1 --2- 172.21.15.1:0/4106157394 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d82156510 0x556d82154050 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0005f40 tx=0x7fc0e0003c20 comp rx=0 tx=0).stop 2024-09-12T01:29:03.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.559+0000 7fc0f05bc740 1 -- 172.21.15.1:0/4106157394 shutdown_connections 2024-09-12T01:29:03.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.559+0000 7fc0f05bc740 1 --2- 172.21.15.1:0/4106157394 >> [v2:172.21.15.104:6834/1547038540,v1:172.21.15.104:6835/1547038540] conn(0x556d82167220 0x556d82168690 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:03.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.559+0000 7fc0f05bc740 1 --2- 172.21.15.1:0/4106157394 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d82156510 0x556d82154050 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:03.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.559+0000 7fc0f05bc740 1 --2- 172.21.15.1:0/4106157394 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d8214bc50 0x556d82153b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:03.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.559+0000 7fc0f05bc740 1 --2- 172.21.15.1:0/4106157394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d8214b330 0x556d821535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:03.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.559+0000 7fc0f05bc740 1 -- 172.21.15.1:0/4106157394 >> 172.21.15.1:0/4106157394 conn(0x556d821478c0 msgr2=0x556d8214d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:03.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.559+0000 7fc0d4c00640 0 client.14946 ms_handle_reset on v2:172.21.15.104:6834/1547038540 2024-09-12T01:29:03.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.559+0000 7fc0f05bc740 1 -- 172.21.15.1:0/4106157394 shutdown_connections 2024-09-12T01:29:03.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.559+0000 7fc0f05bc740 1 -- 172.21.15.1:0/4106157394 wait complete. 2024-09-12T01:29:03.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.559+0000 7fc0f05bc740 20 client.14946 trim_cache size 0 max 16384 2024-09-12T01:29:03.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.578+0000 7fc0f05bc740 1 Processor -- start 2024-09-12T01:29:03.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.578+0000 7fc0f05bc740 1 -- start start 2024-09-12T01:29:03.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.578+0000 7fc0f05bc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d8214b330 0x556d82153ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:03.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.578+0000 7fc0f05bc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d8214bc50 0x556d821508b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:03.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.578+0000 7fc0f05bc740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d82156510 0x556d82150df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:03.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.578+0000 7fc0f05bc740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x556d821669d0 con 0x556d82156510 2024-09-12T01:29:03.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.578+0000 7fc0f05bc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x556d821670a0 con 0x556d8214b330 2024-09-12T01:29:03.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.578+0000 7fc0f05bc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x556d82167770 con 0x556d8214bc50 2024-09-12T01:29:03.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.579+0000 7fc0e6a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d8214b330 0x556d82153ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:03.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.579+0000 7fc0e6a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d8214b330 0x556d82153ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:34182/0 (socket says 172.21.15.1:34182) 2024-09-12T01:29:03.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.579+0000 7fc0e6a00640 1 -- 172.21.15.1:0/2931807929 learned_addr learned my addr 172.21.15.1:0/2931807929 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:03.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.579+0000 7fc0e7400640 1 --2- 172.21.15.1:0/2931807929 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d82156510 0x556d82150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:03.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.579+0000 7fc0de000640 1 --2- 172.21.15.1:0/2931807929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d8214bc50 0x556d821508b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:03.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.579+0000 7fc0e6a00640 1 -- 172.21.15.1:0/2931807929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d8214bc50 msgr2=0x556d821508b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:03.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.579+0000 7fc0e6a00640 1 --2- 172.21.15.1:0/2931807929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d8214bc50 0x556d821508b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:03.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.579+0000 7fc0e6a00640 1 -- 172.21.15.1:0/2931807929 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d82156510 msgr2=0x556d82150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:03.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.579+0000 7fc0e6a00640 1 --2- 172.21.15.1:0/2931807929 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d82156510 0x556d82150df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:03.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.579+0000 7fc0e6a00640 1 -- 172.21.15.1:0/2931807929 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d8214dd00 con 0x556d8214b330 2024-09-12T01:29:03.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.579+0000 7fc0e6a00640 1 --2- 172.21.15.1:0/2931807929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d8214b330 0x556d82153ec0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8054fd0 tx=0x7fc0d803f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9dec073d6df07a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:03.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.580+0000 7fc0d7400640 1 -- 172.21.15.1:0/2931807929 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d8035040 con 0x556d8214b330 2024-09-12T01:29:03.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.580+0000 7fc0d7400640 1 -- 172.21.15.1:0/2931807929 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0d805a070 con 0x556d8214b330 2024-09-12T01:29:03.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.580+0000 7fc0d7400640 1 -- 172.21.15.1:0/2931807929 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d80035f0 con 0x556d8214b330 2024-09-12T01:29:03.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.580+0000 7fc0f05bc740 1 -- 172.21.15.1:0/2931807929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d8214b330 msgr2=0x556d82153ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:03.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.580+0000 7fc0f05bc740 1 --2- 172.21.15.1:0/2931807929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d8214b330 0x556d82153ec0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8054fd0 tx=0x7fc0d803f000 comp rx=0 tx=0).stop 2024-09-12T01:29:03.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.580+0000 7fc0f05bc740 1 -- 172.21.15.1:0/2931807929 shutdown_connections 2024-09-12T01:29:03.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.580+0000 7fc0f05bc740 1 --2- 172.21.15.1:0/2931807929 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d82156510 0x556d82150df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:03.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.580+0000 7fc0f05bc740 1 --2- 172.21.15.1:0/2931807929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d8214bc50 0x556d821508b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:03.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.580+0000 7fc0f05bc740 1 --2- 172.21.15.1:0/2931807929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d8214b330 0x556d82153ec0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:03.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.581+0000 7fc0f05bc740 1 -- 172.21.15.1:0/2931807929 >> 172.21.15.1:0/2931807929 conn(0x556d821478c0 msgr2=0x556d82151f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:03.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.581+0000 7fc0f05bc740 1 -- 172.21.15.1:0/2931807929 shutdown_connections 2024-09-12T01:29:03.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.581+0000 7fc0f05bc740 1 -- 172.21.15.1:0/2931807929 wait complete. 2024-09-12T01:29:03.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.581+0000 7fc0f05bc740 1 -- start start 2024-09-12T01:29:03.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.581+0000 7fc0f05bc740 1 Processor -- start 2024-09-12T01:29:03.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.582+0000 7fc0f05bc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d8214b330 0x556d82170fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:03.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.582+0000 7fc0f05bc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d8214bc50 0x556d821505b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:03.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.582+0000 7fc0f05bc740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d82156510 0x556d82150af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:03.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.582+0000 7fc0f05bc740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x556d82168510 con 0x556d82156510 2024-09-12T01:29:03.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.582+0000 7fc0f05bc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x556d82167e40 con 0x556d8214bc50 2024-09-12T01:29:03.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.582+0000 7fc0f05bc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x556d82168be0 con 0x556d8214b330 2024-09-12T01:29:03.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.582+0000 7fc0f05bc740 10 client.0 _ll_register_callbacks cb 0x7fc0e0004270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:29:03.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.582+0000 7fc0e6a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d8214b330 0x556d82170fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:03.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.582+0000 7fc0e6a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d8214b330 0x556d82170fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:59650/0 (socket says 172.21.15.1:59650) 2024-09-12T01:29:03.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.582+0000 7fc0e6a00640 1 -- 172.21.15.1:0/487591540 learned_addr learned my addr 172.21.15.1:0/487591540 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:03.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.582+0000 7fc0de000640 1 --2- 172.21.15.1:0/487591540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d8214bc50 0x556d821505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:03.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.582+0000 7fc0e7400640 1 --2- 172.21.15.1:0/487591540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d82156510 0x556d82150af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:03.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.583+0000 7fc0e6a00640 1 -- 172.21.15.1:0/487591540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d8214bc50 msgr2=0x556d821505b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:03.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.583+0000 7fc0e6a00640 1 --2- 172.21.15.1:0/487591540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d8214bc50 0x556d821505b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:03.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.583+0000 7fc0e6a00640 1 -- 172.21.15.1:0/487591540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d82156510 msgr2=0x556d82150af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:03.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.583+0000 7fc0e6a00640 1 --2- 172.21.15.1:0/487591540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d82156510 0x556d82150af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:03.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.583+0000 7fc0e6a00640 1 -- 172.21.15.1:0/487591540 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d82151320 con 0x556d8214b330 2024-09-12T01:29:03.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.583+0000 7fc0e6a00640 1 --2- 172.21.15.1:0/487591540 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d8214b330 0x556d82170fa0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc0d802a910 tx=0x7fc0d80058f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=daca64024fb1a123 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:03.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.583+0000 7fc0d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-12T01:29:03.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.583+0000 7fc0d4c00640 1 -- 172.21.15.1:0/487591540 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d8063070 con 0x556d8214b330 2024-09-12T01:29:03.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.583+0000 7fc0f05bc740 1 -- 172.21.15.1:0/487591540 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0d805a070 con 0x556d8214b330 2024-09-12T01:29:03.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.583+0000 7fc0d4c00640 1 -- 172.21.15.1:0/487591540 <== mon.1 v2:172.21.15.104:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fc0d8032df0 con 0x556d8214b330 2024-09-12T01:29:03.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.584+0000 7fc0d4c00640 1 -- 172.21.15.1:0/487591540 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc0d8071050 con 0x556d8214b330 2024-09-12T01:29:03.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.584+0000 7fc0f05bc740 10 client.24979 fetch_fsmap learned FSMap version 28 2024-09-12T01:29:03.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.584+0000 7fc0f05bc740 1 -- 172.21.15.1:0/487591540 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556d821d1930 con 0x556d8214b330 2024-09-12T01:29:03.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.584+0000 7fc0d4c00640 1 -- 172.21.15.1:0/487591540 <== mon.1 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0d805a070 con 0x556d8214b330 2024-09-12T01:29:03.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.584+0000 7fc0d4c00640 1 -- 172.21.15.1:0/487591540 <== mon.1 v2:172.21.15.104:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fc0d8003790 con 0x556d8214b330 2024-09-12T01:29:03.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.584+0000 7fc0f05bc740 10 client.24979 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:29:03.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.584+0000 7fc0f05bc740 10 client.24979 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-12T01:29:03.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.584+0000 7fc0f05bc740 20 client.24979 populate_metadata read hostname 'smithi001' 2024-09-12T01:29:03.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.584+0000 7fc0f05bc740 1 --2- 172.21.15.1:0/487591540 >> [v2:172.21.15.104:6834/1547038540,v1:172.21.15.104:6835/1547038540] conn(0x556d821d5f30 0x556d821d4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:03.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.584+0000 7fc0f05bc740 4 client.24979 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T01:29:03.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.584+0000 7fc0f05bc740 1 -- 172.21.15.1:0/487591540 --> [v2:172.21.15.104:6834/1547038540,v1:172.21.15.104:6835/1547038540] -- command(tid 0: {"prefix": "status"}) -- 0x556d820b8b30 con 0x556d821d5f30 2024-09-12T01:29:03.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.585+0000 7fc0de000640 1 --2- 172.21.15.1:0/487591540 >> [v2:172.21.15.104:6834/1547038540,v1:172.21.15.104:6835/1547038540] conn(0x556d821d5f30 0x556d821d4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:03.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.585+0000 7fc0de000640 1 --2- 172.21.15.1:0/487591540 >> [v2:172.21.15.104:6834/1547038540,v1:172.21.15.104:6835/1547038540] conn(0x556d821d5f30 0x556d821d4c40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f69592455119a7a9 server_cookie=ee423976c2900648 in_seq=0 out_seq=0 2024-09-12T01:29:03.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.585+0000 7fc0d4c00640 10 client.24979 ms_handle_connect on v2:172.21.15.104:6834/1547038540 2024-09-12T01:29:03.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.585+0000 7fc0d4c00640 1 -- 172.21.15.1:0/487591540 <== mds.4 v2:172.21.15.104:6834/1547038540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x556d820b8b30 con 0x556d821d5f30 2024-09-12T01:29:03.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.585+0000 7fc0d4c00640 10 client.24979 handle_command_reply: tid=0 2024-09-12T01:29:03.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.585+0000 7fc0f05bc740 1 client.24979 shutdown 2024-09-12T01:29:03.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.585+0000 7fc0f05bc740 1 -- 172.21.15.1:0/487591540 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d8214b330 msgr2=0x556d82170fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:03.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.585+0000 7fc0f05bc740 1 --2- 172.21.15.1:0/487591540 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d8214b330 0x556d82170fa0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc0d802a910 tx=0x7fc0d80058f0 comp rx=0 tx=0).stop 2024-09-12T01:29:03.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.585+0000 7fc0f05bc740 1 -- 172.21.15.1:0/487591540 shutdown_connections 2024-09-12T01:29:03.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.586+0000 7fc0f05bc740 1 --2- 172.21.15.1:0/487591540 >> [v2:172.21.15.104:6834/1547038540,v1:172.21.15.104:6835/1547038540] conn(0x556d821d5f30 0x556d821d4c40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:03.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.586+0000 7fc0f05bc740 1 --2- 172.21.15.1:0/487591540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d82156510 0x556d82150af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:03.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.586+0000 7fc0f05bc740 1 --2- 172.21.15.1:0/487591540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d8214bc50 0x556d821505b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:03.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.586+0000 7fc0f05bc740 1 --2- 172.21.15.1:0/487591540 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d8214b330 0x556d82170fa0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:03.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.586+0000 7fc0d4c00640 0 client.24979 ms_handle_reset on v2:172.21.15.104:6834/1547038540 2024-09-12T01:29:03.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.586+0000 7fc0f05bc740 1 -- 172.21.15.1:0/487591540 >> 172.21.15.1:0/487591540 conn(0x556d821478c0 msgr2=0x556d8214d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:03.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.586+0000 7fc0f05bc740 1 -- 172.21.15.1:0/487591540 shutdown_connections 2024-09-12T01:29:03.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.586+0000 7fc0f05bc740 1 -- 172.21.15.1:0/487591540 wait complete. 2024-09-12T01:29:03.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.586+0000 7fc0f05bc740 20 client.24979 trim_cache size 0 max 16384 2024-09-12T01:29:03.589 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-12T01:29:03.589 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "c0e4f180-70a4-11ef-bcea-c7b262605968", 2024-09-12T01:29:03.589 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 4, 2024-09-12T01:29:03.589 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24487, 2024-09-12T01:29:03.589 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-09-12T01:29:03.589 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-09-12T01:29:03.589 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-09-12T01:29:03.589 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 79.269812940999998, 2024-09-12T01:29:03.590 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 28, 2024-09-12T01:29:03.590 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 98, 2024-09-12T01:29:03.590 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 96, 2024-09-12T01:29:03.590 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 167.210585146 2024-09-12T01:29:03.590 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-12T01:29:03.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.587+0000 7fc0eda00640 1 -- 172.21.15.1:0/4283365322 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fc0cc0a2c10 msgr2=0x7fc0cc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:03.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.587+0000 7fc0eda00640 1 --2- 172.21.15.1:0/4283365322 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fc0cc0a2c10 0x7fc0cc0a50d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8031550 tx=0x7fc0d8000bc0 comp rx=0 tx=0).stop 2024-09-12T01:29:03.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.587+0000 7fc0eda00640 1 -- 172.21.15.1:0/4283365322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0e8077040 msgr2=0x7fc0e81a4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:03.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.587+0000 7fc0eda00640 1 --2- 172.21.15.1:0/4283365322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0e8077040 0x7fc0e81a4c40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc0d000e9f0 tx=0x7fc0d000eeb0 comp rx=0 tx=0).stop 2024-09-12T01:29:03.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.587+0000 7fc0eda00640 1 -- 172.21.15.1:0/4283365322 shutdown_connections 2024-09-12T01:29:03.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.587+0000 7fc0eda00640 1 --2- 172.21.15.1:0/4283365322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0e8112a30 0x7fc0e819e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:03.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.587+0000 7fc0eda00640 1 --2- 172.21.15.1:0/4283365322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0e8077040 0x7fc0e81a4c40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:03.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.587+0000 7fc0eda00640 1 --2- 172.21.15.1:0/4283365322 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fc0cc0a2c10 0x7fc0cc0a50d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:03.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.587+0000 7fc0eda00640 1 --2- 172.21.15.1:0/4283365322 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc0e8076720 0x7fc0e81a4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:03.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.587+0000 7fc0eda00640 1 -- 172.21.15.1:0/4283365322 >> 172.21.15.1:0/4283365322 conn(0x7fc0e80ffa90 msgr2=0x7fc0e8101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:03.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.587+0000 7fc0eda00640 1 -- 172.21.15.1:0/4283365322 shutdown_connections 2024-09-12T01:29:03.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:03.587+0000 7fc0eda00640 1 -- 172.21.15.1:0/4283365322 wait complete. 2024-09-12T01:29:04.522 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-12T01:29:04.901 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:29:04.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:04 smithi001 ceph-mon[30387]: pgmap v382: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T01:29:04.992 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:04 smithi053 ceph-mon[35158]: pgmap v382: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T01:29:05.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:04 smithi104 ceph-mon[35219]: pgmap v382: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T01:29:05.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.522+0000 7f3c51800640 1 -- 172.21.15.1:0/268477023 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c4c109120 msgr2=0x7f3c4c110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:05.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.522+0000 7f3c51800640 1 --2- 172.21.15.1:0/268477023 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c4c109120 0x7f3c4c110ca0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3c44009ca0 tx=0x7f3c4402a8c0 comp rx=0 tx=0).stop 2024-09-12T01:29:05.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.522+0000 7f3c51800640 1 -- 172.21.15.1:0/268477023 shutdown_connections 2024-09-12T01:29:05.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.522+0000 7f3c51800640 1 --2- 172.21.15.1:0/268477023 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c4c109120 0x7f3c4c110ca0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:05.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.522+0000 7f3c51800640 1 --2- 172.21.15.1:0/268477023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c4c1049a0 0x7f3c4c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:05.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.522+0000 7f3c51800640 1 --2- 172.21.15.1:0/268477023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c4c103ff0 0x7f3c4c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:05.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.522+0000 7f3c51800640 1 -- 172.21.15.1:0/268477023 >> 172.21.15.1:0/268477023 conn(0x7f3c4c0ffa90 msgr2=0x7f3c4c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:05.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.522+0000 7f3c51800640 1 -- 172.21.15.1:0/268477023 shutdown_connections 2024-09-12T01:29:05.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.522+0000 7f3c51800640 1 -- 172.21.15.1:0/268477023 wait complete. 2024-09-12T01:29:05.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.523+0000 7f3c51800640 1 Processor -- start 2024-09-12T01:29:05.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.523+0000 7f3c51800640 1 -- start start 2024-09-12T01:29:05.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.523+0000 7f3c51800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c4c103ff0 0x7f3c4c19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:05.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.523+0000 7f3c51800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c4c1049a0 0x7f3c4c19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:05.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.523+0000 7f3c51800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c4c109120 0x7f3c4c19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:05.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.523+0000 7f3c51800640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4c114180 con 0x7f3c4c103ff0 2024-09-12T01:29:05.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.523+0000 7f3c51800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4c114000 con 0x7f3c4c1049a0 2024-09-12T01:29:05.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.523+0000 7f3c51800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4c114300 con 0x7f3c4c109120 2024-09-12T01:29:05.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.524+0000 7f3c4aa00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c4c103ff0 0x7f3c4c19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:05.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.524+0000 7f3c4b400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c4c109120 0x7f3c4c19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:05.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.524+0000 7f3c43e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c4c1049a0 0x7f3c4c19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:05.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.524+0000 7f3c43e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c4c1049a0 0x7f3c4c19ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:34218/0 (socket says 172.21.15.1:34218) 2024-09-12T01:29:05.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.524+0000 7f3c43e00640 1 -- 172.21.15.1:0/283035162 learned_addr learned my addr 172.21.15.1:0/283035162 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:05.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.524+0000 7f3c4b400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c4c109120 0x7f3c4c19ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:59662/0 (socket says 172.21.15.1:59662) 2024-09-12T01:29:05.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.524+0000 7f3c4aa00640 1 -- 172.21.15.1:0/283035162 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c4c109120 msgr2=0x7f3c4c19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:05.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.524+0000 7f3c4aa00640 1 --2- 172.21.15.1:0/283035162 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c4c109120 0x7f3c4c19ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:05.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.525+0000 7f3c4aa00640 1 -- 172.21.15.1:0/283035162 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c4c1049a0 msgr2=0x7f3c4c19ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:05.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.525+0000 7f3c4aa00640 1 --2- 172.21.15.1:0/283035162 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c4c1049a0 0x7f3c4c19ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:05.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.525+0000 7f3c4aa00640 1 -- 172.21.15.1:0/283035162 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c4c19f5f0 con 0x7f3c4c103ff0 2024-09-12T01:29:05.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.525+0000 7f3c43e00640 1 --2- 172.21.15.1:0/283035162 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c4c1049a0 0x7f3c4c19ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:29:05.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.525+0000 7f3c4aa00640 1 --2- 172.21.15.1:0/283035162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c4c103ff0 0x7f3c4c19a640 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c00e9f0 tx=0x7f3c3c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:05.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.525+0000 7f3c43400640 1 -- 172.21.15.1:0/283035162 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c00ccc0 con 0x7f3c4c103ff0 2024-09-12T01:29:05.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.525+0000 7f3c43400640 1 -- 172.21.15.1:0/283035162 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c3c010910 con 0x7f3c4c103ff0 2024-09-12T01:29:05.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.525+0000 7f3c43400640 1 -- 172.21.15.1:0/283035162 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c018960 con 0x7f3c4c103ff0 2024-09-12T01:29:05.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.525+0000 7f3c51800640 1 -- 172.21.15.1:0/283035162 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c4c19f8e0 con 0x7f3c4c103ff0 2024-09-12T01:29:05.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.526+0000 7f3c51800640 1 -- 172.21.15.1:0/283035162 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c4c1a71c0 con 0x7f3c4c103ff0 2024-09-12T01:29:05.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.529+0000 7f3c543c1740 1 Processor -- start 2024-09-12T01:29:05.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.529+0000 7f3c543c1740 1 -- start start 2024-09-12T01:29:05.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.529+0000 7f3c43400640 1 -- 172.21.15.1:0/283035162 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f3c3c010ac0 con 0x7f3c4c103ff0 2024-09-12T01:29:05.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.529+0000 7f3c543c1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564304ebe330 0x564304ebe710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:05.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.529+0000 7f3c543c1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564304ebec50 0x564304ec8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:05.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.529+0000 7f3c543c1740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564304ec9510 0x564304ecb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:05.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.529+0000 7f3c543c1740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564304e240b0 con 0x564304ec9510 2024-09-12T01:29:05.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.529+0000 7f3c543c1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x564304dcbab0 con 0x564304ebec50 2024-09-12T01:29:05.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.529+0000 7f3c543c1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564304e20070 con 0x564304ebe330 2024-09-12T01:29:05.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.530+0000 7f3c4aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564304ebec50 0x564304ec8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:05.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.530+0000 7f3c4aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564304ebec50 0x564304ec8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:34224/0 (socket says 172.21.15.1:34224) 2024-09-12T01:29:05.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.530+0000 7f3c4aa00640 1 -- 172.21.15.1:0/2534120070 learned_addr learned my addr 172.21.15.1:0/2534120070 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:05.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.530+0000 7f3c43e00640 1 --2- 172.21.15.1:0/2534120070 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564304ec9510 0x564304ecb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:05.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.530+0000 7f3c4b400640 1 --2- 172.21.15.1:0/2534120070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564304ebe330 0x564304ebe710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:05.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.530+0000 7f3c4aa00640 1 -- 172.21.15.1:0/2534120070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564304ebe330 msgr2=0x564304ebe710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:05.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.530+0000 7f3c4aa00640 1 --2- 172.21.15.1:0/2534120070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564304ebe330 0x564304ebe710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:05.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.530+0000 7f3c4aa00640 1 -- 172.21.15.1:0/2534120070 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564304ec9510 msgr2=0x564304ecb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:05.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.530+0000 7f3c4aa00640 1 --2- 172.21.15.1:0/2534120070 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564304ec9510 0x564304ecb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:05.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.530+0000 7f3c4aa00640 1 -- 172.21.15.1:0/2534120070 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564304dd84a0 con 0x564304ebec50 2024-09-12T01:29:05.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.531+0000 7f3c4aa00640 1 --2- 172.21.15.1:0/2534120070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564304ebec50 0x564304ec8fd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0d5bf0 tx=0x7f3c3c0d7670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e08f74ef5251d290 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:05.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.532+0000 7f3c3be00640 1 -- 172.21.15.1:0/2534120070 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c0de070 con 0x564304ebec50 2024-09-12T01:29:05.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.532+0000 7f3c3be00640 1 -- 172.21.15.1:0/2534120070 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c3c0dab00 con 0x564304ebec50 2024-09-12T01:29:05.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.532+0000 7f3c3be00640 1 -- 172.21.15.1:0/2534120070 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c0d9a40 con 0x564304ebec50 2024-09-12T01:29:05.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.532+0000 7f3c43400640 1 --2- 172.21.15.1:0/283035162 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f3c300a2c90 0x7f3c300a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:05.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.532+0000 7f3c43400640 1 -- 172.21.15.1:0/283035162 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f3c3c0cf980 con 0x7f3c4c103ff0 2024-09-12T01:29:05.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.532+0000 7f3c4b400640 1 --2- 172.21.15.1:0/283035162 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f3c300a2c90 0x7f3c300a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:05.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.533+0000 7f3c543c1740 1 -- 172.21.15.1:0/2534120070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564304ebec50 msgr2=0x564304ec8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:05.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.533+0000 7f3c543c1740 1 --2- 172.21.15.1:0/2534120070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564304ebec50 0x564304ec8fd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0d5bf0 tx=0x7f3c3c0d7670 comp rx=0 tx=0).stop 2024-09-12T01:29:05.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.533+0000 7f3c543c1740 1 -- 172.21.15.1:0/2534120070 shutdown_connections 2024-09-12T01:29:05.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.533+0000 7f3c543c1740 1 --2- 172.21.15.1:0/2534120070 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564304ec9510 0x564304ecb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:05.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.533+0000 7f3c543c1740 1 --2- 172.21.15.1:0/2534120070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564304ebec50 0x564304ec8fd0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:05.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.533+0000 7f3c543c1740 1 --2- 172.21.15.1:0/2534120070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564304ebe330 0x564304ebe710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:05.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.533+0000 7f3c543c1740 1 -- 172.21.15.1:0/2534120070 >> 172.21.15.1:0/2534120070 conn(0x564304eba8c0 msgr2=0x564304ebaca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:05.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.533+0000 7f3c4b400640 1 --2- 172.21.15.1:0/283035162 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f3c300a2c90 0x7f3c300a5150 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3c4402f540 tx=0x7f3c4402f3b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:05.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.533+0000 7f3c543c1740 1 -- 172.21.15.1:0/2534120070 shutdown_connections 2024-09-12T01:29:05.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.533+0000 7f3c543c1740 1 -- 172.21.15.1:0/2534120070 wait complete. 2024-09-12T01:29:05.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.534+0000 7f3c543c1740 1 -- start start 2024-09-12T01:29:05.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.534+0000 7f3c543c1740 1 Processor -- start 2024-09-12T01:29:05.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.535+0000 7f3c543c1740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564304ebe330 0x564304ec6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:05.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.535+0000 7f3c543c1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564304ebec50 0x564304ec6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:05.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.535+0000 7f3c543c1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564304ec9510 0x564304ec6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:05.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.535+0000 7f3c543c1740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564304ecc060 con 0x564304ebe330 2024-09-12T01:29:05.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.535+0000 7f3c543c1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x564304dac920 con 0x564304ebec50 2024-09-12T01:29:05.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.535+0000 7f3c543c1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564304ecc1e0 con 0x564304ec9510 2024-09-12T01:29:05.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.535+0000 7f3c543c1740 10 client.0 _ll_register_callbacks cb 0x564304dd08a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:29:05.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.535+0000 7f3c4aa00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564304ebe330 0x564304ec6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:05.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.535+0000 7f3c43e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564304ebec50 0x564304ec6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:05.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.535+0000 7f3c4aa00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564304ebe330 0x564304ec6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58310/0 (socket says 172.21.15.1:58310) 2024-09-12T01:29:05.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.535+0000 7f3c4aa00640 1 -- 172.21.15.1:0/3568240820 learned_addr learned my addr 172.21.15.1:0/3568240820 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:05.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.535+0000 7f3c4b400640 1 --2- 172.21.15.1:0/3568240820 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564304ec9510 0x564304ec6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:05.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.536+0000 7f3c4aa00640 1 -- 172.21.15.1:0/3568240820 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564304ec9510 msgr2=0x564304ec6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:05.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.536+0000 7f3c4aa00640 1 --2- 172.21.15.1:0/3568240820 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564304ec9510 0x564304ec6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:05.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.536+0000 7f3c4aa00640 1 -- 172.21.15.1:0/3568240820 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564304ebec50 msgr2=0x564304ec6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:05.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.536+0000 7f3c4aa00640 1 --2- 172.21.15.1:0/3568240820 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564304ebec50 0x564304ec6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:05.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.536+0000 7f3c4aa00640 1 -- 172.21.15.1:0/3568240820 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564304b952b0 con 0x564304ebe330 2024-09-12T01:29:05.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.536+0000 7f3c43e00640 1 --2- 172.21.15.1:0/3568240820 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564304ebec50 0x564304ec6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:29:05.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.536+0000 7f3c4b400640 1 --2- 172.21.15.1:0/3568240820 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564304ec9510 0x564304ec6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:29:05.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.536+0000 7f3c4aa00640 1 --2- 172.21.15.1:0/3568240820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564304ebe330 0x564304ec6560 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0d7b70 tx=0x7f3c3c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8691fc221a3a9d68 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:05.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.536+0000 7f3c38c00640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-12T01:29:05.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.536+0000 7f3c38c00640 1 -- 172.21.15.1:0/3568240820 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c0de070 con 0x564304ebe330 2024-09-12T01:29:05.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.536+0000 7f3c38c00640 1 -- 172.21.15.1:0/3568240820 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f3c3c0cd190 con 0x564304ebe330 2024-09-12T01:29:05.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.536+0000 7f3c543c1740 1 -- 172.21.15.1:0/3568240820 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c3c0dab00 con 0x564304ebe330 2024-09-12T01:29:05.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.537+0000 7f3c38c00640 1 -- 172.21.15.1:0/3568240820 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3c3c097f30 con 0x564304ebe330 2024-09-12T01:29:05.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.537+0000 7f3c543c1740 10 client.14964 fetch_fsmap learned FSMap version 28 2024-09-12T01:29:05.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.537+0000 7f3c543c1740 1 -- 172.21.15.1:0/3568240820 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564304ed7e30 con 0x564304ebe330 2024-09-12T01:29:05.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.537+0000 7f3c38c00640 1 -- 172.21.15.1:0/3568240820 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c3c0dab00 con 0x564304ebe330 2024-09-12T01:29:05.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.537+0000 7f3c38c00640 1 -- 172.21.15.1:0/3568240820 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f3c3c09bef0 con 0x564304ebe330 2024-09-12T01:29:05.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.537+0000 7f3c543c1740 10 client.14964 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:29:05.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.537+0000 7f3c543c1740 10 client.14964 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T01:29:05.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.537+0000 7f3c543c1740 20 client.14964 populate_metadata read hostname 'smithi001' 2024-09-12T01:29:05.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.537+0000 7f3c543c1740 1 --2- 172.21.15.1:0/3568240820 >> [v2:172.21.15.104:6836/464720247,v1:172.21.15.104:6837/464720247] conn(0x564304ed9f40 0x564304edb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:05.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.537+0000 7f3c543c1740 4 client.14964 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:29:05.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.537+0000 7f3c543c1740 1 -- 172.21.15.1:0/3568240820 --> [v2:172.21.15.104:6836/464720247,v1:172.21.15.104:6837/464720247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564304e31620 con 0x564304ed9f40 2024-09-12T01:29:05.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.538+0000 7f3c43e00640 1 --2- 172.21.15.1:0/3568240820 >> [v2:172.21.15.104:6836/464720247,v1:172.21.15.104:6837/464720247] conn(0x564304ed9f40 0x564304edb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:05.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.538+0000 7f3c43e00640 1 --2- 172.21.15.1:0/3568240820 >> [v2:172.21.15.104:6836/464720247,v1:172.21.15.104:6837/464720247] conn(0x564304ed9f40 0x564304edb3b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c7da5ec583a1d314 server_cookie=830196402e5a806c in_seq=0 out_seq=0 2024-09-12T01:29:05.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.538+0000 7f3c38c00640 10 client.14964 ms_handle_connect on v2:172.21.15.104:6836/464720247 2024-09-12T01:29:05.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.539+0000 7f3c38c00640 1 -- 172.21.15.1:0/3568240820 <== mds.3 v2:172.21.15.104:6836/464720247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564304e31620 con 0x564304ed9f40 2024-09-12T01:29:05.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.539+0000 7f3c38c00640 10 client.14964 handle_command_reply: tid=0 2024-09-12T01:29:05.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.539+0000 7f3c543c1740 1 client.14964 shutdown 2024-09-12T01:29:05.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.539+0000 7f3c543c1740 1 -- 172.21.15.1:0/3568240820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564304ebe330 msgr2=0x564304ec6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:05.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.539+0000 7f3c543c1740 1 --2- 172.21.15.1:0/3568240820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564304ebe330 0x564304ec6560 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0d7b70 tx=0x7f3c3c00cf60 comp rx=0 tx=0).stop 2024-09-12T01:29:05.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.539+0000 7f3c543c1740 1 -- 172.21.15.1:0/3568240820 shutdown_connections 2024-09-12T01:29:05.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.539+0000 7f3c543c1740 1 --2- 172.21.15.1:0/3568240820 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564304ec9510 0x564304ec6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:05.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.539+0000 7f3c543c1740 1 --2- 172.21.15.1:0/3568240820 >> [v2:172.21.15.104:6836/464720247,v1:172.21.15.104:6837/464720247] conn(0x564304ed9f40 0x564304edb3b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:05.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.539+0000 7f3c543c1740 1 --2- 172.21.15.1:0/3568240820 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564304ebec50 0x564304ec6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:05.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.539+0000 7f3c543c1740 1 --2- 172.21.15.1:0/3568240820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564304ebe330 0x564304ec6560 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:05.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.539+0000 7f3c38c00640 0 client.14964 ms_handle_reset on v2:172.21.15.104:6836/464720247 2024-09-12T01:29:05.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.539+0000 7f3c543c1740 1 -- 172.21.15.1:0/3568240820 >> 172.21.15.1:0/3568240820 conn(0x564304eba8c0 msgr2=0x564304ec05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:05.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.540+0000 7f3c543c1740 1 -- 172.21.15.1:0/3568240820 shutdown_connections 2024-09-12T01:29:05.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.540+0000 7f3c543c1740 1 -- 172.21.15.1:0/3568240820 wait complete. 2024-09-12T01:29:05.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.540+0000 7f3c543c1740 20 client.14964 trim_cache size 0 max 16384 2024-09-12T01:29:05.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.559+0000 7f3c543c1740 1 Processor -- start 2024-09-12T01:29:05.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.559+0000 7f3c543c1740 1 -- start start 2024-09-12T01:29:05.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.559+0000 7f3c543c1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564304ebe330 0x564304edb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:05.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.559+0000 7f3c543c1740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564304ebec50 0x564304ec6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:05.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.559+0000 7f3c543c1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564304ec9510 0x564304ec0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:05.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.559+0000 7f3c543c1740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564304ed96f0 con 0x564304ebec50 2024-09-12T01:29:05.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.559+0000 7f3c543c1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x564304ed9dc0 con 0x564304ebe330 2024-09-12T01:29:05.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.559+0000 7f3c543c1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564304eda490 con 0x564304ec9510 2024-09-12T01:29:05.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.560+0000 7f3c43e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564304ebec50 0x564304ec6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:05.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.560+0000 7f3c4b400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564304ec9510 0x564304ec0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:05.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.560+0000 7f3c43e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564304ebec50 0x564304ec6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58320/0 (socket says 172.21.15.1:58320) 2024-09-12T01:29:05.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.560+0000 7f3c43e00640 1 -- 172.21.15.1:0/2526154404 learned_addr learned my addr 172.21.15.1:0/2526154404 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:05.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.560+0000 7f3c4aa00640 1 --2- 172.21.15.1:0/2526154404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564304ebe330 0x564304edb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:05.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.560+0000 7f3c43e00640 1 -- 172.21.15.1:0/2526154404 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564304ec9510 msgr2=0x564304ec0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:05.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.560+0000 7f3c43e00640 1 --2- 172.21.15.1:0/2526154404 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564304ec9510 0x564304ec0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:05.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.560+0000 7f3c43e00640 1 -- 172.21.15.1:0/2526154404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564304ebe330 msgr2=0x564304edb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:05.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.560+0000 7f3c43e00640 1 --2- 172.21.15.1:0/2526154404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564304ebe330 0x564304edb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:05.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.560+0000 7f3c43e00640 1 -- 172.21.15.1:0/2526154404 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564304edb010 con 0x564304ebec50 2024-09-12T01:29:05.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.560+0000 7f3c43e00640 1 --2- 172.21.15.1:0/2526154404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564304ebec50 0x564304ec6e30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3c34009620 tx=0x7f3c34005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1d3f7be3f0f90b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:05.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.561+0000 7f3c3b400640 1 -- 172.21.15.1:0/2526154404 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3400fc80 con 0x564304ebec50 2024-09-12T01:29:05.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.561+0000 7f3c3b400640 1 -- 172.21.15.1:0/2526154404 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c3400fe20 con 0x564304ebec50 2024-09-12T01:29:05.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.561+0000 7f3c3b400640 1 -- 172.21.15.1:0/2526154404 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c34010430 con 0x564304ebec50 2024-09-12T01:29:05.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.561+0000 7f3c543c1740 1 -- 172.21.15.1:0/2526154404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564304ebec50 msgr2=0x564304ec6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:05.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.561+0000 7f3c543c1740 1 --2- 172.21.15.1:0/2526154404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564304ebec50 0x564304ec6e30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3c34009620 tx=0x7f3c34005f90 comp rx=0 tx=0).stop 2024-09-12T01:29:05.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.561+0000 7f3c543c1740 1 -- 172.21.15.1:0/2526154404 shutdown_connections 2024-09-12T01:29:05.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.561+0000 7f3c543c1740 1 --2- 172.21.15.1:0/2526154404 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564304ec9510 0x564304ec0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:05.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.561+0000 7f3c543c1740 1 --2- 172.21.15.1:0/2526154404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564304ebec50 0x564304ec6e30 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:05.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.561+0000 7f3c543c1740 1 --2- 172.21.15.1:0/2526154404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564304ebe330 0x564304edb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:05.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.561+0000 7f3c543c1740 1 -- 172.21.15.1:0/2526154404 >> 172.21.15.1:0/2526154404 conn(0x564304eba8c0 msgr2=0x564304ec7dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:05.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.561+0000 7f3c543c1740 1 -- 172.21.15.1:0/2526154404 shutdown_connections 2024-09-12T01:29:05.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.562+0000 7f3c543c1740 1 -- 172.21.15.1:0/2526154404 wait complete. 2024-09-12T01:29:05.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.562+0000 7f3c543c1740 1 -- start start 2024-09-12T01:29:05.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.562+0000 7f3c543c1740 1 Processor -- start 2024-09-12T01:29:05.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.563+0000 7f3c543c1740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564304ebe330 0x564304ec3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:05.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.563+0000 7f3c543c1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564304ebec50 0x564304ec3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:05.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.563+0000 7f3c543c1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564304ec9510 0x564304ec41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:05.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.563+0000 7f3c543c1740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564304edb230 con 0x564304ebe330 2024-09-12T01:29:05.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.563+0000 7f3c543c1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x564304edab60 con 0x564304ec9510 2024-09-12T01:29:05.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.563+0000 7f3c543c1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564304edb900 con 0x564304ebec50 2024-09-12T01:29:05.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.563+0000 7f3c543c1740 10 client.0 _ll_register_callbacks cb 0x7f3c3c0d5a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:29:05.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.563+0000 7f3c4aa00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564304ebe330 0x564304ec3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:05.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.563+0000 7f3c4aa00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564304ebe330 0x564304ec3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58330/0 (socket says 172.21.15.1:58330) 2024-09-12T01:29:05.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.563+0000 7f3c4aa00640 1 -- 172.21.15.1:0/2227291863 learned_addr learned my addr 172.21.15.1:0/2227291863 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:05.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.563+0000 7f3c4b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564304ec9510 0x564304ec41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:05.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.563+0000 7f3c43e00640 1 --2- 172.21.15.1:0/2227291863 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564304ebec50 0x564304ec3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:05.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.563+0000 7f3c4aa00640 1 -- 172.21.15.1:0/2227291863 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564304ebec50 msgr2=0x564304ec3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:05.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.563+0000 7f3c4aa00640 1 --2- 172.21.15.1:0/2227291863 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564304ebec50 0x564304ec3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:05.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.563+0000 7f3c4aa00640 1 -- 172.21.15.1:0/2227291863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564304ec9510 msgr2=0x564304ec41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:05.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.563+0000 7f3c4aa00640 1 --2- 172.21.15.1:0/2227291863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564304ec9510 0x564304ec41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:05.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.563+0000 7f3c4aa00640 1 -- 172.21.15.1:0/2227291863 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564304ee1ca0 con 0x564304ebe330 2024-09-12T01:29:05.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.564+0000 7f3c4aa00640 1 --2- 172.21.15.1:0/2227291863 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564304ebe330 0x564304ec3740 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0d5210 tx=0x7f3c3c0d5530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8929cd6c0a6da33 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:05.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.564+0000 7f3c38c00640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-12T01:29:05.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.564+0000 7f3c38c00640 1 -- 172.21.15.1:0/2227291863 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c0de070 con 0x564304ebe330 2024-09-12T01:29:05.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.564+0000 7f3c38c00640 1 -- 172.21.15.1:0/2227291863 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f3c3c0a4470 con 0x564304ebe330 2024-09-12T01:29:05.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.564+0000 7f3c543c1740 1 -- 172.21.15.1:0/2227291863 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c3400fe20 con 0x564304ebe330 2024-09-12T01:29:05.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.564+0000 7f3c38c00640 1 -- 172.21.15.1:0/2227291863 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3c3c09b5f0 con 0x564304ebe330 2024-09-12T01:29:05.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.564+0000 7f3c543c1740 10 client.14970 fetch_fsmap learned FSMap version 28 2024-09-12T01:29:05.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.564+0000 7f3c543c1740 1 -- 172.21.15.1:0/2227291863 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564304ee1ca0 con 0x564304ebe330 2024-09-12T01:29:05.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.564+0000 7f3c38c00640 1 -- 172.21.15.1:0/2227291863 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c3400fe20 con 0x564304ebe330 2024-09-12T01:29:05.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.565+0000 7f3c38c00640 1 -- 172.21.15.1:0/2227291863 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f3c3c09b0e0 con 0x564304ebe330 2024-09-12T01:29:05.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.565+0000 7f3c543c1740 10 client.14970 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:29:05.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.565+0000 7f3c543c1740 10 client.14970 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T01:29:05.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.565+0000 7f3c543c1740 20 client.14970 populate_metadata read hostname 'smithi001' 2024-09-12T01:29:05.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.565+0000 7f3c543c1740 1 --2- 172.21.15.1:0/2227291863 >> [v2:172.21.15.104:6836/464720247,v1:172.21.15.104:6837/464720247] conn(0x564304f491b0 0x564304f47ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:05.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.565+0000 7f3c543c1740 4 client.14970 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T01:29:05.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.565+0000 7f3c543c1740 1 -- 172.21.15.1:0/2227291863 --> [v2:172.21.15.104:6836/464720247,v1:172.21.15.104:6837/464720247] -- command(tid 0: {"prefix": "status"}) -- 0x564304e2bb30 con 0x564304f491b0 2024-09-12T01:29:05.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.565+0000 7f3c43e00640 1 --2- 172.21.15.1:0/2227291863 >> [v2:172.21.15.104:6836/464720247,v1:172.21.15.104:6837/464720247] conn(0x564304f491b0 0x564304f47ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:05.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.566+0000 7f3c43e00640 1 --2- 172.21.15.1:0/2227291863 >> [v2:172.21.15.104:6836/464720247,v1:172.21.15.104:6837/464720247] conn(0x564304f491b0 0x564304f47ec0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=109d81dbe6fb704a server_cookie=7a8b6714420d0f5f in_seq=0 out_seq=0 2024-09-12T01:29:05.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.566+0000 7f3c38c00640 10 client.14970 ms_handle_connect on v2:172.21.15.104:6836/464720247 2024-09-12T01:29:05.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.566+0000 7f3c38c00640 1 -- 172.21.15.1:0/2227291863 <== mds.3 v2:172.21.15.104:6836/464720247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x564304e2bb30 con 0x564304f491b0 2024-09-12T01:29:05.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.566+0000 7f3c38c00640 10 client.14970 handle_command_reply: tid=0 2024-09-12T01:29:05.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.566+0000 7f3c543c1740 1 client.14970 shutdown 2024-09-12T01:29:05.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.566+0000 7f3c543c1740 1 -- 172.21.15.1:0/2227291863 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564304ebe330 msgr2=0x564304ec3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:05.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.566+0000 7f3c543c1740 1 --2- 172.21.15.1:0/2227291863 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564304ebe330 0x564304ec3740 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0d5210 tx=0x7f3c3c0d5530 comp rx=0 tx=0).stop 2024-09-12T01:29:05.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.566+0000 7f3c543c1740 1 -- 172.21.15.1:0/2227291863 shutdown_connections 2024-09-12T01:29:05.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.566+0000 7f3c543c1740 1 --2- 172.21.15.1:0/2227291863 >> [v2:172.21.15.104:6836/464720247,v1:172.21.15.104:6837/464720247] conn(0x564304f491b0 0x564304f47ec0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:05.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.566+0000 7f3c543c1740 1 --2- 172.21.15.1:0/2227291863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564304ec9510 0x564304ec41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:05.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.566+0000 7f3c38c00640 0 client.14970 ms_handle_reset on v2:172.21.15.104:6836/464720247 2024-09-12T01:29:05.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.566+0000 7f3c543c1740 1 --2- 172.21.15.1:0/2227291863 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564304ebec50 0x564304ec3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:05.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.566+0000 7f3c543c1740 1 --2- 172.21.15.1:0/2227291863 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564304ebe330 0x564304ec3740 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:05.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.566+0000 7f3c543c1740 1 -- 172.21.15.1:0/2227291863 >> 172.21.15.1:0/2227291863 conn(0x564304eba8c0 msgr2=0x564304ed4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:05.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.567+0000 7f3c543c1740 1 -- 172.21.15.1:0/2227291863 shutdown_connections 2024-09-12T01:29:05.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.567+0000 7f3c543c1740 1 -- 172.21.15.1:0/2227291863 wait complete. 2024-09-12T01:29:05.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.567+0000 7f3c543c1740 20 client.14970 trim_cache size 0 max 16384 2024-09-12T01:29:05.570 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-12T01:29:05.570 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "c0e4f180-70a4-11ef-bcea-c7b262605968", 2024-09-12T01:29:05.571 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 3, 2024-09-12T01:29:05.571 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24493, 2024-09-12T01:29:05.571 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-09-12T01:29:05.571 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-09-12T01:29:05.571 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-09-12T01:29:05.571 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 82.260268663999994, 2024-09-12T01:29:05.571 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 28, 2024-09-12T01:29:05.571 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 98, 2024-09-12T01:29:05.571 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 96, 2024-09-12T01:29:05.571 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 166.64579773200001 2024-09-12T01:29:05.571 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-12T01:29:05.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.567+0000 7f3c51800640 1 -- 172.21.15.1:0/283035162 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f3c300a2c90 msgr2=0x7f3c300a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:05.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.567+0000 7f3c51800640 1 --2- 172.21.15.1:0/283035162 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f3c300a2c90 0x7f3c300a5150 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3c4402f540 tx=0x7f3c4402f3b0 comp rx=0 tx=0).stop 2024-09-12T01:29:05.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.567+0000 7f3c51800640 1 -- 172.21.15.1:0/283035162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c4c103ff0 msgr2=0x7f3c4c19a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:05.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.567+0000 7f3c51800640 1 --2- 172.21.15.1:0/283035162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c4c103ff0 0x7f3c4c19a640 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c00e9f0 tx=0x7f3c3c00eeb0 comp rx=0 tx=0).stop 2024-09-12T01:29:05.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.568+0000 7f3c51800640 1 -- 172.21.15.1:0/283035162 shutdown_connections 2024-09-12T01:29:05.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.568+0000 7f3c51800640 1 --2- 172.21.15.1:0/283035162 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f3c300a2c90 0x7f3c300a5150 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:05.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.568+0000 7f3c51800640 1 --2- 172.21.15.1:0/283035162 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c4c109120 0x7f3c4c19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:05.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.568+0000 7f3c51800640 1 --2- 172.21.15.1:0/283035162 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c4c1049a0 0x7f3c4c19ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:05.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.568+0000 7f3c51800640 1 --2- 172.21.15.1:0/283035162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c4c103ff0 0x7f3c4c19a640 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:05.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.568+0000 7f3c51800640 1 -- 172.21.15.1:0/283035162 >> 172.21.15.1:0/283035162 conn(0x7f3c4c0ffa90 msgr2=0x7f3c4c100580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:05.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.568+0000 7f3c51800640 1 -- 172.21.15.1:0/283035162 shutdown_connections 2024-09-12T01:29:05.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:05.568+0000 7f3c51800640 1 -- 172.21.15.1:0/283035162 wait complete. 2024-09-12T01:29:06.513 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-12T01:29:06.897 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:29:06.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:06 smithi001 ceph-mon[30387]: pgmap v383: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T01:29:06.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:06 smithi053 ceph-mon[35158]: pgmap v383: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T01:29:07.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:06 smithi104 ceph-mon[35219]: pgmap v383: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T01:29:07.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.523+0000 7f43c8800640 1 -- 172.21.15.1:0/3587132389 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43c01049a0 msgr2=0x7f43c01089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:07.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.523+0000 7f43c8800640 1 --2- 172.21.15.1:0/3587132389 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43c01049a0 0x7f43c01089f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f43b0009900 tx=0x7f43b002a3f0 comp rx=0 tx=0).stop 2024-09-12T01:29:07.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.523+0000 7f43c8800640 1 -- 172.21.15.1:0/3587132389 shutdown_connections 2024-09-12T01:29:07.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.523+0000 7f43c8800640 1 --2- 172.21.15.1:0/3587132389 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43c0109120 0x7f43c0110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:07.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.523+0000 7f43c8800640 1 --2- 172.21.15.1:0/3587132389 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43c01049a0 0x7f43c01089f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:07.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.523+0000 7f43c8800640 1 --2- 172.21.15.1:0/3587132389 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43c0103ff0 0x7f43c01043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:07.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.523+0000 7f43c8800640 1 -- 172.21.15.1:0/3587132389 >> 172.21.15.1:0/3587132389 conn(0x7f43c00ffa90 msgr2=0x7f43c0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:07.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.523+0000 7f43c8800640 1 -- 172.21.15.1:0/3587132389 shutdown_connections 2024-09-12T01:29:07.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.524+0000 7f43c8800640 1 -- 172.21.15.1:0/3587132389 wait complete. 2024-09-12T01:29:07.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.524+0000 7f43c8800640 1 Processor -- start 2024-09-12T01:29:07.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.525+0000 7f43c8800640 1 -- start start 2024-09-12T01:29:07.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.525+0000 7f43c8800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43c0103ff0 0x7f43c019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:07.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.525+0000 7f43c8800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43c01049a0 0x7f43c019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:07.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.525+0000 7f43c8800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43c0109120 0x7f43c019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:07.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.525+0000 7f43c8800640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f43c0114190 con 0x7f43c01049a0 2024-09-12T01:29:07.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.525+0000 7f43c8800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f43c0114010 con 0x7f43c0109120 2024-09-12T01:29:07.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.525+0000 7f43c8800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f43c0114310 con 0x7f43c0103ff0 2024-09-12T01:29:07.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.525+0000 7f43c5400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43c01049a0 0x7f43c019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:07.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.525+0000 7f43c6800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43c0109120 0x7f43c019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:07.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.525+0000 7f43c5400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43c01049a0 0x7f43c019abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58346/0 (socket says 172.21.15.1:58346) 2024-09-12T01:29:07.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.525+0000 7f43c5400640 1 -- 172.21.15.1:0/3273115072 learned_addr learned my addr 172.21.15.1:0/3273115072 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:07.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.525+0000 7f43c5e00640 1 --2- 172.21.15.1:0/3273115072 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43c0103ff0 0x7f43c019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:07.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.526+0000 7f43c5400640 1 -- 172.21.15.1:0/3273115072 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43c0103ff0 msgr2=0x7f43c019a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:07.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.526+0000 7f43c5400640 1 --2- 172.21.15.1:0/3273115072 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43c0103ff0 0x7f43c019a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:07.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.526+0000 7f43c5400640 1 -- 172.21.15.1:0/3273115072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43c0109120 msgr2=0x7f43c019ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:07.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.526+0000 7f43c5400640 1 --2- 172.21.15.1:0/3273115072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43c0109120 0x7f43c019ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:07.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.526+0000 7f43c5400640 1 -- 172.21.15.1:0/3273115072 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c019f620 con 0x7f43c01049a0 2024-09-12T01:29:07.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.526+0000 7f43c5e00640 1 --2- 172.21.15.1:0/3273115072 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43c0103ff0 0x7f43c019a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:29:07.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.526+0000 7f43c5400640 1 --2- 172.21.15.1:0/3273115072 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43c01049a0 0x7f43c019abb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f43b0004770 tx=0x7f43b0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:07.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.526+0000 7f43baa00640 1 -- 172.21.15.1:0/3273115072 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b003f070 con 0x7f43c01049a0 2024-09-12T01:29:07.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.526+0000 7f43baa00640 1 -- 172.21.15.1:0/3273115072 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43b0031ca0 con 0x7f43c01049a0 2024-09-12T01:29:07.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.526+0000 7f43c8800640 1 -- 172.21.15.1:0/3273115072 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43c019f8b0 con 0x7f43c01049a0 2024-09-12T01:29:07.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.526+0000 7f43baa00640 1 -- 172.21.15.1:0/3273115072 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b0035030 con 0x7f43c01049a0 2024-09-12T01:29:07.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.527+0000 7f43c8800640 1 -- 172.21.15.1:0/3273115072 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43c01a71f0 con 0x7f43c01049a0 2024-09-12T01:29:07.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.530+0000 7f43baa00640 1 -- 172.21.15.1:0/3273115072 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f43b0042070 con 0x7f43c01049a0 2024-09-12T01:29:07.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.530+0000 7f43cb2d5740 1 Processor -- start 2024-09-12T01:29:07.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.530+0000 7f43cb2d5740 1 -- start start 2024-09-12T01:29:07.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.530+0000 7f43cb2d5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcc7708330 0x55bcc7708710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:07.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.531+0000 7f43cb2d5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcc7708c50 0x55bcc7712fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:07.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.531+0000 7f43cb2d5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcc7713510 0x55bcc7715900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:07.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.531+0000 7f43cb2d5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55bcc766e0b0 con 0x55bcc7708c50 2024-09-12T01:29:07.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.531+0000 7f43cb2d5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55bcc7615ab0 con 0x55bcc7708330 2024-09-12T01:29:07.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.531+0000 7f43cb2d5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55bcc766a070 con 0x55bcc7713510 2024-09-12T01:29:07.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.531+0000 7f43c6800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcc7708330 0x55bcc7708710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:07.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.531+0000 7f43c6800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcc7708330 0x55bcc7708710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:34272/0 (socket says 172.21.15.1:34272) 2024-09-12T01:29:07.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.531+0000 7f43c6800640 1 -- 172.21.15.1:0/1745464878 learned_addr learned my addr 172.21.15.1:0/1745464878 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:07.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.531+0000 7f43c5e00640 1 --2- 172.21.15.1:0/1745464878 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcc7708c50 0x55bcc7712fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:07.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.531+0000 7f43c5400640 1 --2- 172.21.15.1:0/1745464878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcc7713510 0x55bcc7715900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:07.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.531+0000 7f43c6800640 1 -- 172.21.15.1:0/1745464878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcc7713510 msgr2=0x55bcc7715900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:07.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.531+0000 7f43c6800640 1 --2- 172.21.15.1:0/1745464878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcc7713510 0x55bcc7715900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:07.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.531+0000 7f43c6800640 1 -- 172.21.15.1:0/1745464878 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcc7708c50 msgr2=0x55bcc7712fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:07.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.531+0000 7f43c6800640 1 --2- 172.21.15.1:0/1745464878 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcc7708c50 0x55bcc7712fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:07.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.531+0000 7f43c6800640 1 -- 172.21.15.1:0/1745464878 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcc76224a0 con 0x55bcc7708330 2024-09-12T01:29:07.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.532+0000 7f43c5400640 1 --2- 172.21.15.1:0/1745464878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcc7713510 0x55bcc7715900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:29:07.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.532+0000 7f43baa00640 1 --2- 172.21.15.1:0/3273115072 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f43a80a2c10 0x7f43a80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:07.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.532+0000 7f43c6800640 1 --2- 172.21.15.1:0/1745464878 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcc7708330 0x55bcc7708710 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f43bc009f50 tx=0x7f43bc0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f8744b07540e092 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:07.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.532+0000 7f43baa00640 1 -- 172.21.15.1:0/3273115072 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f43b00efba0 con 0x7f43c01049a0 2024-09-12T01:29:07.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.532+0000 7f43c5e00640 1 --2- 172.21.15.1:0/3273115072 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f43a80a2c10 0x7f43a80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:07.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.532+0000 7f43af000640 1 -- 172.21.15.1:0/1745464878 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc004b20 con 0x55bcc7708330 2024-09-12T01:29:07.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.532+0000 7f43af000640 1 -- 172.21.15.1:0/1745464878 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43bc004cc0 con 0x55bcc7708330 2024-09-12T01:29:07.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.532+0000 7f43af000640 1 -- 172.21.15.1:0/1745464878 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc005020 con 0x55bcc7708330 2024-09-12T01:29:07.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.533+0000 7f43cb2d5740 1 -- 172.21.15.1:0/1745464878 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcc7708330 msgr2=0x55bcc7708710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:07.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.533+0000 7f43c5e00640 1 --2- 172.21.15.1:0/3273115072 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f43a80a2c10 0x7f43a80a50d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x55bcc7705720 tx=0x7f43b4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:07.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.533+0000 7f43cb2d5740 1 --2- 172.21.15.1:0/1745464878 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcc7708330 0x55bcc7708710 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f43bc009f50 tx=0x7f43bc0118d0 comp rx=0 tx=0).stop 2024-09-12T01:29:07.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.533+0000 7f43cb2d5740 1 -- 172.21.15.1:0/1745464878 shutdown_connections 2024-09-12T01:29:07.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.533+0000 7f43cb2d5740 1 --2- 172.21.15.1:0/1745464878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcc7713510 0x55bcc7715900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:07.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.533+0000 7f43cb2d5740 1 --2- 172.21.15.1:0/1745464878 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcc7708c50 0x55bcc7712fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:07.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.533+0000 7f43cb2d5740 1 --2- 172.21.15.1:0/1745464878 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcc7708330 0x55bcc7708710 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:07.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.533+0000 7f43cb2d5740 1 -- 172.21.15.1:0/1745464878 >> 172.21.15.1:0/1745464878 conn(0x55bcc77048c0 msgr2=0x55bcc7704ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:07.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.533+0000 7f43cb2d5740 1 -- 172.21.15.1:0/1745464878 shutdown_connections 2024-09-12T01:29:07.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.533+0000 7f43cb2d5740 1 -- 172.21.15.1:0/1745464878 wait complete. 2024-09-12T01:29:07.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.534+0000 7f43cb2d5740 1 -- start start 2024-09-12T01:29:07.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.534+0000 7f43cb2d5740 1 Processor -- start 2024-09-12T01:29:07.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.534+0000 7f43cb2d5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcc7708330 0x55bcc7710520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:07.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.534+0000 7f43cb2d5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcc7708c50 0x55bcc7710a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:07.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.534+0000 7f43cb2d5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcc7713510 0x55bcc7710fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:07.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.534+0000 7f43cb2d5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55bcc7716060 con 0x55bcc7708330 2024-09-12T01:29:07.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.534+0000 7f43cb2d5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55bcc75f6920 con 0x55bcc7713510 2024-09-12T01:29:07.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.534+0000 7f43cb2d5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55bcc77161e0 con 0x55bcc7708c50 2024-09-12T01:29:07.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.535+0000 7f43cb2d5740 10 client.0 _ll_register_callbacks cb 0x55bcc761a8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:29:07.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.535+0000 7f43c6800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcc7713510 0x55bcc7710fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:07.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.535+0000 7f43c6800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcc7713510 0x55bcc7710fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:34278/0 (socket says 172.21.15.1:34278) 2024-09-12T01:29:07.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.535+0000 7f43c6800640 1 -- 172.21.15.1:0/498360331 learned_addr learned my addr 172.21.15.1:0/498360331 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:07.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.535+0000 7f43c5400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcc7708c50 0x55bcc7710a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:07.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.535+0000 7f43c5e00640 1 --2- 172.21.15.1:0/498360331 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcc7708330 0x55bcc7710520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:07.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.535+0000 7f43c6800640 1 -- 172.21.15.1:0/498360331 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcc7708c50 msgr2=0x55bcc7710a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:07.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.535+0000 7f43c6800640 1 --2- 172.21.15.1:0/498360331 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcc7708c50 0x55bcc7710a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:07.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.535+0000 7f43c6800640 1 -- 172.21.15.1:0/498360331 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcc7708330 msgr2=0x55bcc7710520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:07.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.535+0000 7f43c6800640 1 --2- 172.21.15.1:0/498360331 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcc7708330 0x55bcc7710520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:07.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.535+0000 7f43c6800640 1 -- 172.21.15.1:0/498360331 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcc73df2b0 con 0x55bcc7713510 2024-09-12T01:29:07.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.535+0000 7f43c5400640 1 --2- 172.21.15.1:0/498360331 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcc7708c50 0x55bcc7710a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:29:07.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.536+0000 7f43c6800640 1 --2- 172.21.15.1:0/498360331 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcc7713510 0x55bcc7710fa0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f43bc012fd0 tx=0x7f43bc004ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9dabd37680e33107 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:07.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.536+0000 7f43a3400640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-12T01:29:07.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.536+0000 7f43a3400640 1 -- 172.21.15.1:0/498360331 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc029020 con 0x55bcc7713510 2024-09-12T01:29:07.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.536+0000 7f43cb2d5740 1 -- 172.21.15.1:0/498360331 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43bc004cc0 con 0x55bcc7713510 2024-09-12T01:29:07.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.536+0000 7f43a3400640 1 -- 172.21.15.1:0/498360331 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f43bc005000 con 0x55bcc7713510 2024-09-12T01:29:07.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.536+0000 7f43a3400640 1 -- 172.21.15.1:0/498360331 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f43bc004000 con 0x55bcc7713510 2024-09-12T01:29:07.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.536+0000 7f43cb2d5740 10 client.25016 fetch_fsmap learned FSMap version 28 2024-09-12T01:29:07.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.536+0000 7f43cb2d5740 1 -- 172.21.15.1:0/498360331 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bcc7721d20 con 0x55bcc7713510 2024-09-12T01:29:07.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.537+0000 7f43a3400640 1 -- 172.21.15.1:0/498360331 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43bc004cc0 con 0x55bcc7713510 2024-09-12T01:29:07.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.537+0000 7f43a3400640 1 -- 172.21.15.1:0/498360331 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f43bc0257e0 con 0x55bcc7713510 2024-09-12T01:29:07.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.537+0000 7f43cb2d5740 10 client.25016 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:29:07.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.537+0000 7f43cb2d5740 10 client.25016 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-12T01:29:07.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.537+0000 7f43cb2d5740 20 client.25016 populate_metadata read hostname 'smithi001' 2024-09-12T01:29:07.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.537+0000 7f43cb2d5740 1 --2- 172.21.15.1:0/498360331 >> [v2:172.21.15.53:6836/2848743754,v1:172.21.15.53:6837/2848743754] conn(0x55bcc77240f0 0x55bcc7725560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:07.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.537+0000 7f43cb2d5740 4 client.25016 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:29:07.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.537+0000 7f43cb2d5740 1 -- 172.21.15.1:0/498360331 --> [v2:172.21.15.53:6836/2848743754,v1:172.21.15.53:6837/2848743754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bcc767b620 con 0x55bcc77240f0 2024-09-12T01:29:07.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.537+0000 7f43c5e00640 1 --2- 172.21.15.1:0/498360331 >> [v2:172.21.15.53:6836/2848743754,v1:172.21.15.53:6837/2848743754] conn(0x55bcc77240f0 0x55bcc7725560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:07.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.538+0000 7f43c5e00640 1 --2- 172.21.15.1:0/498360331 >> [v2:172.21.15.53:6836/2848743754,v1:172.21.15.53:6837/2848743754] conn(0x55bcc77240f0 0x55bcc7725560 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=191e51c2c69ac6c9 server_cookie=8de0fb7ca3bbc490 in_seq=0 out_seq=0 2024-09-12T01:29:07.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.538+0000 7f43a3400640 10 client.25016 ms_handle_connect on v2:172.21.15.53:6836/2848743754 2024-09-12T01:29:07.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.539+0000 7f43a3400640 1 -- 172.21.15.1:0/498360331 <== mds.2 v2:172.21.15.53:6836/2848743754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bcc767b620 con 0x55bcc77240f0 2024-09-12T01:29:07.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.539+0000 7f43a3400640 10 client.25016 handle_command_reply: tid=0 2024-09-12T01:29:07.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.539+0000 7f43cb2d5740 1 client.25016 shutdown 2024-09-12T01:29:07.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.539+0000 7f43cb2d5740 1 -- 172.21.15.1:0/498360331 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcc7713510 msgr2=0x55bcc7710fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:07.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.539+0000 7f43cb2d5740 1 --2- 172.21.15.1:0/498360331 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcc7713510 0x55bcc7710fa0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f43bc012fd0 tx=0x7f43bc004ef0 comp rx=0 tx=0).stop 2024-09-12T01:29:07.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.539+0000 7f43cb2d5740 1 -- 172.21.15.1:0/498360331 shutdown_connections 2024-09-12T01:29:07.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.539+0000 7f43cb2d5740 1 --2- 172.21.15.1:0/498360331 >> [v2:172.21.15.53:6836/2848743754,v1:172.21.15.53:6837/2848743754] conn(0x55bcc77240f0 0x55bcc7725560 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:07.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.539+0000 7f43cb2d5740 1 --2- 172.21.15.1:0/498360331 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcc7713510 0x55bcc7710fa0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:07.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.539+0000 7f43cb2d5740 1 --2- 172.21.15.1:0/498360331 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcc7708c50 0x55bcc7710a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:07.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.539+0000 7f43cb2d5740 1 --2- 172.21.15.1:0/498360331 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcc7708330 0x55bcc7710520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:07.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.539+0000 7f43a3400640 0 client.25016 ms_handle_reset on v2:172.21.15.53:6836/2848743754 2024-09-12T01:29:07.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.539+0000 7f43cb2d5740 1 -- 172.21.15.1:0/498360331 >> 172.21.15.1:0/498360331 conn(0x55bcc77048c0 msgr2=0x55bcc770a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:07.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.539+0000 7f43cb2d5740 1 -- 172.21.15.1:0/498360331 shutdown_connections 2024-09-12T01:29:07.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.539+0000 7f43cb2d5740 1 -- 172.21.15.1:0/498360331 wait complete. 2024-09-12T01:29:07.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.539+0000 7f43cb2d5740 20 client.25016 trim_cache size 0 max 16384 2024-09-12T01:29:07.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.558+0000 7f43cb2d5740 1 Processor -- start 2024-09-12T01:29:07.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.558+0000 7f43cb2d5740 1 -- start start 2024-09-12T01:29:07.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.558+0000 7f43cb2d5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcc7708330 0x55bcc770fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:07.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.558+0000 7f43cb2d5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcc7708c50 0x55bcc770ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:07.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.558+0000 7f43cb2d5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcc7713510 0x55bcc770f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:07.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.558+0000 7f43cb2d5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55bcc77238a0 con 0x55bcc7713510 2024-09-12T01:29:07.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.558+0000 7f43cb2d5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55bcc7723f70 con 0x55bcc7708c50 2024-09-12T01:29:07.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.558+0000 7f43cb2d5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55bcc7724640 con 0x55bcc7708330 2024-09-12T01:29:07.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.559+0000 7f43c6800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcc7713510 0x55bcc770f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:07.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.559+0000 7f43c5400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcc7708c50 0x55bcc770ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:07.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.559+0000 7f43c5400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcc7708c50 0x55bcc770ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:34282/0 (socket says 172.21.15.1:34282) 2024-09-12T01:29:07.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.559+0000 7f43c5400640 1 -- 172.21.15.1:0/977130534 learned_addr learned my addr 172.21.15.1:0/977130534 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:07.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.559+0000 7f43c5e00640 1 --2- 172.21.15.1:0/977130534 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcc7708330 0x55bcc770fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:07.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.559+0000 7f43c6800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcc7713510 0x55bcc770f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58372/0 (socket says 172.21.15.1:58372) 2024-09-12T01:29:07.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.559+0000 7f43c5400640 1 -- 172.21.15.1:0/977130534 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcc7708330 msgr2=0x55bcc770fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:07.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.559+0000 7f43c5400640 1 --2- 172.21.15.1:0/977130534 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcc7708330 0x55bcc770fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:07.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.559+0000 7f43c5400640 1 -- 172.21.15.1:0/977130534 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcc7713510 msgr2=0x55bcc770f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:07.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.559+0000 7f43c5400640 1 --2- 172.21.15.1:0/977130534 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcc7713510 0x55bcc770f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:07.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.559+0000 7f43c5400640 1 -- 172.21.15.1:0/977130534 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcc7712e90 con 0x55bcc7708c50 2024-09-12T01:29:07.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.559+0000 7f43c6800640 1 --2- 172.21.15.1:0/977130534 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcc7713510 0x55bcc770f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:29:07.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.559+0000 7f43c5400640 1 --2- 172.21.15.1:0/977130534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcc7708c50 0x55bcc770ee10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f43b00bbd20 tx=0x7f43b00dc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d6b7ad680eda555 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:07.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.560+0000 7f43ae600640 1 -- 172.21.15.1:0/977130534 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b0049070 con 0x55bcc7708c50 2024-09-12T01:29:07.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.560+0000 7f43ae600640 1 -- 172.21.15.1:0/977130534 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43b00f4040 con 0x55bcc7708c50 2024-09-12T01:29:07.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.560+0000 7f43ae600640 1 -- 172.21.15.1:0/977130534 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b003b4f0 con 0x55bcc7708c50 2024-09-12T01:29:07.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.560+0000 7f43cb2d5740 1 -- 172.21.15.1:0/977130534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcc7708c50 msgr2=0x55bcc770ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:07.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.561+0000 7f43cb2d5740 1 --2- 172.21.15.1:0/977130534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcc7708c50 0x55bcc770ee10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f43b00bbd20 tx=0x7f43b00dc040 comp rx=0 tx=0).stop 2024-09-12T01:29:07.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.561+0000 7f43cb2d5740 1 -- 172.21.15.1:0/977130534 shutdown_connections 2024-09-12T01:29:07.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.561+0000 7f43cb2d5740 1 --2- 172.21.15.1:0/977130534 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcc7713510 0x55bcc770f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:07.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.561+0000 7f43cb2d5740 1 --2- 172.21.15.1:0/977130534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcc7708c50 0x55bcc770ee10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:07.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.561+0000 7f43cb2d5740 1 --2- 172.21.15.1:0/977130534 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcc7708330 0x55bcc770fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:07.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.561+0000 7f43cb2d5740 1 -- 172.21.15.1:0/977130534 >> 172.21.15.1:0/977130534 conn(0x55bcc77048c0 msgr2=0x55bcc7723190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:07.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.561+0000 7f43cb2d5740 1 -- 172.21.15.1:0/977130534 shutdown_connections 2024-09-12T01:29:07.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.561+0000 7f43cb2d5740 1 -- 172.21.15.1:0/977130534 wait complete. 2024-09-12T01:29:07.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.561+0000 7f43cb2d5740 1 -- start start 2024-09-12T01:29:07.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.561+0000 7f43cb2d5740 1 Processor -- start 2024-09-12T01:29:07.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.562+0000 7f43cb2d5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcc7708330 0x55bcc772de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:07.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.562+0000 7f43cb2d5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcc7708c50 0x55bcc770d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:07.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.562+0000 7f43cb2d5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcc7713510 0x55bcc770d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:07.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.562+0000 7f43cb2d5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55bcc77253e0 con 0x55bcc7708330 2024-09-12T01:29:07.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.562+0000 7f43cb2d5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55bcc7724d10 con 0x55bcc7713510 2024-09-12T01:29:07.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.562+0000 7f43cb2d5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55bcc7725ab0 con 0x55bcc7708c50 2024-09-12T01:29:07.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.562+0000 7f43cb2d5740 10 client.0 _ll_register_callbacks cb 0x7f43bc003df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:29:07.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.562+0000 7f43c5e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcc7708330 0x55bcc772de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:07.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.562+0000 7f43c5e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcc7708330 0x55bcc772de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58386/0 (socket says 172.21.15.1:58386) 2024-09-12T01:29:07.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.562+0000 7f43c5e00640 1 -- 172.21.15.1:0/2362475044 learned_addr learned my addr 172.21.15.1:0/2362475044 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:07.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.562+0000 7f43c5400640 1 --2- 172.21.15.1:0/2362475044 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcc7708c50 0x55bcc770d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:07.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.562+0000 7f43c6800640 1 --2- 172.21.15.1:0/2362475044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcc7713510 0x55bcc770d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:07.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.563+0000 7f43c5e00640 1 -- 172.21.15.1:0/2362475044 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcc7708c50 msgr2=0x55bcc770d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:07.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.563+0000 7f43c5e00640 1 --2- 172.21.15.1:0/2362475044 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcc7708c50 0x55bcc770d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:07.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.563+0000 7f43c5e00640 1 -- 172.21.15.1:0/2362475044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcc7713510 msgr2=0x55bcc770d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:07.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.563+0000 7f43c5e00640 1 --2- 172.21.15.1:0/2362475044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcc7713510 0x55bcc770d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:07.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.563+0000 7f43c5e00640 1 -- 172.21.15.1:0/2362475044 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcc770df30 con 0x55bcc7708330 2024-09-12T01:29:07.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.563+0000 7f43c6800640 1 --2- 172.21.15.1:0/2362475044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcc7713510 0x55bcc770d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:29:07.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.563+0000 7f43c5400640 1 --2- 172.21.15.1:0/2362475044 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcc7708c50 0x55bcc770d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:29:07.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.563+0000 7f43c5e00640 1 --2- 172.21.15.1:0/2362475044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcc7708330 0x55bcc772de90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x55bcc7725d90 tx=0x7f43b4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be94e269c6e88152 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:07.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.563+0000 7f43a3400640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-12T01:29:07.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.563+0000 7f43a3400640 1 -- 172.21.15.1:0/2362475044 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b4005020 con 0x55bcc7708330 2024-09-12T01:29:07.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.563+0000 7f43cb2d5740 1 -- 172.21.15.1:0/2362475044 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43b00f4040 con 0x55bcc7708330 2024-09-12T01:29:07.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.563+0000 7f43a3400640 1 -- 172.21.15.1:0/2362475044 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f43b40051c0 con 0x55bcc7708330 2024-09-12T01:29:07.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.563+0000 7f43a3400640 1 -- 172.21.15.1:0/2362475044 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f43b40053c0 con 0x55bcc7708330 2024-09-12T01:29:07.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.563+0000 7f43cb2d5740 10 client.14988 fetch_fsmap learned FSMap version 28 2024-09-12T01:29:07.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.563+0000 7f43cb2d5740 1 -- 172.21.15.1:0/2362475044 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bcc778e800 con 0x55bcc7708330 2024-09-12T01:29:07.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.564+0000 7f43a3400640 1 -- 172.21.15.1:0/2362475044 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43b00f4040 con 0x55bcc7708330 2024-09-12T01:29:07.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.564+0000 7f43a3400640 1 -- 172.21.15.1:0/2362475044 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f43b402b980 con 0x55bcc7708330 2024-09-12T01:29:07.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.564+0000 7f43cb2d5740 10 client.14988 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:29:07.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.564+0000 7f43cb2d5740 10 client.14988 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-12T01:29:07.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.564+0000 7f43cb2d5740 20 client.14988 populate_metadata read hostname 'smithi001' 2024-09-12T01:29:07.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.564+0000 7f43cb2d5740 1 --2- 172.21.15.1:0/2362475044 >> [v2:172.21.15.53:6836/2848743754,v1:172.21.15.53:6837/2848743754] conn(0x55bcc7793310 0x55bcc7792020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:07.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.564+0000 7f43cb2d5740 4 client.14988 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T01:29:07.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.564+0000 7f43cb2d5740 1 -- 172.21.15.1:0/2362475044 --> [v2:172.21.15.53:6836/2848743754,v1:172.21.15.53:6837/2848743754] -- command(tid 0: {"prefix": "status"}) -- 0x55bcc7675b30 con 0x55bcc7793310 2024-09-12T01:29:07.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.564+0000 7f43c5400640 1 --2- 172.21.15.1:0/2362475044 >> [v2:172.21.15.53:6836/2848743754,v1:172.21.15.53:6837/2848743754] conn(0x55bcc7793310 0x55bcc7792020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:07.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.565+0000 7f43c5400640 1 --2- 172.21.15.1:0/2362475044 >> [v2:172.21.15.53:6836/2848743754,v1:172.21.15.53:6837/2848743754] conn(0x55bcc7793310 0x55bcc7792020 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ec8f67bd438d786b server_cookie=6063835c48fdb7b5 in_seq=0 out_seq=0 2024-09-12T01:29:07.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.565+0000 7f43a3400640 10 client.14988 ms_handle_connect on v2:172.21.15.53:6836/2848743754 2024-09-12T01:29:07.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.565+0000 7f43a3400640 1 -- 172.21.15.1:0/2362475044 <== mds.2 v2:172.21.15.53:6836/2848743754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55bcc7675b30 con 0x55bcc7793310 2024-09-12T01:29:07.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.565+0000 7f43a3400640 10 client.14988 handle_command_reply: tid=0 2024-09-12T01:29:07.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.565+0000 7f43cb2d5740 1 client.14988 shutdown 2024-09-12T01:29:07.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.565+0000 7f43cb2d5740 1 -- 172.21.15.1:0/2362475044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcc7708330 msgr2=0x55bcc772de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:07.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.565+0000 7f43cb2d5740 1 --2- 172.21.15.1:0/2362475044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcc7708330 0x55bcc772de90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x55bcc7725d90 tx=0x7f43b4017000 comp rx=0 tx=0).stop 2024-09-12T01:29:07.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.566+0000 7f43cb2d5740 1 -- 172.21.15.1:0/2362475044 shutdown_connections 2024-09-12T01:29:07.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.566+0000 7f43cb2d5740 1 --2- 172.21.15.1:0/2362475044 >> [v2:172.21.15.53:6836/2848743754,v1:172.21.15.53:6837/2848743754] conn(0x55bcc7793310 0x55bcc7792020 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:07.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.566+0000 7f43cb2d5740 1 --2- 172.21.15.1:0/2362475044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcc7713510 0x55bcc770d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:07.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.566+0000 7f43cb2d5740 1 --2- 172.21.15.1:0/2362475044 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcc7708c50 0x55bcc770d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:07.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.566+0000 7f43cb2d5740 1 --2- 172.21.15.1:0/2362475044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcc7708330 0x55bcc772de90 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:07.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.566+0000 7f43a3400640 0 client.14988 ms_handle_reset on v2:172.21.15.53:6836/2848743754 2024-09-12T01:29:07.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.566+0000 7f43cb2d5740 1 -- 172.21.15.1:0/2362475044 >> 172.21.15.1:0/2362475044 conn(0x55bcc77048c0 msgr2=0x55bcc770ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:07.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.566+0000 7f43cb2d5740 1 -- 172.21.15.1:0/2362475044 shutdown_connections 2024-09-12T01:29:07.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.566+0000 7f43cb2d5740 1 -- 172.21.15.1:0/2362475044 wait complete. 2024-09-12T01:29:07.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.566+0000 7f43cb2d5740 20 client.14988 trim_cache size 0 max 16384 2024-09-12T01:29:07.573 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-12T01:29:07.573 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "c0e4f180-70a4-11ef-bcea-c7b262605968", 2024-09-12T01:29:07.573 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 2, 2024-09-12T01:29:07.573 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24506, 2024-09-12T01:29:07.573 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-09-12T01:29:07.573 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-09-12T01:29:07.574 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-09-12T01:29:07.574 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 85.269388735000007, 2024-09-12T01:29:07.574 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 28, 2024-09-12T01:29:07.574 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 98, 2024-09-12T01:29:07.574 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 96, 2024-09-12T01:29:07.574 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 165.87217381299999 2024-09-12T01:29:07.574 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-12T01:29:07.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.566+0000 7f43c8800640 1 -- 172.21.15.1:0/3273115072 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f43a80a2c10 msgr2=0x7f43a80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:07.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.566+0000 7f43c8800640 1 --2- 172.21.15.1:0/3273115072 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f43a80a2c10 0x7f43a80a50d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x55bcc7705720 tx=0x7f43b4005f90 comp rx=0 tx=0).stop 2024-09-12T01:29:07.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.566+0000 7f43c8800640 1 -- 172.21.15.1:0/3273115072 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43c01049a0 msgr2=0x7f43c019abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:07.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.566+0000 7f43c8800640 1 --2- 172.21.15.1:0/3273115072 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43c01049a0 0x7f43c019abb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f43b0004770 tx=0x7f43b0003c10 comp rx=0 tx=0).stop 2024-09-12T01:29:07.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.567+0000 7f43c8800640 1 -- 172.21.15.1:0/3273115072 shutdown_connections 2024-09-12T01:29:07.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.567+0000 7f43c8800640 1 --2- 172.21.15.1:0/3273115072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43c0109120 0x7f43c019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:07.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.567+0000 7f43c8800640 1 --2- 172.21.15.1:0/3273115072 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43c01049a0 0x7f43c019abb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:07.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.567+0000 7f43c8800640 1 --2- 172.21.15.1:0/3273115072 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f43a80a2c10 0x7f43a80a50d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:07.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.567+0000 7f43c8800640 1 --2- 172.21.15.1:0/3273115072 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43c0103ff0 0x7f43c019a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:07.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.567+0000 7f43c8800640 1 -- 172.21.15.1:0/3273115072 >> 172.21.15.1:0/3273115072 conn(0x7f43c00ffa90 msgr2=0x7f43c0100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:07.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.567+0000 7f43c8800640 1 -- 172.21.15.1:0/3273115072 shutdown_connections 2024-09-12T01:29:07.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:07.567+0000 7f43c8800640 1 -- 172.21.15.1:0/3273115072 wait complete. 2024-09-12T01:29:08.533 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-12T01:29:08.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:08 smithi001 ceph-mon[30387]: pgmap v384: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-12T01:29:08.913 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:29:08.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:08 smithi053 ceph-mon[35158]: pgmap v384: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-12T01:29:09.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:08 smithi104 ceph-mon[35219]: pgmap v384: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-12T01:29:09.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.547+0000 7f7b1e400640 1 -- 172.21.15.1:0/1693507523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b18106f10 msgr2=0x7f7b1810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:09.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.547+0000 7f7b1e400640 1 --2- 172.21.15.1:0/1693507523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b18106f10 0x7f7b1810ea90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7b10009ca0 tx=0x7f7b1002a8c0 comp rx=0 tx=0).stop 2024-09-12T01:29:09.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.547+0000 7f7b1e400640 1 -- 172.21.15.1:0/1693507523 shutdown_connections 2024-09-12T01:29:09.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.547+0000 7f7b1e400640 1 --2- 172.21.15.1:0/1693507523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b18106f10 0x7f7b1810ea90 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:09.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.547+0000 7f7b1e400640 1 --2- 172.21.15.1:0/1693507523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b18102790 0x7f7b181067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:09.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.547+0000 7f7b1e400640 1 --2- 172.21.15.1:0/1693507523 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b18101de0 0x7f7b181021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:09.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.547+0000 7f7b1e400640 1 -- 172.21.15.1:0/1693507523 >> 172.21.15.1:0/1693507523 conn(0x7f7b180fd8c0 msgr2=0x7f7b180ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:09.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.547+0000 7f7b1e400640 1 -- 172.21.15.1:0/1693507523 shutdown_connections 2024-09-12T01:29:09.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.547+0000 7f7b1e400640 1 -- 172.21.15.1:0/1693507523 wait complete. 2024-09-12T01:29:09.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.548+0000 7f7b1e400640 1 Processor -- start 2024-09-12T01:29:09.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.549+0000 7f7b1e400640 1 -- start start 2024-09-12T01:29:09.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.549+0000 7f7b1e400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b18101de0 0x7f7b18198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:09.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.549+0000 7f7b1e400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b18102790 0x7f7b181989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:09.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.549+0000 7f7b1e400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b18106f10 0x7f7b1819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:09.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.549+0000 7f7b1e400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7b18111f70 con 0x7f7b18101de0 2024-09-12T01:29:09.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.549+0000 7f7b1e400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7b18111df0 con 0x7f7b18102790 2024-09-12T01:29:09.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.549+0000 7f7b1e400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7b181120f0 con 0x7f7b18106f10 2024-09-12T01:29:09.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.549+0000 7f7b16a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b18102790 0x7f7b181989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:09.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.549+0000 7f7b16a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b18102790 0x7f7b181989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:35652/0 (socket says 172.21.15.1:35652) 2024-09-12T01:29:09.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.549+0000 7f7b17400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b18101de0 0x7f7b18198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:09.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.549+0000 7f7b16a00640 1 -- 172.21.15.1:0/159833 learned_addr learned my addr 172.21.15.1:0/159833 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:09.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.549+0000 7f7b17e00640 1 --2- 172.21.15.1:0/159833 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b18106f10 0x7f7b1819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:09.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.550+0000 7f7b16a00640 1 -- 172.21.15.1:0/159833 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b18106f10 msgr2=0x7f7b1819cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:09.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.550+0000 7f7b16a00640 1 --2- 172.21.15.1:0/159833 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b18106f10 0x7f7b1819cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:09.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.550+0000 7f7b16a00640 1 -- 172.21.15.1:0/159833 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b18101de0 msgr2=0x7f7b18198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:09.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.550+0000 7f7b16a00640 1 --2- 172.21.15.1:0/159833 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b18101de0 0x7f7b18198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:09.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.550+0000 7f7b16a00640 1 -- 172.21.15.1:0/159833 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b1819d420 con 0x7f7b18102790 2024-09-12T01:29:09.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.550+0000 7f7b16a00640 1 --2- 172.21.15.1:0/159833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b18102790 0x7f7b181989b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7b0400e9f0 tx=0x7f7b0400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:09.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.550+0000 7f7b0fe00640 1 -- 172.21.15.1:0/159833 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0400cd80 con 0x7f7b18102790 2024-09-12T01:29:09.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.550+0000 7f7b1e400640 1 -- 172.21.15.1:0/159833 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b1819d710 con 0x7f7b18102790 2024-09-12T01:29:09.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.550+0000 7f7b0fe00640 1 -- 172.21.15.1:0/159833 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b04010910 con 0x7f7b18102790 2024-09-12T01:29:09.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.550+0000 7f7b0fe00640 1 -- 172.21.15.1:0/159833 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b040188f0 con 0x7f7b18102790 2024-09-12T01:29:09.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.551+0000 7f7b1e400640 1 -- 172.21.15.1:0/159833 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b181a4ff0 con 0x7f7b18102790 2024-09-12T01:29:09.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.554+0000 7f7b20fda740 1 Processor -- start 2024-09-12T01:29:09.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.554+0000 7f7b20fda740 1 -- start start 2024-09-12T01:29:09.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.554+0000 7f7b20fda740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610a20f5740 0x5610a20f5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:09.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.554+0000 7f7b20fda740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610a20f6060 0x5610a21003e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:09.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.554+0000 7f7b20fda740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610a2100920 0x5610a2102d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:09.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.554+0000 7f7b20fda740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5610a20570b0 con 0x5610a20f6060 2024-09-12T01:29:09.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.554+0000 7f7b20fda740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5610a1ffeab0 con 0x5610a20f5740 2024-09-12T01:29:09.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.554+0000 7f7b20fda740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5610a2053070 con 0x5610a2100920 2024-09-12T01:29:09.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.554+0000 7f7b0fe00640 1 -- 172.21.15.1:0/159833 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f7b04010ac0 con 0x7f7b18102790 2024-09-12T01:29:09.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.554+0000 7f7b17400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610a20f6060 0x5610a21003e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:09.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.554+0000 7f7b17e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610a20f5740 0x5610a20f5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:09.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.554+0000 7f7b17400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610a20f6060 0x5610a21003e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40968/0 (socket says 172.21.15.1:40968) 2024-09-12T01:29:09.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.554+0000 7f7b17400640 1 -- 172.21.15.1:0/3404907258 learned_addr learned my addr 172.21.15.1:0/3404907258 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:09.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.555+0000 7f7b16a00640 1 --2- 172.21.15.1:0/3404907258 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610a2100920 0x5610a2102d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:09.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.555+0000 7f7b17e00640 1 -- 172.21.15.1:0/3404907258 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610a2100920 msgr2=0x5610a2102d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:09.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.555+0000 7f7b17e00640 1 --2- 172.21.15.1:0/3404907258 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610a2100920 0x5610a2102d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:09.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.555+0000 7f7b17e00640 1 -- 172.21.15.1:0/3404907258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610a20f6060 msgr2=0x5610a21003e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:09.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.555+0000 7f7b17e00640 1 --2- 172.21.15.1:0/3404907258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610a20f6060 0x5610a21003e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:09.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.555+0000 7f7b17e00640 1 -- 172.21.15.1:0/3404907258 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610a200b4a0 con 0x5610a20f5740 2024-09-12T01:29:09.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.555+0000 7f7b16a00640 1 --2- 172.21.15.1:0/3404907258 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610a2100920 0x5610a2102d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:29:09.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.555+0000 7f7b17e00640 1 --2- 172.21.15.1:0/3404907258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610a20f5740 0x5610a20f5b20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7b10006010 tx=0x7f7b10005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b225d74edc206dc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:09.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.556+0000 7f7b0cc00640 1 -- 172.21.15.1:0/3404907258 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b10031db0 con 0x5610a20f5740 2024-09-12T01:29:09.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.556+0000 7f7b0cc00640 1 -- 172.21.15.1:0/3404907258 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b10036040 con 0x5610a20f5740 2024-09-12T01:29:09.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.556+0000 7f7b0cc00640 1 -- 172.21.15.1:0/3404907258 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b100042a0 con 0x5610a20f5740 2024-09-12T01:29:09.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.556+0000 7f7b0fe00640 1 --2- 172.21.15.1:0/159833 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f7afc0a2c10 0x7f7afc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:09.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.556+0000 7f7b0fe00640 1 -- 172.21.15.1:0/159833 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f7b040cf920 con 0x7f7b18102790 2024-09-12T01:29:09.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.556+0000 7f7b17400640 1 --2- 172.21.15.1:0/159833 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f7afc0a2c10 0x7f7afc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:09.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.556+0000 7f7b20fda740 1 -- 172.21.15.1:0/3404907258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610a20f5740 msgr2=0x5610a20f5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:09.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.556+0000 7f7b20fda740 1 --2- 172.21.15.1:0/3404907258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610a20f5740 0x5610a20f5b20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7b10006010 tx=0x7f7b10005f10 comp rx=0 tx=0).stop 2024-09-12T01:29:09.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.557+0000 7f7b20fda740 1 -- 172.21.15.1:0/3404907258 shutdown_connections 2024-09-12T01:29:09.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.557+0000 7f7b20fda740 1 --2- 172.21.15.1:0/3404907258 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610a2100920 0x5610a2102d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:09.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.557+0000 7f7b20fda740 1 --2- 172.21.15.1:0/3404907258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610a20f6060 0x5610a21003e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:09.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.557+0000 7f7b20fda740 1 --2- 172.21.15.1:0/3404907258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610a20f5740 0x5610a20f5b20 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:09.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.557+0000 7f7b20fda740 1 -- 172.21.15.1:0/3404907258 >> 172.21.15.1:0/3404907258 conn(0x5610a20f1cd0 msgr2=0x5610a20f20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:09.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.557+0000 7f7b20fda740 1 -- 172.21.15.1:0/3404907258 shutdown_connections 2024-09-12T01:29:09.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.557+0000 7f7b20fda740 1 -- 172.21.15.1:0/3404907258 wait complete. 2024-09-12T01:29:09.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.557+0000 7f7b17400640 1 --2- 172.21.15.1:0/159833 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f7afc0a2c10 0x7f7afc0a50d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x5610a20f2b30 tx=0x7f7b08005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:09.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.557+0000 7f7b20fda740 1 -- start start 2024-09-12T01:29:09.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.558+0000 7f7b20fda740 1 Processor -- start 2024-09-12T01:29:09.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.558+0000 7f7b20fda740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610a20f5740 0x5610a20fd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:09.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.558+0000 7f7b20fda740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610a20f6060 0x5610a20fdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:09.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.558+0000 7f7b20fda740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610a2100920 0x5610a20fe230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:09.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.558+0000 7f7b20fda740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5610a2103410 con 0x5610a20f5740 2024-09-12T01:29:09.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.558+0000 7f7b20fda740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5610a1fdf920 con 0x5610a20f6060 2024-09-12T01:29:09.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.558+0000 7f7b20fda740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5610a2103590 con 0x5610a2100920 2024-09-12T01:29:09.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.558+0000 7f7b20fda740 10 client.0 _ll_register_callbacks cb 0x5610a20038a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:29:09.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.559+0000 7f7b16a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610a20f6060 0x5610a20fdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:09.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.559+0000 7f7b17400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610a20f5740 0x5610a20fd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:09.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.559+0000 7f7b16a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610a20f6060 0x5610a20fdcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:35674/0 (socket says 172.21.15.1:35674) 2024-09-12T01:29:09.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.559+0000 7f7b16a00640 1 -- 172.21.15.1:0/1509414375 learned_addr learned my addr 172.21.15.1:0/1509414375 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:09.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.559+0000 7f7b17e00640 1 --2- 172.21.15.1:0/1509414375 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610a2100920 0x5610a20fe230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:09.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.559+0000 7f7b16a00640 1 -- 172.21.15.1:0/1509414375 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610a2100920 msgr2=0x5610a20fe230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:09.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.559+0000 7f7b16a00640 1 --2- 172.21.15.1:0/1509414375 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610a2100920 0x5610a20fe230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:09.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.559+0000 7f7b16a00640 1 -- 172.21.15.1:0/1509414375 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610a20f5740 msgr2=0x5610a20fd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:09.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.559+0000 7f7b16a00640 1 --2- 172.21.15.1:0/1509414375 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610a20f5740 0x5610a20fd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:09.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.559+0000 7f7b16a00640 1 -- 172.21.15.1:0/1509414375 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610a1dc82b0 con 0x5610a20f6060 2024-09-12T01:29:09.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.559+0000 7f7b16a00640 1 --2- 172.21.15.1:0/1509414375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610a20f6060 0x5610a20fdcf0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7b040986b0 tx=0x7f7b0409be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e4f7c8a4fb63676 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:09.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.559+0000 7f7b01600640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-12T01:29:09.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.559+0000 7f7b01600640 1 -- 172.21.15.1:0/1509414375 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0409b330 con 0x5610a20f6060 2024-09-12T01:29:09.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.559+0000 7f7b20fda740 1 -- 172.21.15.1:0/1509414375 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b10036040 con 0x5610a20f6060 2024-09-12T01:29:09.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.559+0000 7f7b01600640 1 -- 172.21.15.1:0/1509414375 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f7b04010430 con 0x5610a20f6060 2024-09-12T01:29:09.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.560+0000 7f7b01600640 1 -- 172.21.15.1:0/1509414375 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7b04010630 con 0x5610a20f6060 2024-09-12T01:29:09.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.560+0000 7f7b20fda740 10 client.25034 fetch_fsmap learned FSMap version 28 2024-09-12T01:29:09.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.560+0000 7f7b20fda740 1 -- 172.21.15.1:0/1509414375 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5610a210efc0 con 0x5610a20f6060 2024-09-12T01:29:09.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.560+0000 7f7b01600640 1 -- 172.21.15.1:0/1509414375 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b10036040 con 0x5610a20f6060 2024-09-12T01:29:09.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.560+0000 7f7b01600640 1 -- 172.21.15.1:0/1509414375 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f7b04018bb0 con 0x5610a20f6060 2024-09-12T01:29:09.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.560+0000 7f7b20fda740 10 client.25034 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:29:09.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.560+0000 7f7b20fda740 10 client.25034 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T01:29:09.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.560+0000 7f7b20fda740 20 client.25034 populate_metadata read hostname 'smithi001' 2024-09-12T01:29:09.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.560+0000 7f7b20fda740 1 --2- 172.21.15.1:0/1509414375 >> [v2:172.21.15.104:6838/1459976041,v1:172.21.15.104:6839/1459976041] conn(0x5610a2111550 0x5610a21129c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:09.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.560+0000 7f7b20fda740 4 client.25034 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:29:09.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.560+0000 7f7b20fda740 1 -- 172.21.15.1:0/1509414375 --> [v2:172.21.15.104:6838/1459976041,v1:172.21.15.104:6839/1459976041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610a2064620 con 0x5610a2111550 2024-09-12T01:29:09.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.561+0000 7f7b17400640 1 --2- 172.21.15.1:0/1509414375 >> [v2:172.21.15.104:6838/1459976041,v1:172.21.15.104:6839/1459976041] conn(0x5610a2111550 0x5610a21129c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:09.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.561+0000 7f7b17400640 1 --2- 172.21.15.1:0/1509414375 >> [v2:172.21.15.104:6838/1459976041,v1:172.21.15.104:6839/1459976041] conn(0x5610a2111550 0x5610a21129c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5f7fbd21f8b67a62 server_cookie=2422e1bdcb99e1a4 in_seq=0 out_seq=0 2024-09-12T01:29:09.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.561+0000 7f7b01600640 10 client.25034 ms_handle_connect on v2:172.21.15.104:6838/1459976041 2024-09-12T01:29:09.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.562+0000 7f7b01600640 1 -- 172.21.15.1:0/1509414375 <== mds.1 v2:172.21.15.104:6838/1459976041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5610a2064620 con 0x5610a2111550 2024-09-12T01:29:09.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.562+0000 7f7b01600640 10 client.25034 handle_command_reply: tid=0 2024-09-12T01:29:09.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.562+0000 7f7b20fda740 1 client.25034 shutdown 2024-09-12T01:29:09.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.562+0000 7f7b20fda740 1 -- 172.21.15.1:0/1509414375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610a20f6060 msgr2=0x5610a20fdcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:09.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.562+0000 7f7b20fda740 1 --2- 172.21.15.1:0/1509414375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610a20f6060 0x5610a20fdcf0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7b040986b0 tx=0x7f7b0409be20 comp rx=0 tx=0).stop 2024-09-12T01:29:09.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.562+0000 7f7b20fda740 1 -- 172.21.15.1:0/1509414375 shutdown_connections 2024-09-12T01:29:09.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.562+0000 7f7b20fda740 1 --2- 172.21.15.1:0/1509414375 >> [v2:172.21.15.104:6838/1459976041,v1:172.21.15.104:6839/1459976041] conn(0x5610a2111550 0x5610a21129c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:09.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.562+0000 7f7b20fda740 1 --2- 172.21.15.1:0/1509414375 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610a2100920 0x5610a20fe230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:09.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.563+0000 7f7b20fda740 1 --2- 172.21.15.1:0/1509414375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610a20f6060 0x5610a20fdcf0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:09.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.563+0000 7f7b20fda740 1 --2- 172.21.15.1:0/1509414375 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610a20f5740 0x5610a20fd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:09.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.563+0000 7f7b20fda740 1 -- 172.21.15.1:0/1509414375 >> 172.21.15.1:0/1509414375 conn(0x5610a20f1cd0 msgr2=0x5610a20eca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:09.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.563+0000 7f7b01600640 0 client.25034 ms_handle_reset on v2:172.21.15.104:6838/1459976041 2024-09-12T01:29:09.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.563+0000 7f7b20fda740 1 -- 172.21.15.1:0/1509414375 shutdown_connections 2024-09-12T01:29:09.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.563+0000 7f7b20fda740 1 -- 172.21.15.1:0/1509414375 wait complete. 2024-09-12T01:29:09.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.563+0000 7f7b20fda740 20 client.25034 trim_cache size 0 max 16384 2024-09-12T01:29:09.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.582+0000 7f7b20fda740 1 Processor -- start 2024-09-12T01:29:09.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.582+0000 7f7b20fda740 1 -- start start 2024-09-12T01:29:09.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.582+0000 7f7b20fda740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610a20f5740 0x5610a20f20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:09.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.582+0000 7f7b20fda740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610a20f6060 0x5610a20faa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:09.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.582+0000 7f7b20fda740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610a2100920 0x5610a20faf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:09.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.582+0000 7f7b20fda740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5610a2110d00 con 0x5610a20f5740 2024-09-12T01:29:09.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.582+0000 7f7b20fda740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5610a21113d0 con 0x5610a20f6060 2024-09-12T01:29:09.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.582+0000 7f7b20fda740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5610a2111aa0 con 0x5610a2100920 2024-09-12T01:29:09.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.582+0000 7f7b17400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610a20f5740 0x5610a20f20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:09.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.582+0000 7f7b17400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610a20f5740 0x5610a20f20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40988/0 (socket says 172.21.15.1:40988) 2024-09-12T01:29:09.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.582+0000 7f7b17400640 1 -- 172.21.15.1:0/1843512611 learned_addr learned my addr 172.21.15.1:0/1843512611 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:09.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.582+0000 7f7b16a00640 1 --2- 172.21.15.1:0/1843512611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610a20f6060 0x5610a20faa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:09.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.582+0000 7f7b17e00640 1 --2- 172.21.15.1:0/1843512611 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610a2100920 0x5610a20faf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:09.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.583+0000 7f7b17400640 1 -- 172.21.15.1:0/1843512611 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610a2100920 msgr2=0x5610a20faf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:09.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.583+0000 7f7b17400640 1 --2- 172.21.15.1:0/1843512611 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610a2100920 0x5610a20faf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:09.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.583+0000 7f7b17400640 1 -- 172.21.15.1:0/1843512611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610a20f6060 msgr2=0x5610a20faa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:09.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.583+0000 7f7b17400640 1 --2- 172.21.15.1:0/1843512611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610a20f6060 0x5610a20faa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:09.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.583+0000 7f7b17400640 1 -- 172.21.15.1:0/1843512611 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610a2100120 con 0x5610a20f5740 2024-09-12T01:29:09.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.583+0000 7f7b17400640 1 --2- 172.21.15.1:0/1843512611 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610a20f5740 0x5610a20f20b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x5610a2113190 tx=0x7f7b0801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=169d7ab71f5176bf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:09.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.583+0000 7f7b03e00640 1 -- 172.21.15.1:0/1843512611 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0801dbd0 con 0x5610a20f5740 2024-09-12T01:29:09.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.583+0000 7f7b03e00640 1 -- 172.21.15.1:0/1843512611 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b0801dd70 con 0x5610a20f5740 2024-09-12T01:29:09.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.583+0000 7f7b03e00640 1 -- 172.21.15.1:0/1843512611 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0802b430 con 0x5610a20f5740 2024-09-12T01:29:09.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.584+0000 7f7b20fda740 1 -- 172.21.15.1:0/1843512611 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610a20f5740 msgr2=0x5610a20f20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:09.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.584+0000 7f7b20fda740 1 --2- 172.21.15.1:0/1843512611 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610a20f5740 0x5610a20f20b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x5610a2113190 tx=0x7f7b0801b000 comp rx=0 tx=0).stop 2024-09-12T01:29:09.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.584+0000 7f7b20fda740 1 -- 172.21.15.1:0/1843512611 shutdown_connections 2024-09-12T01:29:09.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.584+0000 7f7b20fda740 1 --2- 172.21.15.1:0/1843512611 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610a2100920 0x5610a20faf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:09.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.584+0000 7f7b20fda740 1 --2- 172.21.15.1:0/1843512611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610a20f6060 0x5610a20faa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:09.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.584+0000 7f7b20fda740 1 --2- 172.21.15.1:0/1843512611 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610a20f5740 0x5610a20f20b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:09.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.584+0000 7f7b20fda740 1 -- 172.21.15.1:0/1843512611 >> 172.21.15.1:0/1843512611 conn(0x5610a20f1cd0 msgr2=0x5610a20fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:09.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.584+0000 7f7b20fda740 1 -- 172.21.15.1:0/1843512611 shutdown_connections 2024-09-12T01:29:09.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.584+0000 7f7b20fda740 1 -- 172.21.15.1:0/1843512611 wait complete. 2024-09-12T01:29:09.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.585+0000 7f7b20fda740 1 -- start start 2024-09-12T01:29:09.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.585+0000 7f7b20fda740 1 Processor -- start 2024-09-12T01:29:09.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.585+0000 7f7b20fda740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610a20f5740 0x5610a2114b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:09.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.585+0000 7f7b20fda740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610a20f6060 0x5610a2115090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:09.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.585+0000 7f7b20fda740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610a2100920 0x5610a21155d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:09.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.585+0000 7f7b20fda740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5610a2112840 con 0x5610a20f6060 2024-09-12T01:29:09.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.585+0000 7f7b20fda740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5610a2112170 con 0x5610a2100920 2024-09-12T01:29:09.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.585+0000 7f7b20fda740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5610a2112f10 con 0x5610a20f5740 2024-09-12T01:29:09.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.585+0000 7f7b20fda740 10 client.0 _ll_register_callbacks cb 0x7f7b0409b640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:29:09.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.586+0000 7f7b17400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610a20f5740 0x5610a2114b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:09.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.586+0000 7f7b16a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610a20f6060 0x5610a2115090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:09.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.586+0000 7f7b17e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610a2100920 0x5610a21155d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:09.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.586+0000 7f7b17400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610a20f5740 0x5610a2114b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:36132/0 (socket says 172.21.15.1:36132) 2024-09-12T01:29:09.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.586+0000 7f7b17400640 1 -- 172.21.15.1:0/1823661883 learned_addr learned my addr 172.21.15.1:0/1823661883 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:09.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.586+0000 7f7b17400640 1 -- 172.21.15.1:0/1823661883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610a2100920 msgr2=0x5610a21155d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:09.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.586+0000 7f7b17400640 1 --2- 172.21.15.1:0/1823661883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610a2100920 0x5610a21155d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:09.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.586+0000 7f7b17400640 1 -- 172.21.15.1:0/1823661883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610a20f6060 msgr2=0x5610a2115090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:09.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.586+0000 7f7b17400640 1 --2- 172.21.15.1:0/1823661883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610a20f6060 0x5610a2115090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.586+0000 7f7b17400640 1 -- 172.21.15.1:0/1823661883 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610a21194c0 con 0x5610a20f5740 2024-09-12T01:29:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.586+0000 7f7b16a00640 1 --2- 172.21.15.1:0/1823661883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610a20f6060 0x5610a2115090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:29:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.586+0000 7f7b17e00640 1 --2- 172.21.15.1:0/1823661883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610a2100920 0x5610a21155d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:29:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.586+0000 7f7b17400640 1 --2- 172.21.15.1:0/1823661883 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610a20f5740 0x5610a2114b50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7b08009460 tx=0x7f7b08004a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cc8cb39d20b3502 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.586+0000 7f7b01600640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-12T01:29:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.587+0000 7f7b01600640 1 -- 172.21.15.1:0/1823661883 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b08004de0 con 0x5610a20f5740 2024-09-12T01:29:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.587+0000 7f7b20fda740 1 -- 172.21.15.1:0/1823661883 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b0801dd70 con 0x5610a20f5740 2024-09-12T01:29:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.587+0000 7f7b01600640 1 -- 172.21.15.1:0/1823661883 <== mon.1 v2:172.21.15.104:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f7b08004f80 con 0x5610a20f5740 2024-09-12T01:29:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.587+0000 7f7b01600640 1 -- 172.21.15.1:0/1823661883 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7b08044050 con 0x5610a20f5740 2024-09-12T01:29:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.587+0000 7f7b20fda740 10 client.25027 fetch_fsmap learned FSMap version 28 2024-09-12T01:29:09.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.587+0000 7f7b20fda740 1 -- 172.21.15.1:0/1823661883 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5610a217c0c0 con 0x5610a20f5740 2024-09-12T01:29:09.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.587+0000 7f7b01600640 1 -- 172.21.15.1:0/1823661883 <== mon.1 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b0801dd70 con 0x5610a20f5740 2024-09-12T01:29:09.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.588+0000 7f7b01600640 1 -- 172.21.15.1:0/1823661883 <== mon.1 v2:172.21.15.104:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f7b08033440 con 0x5610a20f5740 2024-09-12T01:29:09.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.588+0000 7f7b20fda740 10 client.25027 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:29:09.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.588+0000 7f7b20fda740 10 client.25027 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T01:29:09.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.588+0000 7f7b20fda740 20 client.25027 populate_metadata read hostname 'smithi001' 2024-09-12T01:29:09.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.588+0000 7f7b20fda740 1 --2- 172.21.15.1:0/1823661883 >> [v2:172.21.15.104:6838/1459976041,v1:172.21.15.104:6839/1459976041] conn(0x5610a2180a00 0x5610a217f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:09.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.588+0000 7f7b20fda740 4 client.25027 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T01:29:09.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.588+0000 7f7b20fda740 1 -- 172.21.15.1:0/1823661883 --> [v2:172.21.15.104:6838/1459976041,v1:172.21.15.104:6839/1459976041] -- command(tid 0: {"prefix": "status"}) -- 0x5610a205eb30 con 0x5610a2180a00 2024-09-12T01:29:09.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.588+0000 7f7b16a00640 1 --2- 172.21.15.1:0/1823661883 >> [v2:172.21.15.104:6838/1459976041,v1:172.21.15.104:6839/1459976041] conn(0x5610a2180a00 0x5610a217f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:09.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.589+0000 7f7b16a00640 1 --2- 172.21.15.1:0/1823661883 >> [v2:172.21.15.104:6838/1459976041,v1:172.21.15.104:6839/1459976041] conn(0x5610a2180a00 0x5610a217f710 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=783b13c17c317e01 server_cookie=7bb8dc70dc3f52d1 in_seq=0 out_seq=0 2024-09-12T01:29:09.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.589+0000 7f7b01600640 10 client.25027 ms_handle_connect on v2:172.21.15.104:6838/1459976041 2024-09-12T01:29:09.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.589+0000 7f7b01600640 1 -- 172.21.15.1:0/1823661883 <== mds.1 v2:172.21.15.104:6838/1459976041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5610a205eb30 con 0x5610a2180a00 2024-09-12T01:29:09.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.589+0000 7f7b01600640 10 client.25027 handle_command_reply: tid=0 2024-09-12T01:29:09.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.589+0000 7f7b20fda740 1 client.25027 shutdown 2024-09-12T01:29:09.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.589+0000 7f7b20fda740 1 -- 172.21.15.1:0/1823661883 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610a20f5740 msgr2=0x5610a2114b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:09.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.589+0000 7f7b20fda740 1 --2- 172.21.15.1:0/1823661883 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610a20f5740 0x5610a2114b50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7b08009460 tx=0x7f7b08004a00 comp rx=0 tx=0).stop 2024-09-12T01:29:09.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.589+0000 7f7b20fda740 1 -- 172.21.15.1:0/1823661883 shutdown_connections 2024-09-12T01:29:09.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.589+0000 7f7b20fda740 1 --2- 172.21.15.1:0/1823661883 >> [v2:172.21.15.104:6838/1459976041,v1:172.21.15.104:6839/1459976041] conn(0x5610a2180a00 0x5610a217f710 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:09.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.589+0000 7f7b20fda740 1 --2- 172.21.15.1:0/1823661883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610a2100920 0x5610a21155d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:09.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.589+0000 7f7b20fda740 1 --2- 172.21.15.1:0/1823661883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610a20f6060 0x5610a2115090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:09.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.589+0000 7f7b20fda740 1 --2- 172.21.15.1:0/1823661883 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5610a20f5740 0x5610a2114b50 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:09.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.589+0000 7f7b20fda740 1 -- 172.21.15.1:0/1823661883 >> 172.21.15.1:0/1823661883 conn(0x5610a20f1cd0 msgr2=0x5610a20fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:09.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.589+0000 7f7b01600640 0 client.25027 ms_handle_reset on v2:172.21.15.104:6838/1459976041 2024-09-12T01:29:09.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.589+0000 7f7b20fda740 1 -- 172.21.15.1:0/1823661883 shutdown_connections 2024-09-12T01:29:09.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.590+0000 7f7b20fda740 1 -- 172.21.15.1:0/1823661883 wait complete. 2024-09-12T01:29:09.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.590+0000 7f7b20fda740 20 client.25027 trim_cache size 0 max 16384 2024-09-12T01:29:09.595 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-12T01:29:09.595 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "c0e4f180-70a4-11ef-bcea-c7b262605968", 2024-09-12T01:29:09.595 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 1, 2024-09-12T01:29:09.595 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24511, 2024-09-12T01:29:09.595 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-09-12T01:29:09.595 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-09-12T01:29:09.595 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-09-12T01:29:09.595 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 88.298729523000006, 2024-09-12T01:29:09.595 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 28, 2024-09-12T01:29:09.595 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 98, 2024-09-12T01:29:09.595 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 96, 2024-09-12T01:29:09.595 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 165.302842849 2024-09-12T01:29:09.596 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-12T01:29:09.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.590+0000 7f7b1e400640 1 -- 172.21.15.1:0/159833 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f7afc0a2c10 msgr2=0x7f7afc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:09.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.590+0000 7f7b1e400640 1 --2- 172.21.15.1:0/159833 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f7afc0a2c10 0x7f7afc0a50d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x5610a20f2b30 tx=0x7f7b08005f90 comp rx=0 tx=0).stop 2024-09-12T01:29:09.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.590+0000 7f7b1e400640 1 -- 172.21.15.1:0/159833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b18102790 msgr2=0x7f7b181989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:09.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.590+0000 7f7b1e400640 1 --2- 172.21.15.1:0/159833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b18102790 0x7f7b181989b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7b0400e9f0 tx=0x7f7b0400eeb0 comp rx=0 tx=0).stop 2024-09-12T01:29:09.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.590+0000 7f7b1e400640 1 -- 172.21.15.1:0/159833 shutdown_connections 2024-09-12T01:29:09.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.590+0000 7f7b1e400640 1 --2- 172.21.15.1:0/159833 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f7afc0a2c10 0x7f7afc0a50d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:09.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.590+0000 7f7b1e400640 1 --2- 172.21.15.1:0/159833 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b18106f10 0x7f7b1819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:09.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.590+0000 7f7b1e400640 1 --2- 172.21.15.1:0/159833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b18102790 0x7f7b181989b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:09.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.590+0000 7f7b1e400640 1 --2- 172.21.15.1:0/159833 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b18101de0 0x7f7b18198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:09.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.590+0000 7f7b1e400640 1 -- 172.21.15.1:0/159833 >> 172.21.15.1:0/159833 conn(0x7f7b180fd8c0 msgr2=0x7f7b180feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:09.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.591+0000 7f7b1e400640 1 -- 172.21.15.1:0/159833 shutdown_connections 2024-09-12T01:29:09.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:09.591+0000 7f7b1e400640 1 -- 172.21.15.1:0/159833 wait complete. 2024-09-12T01:29:10.553 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-12T01:29:10.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:10 smithi001 ceph-mon[30387]: pgmap v385: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T01:29:10.937 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:29:10.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:10 smithi053 ceph-mon[35158]: pgmap v385: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T01:29:11.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:10 smithi104 ceph-mon[35219]: pgmap v385: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T01:29:11.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.494+0000 7f8118a00640 1 -- 172.21.15.1:0/3736050831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81141019d0 msgr2=0x7f8114101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:11.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.494+0000 7f8118a00640 1 --2- 172.21.15.1:0/3736050831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81141019d0 0x7f8114101e30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8100009900 tx=0x7f810002a3f0 comp rx=0 tx=0).stop 2024-09-12T01:29:11.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.494+0000 7f8118a00640 1 -- 172.21.15.1:0/3736050831 shutdown_connections 2024-09-12T01:29:11.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.494+0000 7f8118a00640 1 --2- 172.21.15.1:0/3736050831 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8114102370 0x7f811410e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:11.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.494+0000 7f8118a00640 1 --2- 172.21.15.1:0/3736050831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81141019d0 0x7f8114101e30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:11.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.494+0000 7f8118a00640 1 --2- 172.21.15.1:0/3736050831 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81141079d0 0x7f8114107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:11.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.494+0000 7f8118a00640 1 -- 172.21.15.1:0/3736050831 >> 172.21.15.1:0/3736050831 conn(0x7f81140fd710 msgr2=0x7f81140ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:11.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.494+0000 7f8118a00640 1 -- 172.21.15.1:0/3736050831 shutdown_connections 2024-09-12T01:29:11.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.494+0000 7f8118a00640 1 -- 172.21.15.1:0/3736050831 wait complete. 2024-09-12T01:29:11.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.496+0000 7f8118a00640 1 Processor -- start 2024-09-12T01:29:11.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.496+0000 7f8118a00640 1 -- start start 2024-09-12T01:29:11.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.496+0000 7f8118a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81141019d0 0x7f811419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:11.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.496+0000 7f8118a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8114102370 0x7f811419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:11.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.496+0000 7f8118a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81141079d0 0x7f811419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:11.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.496+0000 7f8118a00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8114111de0 con 0x7f8114102370 2024-09-12T01:29:11.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.496+0000 7f8118a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8114111c60 con 0x7f81141079d0 2024-09-12T01:29:11.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.496+0000 7f8118a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8114111f60 con 0x7f81141019d0 2024-09-12T01:29:11.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.497+0000 7f8112a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8114102370 0x7f811419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:11.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.497+0000 7f8112a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8114102370 0x7f811419aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41018/0 (socket says 172.21.15.1:41018) 2024-09-12T01:29:11.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.497+0000 7f8112a00640 1 -- 172.21.15.1:0/3305048876 learned_addr learned my addr 172.21.15.1:0/3305048876 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:11.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.497+0000 7f8113400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81141019d0 0x7f811419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:11.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.497+0000 7f8113e00640 1 --2- 172.21.15.1:0/3305048876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81141079d0 0x7f811419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:11.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.497+0000 7f8112a00640 1 -- 172.21.15.1:0/3305048876 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81141019d0 msgr2=0x7f811419a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:11.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.497+0000 7f8112a00640 1 --2- 172.21.15.1:0/3305048876 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81141019d0 0x7f811419a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:11.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.497+0000 7f8112a00640 1 -- 172.21.15.1:0/3305048876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81141079d0 msgr2=0x7f811419eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:11.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.497+0000 7f8112a00640 1 --2- 172.21.15.1:0/3305048876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81141079d0 0x7f811419eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:11.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.497+0000 7f8112a00640 1 -- 172.21.15.1:0/3305048876 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f811419f480 con 0x7f8114102370 2024-09-12T01:29:11.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.497+0000 7f8113400640 1 --2- 172.21.15.1:0/3305048876 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81141019d0 0x7f811419a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:29:11.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.498+0000 7f8112a00640 1 --2- 172.21.15.1:0/3305048876 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8114102370 0x7f811419aa10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f810002a9e0 tx=0x7f8100003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:11.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.498+0000 7f810be00640 1 -- 172.21.15.1:0/3305048876 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810003f070 con 0x7f8114102370 2024-09-12T01:29:11.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.498+0000 7f8118a00640 1 -- 172.21.15.1:0/3305048876 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f811419f710 con 0x7f8114102370 2024-09-12T01:29:11.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.498+0000 7f810be00640 1 -- 172.21.15.1:0/3305048876 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8100031c30 con 0x7f8114102370 2024-09-12T01:29:11.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.498+0000 7f810be00640 1 -- 172.21.15.1:0/3305048876 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8100035030 con 0x7f8114102370 2024-09-12T01:29:11.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.498+0000 7f8118a00640 1 -- 172.21.15.1:0/3305048876 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81141a7050 con 0x7f8114102370 2024-09-12T01:29:11.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.502+0000 7f811cc66740 1 Processor -- start 2024-09-12T01:29:11.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.502+0000 7f810be00640 1 -- 172.21.15.1:0/3305048876 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f8100042070 con 0x7f8114102370 2024-09-12T01:29:11.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.502+0000 7f811cc66740 1 -- start start 2024-09-12T01:29:11.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.502+0000 7f811cc66740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c1bba1d4e0 0x55c1bba1d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:11.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.502+0000 7f811cc66740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1bba1de00 0x55c1bba28180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:11.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.502+0000 7f811cc66740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1bba286c0 0x55c1bba2aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:11.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.502+0000 7f811cc66740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c1bb9830b0 con 0x55c1bba1d4e0 2024-09-12T01:29:11.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.502+0000 7f811cc66740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55c1bb92aab0 con 0x55c1bba1de00 2024-09-12T01:29:11.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.502+0000 7f811cc66740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c1bb97f070 con 0x55c1bba286c0 2024-09-12T01:29:11.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.503+0000 7f8113e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c1bba1d4e0 0x55c1bba1d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:11.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.503+0000 7f8113e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c1bba1d4e0 0x55c1bba1d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41034/0 (socket says 172.21.15.1:41034) 2024-09-12T01:29:11.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.503+0000 7f8113e00640 1 -- 172.21.15.1:0/2527126216 learned_addr learned my addr 172.21.15.1:0/2527126216 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:11.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.503+0000 7f8112a00640 1 --2- 172.21.15.1:0/2527126216 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1bba286c0 0x55c1bba2aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:11.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.503+0000 7f8113400640 1 --2- 172.21.15.1:0/2527126216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1bba1de00 0x55c1bba28180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:11.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.503+0000 7f8113e00640 1 -- 172.21.15.1:0/2527126216 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1bba286c0 msgr2=0x55c1bba2aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:11.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.503+0000 7f8113e00640 1 --2- 172.21.15.1:0/2527126216 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1bba286c0 0x55c1bba2aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:11.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.503+0000 7f8113e00640 1 -- 172.21.15.1:0/2527126216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1bba1de00 msgr2=0x55c1bba28180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:11.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.503+0000 7f8113e00640 1 --2- 172.21.15.1:0/2527126216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1bba1de00 0x55c1bba28180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:11.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.503+0000 7f8113e00640 1 -- 172.21.15.1:0/2527126216 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1bb9374a0 con 0x55c1bba1d4e0 2024-09-12T01:29:11.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.503+0000 7f8112a00640 1 --2- 172.21.15.1:0/2527126216 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1bba286c0 0x55c1bba2aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:29:11.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.503+0000 7f8113e00640 1 --2- 172.21.15.1:0/2527126216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c1bba1d4e0 0x55c1bba1d8c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f810c009fd0 tx=0x7f810c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad57ab2df816ab2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:11.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.504+0000 7f810be00640 1 --2- 172.21.15.1:0/3305048876 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f80f80a2c90 0x7f80f80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:11.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.504+0000 7f810be00640 1 -- 172.21.15.1:0/3305048876 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f81000ef770 con 0x7f8114102370 2024-09-12T01:29:11.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.504+0000 7f80ffe00640 1 -- 172.21.15.1:0/2527126216 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c004d20 con 0x55c1bba1d4e0 2024-09-12T01:29:11.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.504+0000 7f80ffe00640 1 -- 172.21.15.1:0/2527126216 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f810c004050 con 0x55c1bba1d4e0 2024-09-12T01:29:11.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.504+0000 7f80ffe00640 1 -- 172.21.15.1:0/2527126216 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c005260 con 0x55c1bba1d4e0 2024-09-12T01:29:11.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.504+0000 7f8113400640 1 --2- 172.21.15.1:0/3305048876 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f80f80a2c90 0x7f80f80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:11.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.505+0000 7f811cc66740 1 -- 172.21.15.1:0/2527126216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c1bba1d4e0 msgr2=0x55c1bba1d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:11.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.505+0000 7f811cc66740 1 --2- 172.21.15.1:0/2527126216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c1bba1d4e0 0x55c1bba1d8c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f810c009fd0 tx=0x7f810c0118d0 comp rx=0 tx=0).stop 2024-09-12T01:29:11.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.505+0000 7f8113400640 1 --2- 172.21.15.1:0/3305048876 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f80f80a2c90 0x7f80f80a5150 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x55c1bba1a8d0 tx=0x7f8104005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:11.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.505+0000 7f811cc66740 1 -- 172.21.15.1:0/2527126216 shutdown_connections 2024-09-12T01:29:11.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.505+0000 7f811cc66740 1 --2- 172.21.15.1:0/2527126216 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1bba286c0 0x55c1bba2aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:11.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.505+0000 7f811cc66740 1 --2- 172.21.15.1:0/2527126216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1bba1de00 0x55c1bba28180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:11.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.505+0000 7f811cc66740 1 --2- 172.21.15.1:0/2527126216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c1bba1d4e0 0x55c1bba1d8c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:11.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.505+0000 7f811cc66740 1 -- 172.21.15.1:0/2527126216 >> 172.21.15.1:0/2527126216 conn(0x55c1bba19a70 msgr2=0x55c1bba19e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:11.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.505+0000 7f811cc66740 1 -- 172.21.15.1:0/2527126216 shutdown_connections 2024-09-12T01:29:11.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.505+0000 7f811cc66740 1 -- 172.21.15.1:0/2527126216 wait complete. 2024-09-12T01:29:11.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.506+0000 7f811cc66740 1 -- start start 2024-09-12T01:29:11.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.506+0000 7f811cc66740 1 Processor -- start 2024-09-12T01:29:11.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.506+0000 7f811cc66740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c1bba1d4e0 0x55c1bba255b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:11.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.506+0000 7f811cc66740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1bba1de00 0x55c1bba25af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:11.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.506+0000 7f811cc66740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1bba286c0 0x55c1bba26030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:11.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.506+0000 7f811cc66740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c1bba2b330 con 0x55c1bba1d4e0 2024-09-12T01:29:11.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.506+0000 7f811cc66740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55c1bb90b920 con 0x55c1bba286c0 2024-09-12T01:29:11.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.506+0000 7f811cc66740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c1bba2b4b0 con 0x55c1bba1de00 2024-09-12T01:29:11.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.507+0000 7f811cc66740 10 client.0 _ll_register_callbacks cb 0x55c1bb92f8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:29:11.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.507+0000 7f8112a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1bba1de00 0x55c1bba25af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:11.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.507+0000 7f8112a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1bba1de00 0x55c1bba25af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:36184/0 (socket says 172.21.15.1:36184) 2024-09-12T01:29:11.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.507+0000 7f8113400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c1bba1d4e0 0x55c1bba255b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:11.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.507+0000 7f8112a00640 1 -- 172.21.15.1:0/288972285 learned_addr learned my addr 172.21.15.1:0/288972285 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:11.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.507+0000 7f8113e00640 1 --2- 172.21.15.1:0/288972285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1bba286c0 0x55c1bba26030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:11.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.507+0000 7f8112a00640 1 -- 172.21.15.1:0/288972285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1bba286c0 msgr2=0x55c1bba26030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:11.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.507+0000 7f8112a00640 1 --2- 172.21.15.1:0/288972285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1bba286c0 0x55c1bba26030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:11.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.507+0000 7f8112a00640 1 -- 172.21.15.1:0/288972285 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c1bba1d4e0 msgr2=0x55c1bba255b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:11.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.507+0000 7f8112a00640 1 --2- 172.21.15.1:0/288972285 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c1bba1d4e0 0x55c1bba255b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:11.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.507+0000 7f8112a00640 1 -- 172.21.15.1:0/288972285 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1bb6f42b0 con 0x55c1bba1de00 2024-09-12T01:29:11.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.507+0000 7f8113400640 1 --2- 172.21.15.1:0/288972285 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c1bba1d4e0 0x55c1bba255b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:29:11.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.507+0000 7f8112a00640 1 --2- 172.21.15.1:0/288972285 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1bba1de00 0x55c1bba25af0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8100006010 tx=0x7f8100009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2afa972da7ccab09 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:11.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.507+0000 7f80fcc00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-12T01:29:11.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.507+0000 7f80fcc00640 1 -- 172.21.15.1:0/288972285 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8100049070 con 0x55c1bba1de00 2024-09-12T01:29:11.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.508+0000 7f811cc66740 1 -- 172.21.15.1:0/288972285 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f810c004050 con 0x55c1bba1de00 2024-09-12T01:29:11.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.508+0000 7f80fcc00640 1 -- 172.21.15.1:0/288972285 <== mon.1 v2:172.21.15.104:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f81000ecf80 con 0x55c1bba1de00 2024-09-12T01:29:11.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.508+0000 7f80fcc00640 1 -- 172.21.15.1:0/288972285 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f81000bb900 con 0x55c1bba1de00 2024-09-12T01:29:11.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.508+0000 7f811cc66740 10 client.25039 fetch_fsmap learned FSMap version 28 2024-09-12T01:29:11.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.508+0000 7f811cc66740 1 -- 172.21.15.1:0/288972285 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c1bba2b390 con 0x55c1bba1de00 2024-09-12T01:29:11.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.508+0000 7f80fcc00640 1 -- 172.21.15.1:0/288972285 <== mon.1 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f810c004050 con 0x55c1bba1de00 2024-09-12T01:29:11.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.508+0000 7f80fcc00640 1 -- 172.21.15.1:0/288972285 <== mon.1 v2:172.21.15.104:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f810004c060 con 0x55c1bba1de00 2024-09-12T01:29:11.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.508+0000 7f811cc66740 10 client.25039 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:29:11.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.508+0000 7f811cc66740 10 client.25039 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T01:29:11.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.508+0000 7f811cc66740 20 client.25039 populate_metadata read hostname 'smithi001' 2024-09-12T01:29:11.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.508+0000 7f811cc66740 1 --2- 172.21.15.1:0/288972285 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55c1bba39660 0x55c1bba3aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:11.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.508+0000 7f811cc66740 4 client.25039 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:29:11.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.508+0000 7f811cc66740 1 -- 172.21.15.1:0/288972285 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1bb990620 con 0x55c1bba39660 2024-09-12T01:29:11.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.509+0000 7f8113400640 1 --2- 172.21.15.1:0/288972285 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55c1bba39660 0x55c1bba3aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:11.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.509+0000 7f8113400640 1 --2- 172.21.15.1:0/288972285 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55c1bba39660 0x55c1bba3aad0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99fde2e96e837d7f server_cookie=208f36bbb2686965 in_seq=0 out_seq=0 2024-09-12T01:29:11.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.509+0000 7f80fcc00640 10 client.25039 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:29:11.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.510+0000 7f80fcc00640 1 -- 172.21.15.1:0/288972285 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c1bb990620 con 0x55c1bba39660 2024-09-12T01:29:11.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.510+0000 7f80fcc00640 10 client.25039 handle_command_reply: tid=0 2024-09-12T01:29:11.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.510+0000 7f811cc66740 1 client.25039 shutdown 2024-09-12T01:29:11.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.510+0000 7f811cc66740 1 -- 172.21.15.1:0/288972285 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1bba1de00 msgr2=0x55c1bba25af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:11.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.510+0000 7f811cc66740 1 --2- 172.21.15.1:0/288972285 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1bba1de00 0x55c1bba25af0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8100006010 tx=0x7f8100009560 comp rx=0 tx=0).stop 2024-09-12T01:29:11.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.511+0000 7f811cc66740 1 -- 172.21.15.1:0/288972285 shutdown_connections 2024-09-12T01:29:11.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.511+0000 7f811cc66740 1 --2- 172.21.15.1:0/288972285 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55c1bba39660 0x55c1bba3aad0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:11.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.511+0000 7f811cc66740 1 --2- 172.21.15.1:0/288972285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1bba286c0 0x55c1bba26030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:11.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.511+0000 7f80fcc00640 0 client.25039 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:29:11.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.511+0000 7f811cc66740 1 --2- 172.21.15.1:0/288972285 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1bba1de00 0x55c1bba25af0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:11.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.511+0000 7f811cc66740 1 --2- 172.21.15.1:0/288972285 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c1bba1d4e0 0x55c1bba255b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:11.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.511+0000 7f811cc66740 1 -- 172.21.15.1:0/288972285 >> 172.21.15.1:0/288972285 conn(0x55c1bba19a70 msgr2=0x55c1bba1f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:11.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.511+0000 7f811cc66740 1 -- 172.21.15.1:0/288972285 shutdown_connections 2024-09-12T01:29:11.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.511+0000 7f811cc66740 1 -- 172.21.15.1:0/288972285 wait complete. 2024-09-12T01:29:11.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.511+0000 7f811cc66740 20 client.25039 trim_cache size 0 max 16384 2024-09-12T01:29:11.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.538+0000 7f811cc66740 1 Processor -- start 2024-09-12T01:29:11.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.538+0000 7f811cc66740 1 -- start start 2024-09-12T01:29:11.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.538+0000 7f811cc66740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1bba1d4e0 0x55c1bba1ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:11.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.538+0000 7f811cc66740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c1bba1de00 0x55c1bba23e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:11.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.538+0000 7f811cc66740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1bba286c0 0x55c1bba243c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:11.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.538+0000 7f811cc66740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c1bba38e10 con 0x55c1bba1de00 2024-09-12T01:29:11.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.538+0000 7f811cc66740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55c1bba394e0 con 0x55c1bba1d4e0 2024-09-12T01:29:11.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.538+0000 7f811cc66740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c1bba39bb0 con 0x55c1bba286c0 2024-09-12T01:29:11.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.539+0000 7f8113400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1bba1d4e0 0x55c1bba1ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:11.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.539+0000 7f8112a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c1bba1de00 0x55c1bba23e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:11.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.539+0000 7f8113400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1bba1d4e0 0x55c1bba1ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:35714/0 (socket says 172.21.15.1:35714) 2024-09-12T01:29:11.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.539+0000 7f8113400640 1 -- 172.21.15.1:0/90533489 learned_addr learned my addr 172.21.15.1:0/90533489 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:11.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.539+0000 7f8113e00640 1 --2- 172.21.15.1:0/90533489 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1bba286c0 0x55c1bba243c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:11.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.539+0000 7f8112a00640 1 -- 172.21.15.1:0/90533489 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1bba286c0 msgr2=0x55c1bba243c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:11.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.539+0000 7f8112a00640 1 --2- 172.21.15.1:0/90533489 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1bba286c0 0x55c1bba243c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:11.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.539+0000 7f8112a00640 1 -- 172.21.15.1:0/90533489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1bba1d4e0 msgr2=0x55c1bba1ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:11.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.539+0000 7f8112a00640 1 --2- 172.21.15.1:0/90533489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1bba1d4e0 0x55c1bba1ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:11.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.539+0000 7f8112a00640 1 -- 172.21.15.1:0/90533489 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1bba23b00 con 0x55c1bba1de00 2024-09-12T01:29:11.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.539+0000 7f8113400640 1 --2- 172.21.15.1:0/90533489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1bba1d4e0 0x55c1bba1ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:29:11.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.539+0000 7f8112a00640 1 --2- 172.21.15.1:0/90533489 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c1bba1de00 0x55c1bba23e80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f81000b8500 tx=0x7f81000bff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6303d39814639fbf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:11.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.540+0000 7f80ff400640 1 -- 172.21.15.1:0/90533489 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8100049040 con 0x55c1bba1de00 2024-09-12T01:29:11.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.540+0000 7f80ff400640 1 -- 172.21.15.1:0/90533489 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8100031de0 con 0x55c1bba1de00 2024-09-12T01:29:11.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.540+0000 7f80ff400640 1 -- 172.21.15.1:0/90533489 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8100005d50 con 0x55c1bba1de00 2024-09-12T01:29:11.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.540+0000 7f811cc66740 1 -- 172.21.15.1:0/90533489 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c1bba1de00 msgr2=0x55c1bba23e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:11.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.540+0000 7f811cc66740 1 --2- 172.21.15.1:0/90533489 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c1bba1de00 0x55c1bba23e80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f81000b8500 tx=0x7f81000bff90 comp rx=0 tx=0).stop 2024-09-12T01:29:11.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.541+0000 7f811cc66740 1 -- 172.21.15.1:0/90533489 shutdown_connections 2024-09-12T01:29:11.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.541+0000 7f811cc66740 1 --2- 172.21.15.1:0/90533489 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1bba286c0 0x55c1bba243c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:11.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.541+0000 7f811cc66740 1 --2- 172.21.15.1:0/90533489 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c1bba1de00 0x55c1bba23e80 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:11.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.541+0000 7f811cc66740 1 --2- 172.21.15.1:0/90533489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1bba1d4e0 0x55c1bba1ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:11.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.541+0000 7f811cc66740 1 -- 172.21.15.1:0/90533489 >> 172.21.15.1:0/90533489 conn(0x55c1bba19a70 msgr2=0x55c1bba38210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:11.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.541+0000 7f811cc66740 1 -- 172.21.15.1:0/90533489 shutdown_connections 2024-09-12T01:29:11.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.541+0000 7f811cc66740 1 -- 172.21.15.1:0/90533489 wait complete. 2024-09-12T01:29:11.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.541+0000 7f811cc66740 1 -- start start 2024-09-12T01:29:11.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.541+0000 7f811cc66740 1 Processor -- start 2024-09-12T01:29:11.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.542+0000 7f811cc66740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c1bba1d4e0 0x55c1bba43450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:11.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.542+0000 7f811cc66740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1bba1de00 0x55c1bba22220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:11.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.542+0000 7f811cc66740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1bba286c0 0x55c1bba22760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:11.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.542+0000 7f811cc66740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c1bba3a950 con 0x55c1bba1d4e0 2024-09-12T01:29:11.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.542+0000 7f811cc66740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55c1bba3a280 con 0x55c1bba286c0 2024-09-12T01:29:11.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.542+0000 7f811cc66740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c1bba3b020 con 0x55c1bba1de00 2024-09-12T01:29:11.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.542+0000 7f811cc66740 10 client.0 _ll_register_callbacks cb 0x7f8100009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:29:11.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.542+0000 7f8113400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c1bba1d4e0 0x55c1bba43450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:11.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.542+0000 7f8112a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1bba1de00 0x55c1bba22220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:11.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.542+0000 7f8113400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c1bba1d4e0 0x55c1bba43450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41052/0 (socket says 172.21.15.1:41052) 2024-09-12T01:29:11.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.542+0000 7f8113e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1bba286c0 0x55c1bba22760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:11.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.542+0000 7f8113400640 1 -- 172.21.15.1:0/3466199356 learned_addr learned my addr 172.21.15.1:0/3466199356 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:11.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.543+0000 7f8112a00640 1 -- 172.21.15.1:0/3466199356 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1bba286c0 msgr2=0x55c1bba22760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:11.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.543+0000 7f8112a00640 1 --2- 172.21.15.1:0/3466199356 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1bba286c0 0x55c1bba22760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:11.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.543+0000 7f8112a00640 1 -- 172.21.15.1:0/3466199356 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c1bba1d4e0 msgr2=0x55c1bba43450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:11.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.543+0000 7f8112a00640 1 --2- 172.21.15.1:0/3466199356 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c1bba1d4e0 0x55c1bba43450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:11.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.543+0000 7f8112a00640 1 -- 172.21.15.1:0/3466199356 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1bba22f70 con 0x55c1bba1de00 2024-09-12T01:29:11.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.543+0000 7f8113400640 1 --2- 172.21.15.1:0/3466199356 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c1bba1d4e0 0x55c1bba43450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:29:11.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.543+0000 7f8112a00640 1 --2- 172.21.15.1:0/3466199356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1bba1de00 0x55c1bba22220 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f81000b8500 tx=0x7f81000bb1b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7c0568a69e5a37 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:11.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.543+0000 7f80fcc00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-12T01:29:11.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.543+0000 7f80fcc00640 1 -- 172.21.15.1:0/3466199356 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8100049070 con 0x55c1bba1de00 2024-09-12T01:29:11.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.543+0000 7f80fcc00640 1 -- 172.21.15.1:0/3466199356 <== mon.1 v2:172.21.15.104:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f8100031960 con 0x55c1bba1de00 2024-09-12T01:29:11.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.543+0000 7f811cc66740 1 -- 172.21.15.1:0/3466199356 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8100031de0 con 0x55c1bba1de00 2024-09-12T01:29:11.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.544+0000 7f80fcc00640 1 -- 172.21.15.1:0/3466199356 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f81000bb490 con 0x55c1bba1de00 2024-09-12T01:29:11.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.544+0000 7f811cc66740 10 client.25045 fetch_fsmap learned FSMap version 28 2024-09-12T01:29:11.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.544+0000 7f811cc66740 1 -- 172.21.15.1:0/3466199356 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c1bba22f70 con 0x55c1bba1de00 2024-09-12T01:29:11.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.544+0000 7f80fcc00640 1 -- 172.21.15.1:0/3466199356 <== mon.1 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8100031de0 con 0x55c1bba1de00 2024-09-12T01:29:11.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.544+0000 7f80fcc00640 1 -- 172.21.15.1:0/3466199356 <== mon.1 v2:172.21.15.104:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f81000ba9f0 con 0x55c1bba1de00 2024-09-12T01:29:11.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.544+0000 7f811cc66740 10 client.25045 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:29:11.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.544+0000 7f811cc66740 10 client.25045 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T01:29:11.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.544+0000 7f811cc66740 20 client.25045 populate_metadata read hostname 'smithi001' 2024-09-12T01:29:11.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.544+0000 7f811cc66740 1 --2- 172.21.15.1:0/3466199356 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55c1bbaa80a0 0x55c1bbaa6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:11.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.544+0000 7f811cc66740 4 client.25045 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T01:29:11.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.544+0000 7f811cc66740 1 -- 172.21.15.1:0/3466199356 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "status"}) -- 0x55c1bb98ab30 con 0x55c1bbaa80a0 2024-09-12T01:29:11.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.545+0000 7f8113400640 1 --2- 172.21.15.1:0/3466199356 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55c1bbaa80a0 0x55c1bbaa6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:11.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.545+0000 7f8113400640 1 --2- 172.21.15.1:0/3466199356 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55c1bbaa80a0 0x55c1bbaa6db0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85e7d4ca56e93549 server_cookie=fb85ad861c73d084 in_seq=0 out_seq=0 2024-09-12T01:29:11.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.545+0000 7f80fcc00640 10 client.25045 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:29:11.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.545+0000 7f80fcc00640 1 -- 172.21.15.1:0/3466199356 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55c1bb98ab30 con 0x55c1bbaa80a0 2024-09-12T01:29:11.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.545+0000 7f80fcc00640 10 client.25045 handle_command_reply: tid=0 2024-09-12T01:29:11.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.545+0000 7f811cc66740 1 client.25045 shutdown 2024-09-12T01:29:11.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.546+0000 7f811cc66740 1 -- 172.21.15.1:0/3466199356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1bba1de00 msgr2=0x55c1bba22220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:11.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.546+0000 7f811cc66740 1 --2- 172.21.15.1:0/3466199356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1bba1de00 0x55c1bba22220 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f81000b8500 tx=0x7f81000bb1b0 comp rx=0 tx=0).stop 2024-09-12T01:29:11.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.546+0000 7f811cc66740 1 -- 172.21.15.1:0/3466199356 shutdown_connections 2024-09-12T01:29:11.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.546+0000 7f811cc66740 1 --2- 172.21.15.1:0/3466199356 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55c1bbaa80a0 0x55c1bbaa6db0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:11.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.546+0000 7f811cc66740 1 --2- 172.21.15.1:0/3466199356 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1bba286c0 0x55c1bba22760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:11.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.546+0000 7f811cc66740 1 --2- 172.21.15.1:0/3466199356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1bba1de00 0x55c1bba22220 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:11.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.546+0000 7f811cc66740 1 --2- 172.21.15.1:0/3466199356 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c1bba1d4e0 0x55c1bba43450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:11.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.546+0000 7f80fcc00640 0 client.25045 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:29:11.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.546+0000 7f811cc66740 1 -- 172.21.15.1:0/3466199356 >> 172.21.15.1:0/3466199356 conn(0x55c1bba19a70 msgr2=0x55c1bba23d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:11.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.546+0000 7f811cc66740 1 -- 172.21.15.1:0/3466199356 shutdown_connections 2024-09-12T01:29:11.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.546+0000 7f811cc66740 1 -- 172.21.15.1:0/3466199356 wait complete. 2024-09-12T01:29:11.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.546+0000 7f811cc66740 20 client.25045 trim_cache size 0 max 16384 2024-09-12T01:29:11.553 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-12T01:29:11.553 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "c0e4f180-70a4-11ef-bcea-c7b262605968", 2024-09-12T01:29:11.553 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 0, 2024-09-12T01:29:11.553 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24545, 2024-09-12T01:29:11.553 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-09-12T01:29:11.554 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-09-12T01:29:11.554 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-09-12T01:29:11.554 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 142.07464578599999, 2024-09-12T01:29:11.554 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 28, 2024-09-12T01:29:11.554 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 98, 2024-09-12T01:29:11.554 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 96, 2024-09-12T01:29:11.554 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 156.67921582100001 2024-09-12T01:29:11.554 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-12T01:29:11.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.546+0000 7f8118a00640 1 -- 172.21.15.1:0/3305048876 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f80f80a2c90 msgr2=0x7f80f80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:11.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.546+0000 7f8118a00640 1 --2- 172.21.15.1:0/3305048876 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f80f80a2c90 0x7f80f80a5150 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x55c1bba1a8d0 tx=0x7f8104005f90 comp rx=0 tx=0).stop 2024-09-12T01:29:11.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.546+0000 7f8118a00640 1 -- 172.21.15.1:0/3305048876 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8114102370 msgr2=0x7f811419aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:11.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.546+0000 7f8118a00640 1 --2- 172.21.15.1:0/3305048876 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8114102370 0x7f811419aa10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f810002a9e0 tx=0x7f8100003c10 comp rx=0 tx=0).stop 2024-09-12T01:29:11.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.547+0000 7f8118a00640 1 -- 172.21.15.1:0/3305048876 shutdown_connections 2024-09-12T01:29:11.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.547+0000 7f8118a00640 1 --2- 172.21.15.1:0/3305048876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81141079d0 0x7f811419eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:11.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.547+0000 7f8118a00640 1 --2- 172.21.15.1:0/3305048876 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8114102370 0x7f811419aa10 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:11.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.547+0000 7f8118a00640 1 --2- 172.21.15.1:0/3305048876 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f80f80a2c90 0x7f80f80a5150 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:11.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.547+0000 7f8118a00640 1 --2- 172.21.15.1:0/3305048876 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81141019d0 0x7f811419a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:11.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.547+0000 7f8118a00640 1 -- 172.21.15.1:0/3305048876 >> 172.21.15.1:0/3305048876 conn(0x7f81140fd710 msgr2=0x7f811410d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:11.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.547+0000 7f8118a00640 1 -- 172.21.15.1:0/3305048876 shutdown_connections 2024-09-12T01:29:11.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:11.547+0000 7f8118a00640 1 -- 172.21.15.1:0/3305048876 wait complete. 2024-09-12T01:29:12.484 INFO:tasks.cephadm:Setup complete, yielding 2024-09-12T01:29:12.484 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-12T01:29:12.494 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi001.front.sepia.ceph.com 2024-09-12T01:29:12.494 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-12T01:29:12.873 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:29:12.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:12 smithi001 ceph-mon[30387]: pgmap v386: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T01:29:12.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:12 smithi053 ceph-mon[35158]: pgmap v386: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T01:29:13.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:12 smithi104 ceph-mon[35219]: pgmap v386: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T01:29:13.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.445+0000 7fe040e00640 1 -- 172.21.15.1:0/1996689952 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe03c103e00 msgr2=0x7fe03c104200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:13.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.445+0000 7fe040e00640 1 --2- 172.21.15.1:0/1996689952 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe03c103e00 0x7fe03c104200 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe02c009860 tx=0x7fe02c02a340 comp rx=0 tx=0).stop 2024-09-12T01:29:13.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.445+0000 7fe040e00640 1 -- 172.21.15.1:0/1996689952 shutdown_connections 2024-09-12T01:29:13.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.445+0000 7fe040e00640 1 --2- 172.21.15.1:0/1996689952 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe03c105920 0x7fe03c10c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:13.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.445+0000 7fe040e00640 1 --2- 172.21.15.1:0/1996689952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe03c104f60 0x7fe03c1053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:13.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.445+0000 7fe040e00640 1 --2- 172.21.15.1:0/1996689952 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe03c103e00 0x7fe03c104200 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:13.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.445+0000 7fe040e00640 1 -- 172.21.15.1:0/1996689952 >> 172.21.15.1:0/1996689952 conn(0x7fe03c0ff5b0 msgr2=0x7fe03c1019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:13.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.445+0000 7fe040e00640 1 -- 172.21.15.1:0/1996689952 shutdown_connections 2024-09-12T01:29:13.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.445+0000 7fe040e00640 1 -- 172.21.15.1:0/1996689952 wait complete. 2024-09-12T01:29:13.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.446+0000 7fe040e00640 1 Processor -- start 2024-09-12T01:29:13.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.446+0000 7fe040e00640 1 -- start start 2024-09-12T01:29:13.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.446+0000 7fe040e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe03c103e00 0x7fe03c079fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:13.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.446+0000 7fe040e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe03c104f60 0x7fe03c07a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:13.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.446+0000 7fe040e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe03c105920 0x7fe03c076b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:13.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.446+0000 7fe040e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe03c10fa90 con 0x7fe03c103e00 2024-09-12T01:29:13.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.446+0000 7fe040e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe03c10f910 con 0x7fe03c105920 2024-09-12T01:29:13.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.446+0000 7fe040e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe03c10fc10 con 0x7fe03c104f60 2024-09-12T01:29:13.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.447+0000 7fe03a800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe03c105920 0x7fe03c076b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:13.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.447+0000 7fe039400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe03c104f60 0x7fe03c07a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:13.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.447+0000 7fe039e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe03c103e00 0x7fe03c079fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:13.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.447+0000 7fe03a800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe03c105920 0x7fe03c076b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:35742/0 (socket says 172.21.15.1:35742) 2024-09-12T01:29:13.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.447+0000 7fe039400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe03c104f60 0x7fe03c07a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:36218/0 (socket says 172.21.15.1:36218) 2024-09-12T01:29:13.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.447+0000 7fe03a800640 1 -- 172.21.15.1:0/2036871519 learned_addr learned my addr 172.21.15.1:0/2036871519 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:13.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.447+0000 7fe039400640 1 -- 172.21.15.1:0/2036871519 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe03c105920 msgr2=0x7fe03c076b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:13.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.447+0000 7fe039400640 1 --2- 172.21.15.1:0/2036871519 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe03c105920 0x7fe03c076b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:13.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.447+0000 7fe039400640 1 -- 172.21.15.1:0/2036871519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe03c103e00 msgr2=0x7fe03c079fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:13.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.447+0000 7fe039400640 1 --2- 172.21.15.1:0/2036871519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe03c103e00 0x7fe03c079fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:13.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.447+0000 7fe039400640 1 -- 172.21.15.1:0/2036871519 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe03c0773c0 con 0x7fe03c104f60 2024-09-12T01:29:13.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.447+0000 7fe039400640 1 --2- 172.21.15.1:0/2036871519 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe03c104f60 0x7fe03c07a4e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe02800e9c0 tx=0x7fe02800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:13.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.448+0000 7fe032a00640 1 -- 172.21.15.1:0/2036871519 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02800ccc0 con 0x7fe03c104f60 2024-09-12T01:29:13.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.448+0000 7fe040e00640 1 -- 172.21.15.1:0/2036871519 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe03c1a6540 con 0x7fe03c104f60 2024-09-12T01:29:13.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.448+0000 7fe032a00640 1 -- 172.21.15.1:0/2036871519 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0280035f0 con 0x7fe03c104f60 2024-09-12T01:29:13.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.448+0000 7fe032a00640 1 -- 172.21.15.1:0/2036871519 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02800ce60 con 0x7fe03c104f60 2024-09-12T01:29:13.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.448+0000 7fe040e00640 1 -- 172.21.15.1:0/2036871519 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe03c1a6990 con 0x7fe03c104f60 2024-09-12T01:29:13.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.449+0000 7fe040e00640 1 -- 172.21.15.1:0/2036871519 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe03c10b190 con 0x7fe03c104f60 2024-09-12T01:29:13.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.451+0000 7fe032a00640 1 -- 172.21.15.1:0/2036871519 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fe0280037a0 con 0x7fe03c104f60 2024-09-12T01:29:13.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.452+0000 7fe032a00640 1 --2- 172.21.15.1:0/2036871519 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe0200a2c10 0x7fe0200a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:13.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.453+0000 7fe032a00640 1 -- 172.21.15.1:0/2036871519 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fe0280cf9d0 con 0x7fe03c104f60 2024-09-12T01:29:13.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.453+0000 7fe039e00640 1 --2- 172.21.15.1:0/2036871519 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe0200a2c10 0x7fe0200a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:13.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.453+0000 7fe039e00640 1 --2- 172.21.15.1:0/2036871519 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe0200a2c10 0x7fe0200a50d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe02c009830 tx=0x7fe02c030550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:13.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.457+0000 7fe032a00640 1 -- 172.21.15.1:0/2036871519 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe02809c3e0 con 0x7fe03c104f60 2024-09-12T01:29:13.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.665+0000 7fe040e00640 1 -- 172.21.15.1:0/2036871519 --> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fe03c062630 con 0x7fe0200a2c10 2024-09-12T01:29:13.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.667+0000 7fe032a00640 1 -- 172.21.15.1:0/2036871519 <== mgr.14150 v2:172.21.15.1:6800/1928662456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fe03c062630 con 0x7fe0200a2c10 2024-09-12T01:29:13.666 INFO:teuthology.orchestra.run.smithi001.stdout:Backend: cephadm 2024-09-12T01:29:13.667 INFO:teuthology.orchestra.run.smithi001.stdout:Available: Yes 2024-09-12T01:29:13.667 INFO:teuthology.orchestra.run.smithi001.stdout:Paused: No 2024-09-12T01:29:13.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.669+0000 7fe040e00640 1 -- 172.21.15.1:0/2036871519 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe0200a2c10 msgr2=0x7fe0200a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:13.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.669+0000 7fe040e00640 1 --2- 172.21.15.1:0/2036871519 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe0200a2c10 0x7fe0200a50d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe02c009830 tx=0x7fe02c030550 comp rx=0 tx=0).stop 2024-09-12T01:29:13.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.669+0000 7fe040e00640 1 -- 172.21.15.1:0/2036871519 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe03c104f60 msgr2=0x7fe03c07a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:13.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.669+0000 7fe040e00640 1 --2- 172.21.15.1:0/2036871519 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe03c104f60 0x7fe03c07a4e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe02800e9c0 tx=0x7fe02800ee80 comp rx=0 tx=0).stop 2024-09-12T01:29:13.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.669+0000 7fe040e00640 1 -- 172.21.15.1:0/2036871519 shutdown_connections 2024-09-12T01:29:13.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.669+0000 7fe040e00640 1 --2- 172.21.15.1:0/2036871519 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe03c105920 0x7fe03c076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:13.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.669+0000 7fe040e00640 1 --2- 172.21.15.1:0/2036871519 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe0200a2c10 0x7fe0200a50d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:13.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.669+0000 7fe040e00640 1 --2- 172.21.15.1:0/2036871519 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe03c104f60 0x7fe03c07a4e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:13.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.669+0000 7fe040e00640 1 --2- 172.21.15.1:0/2036871519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe03c103e00 0x7fe03c079fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:13.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.669+0000 7fe040e00640 1 -- 172.21.15.1:0/2036871519 >> 172.21.15.1:0/2036871519 conn(0x7fe03c0ff5b0 msgr2=0x7fe03c108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:13.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.669+0000 7fe040e00640 1 -- 172.21.15.1:0/2036871519 shutdown_connections 2024-09-12T01:29:13.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:13.669+0000 7fe040e00640 1 -- 172.21.15.1:0/2036871519 wait complete. 2024-09-12T01:29:14.518 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-12T01:29:14.900 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:14 smithi001 ceph-mon[30387]: pgmap v387: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T01:29:14.936 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:29:14.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:14 smithi053 ceph-mon[35158]: pgmap v387: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T01:29:15.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:14 smithi104 ceph-mon[35219]: pgmap v387: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T01:29:15.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.611+0000 7f8e1d800640 1 -- 172.21.15.1:0/3974130469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8e18103e00 msgr2=0x7f8e18104200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:15.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.611+0000 7f8e1d800640 1 --2- 172.21.15.1:0/3974130469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8e18103e00 0x7f8e18104200 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8e08009900 tx=0x7f8e0802a3f0 comp rx=0 tx=0).stop 2024-09-12T01:29:15.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.611+0000 7f8e1d800640 1 -- 172.21.15.1:0/3974130469 shutdown_connections 2024-09-12T01:29:15.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.611+0000 7f8e1d800640 1 --2- 172.21.15.1:0/3974130469 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e18105920 0x7f8e1810c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:15.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.611+0000 7f8e1d800640 1 --2- 172.21.15.1:0/3974130469 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e18104f60 0x7f8e181053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:15.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.611+0000 7f8e1d800640 1 --2- 172.21.15.1:0/3974130469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8e18103e00 0x7f8e18104200 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:15.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.611+0000 7f8e1d800640 1 -- 172.21.15.1:0/3974130469 >> 172.21.15.1:0/3974130469 conn(0x7f8e180ff5b0 msgr2=0x7f8e181019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:15.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.611+0000 7f8e1d800640 1 -- 172.21.15.1:0/3974130469 shutdown_connections 2024-09-12T01:29:15.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.612+0000 7f8e1d800640 1 -- 172.21.15.1:0/3974130469 wait complete. 2024-09-12T01:29:15.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.613+0000 7f8e1d800640 1 Processor -- start 2024-09-12T01:29:15.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.613+0000 7f8e1d800640 1 -- start start 2024-09-12T01:29:15.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.613+0000 7f8e1d800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e18103e00 0x7f8e18079fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:15.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.613+0000 7f8e1d800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e18104f60 0x7f8e1807a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:15.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.613+0000 7f8e1d800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8e18105920 0x7f8e18076b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:15.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.613+0000 7f8e1d800640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1810fb00 con 0x7f8e18104f60 2024-09-12T01:29:15.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.613+0000 7f8e1d800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1810f980 con 0x7f8e18103e00 2024-09-12T01:29:15.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.613+0000 7f8e1d800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1810fc80 con 0x7f8e18105920 2024-09-12T01:29:15.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.613+0000 7f8e16a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e18103e00 0x7f8e18079fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:15.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.613+0000 7f8e16000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e18104f60 0x7f8e1807a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:15.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.613+0000 7f8e17400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8e18105920 0x7f8e18076b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:15.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.613+0000 7f8e16000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e18104f60 0x7f8e1807a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41094/0 (socket says 172.21.15.1:41094) 2024-09-12T01:29:15.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.613+0000 7f8e16000640 1 -- 172.21.15.1:0/3020606790 learned_addr learned my addr 172.21.15.1:0/3020606790 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:15.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.613+0000 7f8e17400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8e18105920 0x7f8e18076b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:36246/0 (socket says 172.21.15.1:36246) 2024-09-12T01:29:15.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.614+0000 7f8e17400640 1 -- 172.21.15.1:0/3020606790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e18103e00 msgr2=0x7f8e18079fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:15.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.614+0000 7f8e17400640 1 --2- 172.21.15.1:0/3020606790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e18103e00 0x7f8e18079fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:15.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.614+0000 7f8e17400640 1 -- 172.21.15.1:0/3020606790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e18104f60 msgr2=0x7f8e1807a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:15.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.614+0000 7f8e17400640 1 --2- 172.21.15.1:0/3020606790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e18104f60 0x7f8e1807a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:15.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.614+0000 7f8e17400640 1 -- 172.21.15.1:0/3020606790 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e180773c0 con 0x7f8e18105920 2024-09-12T01:29:15.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.614+0000 7f8e16000640 1 --2- 172.21.15.1:0/3020606790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e18104f60 0x7f8e1807a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:29:15.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.614+0000 7f8e17400640 1 --2- 172.21.15.1:0/3020606790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8e18105920 0x7f8e18076b00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8e10009fd0 tx=0x7f8e1000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:15.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.614+0000 7f8e0f400640 1 -- 172.21.15.1:0/3020606790 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1000ce20 con 0x7f8e18105920 2024-09-12T01:29:15.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.615+0000 7f8e0f400640 1 -- 172.21.15.1:0/3020606790 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e10005020 con 0x7f8e18105920 2024-09-12T01:29:15.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.615+0000 7f8e0f400640 1 -- 172.21.15.1:0/3020606790 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e10010720 con 0x7f8e18105920 2024-09-12T01:29:15.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.615+0000 7f8e1d800640 1 -- 172.21.15.1:0/3020606790 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e181a6500 con 0x7f8e18105920 2024-09-12T01:29:15.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.615+0000 7f8e1d800640 1 -- 172.21.15.1:0/3020606790 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e181a68a0 con 0x7f8e18105920 2024-09-12T01:29:15.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.616+0000 7f8e1d800640 1 -- 172.21.15.1:0/3020606790 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e18104200 con 0x7f8e18105920 2024-09-12T01:29:15.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.618+0000 7f8e0f400640 1 -- 172.21.15.1:0/3020606790 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f8e10005540 con 0x7f8e18105920 2024-09-12T01:29:15.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.619+0000 7f8e0f400640 1 --2- 172.21.15.1:0/3020606790 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f8dfc0a2c90 0x7f8dfc0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:15.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.619+0000 7f8e0f400640 1 -- 172.21.15.1:0/3020606790 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f8e100cfc30 con 0x7f8e18105920 2024-09-12T01:29:15.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.619+0000 7f8e16a00640 1 --2- 172.21.15.1:0/3020606790 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f8dfc0a2c90 0x7f8dfc0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:15.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.620+0000 7f8e16a00640 1 --2- 172.21.15.1:0/3020606790 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f8dfc0a2c90 0x7f8dfc0a5150 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8e08031440 tx=0x7f8e080315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:15.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.621+0000 7f8e0f400640 1 -- 172.21.15.1:0/3020606790 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8e1009c6d0 con 0x7f8e18105920 2024-09-12T01:29:15.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.649+0000 7f8e0f400640 1 -- 172.21.15.1:0/3020606790 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f8e10098640 con 0x7f8e18105920 2024-09-12T01:29:15.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.821+0000 7f8e1d800640 1 -- 172.21.15.1:0/3020606790 --> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f8e18062630 con 0x7f8dfc0a2c90 2024-09-12T01:29:15.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.834+0000 7f8e0f400640 1 -- 172.21.15.1:0/3020606790 <== mgr.14150 v2:172.21.15.1:6800/1928662456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4092 (secure 0 0 0) 0x7f8e18062630 con 0x7f8dfc0a2c90 2024-09-12T01:29:15.833 INFO:teuthology.orchestra.run.smithi001.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-12T01:29:15.833 INFO:teuthology.orchestra.run.smithi001.stdout:mds.a smithi001 running (2m) 23s ago 2m 16.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 7f31d7243330 2024-09-12T01:29:15.834 INFO:teuthology.orchestra.run.smithi001.stdout:mds.b smithi053 running (2m) 26s ago 2m 26.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 be93d6a77561 2024-09-12T01:29:15.834 INFO:teuthology.orchestra.run.smithi001.stdout:mds.c smithi104 running (2m) 26s ago 2m 18.6M - 19.1.1-298-gb53b6d3e 54467abc55f8 fdc976a557b2 2024-09-12T01:29:15.834 INFO:teuthology.orchestra.run.smithi001.stdout:mds.d smithi001 running (3m) 23s ago 3m 13.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 d1561eab7856 2024-09-12T01:29:15.834 INFO:teuthology.orchestra.run.smithi001.stdout:mds.e smithi053 running (3m) 26s ago 3m 19.2M - 19.1.1-298-gb53b6d3e 54467abc55f8 a74684ef297f 2024-09-12T01:29:15.834 INFO:teuthology.orchestra.run.smithi001.stdout:mds.f smithi104 running (2m) 26s ago 2m 17.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 053c647063ac 2024-09-12T01:29:15.834 INFO:teuthology.orchestra.run.smithi001.stdout:mds.g smithi001 running (2m) 23s ago 2m 16.3M - 19.1.1-298-gb53b6d3e 54467abc55f8 801be67e9eb3 2024-09-12T01:29:15.834 INFO:teuthology.orchestra.run.smithi001.stdout:mds.h smithi053 running (2m) 26s ago 2m 19.7M - 19.1.1-298-gb53b6d3e 54467abc55f8 c5a1c644c4e5 2024-09-12T01:29:15.834 INFO:teuthology.orchestra.run.smithi001.stdout:mds.i smithi104 running (2m) 26s ago 2m 20.3M - 19.1.1-298-gb53b6d3e 54467abc55f8 ee23f250750b 2024-09-12T01:29:15.834 INFO:teuthology.orchestra.run.smithi001.stdout:mds.j smithi001 running (3m) 23s ago 3m 13.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 e2640934b85c 2024-09-12T01:29:15.834 INFO:teuthology.orchestra.run.smithi001.stdout:mds.k smithi053 running (3m) 26s ago 3m 18.2M - 19.1.1-298-gb53b6d3e 54467abc55f8 cc65a2bf2316 2024-09-12T01:29:15.834 INFO:teuthology.orchestra.run.smithi001.stdout:mds.l smithi104 running (3m) 26s ago 3m 18.2M - 19.1.1-298-gb53b6d3e 54467abc55f8 53a28893b85e 2024-09-12T01:29:15.834 INFO:teuthology.orchestra.run.smithi001.stdout:mgr.x smithi001 *:9283,8765 running (10m) 23s ago 10m 510M - 19.1.1-298-gb53b6d3e 54467abc55f8 37d4931d977d 2024-09-12T01:29:15.834 INFO:teuthology.orchestra.run.smithi001.stdout:mgr.y smithi053 *:8443,8765 running (8m) 26s ago 8m 443M - 19.1.1-298-gb53b6d3e 54467abc55f8 d40466187460 2024-09-12T01:29:15.834 INFO:teuthology.orchestra.run.smithi001.stdout:mgr.z smithi104 *:8443,8765 running (8m) 26s ago 8m 444M - 19.1.1-298-gb53b6d3e 54467abc55f8 240430cd19ba 2024-09-12T01:29:15.835 INFO:teuthology.orchestra.run.smithi001.stdout:mon.a smithi001 running (10m) 23s ago 10m 49.4M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 72a843457ce6 2024-09-12T01:29:15.835 INFO:teuthology.orchestra.run.smithi001.stdout:mon.b smithi053 running (9m) 26s ago 9m 46.9M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 a344fc8ebfb6 2024-09-12T01:29:15.835 INFO:teuthology.orchestra.run.smithi001.stdout:mon.c smithi104 running (9m) 26s ago 9m 42.0M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 b8b8b09583ac 2024-09-12T01:29:15.835 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0 smithi001 running (8m) 23s ago 8m 52.8M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 5f728dbef0a6 2024-09-12T01:29:15.835 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1 smithi053 running (8m) 26s ago 8m 57.7M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 24dedaea265f 2024-09-12T01:29:15.835 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2 smithi104 running (7m) 26s ago 7m 58.0M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 7efa77d25e68 2024-09-12T01:29:15.835 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3 smithi001 running (7m) 23s ago 7m 52.9M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 9010efe13821 2024-09-12T01:29:15.835 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4 smithi053 running (6m) 26s ago 6m 44.6M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 3831e199beb4 2024-09-12T01:29:15.835 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5 smithi104 running (6m) 26s ago 6m 55.4M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 5219a8f6c82f 2024-09-12T01:29:15.835 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6 smithi001 running (5m) 23s ago 5m 42.0M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 f2b9464762aa 2024-09-12T01:29:15.835 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7 smithi053 running (5m) 26s ago 5m 54.1M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 c2ca342fd2b5 2024-09-12T01:29:15.835 INFO:teuthology.orchestra.run.smithi001.stdout:osd.8 smithi104 running (4m) 26s ago 4m 45.5M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 6a4500edb1be 2024-09-12T01:29:15.835 INFO:teuthology.orchestra.run.smithi001.stdout:osd.9 smithi001 running (4m) 23s ago 4m 42.2M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 df85cdd4409f 2024-09-12T01:29:15.835 INFO:teuthology.orchestra.run.smithi001.stdout:osd.10 smithi053 running (3m) 26s ago 3m 53.0M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 c9ea596a889b 2024-09-12T01:29:15.836 INFO:teuthology.orchestra.run.smithi001.stdout:osd.11 smithi104 running (3m) 26s ago 3m 49.0M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 de906ca44a39 2024-09-12T01:29:15.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.836+0000 7f8e1d800640 1 -- 172.21.15.1:0/3020606790 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f8dfc0a2c90 msgr2=0x7f8dfc0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:15.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.836+0000 7f8e1d800640 1 --2- 172.21.15.1:0/3020606790 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f8dfc0a2c90 0x7f8dfc0a5150 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8e08031440 tx=0x7f8e080315b0 comp rx=0 tx=0).stop 2024-09-12T01:29:15.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.836+0000 7f8e1d800640 1 -- 172.21.15.1:0/3020606790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8e18105920 msgr2=0x7f8e18076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:15.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.836+0000 7f8e1d800640 1 --2- 172.21.15.1:0/3020606790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8e18105920 0x7f8e18076b00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8e10009fd0 tx=0x7f8e1000ef10 comp rx=0 tx=0).stop 2024-09-12T01:29:15.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.836+0000 7f8e1d800640 1 -- 172.21.15.1:0/3020606790 shutdown_connections 2024-09-12T01:29:15.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.836+0000 7f8e1d800640 1 --2- 172.21.15.1:0/3020606790 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f8dfc0a2c90 0x7f8dfc0a5150 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:15.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.836+0000 7f8e1d800640 1 --2- 172.21.15.1:0/3020606790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8e18105920 0x7f8e18076b00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:15.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.836+0000 7f8e1d800640 1 --2- 172.21.15.1:0/3020606790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e18104f60 0x7f8e1807a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:15.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.836+0000 7f8e1d800640 1 --2- 172.21.15.1:0/3020606790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e18103e00 0x7f8e18079fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:15.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.836+0000 7f8e1d800640 1 -- 172.21.15.1:0/3020606790 >> 172.21.15.1:0/3020606790 conn(0x7f8e180ff5b0 msgr2=0x7f8e181082f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:15.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.837+0000 7f8e1d800640 1 -- 172.21.15.1:0/3020606790 shutdown_connections 2024-09-12T01:29:15.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:15.837+0000 7f8e1d800640 1 -- 172.21.15.1:0/3020606790 wait complete. 2024-09-12T01:29:15.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:15 smithi001 ceph-mon[30387]: from='client.25051 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:29:15.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:15 smithi001 ceph-mon[30387]: pgmap v388: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T01:29:15.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:15 smithi053 ceph-mon[35158]: from='client.25051 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:29:15.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:15 smithi053 ceph-mon[35158]: pgmap v388: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T01:29:16.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:15 smithi104 ceph-mon[35219]: from='client.25051 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:29:16.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:15 smithi104 ceph-mon[35219]: pgmap v388: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T01:29:16.646 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-12T01:29:16.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:16 smithi001 ceph-mon[30387]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-09-12T01:29:16.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:16 smithi001 ceph-mon[30387]: from='client.25057 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:29:16.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:16 smithi053 ceph-mon[35158]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-09-12T01:29:16.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:16 smithi053 ceph-mon[35158]: from='client.25057 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:29:17.027 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:29:17.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:16 smithi104 ceph-mon[35219]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-09-12T01:29:17.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:16 smithi104 ceph-mon[35219]: from='client.25057 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:29:17.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.670+0000 7f5f13e00640 1 -- 172.21.15.1:0/2013464180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f14074bf0 msgr2=0x7f5f14075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:17.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.670+0000 7f5f13e00640 1 --2- 172.21.15.1:0/2013464180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f14074bf0 0x7f5f14075050 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5f00009900 tx=0x7f5f0002a480 comp rx=0 tx=0).stop 2024-09-12T01:29:17.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.670+0000 7f5f13e00640 1 -- 172.21.15.1:0/2013464180 shutdown_connections 2024-09-12T01:29:17.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.670+0000 7f5f13e00640 1 --2- 172.21.15.1:0/2013464180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f141098e0 0x7f5f1410bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:17.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.670+0000 7f5f13e00640 1 --2- 172.21.15.1:0/2013464180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f14074bf0 0x7f5f14075050 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:17.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.670+0000 7f5f13e00640 1 --2- 172.21.15.1:0/2013464180 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f140761c0 0x7f5f14074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:17.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.670+0000 7f5f13e00640 1 -- 172.21.15.1:0/2013464180 >> 172.21.15.1:0/2013464180 conn(0x7f5f140fd1f0 msgr2=0x7f5f140ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:17.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.670+0000 7f5f13e00640 1 -- 172.21.15.1:0/2013464180 shutdown_connections 2024-09-12T01:29:17.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.670+0000 7f5f13e00640 1 -- 172.21.15.1:0/2013464180 wait complete. 2024-09-12T01:29:17.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.671+0000 7f5f13e00640 1 Processor -- start 2024-09-12T01:29:17.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.671+0000 7f5f13e00640 1 -- start start 2024-09-12T01:29:17.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.672+0000 7f5f13e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f14074bf0 0x7f5f14199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:17.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.672+0000 7f5f13e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f140761c0 0x7f5f1419a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:17.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.672+0000 7f5f13e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f141098e0 0x7f5f141a1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:17.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.672+0000 7f5f13e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1410f8c0 con 0x7f5f141098e0 2024-09-12T01:29:17.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.672+0000 7f5f13e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1410f740 con 0x7f5f14074bf0 2024-09-12T01:29:17.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.672+0000 7f5f13e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1410fa40 con 0x7f5f140761c0 2024-09-12T01:29:17.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.672+0000 7f5f13400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f141098e0 0x7f5f141a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:17.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.672+0000 7f5f12000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f140761c0 0x7f5f1419a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:17.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.672+0000 7f5f12000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f140761c0 0x7f5f1419a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:36264/0 (socket says 172.21.15.1:36264) 2024-09-12T01:29:17.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.672+0000 7f5f12000640 1 -- 172.21.15.1:0/1983043190 learned_addr learned my addr 172.21.15.1:0/1983043190 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:17.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.672+0000 7f5f12a00640 1 --2- 172.21.15.1:0/1983043190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f14074bf0 0x7f5f14199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:17.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.673+0000 7f5f12000640 1 -- 172.21.15.1:0/1983043190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f14074bf0 msgr2=0x7f5f14199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:17.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.673+0000 7f5f12000640 1 --2- 172.21.15.1:0/1983043190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f14074bf0 0x7f5f14199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:17.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.673+0000 7f5f12000640 1 -- 172.21.15.1:0/1983043190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f141098e0 msgr2=0x7f5f141a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:17.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.673+0000 7f5f12000640 1 --2- 172.21.15.1:0/1983043190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f141098e0 0x7f5f141a1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:17.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.673+0000 7f5f12000640 1 -- 172.21.15.1:0/1983043190 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f141a1af0 con 0x7f5f140761c0 2024-09-12T01:29:17.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.673+0000 7f5f12a00640 1 --2- 172.21.15.1:0/1983043190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f14074bf0 0x7f5f14199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:29:17.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.673+0000 7f5f13400640 1 --2- 172.21.15.1:0/1983043190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f141098e0 0x7f5f141a1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:29:17.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.673+0000 7f5f12000640 1 --2- 172.21.15.1:0/1983043190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f140761c0 0x7f5f1419a400 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5f00009230 tx=0x7f5f000042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:17.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.673+0000 7f5f0b400640 1 -- 172.21.15.1:0/1983043190 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0003f030 con 0x7f5f140761c0 2024-09-12T01:29:17.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.673+0000 7f5f13e00640 1 -- 172.21.15.1:0/1983043190 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f141a1d80 con 0x7f5f140761c0 2024-09-12T01:29:17.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.673+0000 7f5f0b400640 1 -- 172.21.15.1:0/1983043190 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f00003ce0 con 0x7f5f140761c0 2024-09-12T01:29:17.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.673+0000 7f5f0b400640 1 -- 172.21.15.1:0/1983043190 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f00035070 con 0x7f5f140761c0 2024-09-12T01:29:17.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.674+0000 7f5f13e00640 1 -- 172.21.15.1:0/1983043190 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f141a2250 con 0x7f5f140761c0 2024-09-12T01:29:17.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.675+0000 7f5f13e00640 1 -- 172.21.15.1:0/1983043190 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f141086a0 con 0x7f5f140761c0 2024-09-12T01:29:17.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.676+0000 7f5f0b400640 1 -- 172.21.15.1:0/1983043190 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f5f00042030 con 0x7f5f140761c0 2024-09-12T01:29:17.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.677+0000 7f5f0b400640 1 --2- 172.21.15.1:0/1983043190 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f5ef80a2c30 0x7f5ef80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:17.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.678+0000 7f5f0b400640 1 -- 172.21.15.1:0/1983043190 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f5f000ef820 con 0x7f5f140761c0 2024-09-12T01:29:17.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.678+0000 7f5f12a00640 1 --2- 172.21.15.1:0/1983043190 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f5ef80a2c30 0x7f5ef80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:17.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.678+0000 7f5f12a00640 1 --2- 172.21.15.1:0/1983043190 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f5ef80a2c30 0x7f5ef80a50f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5f04009fc0 tx=0x7f5f04005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:17.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.681+0000 7f5f0b400640 1 -- 172.21.15.1:0/1983043190 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5f000bbd40 con 0x7f5f140761c0 2024-09-12T01:29:17.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.881+0000 7f5f13e00640 1 -- 172.21.15.1:0/1983043190 --> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f5f14062630 con 0x7f5ef80a2c30 2024-09-12T01:29:17.881 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:17 smithi001 ceph-mon[30387]: pgmap v389: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-12T01:29:17.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.887+0000 7f5f0b400640 1 -- 172.21.15.1:0/1983043190 <== mgr.14150 v2:172.21.15.1:6800/1928662456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f5f14062630 con 0x7f5ef80a2c30 2024-09-12T01:29:17.889 INFO:teuthology.orchestra.run.smithi001.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-12T01:29:17.889 INFO:teuthology.orchestra.run.smithi001.stdout:mds.all 12/12 28s ago 3m smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi104=c;smithi104=f;smithi104=i;smithi104=l;count:12 2024-09-12T01:29:17.889 INFO:teuthology.orchestra.run.smithi001.stdout:mgr 3/3 28s ago 9m smithi001=x;smithi053=y;smithi104=z;count:3 2024-09-12T01:29:17.889 INFO:teuthology.orchestra.run.smithi001.stdout:mon 3/3 28s ago 9m smithi001:172.21.15.1=a;smithi053:172.21.15.53=b;smithi104:172.21.15.104=c;count:3 2024-09-12T01:29:17.889 INFO:teuthology.orchestra.run.smithi001.stdout:osd 12 28s ago - 2024-09-12T01:29:17.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.890+0000 7f5f13e00640 1 -- 172.21.15.1:0/1983043190 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f5ef80a2c30 msgr2=0x7f5ef80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:17.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.890+0000 7f5f13e00640 1 --2- 172.21.15.1:0/1983043190 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f5ef80a2c30 0x7f5ef80a50f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5f04009fc0 tx=0x7f5f04005f90 comp rx=0 tx=0).stop 2024-09-12T01:29:17.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.890+0000 7f5f13e00640 1 -- 172.21.15.1:0/1983043190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f140761c0 msgr2=0x7f5f1419a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:17.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.890+0000 7f5f13e00640 1 --2- 172.21.15.1:0/1983043190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f140761c0 0x7f5f1419a400 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5f00009230 tx=0x7f5f000042f0 comp rx=0 tx=0).stop 2024-09-12T01:29:17.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.890+0000 7f5f13e00640 1 -- 172.21.15.1:0/1983043190 shutdown_connections 2024-09-12T01:29:17.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.890+0000 7f5f13e00640 1 --2- 172.21.15.1:0/1983043190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f141098e0 0x7f5f141a1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:17.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.890+0000 7f5f13e00640 1 --2- 172.21.15.1:0/1983043190 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f5ef80a2c30 0x7f5ef80a50f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:17.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.890+0000 7f5f13e00640 1 --2- 172.21.15.1:0/1983043190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f140761c0 0x7f5f1419a400 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:17.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.890+0000 7f5f13e00640 1 --2- 172.21.15.1:0/1983043190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f14074bf0 0x7f5f14199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:17.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.890+0000 7f5f13e00640 1 -- 172.21.15.1:0/1983043190 >> 172.21.15.1:0/1983043190 conn(0x7f5f140fd1f0 msgr2=0x7f5f14105870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:17.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.890+0000 7f5f13e00640 1 -- 172.21.15.1:0/1983043190 shutdown_connections 2024-09-12T01:29:17.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:17.890+0000 7f5f13e00640 1 -- 172.21.15.1:0/1983043190 wait complete. 2024-09-12T01:29:17.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:17 smithi053 ceph-mon[35158]: pgmap v389: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-12T01:29:18.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:17 smithi104 ceph-mon[35219]: pgmap v389: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-12T01:29:18.653 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-12T01:29:18.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:18 smithi001 ceph-mon[30387]: from='client.25063 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:29:18.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:18 smithi053 ceph-mon[35158]: from='client.25063 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:29:19.034 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:29:19.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:18 smithi104 ceph-mon[35219]: from='client.25063 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:29:19.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.696+0000 7f76e9600640 1 -- 172.21.15.1:0/1783906843 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76e4102c40 msgr2=0x7f76e41030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:19.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.696+0000 7f76e9600640 1 --2- 172.21.15.1:0/1783906843 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76e4102c40 0x7f76e41030c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f76d0009900 tx=0x7f76d002a3f0 comp rx=0 tx=0).stop 2024-09-12T01:29:19.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.696+0000 7f76e9600640 1 -- 172.21.15.1:0/1783906843 shutdown_connections 2024-09-12T01:29:19.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.696+0000 7f76e9600640 1 --2- 172.21.15.1:0/1783906843 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76e4103600 0x7f76e4109e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:19.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.696+0000 7f76e9600640 1 --2- 172.21.15.1:0/1783906843 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76e4102c40 0x7f76e41030c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:19.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.696+0000 7f76e9600640 1 --2- 172.21.15.1:0/1783906843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76e4101a40 0x7f76e4101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:19.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.696+0000 7f76e9600640 1 -- 172.21.15.1:0/1783906843 >> 172.21.15.1:0/1783906843 conn(0x7f76e40fd1f0 msgr2=0x7f76e40ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:19.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.696+0000 7f76e9600640 1 -- 172.21.15.1:0/1783906843 shutdown_connections 2024-09-12T01:29:19.696 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:19 smithi001 ceph-mon[30387]: pgmap v390: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-12T01:29:19.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.696+0000 7f76e9600640 1 -- 172.21.15.1:0/1783906843 wait complete. 2024-09-12T01:29:19.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.697+0000 7f76e9600640 1 Processor -- start 2024-09-12T01:29:19.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.697+0000 7f76e9600640 1 -- start start 2024-09-12T01:29:19.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.697+0000 7f76e9600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76e4101a40 0x7f76e4075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:19.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.697+0000 7f76e9600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76e4102c40 0x7f76e40745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:19.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.697+0000 7f76e9600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76e4103600 0x7f76e4074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:19.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.697+0000 7f76e9600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f76e410d6f0 con 0x7f76e4103600 2024-09-12T01:29:19.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.697+0000 7f76e9600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f76e410d570 con 0x7f76e4102c40 2024-09-12T01:29:19.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.697+0000 7f76e9600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f76e410d870 con 0x7f76e4101a40 2024-09-12T01:29:19.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.698+0000 7f76e3400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76e4102c40 0x7f76e40745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:19.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.698+0000 7f76e3400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76e4102c40 0x7f76e40745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:57692/0 (socket says 172.21.15.1:57692) 2024-09-12T01:29:19.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.698+0000 7f76e3400640 1 -- 172.21.15.1:0/2775547552 learned_addr learned my addr 172.21.15.1:0/2775547552 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:19.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.698+0000 7f76e8c00640 1 --2- 172.21.15.1:0/2775547552 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76e4103600 0x7f76e4074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:19.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.698+0000 7f76e3e00640 1 --2- 172.21.15.1:0/2775547552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76e4101a40 0x7f76e4075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:19.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.698+0000 7f76e3400640 1 -- 172.21.15.1:0/2775547552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76e4101a40 msgr2=0x7f76e4075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:19.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.698+0000 7f76e3400640 1 --2- 172.21.15.1:0/2775547552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76e4101a40 0x7f76e4075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:19.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.698+0000 7f76e3400640 1 -- 172.21.15.1:0/2775547552 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76e4103600 msgr2=0x7f76e4074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:19.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.698+0000 7f76e3400640 1 --2- 172.21.15.1:0/2775547552 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76e4103600 0x7f76e4074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:19.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.698+0000 7f76e3400640 1 -- 172.21.15.1:0/2775547552 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e40753e0 con 0x7f76e4102c40 2024-09-12T01:29:19.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.698+0000 7f76e3e00640 1 --2- 172.21.15.1:0/2775547552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76e4101a40 0x7f76e4075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:29:19.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.699+0000 7f76e3400640 1 --2- 172.21.15.1:0/2775547552 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76e4102c40 0x7f76e40745e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f76d0031440 tx=0x7f76d0032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:19.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.699+0000 7f76e0c00640 1 -- 172.21.15.1:0/2775547552 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d003f030 con 0x7f76e4102c40 2024-09-12T01:29:19.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.699+0000 7f76e0c00640 1 -- 172.21.15.1:0/2775547552 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76d0003ce0 con 0x7f76e4102c40 2024-09-12T01:29:19.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.699+0000 7f76e9600640 1 -- 172.21.15.1:0/2775547552 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76e41a1ea0 con 0x7f76e4102c40 2024-09-12T01:29:19.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.699+0000 7f76e0c00640 1 -- 172.21.15.1:0/2775547552 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d0035070 con 0x7f76e4102c40 2024-09-12T01:29:19.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.699+0000 7f76e9600640 1 -- 172.21.15.1:0/2775547552 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76e41a2290 con 0x7f76e4102c40 2024-09-12T01:29:19.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.700+0000 7f76e9600640 1 -- 172.21.15.1:0/2775547552 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76e4108e40 con 0x7f76e4102c40 2024-09-12T01:29:19.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.702+0000 7f76e0c00640 1 -- 172.21.15.1:0/2775547552 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f76d0042030 con 0x7f76e4102c40 2024-09-12T01:29:19.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.703+0000 7f76e0c00640 1 --2- 172.21.15.1:0/2775547552 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f76c80a2b80 0x7f76c80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:19.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.703+0000 7f76e0c00640 1 -- 172.21.15.1:0/2775547552 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f76d00ef4e0 con 0x7f76e4102c40 2024-09-12T01:29:19.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.703+0000 7f76e3e00640 1 --2- 172.21.15.1:0/2775547552 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f76c80a2b80 0x7f76c80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:19.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.704+0000 7f76e3e00640 1 --2- 172.21.15.1:0/2775547552 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f76c80a2b80 0x7f76c80a5040 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f76e4102aa0 tx=0x7f76d4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:19.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.708+0000 7f76e0c00640 1 -- 172.21.15.1:0/2775547552 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f76d00bc050 con 0x7f76e4102c40 2024-09-12T01:29:19.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.898+0000 7f76e9600640 1 -- 172.21.15.1:0/2775547552 --> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f76e4062630 con 0x7f76c80a2b80 2024-09-12T01:29:19.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.900+0000 7f76e0c00640 1 -- 172.21.15.1:0/2775547552 <== mgr.14150 v2:172.21.15.1:6800/1928662456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f76e4062630 con 0x7f76c80a2b80 2024-09-12T01:29:19.900 INFO:teuthology.orchestra.run.smithi001.stdout:HOST ADDR LABELS STATUS 2024-09-12T01:29:19.900 INFO:teuthology.orchestra.run.smithi001.stdout:smithi001 172.21.15.1 2024-09-12T01:29:19.900 INFO:teuthology.orchestra.run.smithi001.stdout:smithi053 172.21.15.53 2024-09-12T01:29:19.900 INFO:teuthology.orchestra.run.smithi001.stdout:smithi104 172.21.15.104 2024-09-12T01:29:19.900 INFO:teuthology.orchestra.run.smithi001.stdout:3 hosts in cluster 2024-09-12T01:29:19.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.902+0000 7f76e9600640 1 -- 172.21.15.1:0/2775547552 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f76c80a2b80 msgr2=0x7f76c80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:19.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.902+0000 7f76e9600640 1 --2- 172.21.15.1:0/2775547552 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f76c80a2b80 0x7f76c80a5040 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f76e4102aa0 tx=0x7f76d4005f90 comp rx=0 tx=0).stop 2024-09-12T01:29:19.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.902+0000 7f76e9600640 1 -- 172.21.15.1:0/2775547552 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76e4102c40 msgr2=0x7f76e40745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:19.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.902+0000 7f76e9600640 1 --2- 172.21.15.1:0/2775547552 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76e4102c40 0x7f76e40745e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f76d0031440 tx=0x7f76d0032ce0 comp rx=0 tx=0).stop 2024-09-12T01:29:19.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.902+0000 7f76e9600640 1 -- 172.21.15.1:0/2775547552 shutdown_connections 2024-09-12T01:29:19.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.902+0000 7f76e9600640 1 --2- 172.21.15.1:0/2775547552 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76e4103600 0x7f76e4074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:19.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.902+0000 7f76e9600640 1 --2- 172.21.15.1:0/2775547552 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76e4102c40 0x7f76e40745e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:19.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.902+0000 7f76e9600640 1 --2- 172.21.15.1:0/2775547552 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f76c80a2b80 0x7f76c80a5040 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:19.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.902+0000 7f76e9600640 1 --2- 172.21.15.1:0/2775547552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76e4101a40 0x7f76e4075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:19.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.902+0000 7f76e9600640 1 -- 172.21.15.1:0/2775547552 >> 172.21.15.1:0/2775547552 conn(0x7f76e40fd1f0 msgr2=0x7f76e4106010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:19.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.902+0000 7f76e9600640 1 -- 172.21.15.1:0/2775547552 shutdown_connections 2024-09-12T01:29:19.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:19.902+0000 7f76e9600640 1 -- 172.21.15.1:0/2775547552 wait complete. 2024-09-12T01:29:19.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:19 smithi053 ceph-mon[35158]: pgmap v390: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-12T01:29:20.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:19 smithi104 ceph-mon[35219]: pgmap v390: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-12T01:29:20.840 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-12T01:29:20.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:20 smithi001 ceph-mon[30387]: from='client.25082 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:29:20.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:20 smithi053 ceph-mon[35158]: from='client.25082 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:29:21.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:20 smithi104 ceph-mon[35219]: from='client.25082 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:29:21.221 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:29:21.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:21.813+0000 7fe880a00640 1 -- 172.21.15.1:0/4092926302 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe87c101c00 msgr2=0x7fe87c102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:21.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:21.813+0000 7fe880a00640 1 --2- 172.21.15.1:0/4092926302 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe87c101c00 0x7fe87c102000 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe86c009900 tx=0x7fe86c02a410 comp rx=0 tx=0).stop 2024-09-12T01:29:21.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:21.813+0000 7fe880a00640 1 -- 172.21.15.1:0/4092926302 shutdown_connections 2024-09-12T01:29:21.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:21.813+0000 7fe880a00640 1 --2- 172.21.15.1:0/4092926302 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe87c1037c0 0x7fe87c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:21.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:21.813+0000 7fe880a00640 1 --2- 172.21.15.1:0/4092926302 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe87c102e00 0x7fe87c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:21.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:21.813+0000 7fe880a00640 1 --2- 172.21.15.1:0/4092926302 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe87c101c00 0x7fe87c102000 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:21.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:21.813+0000 7fe880a00640 1 -- 172.21.15.1:0/4092926302 >> 172.21.15.1:0/4092926302 conn(0x7fe87c0fd390 msgr2=0x7fe87c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:21.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:21.813+0000 7fe880a00640 1 -- 172.21.15.1:0/4092926302 shutdown_connections 2024-09-12T01:29:21.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:21.813+0000 7fe880a00640 1 -- 172.21.15.1:0/4092926302 wait complete. 2024-09-12T01:29:21.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:21.815+0000 7fe880a00640 1 Processor -- start 2024-09-12T01:29:21.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:21.815+0000 7fe880a00640 1 -- start start 2024-09-12T01:29:21.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:21.815+0000 7fe880a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe87c101c00 0x7fe87c19a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:21.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:21.815+0000 7fe880a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe87c102e00 0x7fe87c19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:21.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:21.815+0000 7fe880a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe87c1037c0 0x7fe87c1a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:21.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:21.815+0000 7fe880a00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe87c10d860 con 0x7fe87c102e00 2024-09-12T01:29:21.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:21.815+0000 7fe880a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe87c10d6e0 con 0x7fe87c1037c0 2024-09-12T01:29:21.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:21.815+0000 7fe880a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe87c10d9e0 con 0x7fe87c101c00 2024-09-12T01:29:21.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:21.815+0000 7fe879400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe87c102e00 0x7fe87c19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:21.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:21.816+0000 7fe879400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe87c102e00 0x7fe87c19a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55028/0 (socket says 172.21.15.1:55028) 2024-09-12T01:29:21.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:21.816+0000 7fe879400640 1 -- 172.21.15.1:0/259447345 learned_addr learned my addr 172.21.15.1:0/259447345 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:21.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:21.816+0000 7fe87a800640 1 --2- 172.21.15.1:0/259447345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe87c1037c0 0x7fe87c1a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:21.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:21.816+0000 7fe879e00640 1 --2- 172.21.15.1:0/259447345 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe87c101c00 0x7fe87c19a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:21.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:21.816+0000 7fe879400640 1 -- 172.21.15.1:0/259447345 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe87c101c00 msgr2=0x7fe87c19a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:21.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:21.816+0000 7fe879400640 1 --2- 172.21.15.1:0/259447345 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe87c101c00 0x7fe87c19a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:21.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:21.816+0000 7fe879400640 1 -- 172.21.15.1:0/259447345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe87c1037c0 msgr2=0x7fe87c1a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:21.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:21.816+0000 7fe879400640 1 --2- 172.21.15.1:0/259447345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe87c1037c0 0x7fe87c1a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:21.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:21.816+0000 7fe879400640 1 -- 172.21.15.1:0/259447345 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe87c1a1d90 con 0x7fe87c102e00 2024-09-12T01:29:21.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:21.816+0000 7fe879e00640 1 --2- 172.21.15.1:0/259447345 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe87c101c00 0x7fe87c19a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:29:21.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:21.816+0000 7fe87a800640 1 --2- 172.21.15.1:0/259447345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe87c1037c0 0x7fe87c1a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:29:21.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:21.817+0000 7fe879400640 1 --2- 172.21.15.1:0/259447345 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe87c102e00 0x7fe87c19a6a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe86800ea60 tx=0x7fe86800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:21.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:21.817+0000 7fe872a00640 1 -- 172.21.15.1:0/259447345 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86800cde0 con 0x7fe87c102e00 2024-09-12T01:29:21.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:21.817+0000 7fe872a00640 1 -- 172.21.15.1:0/259447345 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8680035f0 con 0x7fe87c102e00 2024-09-12T01:29:21.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:21.817+0000 7fe872a00640 1 -- 172.21.15.1:0/259447345 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe868010670 con 0x7fe87c102e00 2024-09-12T01:29:21.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:21.817+0000 7fe880a00640 1 -- 172.21.15.1:0/259447345 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe87c1a2080 con 0x7fe87c102e00 2024-09-12T01:29:21.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:21.817+0000 7fe880a00640 1 -- 172.21.15.1:0/259447345 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe87c1a2570 con 0x7fe87c102e00 2024-09-12T01:29:21.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:21.818+0000 7fe880a00640 1 -- 172.21.15.1:0/259447345 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe87c1a2830 con 0x7fe87c102e00 2024-09-12T01:29:21.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:21.820+0000 7fe872a00640 1 -- 172.21.15.1:0/259447345 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fe8680037a0 con 0x7fe87c102e00 2024-09-12T01:29:21.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:21.821+0000 7fe872a00640 1 --2- 172.21.15.1:0/259447345 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe8600a2c30 0x7fe8600a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:21.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:21.821+0000 7fe872a00640 1 -- 172.21.15.1:0/259447345 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fe8680cf680 con 0x7fe87c102e00 2024-09-12T01:29:21.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:21.822+0000 7fe879e00640 1 --2- 172.21.15.1:0/259447345 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe8600a2c30 0x7fe8600a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:21.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:21.822+0000 7fe879e00640 1 --2- 172.21.15.1:0/259447345 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe8600a2c30 0x7fe8600a50f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe86c005ab0 tx=0x7fe86c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:21.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:21.826+0000 7fe872a00640 1 -- 172.21.15.1:0/259447345 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe86809c1f0 con 0x7fe87c102e00 2024-09-12T01:29:21.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:21 smithi001 ceph-mon[30387]: pgmap v391: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-12T01:29:21.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:21 smithi053 ceph-mon[35158]: pgmap v391: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-12T01:29:22.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:22.017+0000 7fe880a00640 1 -- 172.21.15.1:0/259447345 --> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fe87c1a2a00 con 0x7fe8600a2c30 2024-09-12T01:29:22.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:22.019+0000 7fe872a00640 1 -- 172.21.15.1:0/259447345 <== mgr.14150 v2:172.21.15.1:6800/1928662456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fe87c1a2a00 con 0x7fe8600a2c30 2024-09-12T01:29:22.019 INFO:teuthology.orchestra.run.smithi001.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-12T01:29:22.019 INFO:teuthology.orchestra.run.smithi001.stdout:smithi001 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513000D480DGN 447G No 4m ago Has a FileSystem, LVM detected 2024-09-12T01:29:22.019 INFO:teuthology.orchestra.run.smithi001.stdout:smithi053 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300G8480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-09-12T01:29:22.019 INFO:teuthology.orchestra.run.smithi001.stdout:smithi104 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NW400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-12T01:29:22.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:22.021+0000 7fe880a00640 1 -- 172.21.15.1:0/259447345 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe8600a2c30 msgr2=0x7fe8600a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:22.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:22.021+0000 7fe880a00640 1 --2- 172.21.15.1:0/259447345 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe8600a2c30 0x7fe8600a50f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe86c005ab0 tx=0x7fe86c000bc0 comp rx=0 tx=0).stop 2024-09-12T01:29:22.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:22.021+0000 7fe880a00640 1 -- 172.21.15.1:0/259447345 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe87c102e00 msgr2=0x7fe87c19a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:22.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:22.021+0000 7fe880a00640 1 --2- 172.21.15.1:0/259447345 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe87c102e00 0x7fe87c19a6a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe86800ea60 tx=0x7fe86800ef20 comp rx=0 tx=0).stop 2024-09-12T01:29:22.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:22.022+0000 7fe880a00640 1 -- 172.21.15.1:0/259447345 shutdown_connections 2024-09-12T01:29:22.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:22.022+0000 7fe880a00640 1 --2- 172.21.15.1:0/259447345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe87c1037c0 0x7fe87c1a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:22.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:22.022+0000 7fe880a00640 1 --2- 172.21.15.1:0/259447345 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe87c102e00 0x7fe87c19a6a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:22.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:22.022+0000 7fe880a00640 1 --2- 172.21.15.1:0/259447345 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe8600a2c30 0x7fe8600a50f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:22.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:22.022+0000 7fe880a00640 1 --2- 172.21.15.1:0/259447345 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe87c101c00 0x7fe87c19a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:22.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:22.022+0000 7fe880a00640 1 -- 172.21.15.1:0/259447345 >> 172.21.15.1:0/259447345 conn(0x7fe87c0fd390 msgr2=0x7fe87c1085a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:22.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:22.022+0000 7fe880a00640 1 -- 172.21.15.1:0/259447345 shutdown_connections 2024-09-12T01:29:22.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:22.022+0000 7fe880a00640 1 -- 172.21.15.1:0/259447345 wait complete. 2024-09-12T01:29:22.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:21 smithi104 ceph-mon[35219]: pgmap v391: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-12T01:29:22.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:22 smithi001 ceph-mon[30387]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:29:22.854 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-12T01:29:22.863 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi001.front.sepia.ceph.com 2024-09-12T01:29:22.863 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-12T01:29:22.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:22 smithi053 ceph-mon[35158]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:29:23.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:22 smithi104 ceph-mon[35219]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:29:23.239 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:29:23.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:23.794+0000 7eff65400640 1 -- 172.21.15.1:0/3908705625 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff60078790 msgr2=0x7eff60079030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:23.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:23.794+0000 7eff65400640 1 --2- 172.21.15.1:0/3908705625 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff60078790 0x7eff60079030 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7eff58009ca0 tx=0x7eff5802a8c0 comp rx=0 tx=0).stop 2024-09-12T01:29:23.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:23.795+0000 7eff65400640 1 -- 172.21.15.1:0/3908705625 shutdown_connections 2024-09-12T01:29:23.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:23.795+0000 7eff65400640 1 --2- 172.21.15.1:0/3908705625 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff60078790 0x7eff60079030 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:23.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:23.795+0000 7eff65400640 1 --2- 172.21.15.1:0/3908705625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff60077df0 0x7eff60078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:23.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:23.795+0000 7eff65400640 1 --2- 172.21.15.1:0/3908705625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff60076ba0 0x7eff60076fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:23.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:23.795+0000 7eff65400640 1 -- 172.21.15.1:0/3908705625 >> 172.21.15.1:0/3908705625 conn(0x7eff600ff5b0 msgr2=0x7eff601019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:23.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:23.795+0000 7eff65400640 1 -- 172.21.15.1:0/3908705625 shutdown_connections 2024-09-12T01:29:23.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:23.795+0000 7eff65400640 1 -- 172.21.15.1:0/3908705625 wait complete. 2024-09-12T01:29:23.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:23.796+0000 7eff65400640 1 Processor -- start 2024-09-12T01:29:23.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:23.796+0000 7eff65400640 1 -- start start 2024-09-12T01:29:23.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:23.796+0000 7eff65400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff60076ba0 0x7eff6019e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:23.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:23.796+0000 7eff65400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff60077df0 0x7eff6019e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:23.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:23.796+0000 7eff65400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff60078790 0x7eff601a5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:23.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:23.796+0000 7eff65400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7eff60113ea0 con 0x7eff60078790 2024-09-12T01:29:23.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:23.796+0000 7eff65400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7eff60113d20 con 0x7eff60076ba0 2024-09-12T01:29:23.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:23.796+0000 7eff65400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7eff60114020 con 0x7eff60077df0 2024-09-12T01:29:23.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:23.797+0000 7eff5f200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff60078790 0x7eff601a5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:23.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:23.797+0000 7eff5e800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff60076ba0 0x7eff6019e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:23.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:23.797+0000 7eff5f200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff60078790 0x7eff601a5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55046/0 (socket says 172.21.15.1:55046) 2024-09-12T01:29:23.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:23.797+0000 7eff5f200640 1 -- 172.21.15.1:0/2781956474 learned_addr learned my addr 172.21.15.1:0/2781956474 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:23.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:23.797+0000 7eff5e800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff60076ba0 0x7eff6019e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:57710/0 (socket says 172.21.15.1:57710) 2024-09-12T01:29:23.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:23.797+0000 7eff57e00640 1 --2- 172.21.15.1:0/2781956474 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff60077df0 0x7eff6019e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:23.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:23.797+0000 7eff5f200640 1 -- 172.21.15.1:0/2781956474 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff60077df0 msgr2=0x7eff6019e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:23.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:23.797+0000 7eff5f200640 1 --2- 172.21.15.1:0/2781956474 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff60077df0 0x7eff6019e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:23.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:23.797+0000 7eff5f200640 1 -- 172.21.15.1:0/2781956474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff60076ba0 msgr2=0x7eff6019e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:23.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:23.797+0000 7eff5f200640 1 --2- 172.21.15.1:0/2781956474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff60076ba0 0x7eff6019e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:23.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:23.797+0000 7eff5f200640 1 -- 172.21.15.1:0/2781956474 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff601a60a0 con 0x7eff60078790 2024-09-12T01:29:23.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:23.797+0000 7eff5f200640 1 --2- 172.21.15.1:0/2781956474 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff60078790 0x7eff601a5a30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7eff58031ae0 tx=0x7eff58031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:23.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:23.798+0000 7eff57400640 1 -- 172.21.15.1:0/2781956474 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff58003b70 con 0x7eff60078790 2024-09-12T01:29:23.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:23.798+0000 7eff65400640 1 -- 172.21.15.1:0/2781956474 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff601a6330 con 0x7eff60078790 2024-09-12T01:29:23.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:23.798+0000 7eff57400640 1 -- 172.21.15.1:0/2781956474 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff58003d10 con 0x7eff60078790 2024-09-12T01:29:23.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:23.798+0000 7eff57400640 1 -- 172.21.15.1:0/2781956474 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5802fc40 con 0x7eff60078790 2024-09-12T01:29:23.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:23.798+0000 7eff65400640 1 -- 172.21.15.1:0/2781956474 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff601a6870 con 0x7eff60078790 2024-09-12T01:29:23.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:23.799+0000 7eff65400640 1 -- 172.21.15.1:0/2781956474 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff60110780 con 0x7eff60078790 2024-09-12T01:29:23.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:23.801+0000 7eff57400640 1 -- 172.21.15.1:0/2781956474 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7eff58004050 con 0x7eff60078790 2024-09-12T01:29:23.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:23.802+0000 7eff57400640 1 --2- 172.21.15.1:0/2781956474 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7eff440a2bb0 0x7eff440a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:23.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:23.802+0000 7eff57400640 1 -- 172.21.15.1:0/2781956474 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7eff580f0420 con 0x7eff60078790 2024-09-12T01:29:23.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:23.802+0000 7eff5e800640 1 --2- 172.21.15.1:0/2781956474 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7eff440a2bb0 0x7eff440a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:23.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:23.803+0000 7eff5e800640 1 --2- 172.21.15.1:0/2781956474 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7eff440a2bb0 0x7eff440a5070 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7eff50009bd0 tx=0x7eff50005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:23.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:23.806+0000 7eff57400640 1 -- 172.21.15.1:0/2781956474 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7eff580bd110 con 0x7eff60078790 2024-09-12T01:29:23.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:23 smithi001 ceph-mon[30387]: pgmap v392: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-12T01:29:23.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:23 smithi053 ceph-mon[35158]: pgmap v392: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-12T01:29:24.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:24.026+0000 7eff65400640 1 -- 172.21.15.1:0/2781956474 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7eff60062df0 con 0x7eff60078790 2024-09-12T01:29:24.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:24.026+0000 7eff57400640 1 -- 172.21.15.1:0/2781956474 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+3122 (secure 0 0 0) 0x7eff580bca10 con 0x7eff60078790 2024-09-12T01:29:24.026 INFO:teuthology.orchestra.run.smithi001.stdout:e28 2024-09-12T01:29:24.027 INFO:teuthology.orchestra.run.smithi001.stdout:btime 2024-09-12T01:27:46:332106+0000 2024-09-12T01:29:24.027 INFO:teuthology.orchestra.run.smithi001.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T01:29:24.027 INFO:teuthology.orchestra.run.smithi001.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T01:29:24.027 INFO:teuthology.orchestra.run.smithi001.stdout:legacy client fscid: 1 2024-09-12T01:29:24.027 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:29:24.027 INFO:teuthology.orchestra.run.smithi001.stdout:Filesystem 'cephfs' (1) 2024-09-12T01:29:24.027 INFO:teuthology.orchestra.run.smithi001.stdout:fs_name cephfs 2024-09-12T01:29:24.027 INFO:teuthology.orchestra.run.smithi001.stdout:epoch 28 2024-09-12T01:29:24.027 INFO:teuthology.orchestra.run.smithi001.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-12T01:29:24.027 INFO:teuthology.orchestra.run.smithi001.stdout:created 2024-09-12T01:26:48.555447+0000 2024-09-12T01:29:24.027 INFO:teuthology.orchestra.run.smithi001.stdout:modified 2024-09-12T01:27:46.206001+0000 2024-09-12T01:29:24.027 INFO:teuthology.orchestra.run.smithi001.stdout:tableserver 0 2024-09-12T01:29:24.027 INFO:teuthology.orchestra.run.smithi001.stdout:root 0 2024-09-12T01:29:24.028 INFO:teuthology.orchestra.run.smithi001.stdout:session_timeout 300 2024-09-12T01:29:24.028 INFO:teuthology.orchestra.run.smithi001.stdout:session_autoclose 300 2024-09-12T01:29:24.028 INFO:teuthology.orchestra.run.smithi001.stdout:max_file_size 1099511627776 2024-09-12T01:29:24.028 INFO:teuthology.orchestra.run.smithi001.stdout:max_xattr_size 65536 2024-09-12T01:29:24.028 INFO:teuthology.orchestra.run.smithi001.stdout:required_client_features {} 2024-09-12T01:29:24.028 INFO:teuthology.orchestra.run.smithi001.stdout:last_failure 0 2024-09-12T01:29:24.028 INFO:teuthology.orchestra.run.smithi001.stdout:last_failure_osd_epoch 0 2024-09-12T01:29:24.028 INFO:teuthology.orchestra.run.smithi001.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T01:29:24.028 INFO:teuthology.orchestra.run.smithi001.stdout:max_mds 5 2024-09-12T01:29:24.028 INFO:teuthology.orchestra.run.smithi001.stdout:in 0,1,2,3,4 2024-09-12T01:29:24.028 INFO:teuthology.orchestra.run.smithi001.stdout:up {0=24545,1=24511,2=24506,3=24493,4=24487} 2024-09-12T01:29:24.028 INFO:teuthology.orchestra.run.smithi001.stdout:failed 2024-09-12T01:29:24.028 INFO:teuthology.orchestra.run.smithi001.stdout:damaged 2024-09-12T01:29:24.028 INFO:teuthology.orchestra.run.smithi001.stdout:stopped 2024-09-12T01:29:24.029 INFO:teuthology.orchestra.run.smithi001.stdout:data_pools [3,4] 2024-09-12T01:29:24.029 INFO:teuthology.orchestra.run.smithi001.stdout:metadata_pool 2 2024-09-12T01:29:24.029 INFO:teuthology.orchestra.run.smithi001.stdout:inline_data disabled 2024-09-12T01:29:24.029 INFO:teuthology.orchestra.run.smithi001.stdout:balancer 2024-09-12T01:29:24.029 INFO:teuthology.orchestra.run.smithi001.stdout:bal_rank_mask -1 2024-09-12T01:29:24.029 INFO:teuthology.orchestra.run.smithi001.stdout:standby_count_wanted 0 2024-09-12T01:29:24.029 INFO:teuthology.orchestra.run.smithi001.stdout:qdb_cluster leader: 24545 members: 24487,24493,24506,24545,24511 2024-09-12T01:29:24.029 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.b{0:24545} state up:active seq 5 addr [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T01:29:24.029 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.e{0:24482} state up:standby-replay seq 1 addr [v2:172.21.15.53:6834/2032381725,v1:172.21.15.53:6835/2032381725] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T01:29:24.029 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.i{1:24511} state up:active seq 21 addr [v2:172.21.15.104:6838/1459976041,v1:172.21.15.104:6839/1459976041] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T01:29:24.029 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.k{1:24446} state up:standby-replay seq 1 addr [v2:172.21.15.53:6832/2930310711,v1:172.21.15.53:6833/2930310711] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T01:29:24.029 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.h{2:24506} state up:active seq 22 addr [v2:172.21.15.53:6836/2848743754,v1:172.21.15.53:6837/2848743754] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T01:29:24.029 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.l{2:24439} state up:standby-replay seq 1 addr [v2:172.21.15.104:6832/2071825633,v1:172.21.15.104:6833/2071825633] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T01:29:24.029 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.c{3:24493} state up:active seq 23 addr [v2:172.21.15.104:6836/464720247,v1:172.21.15.104:6837/464720247] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T01:29:24.029 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.a{3:14493} state up:standby-replay seq 1 addr [v2:172.21.15.1:6840/2464781973,v1:172.21.15.1:6841/2464781973] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T01:29:24.030 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.f{4:24487} state up:active seq 23 addr [v2:172.21.15.104:6834/1547038540,v1:172.21.15.104:6835/1547038540] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T01:29:24.030 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.g{4:14481} state up:standby-replay seq 1 addr [v2:172.21.15.1:6838/3651849993,v1:172.21.15.1:6839/3651849993] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T01:29:24.030 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:29:24.030 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:29:24.030 INFO:teuthology.orchestra.run.smithi001.stdout:Standby daemons: 2024-09-12T01:29:24.030 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:29:24.030 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.j{-1:14430} state up:standby seq 1 addr [v2:172.21.15.1:6834/572192306,v1:172.21.15.1:6835/572192306] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T01:29:24.030 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.d{-1:14442} state up:standby seq 1 addr [v2:172.21.15.1:6836/1050734277,v1:172.21.15.1:6837/1050734277] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T01:29:24.031 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 28 2024-09-12T01:29:24.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:24.029+0000 7eff65400640 1 -- 172.21.15.1:0/2781956474 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7eff440a2bb0 msgr2=0x7eff440a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:24.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:24.029+0000 7eff65400640 1 --2- 172.21.15.1:0/2781956474 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7eff440a2bb0 0x7eff440a5070 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7eff50009bd0 tx=0x7eff50005f90 comp rx=0 tx=0).stop 2024-09-12T01:29:24.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:24.029+0000 7eff65400640 1 -- 172.21.15.1:0/2781956474 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff60078790 msgr2=0x7eff601a5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:24.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:24.029+0000 7eff65400640 1 --2- 172.21.15.1:0/2781956474 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff60078790 0x7eff601a5a30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7eff58031ae0 tx=0x7eff58031ee0 comp rx=0 tx=0).stop 2024-09-12T01:29:24.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:24.029+0000 7eff65400640 1 -- 172.21.15.1:0/2781956474 shutdown_connections 2024-09-12T01:29:24.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:24.029+0000 7eff65400640 1 --2- 172.21.15.1:0/2781956474 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff60078790 0x7eff601a5a30 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:24.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:24.029+0000 7eff65400640 1 --2- 172.21.15.1:0/2781956474 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7eff440a2bb0 0x7eff440a5070 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:24.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:24.029+0000 7eff65400640 1 --2- 172.21.15.1:0/2781956474 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff60077df0 0x7eff6019e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:24.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:24.029+0000 7eff65400640 1 --2- 172.21.15.1:0/2781956474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff60076ba0 0x7eff6019e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:24.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:24.029+0000 7eff65400640 1 -- 172.21.15.1:0/2781956474 >> 172.21.15.1:0/2781956474 conn(0x7eff600ff5b0 msgr2=0x7eff60101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:24.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:24.029+0000 7eff65400640 1 -- 172.21.15.1:0/2781956474 shutdown_connections 2024-09-12T01:29:24.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:24.029+0000 7eff65400640 1 -- 172.21.15.1:0/2781956474 wait complete. 2024-09-12T01:29:24.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:23 smithi104 ceph-mon[35219]: pgmap v392: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-12T01:29:24.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:24 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/2781956474' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-12T01:29:24.990 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-12T01:29:24.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:24 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/2781956474' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-12T01:29:25.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:24 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/2781956474' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-12T01:29:25.369 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:29:25.820 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:25 smithi001 ceph-mon[30387]: pgmap v393: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-12T01:29:25.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:25 smithi053 ceph-mon[35158]: pgmap v393: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-12T01:29:26.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:25 smithi104 ceph-mon[35219]: pgmap v393: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-12T01:29:26.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.124+0000 7f9569000640 1 -- 172.21.15.1:0/1535552377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9564105810 msgr2=0x7f956410c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:26.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.124+0000 7f9569000640 1 --2- 172.21.15.1:0/1535552377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9564105810 0x7f956410c0a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f955c009ca0 tx=0x7f955c02a8c0 comp rx=0 tx=0).stop 2024-09-12T01:29:26.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.124+0000 7f9569000640 1 -- 172.21.15.1:0/1535552377 shutdown_connections 2024-09-12T01:29:26.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.124+0000 7f9569000640 1 --2- 172.21.15.1:0/1535552377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9564105810 0x7f956410c0a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:26.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.124+0000 7f9569000640 1 --2- 172.21.15.1:0/1535552377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9564104e50 0x7f95641052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:26.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.124+0000 7f9569000640 1 --2- 172.21.15.1:0/1535552377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9564103c50 0x7f9564104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:26.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.124+0000 7f9569000640 1 -- 172.21.15.1:0/1535552377 >> 172.21.15.1:0/1535552377 conn(0x7f95640ff3e0 msgr2=0x7f9564101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:26.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.124+0000 7f9569000640 1 -- 172.21.15.1:0/1535552377 shutdown_connections 2024-09-12T01:29:26.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.124+0000 7f9569000640 1 -- 172.21.15.1:0/1535552377 wait complete. 2024-09-12T01:29:26.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.125+0000 7f9569000640 1 Processor -- start 2024-09-12T01:29:26.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.125+0000 7f9569000640 1 -- start start 2024-09-12T01:29:26.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.125+0000 7f9569000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9564103c50 0x7f9564199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:26.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.125+0000 7f9569000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9564104e50 0x7f956419a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:26.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.125+0000 7f9569000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9564105810 0x7f95641a1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:26.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.125+0000 7f9569000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f956410f920 con 0x7f9564104e50 2024-09-12T01:29:26.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.125+0000 7f9569000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f956410f7a0 con 0x7f9564103c50 2024-09-12T01:29:26.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.125+0000 7f9569000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f956410faa0 con 0x7f9564105810 2024-09-12T01:29:26.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.126+0000 7f9562a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9564104e50 0x7f956419a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:26.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.126+0000 7f9563400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9564103c50 0x7f9564199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:26.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.126+0000 7f9563400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9564103c50 0x7f9564199e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:57724/0 (socket says 172.21.15.1:57724) 2024-09-12T01:29:26.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.126+0000 7f9563400640 1 -- 172.21.15.1:0/2751349351 learned_addr learned my addr 172.21.15.1:0/2751349351 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:26.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.126+0000 7f9563e00640 1 --2- 172.21.15.1:0/2751349351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9564105810 0x7f95641a1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:26.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.126+0000 7f9562a00640 1 -- 172.21.15.1:0/2751349351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9564105810 msgr2=0x7f95641a1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:26.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.126+0000 7f9562a00640 1 --2- 172.21.15.1:0/2751349351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9564105810 0x7f95641a1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:26.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.126+0000 7f9562a00640 1 -- 172.21.15.1:0/2751349351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9564103c50 msgr2=0x7f9564199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:26.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.126+0000 7f9562a00640 1 --2- 172.21.15.1:0/2751349351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9564103c50 0x7f9564199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:26.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.126+0000 7f9562a00640 1 -- 172.21.15.1:0/2751349351 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95641a1ab0 con 0x7f9564104e50 2024-09-12T01:29:26.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.126+0000 7f9563400640 1 --2- 172.21.15.1:0/2751349351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9564103c50 0x7f9564199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:29:26.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.126+0000 7f9563e00640 1 --2- 172.21.15.1:0/2751349351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9564105810 0x7f95641a1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:29:26.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.127+0000 7f9562a00640 1 --2- 172.21.15.1:0/2751349351 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9564104e50 0x7f956419a3c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9550005fd0 tx=0x7f955000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:26.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.127+0000 7f955be00640 1 -- 172.21.15.1:0/2751349351 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9550009c30 con 0x7f9564104e50 2024-09-12T01:29:26.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.127+0000 7f955be00640 1 -- 172.21.15.1:0/2751349351 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9550009dd0 con 0x7f9564104e50 2024-09-12T01:29:26.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.127+0000 7f9569000640 1 -- 172.21.15.1:0/2751349351 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95641a1da0 con 0x7f9564104e50 2024-09-12T01:29:26.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.127+0000 7f955be00640 1 -- 172.21.15.1:0/2751349351 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9550010460 con 0x7f9564104e50 2024-09-12T01:29:26.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.127+0000 7f9569000640 1 -- 172.21.15.1:0/2751349351 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95641a2260 con 0x7f9564104e50 2024-09-12T01:29:26.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.128+0000 7f9569000640 1 -- 172.21.15.1:0/2751349351 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f956410af20 con 0x7f9564104e50 2024-09-12T01:29:26.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.131+0000 7f955be00640 1 -- 172.21.15.1:0/2751349351 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f9550003f50 con 0x7f9564104e50 2024-09-12T01:29:26.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.132+0000 7f955be00640 1 --2- 172.21.15.1:0/2751349351 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f95480a2c30 0x7f95480a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:26.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.132+0000 7f955be00640 1 -- 172.21.15.1:0/2751349351 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f95500ceb70 con 0x7f9564104e50 2024-09-12T01:29:26.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.133+0000 7f9563400640 1 --2- 172.21.15.1:0/2751349351 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f95480a2c30 0x7f95480a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:26.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.133+0000 7f9563400640 1 --2- 172.21.15.1:0/2751349351 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f95480a2c30 0x7f95480a50f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f95540058c0 tx=0x7f9554005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:26.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.136+0000 7f955be00640 1 -- 172.21.15.1:0/2751349351 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9550017020 con 0x7f9564104e50 2024-09-12T01:29:26.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.321+0000 7f9569000640 1 -- 172.21.15.1:0/2751349351 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f9564062e90 con 0x7f9564104e50 2024-09-12T01:29:26.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.322+0000 7f955be00640 1 -- 172.21.15.1:0/2751349351 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v98) ==== 56+0+5616 (secure 0 0 0) 0x7f955009b090 con 0x7f9564104e50 2024-09-12T01:29:26.322 INFO:teuthology.orchestra.run.smithi001.stdout:epoch 98 2024-09-12T01:29:26.323 INFO:teuthology.orchestra.run.smithi001.stdout:fsid c0e4f180-70a4-11ef-bcea-c7b262605968 2024-09-12T01:29:26.323 INFO:teuthology.orchestra.run.smithi001.stdout:created 2024-09-12T01:18:19.337406+0000 2024-09-12T01:29:26.323 INFO:teuthology.orchestra.run.smithi001.stdout:modified 2024-09-12T01:28:17.407774+0000 2024-09-12T01:29:26.323 INFO:teuthology.orchestra.run.smithi001.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-12T01:29:26.323 INFO:teuthology.orchestra.run.smithi001.stdout:crush_version 27 2024-09-12T01:29:26.323 INFO:teuthology.orchestra.run.smithi001.stdout:full_ratio 0.95 2024-09-12T01:29:26.324 INFO:teuthology.orchestra.run.smithi001.stdout:backfillfull_ratio 0.9 2024-09-12T01:29:26.324 INFO:teuthology.orchestra.run.smithi001.stdout:nearfull_ratio 0.85 2024-09-12T01:29:26.324 INFO:teuthology.orchestra.run.smithi001.stdout:require_min_compat_client luminous 2024-09-12T01:29:26.324 INFO:teuthology.orchestra.run.smithi001.stdout:min_compat_client luminous 2024-09-12T01:29:26.324 INFO:teuthology.orchestra.run.smithi001.stdout:require_osd_release squid 2024-09-12T01:29:26.324 INFO:teuthology.orchestra.run.smithi001.stdout:stretch_mode_enabled false 2024-09-12T01:29:26.324 INFO:teuthology.orchestra.run.smithi001.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-12T01:29:26.325 INFO:teuthology.orchestra.run.smithi001.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 87 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-12T01:29:26.325 INFO:teuthology.orchestra.run.smithi001.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 87 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-09-12T01:29:26.325 INFO:teuthology.orchestra.run.smithi001.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 94 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-12T01:29:26.325 INFO:teuthology.orchestra.run.smithi001.stdout:max_osd 12 2024-09-12T01:29:26.325 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0 up in weight 1 up_from 8 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6802/1125335136,v1:172.21.15.1:6803/1125335136] [v2:172.21.15.1:6804/1125335136,v1:172.21.15.1:6805/1125335136] exists,up b9a396ae-bf97-462f-9ef9-b10ef6b9d337 2024-09-12T01:29:26.325 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1 up in weight 1 up_from 13 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6800/3131049165,v1:172.21.15.53:6801/3131049165] [v2:172.21.15.53:6802/3131049165,v1:172.21.15.53:6803/3131049165] exists,up dfe7a8a3-3700-4789-ac64-88660de9e0d9 2024-09-12T01:29:26.326 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2 up in weight 1 up_from 18 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.104:6800/1133285636,v1:172.21.15.104:6801/1133285636] [v2:172.21.15.104:6802/1133285636,v1:172.21.15.104:6803/1133285636] exists,up 5d9d3603-ba7c-4115-9267-12abcb20d55b 2024-09-12T01:29:26.326 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3 up in weight 1 up_from 25 up_thru 97 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6810/1247468803,v1:172.21.15.1:6811/1247468803] [v2:172.21.15.1:6812/1247468803,v1:172.21.15.1:6813/1247468803] exists,up 1e063c66-41b4-4db1-a964-28e122ce628e 2024-09-12T01:29:26.326 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4 up in weight 1 up_from 33 up_thru 95 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6808/2673413639,v1:172.21.15.53:6809/2673413639] [v2:172.21.15.53:6810/2673413639,v1:172.21.15.53:6811/2673413639] exists,up fe89ece0-2171-45d5-a06d-e400f77233e5 2024-09-12T01:29:26.326 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5 up in weight 1 up_from 38 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.104:6808/3131498790,v1:172.21.15.104:6809/3131498790] [v2:172.21.15.104:6810/3131498790,v1:172.21.15.104:6811/3131498790] exists,up daab795c-9942-4d16-aa34-8bba30a4b99b 2024-09-12T01:29:26.326 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6 up in weight 1 up_from 46 up_thru 95 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6818/3413067918,v1:172.21.15.1:6819/3413067918] [v2:172.21.15.1:6820/3413067918,v1:172.21.15.1:6821/3413067918] exists,up f31d8633-e435-499b-9d46-0aa11a2481cd 2024-09-12T01:29:26.326 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7 up in weight 1 up_from 51 up_thru 95 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6816/2841484136,v1:172.21.15.53:6817/2841484136] [v2:172.21.15.53:6818/2841484136,v1:172.21.15.53:6819/2841484136] exists,up e64da49e-d31e-4801-b6cc-634a233b4761 2024-09-12T01:29:26.327 INFO:teuthology.orchestra.run.smithi001.stdout:osd.8 up in weight 1 up_from 59 up_thru 95 down_at 0 last_clean_interval [0,0) [v2:172.21.15.104:6816/941453547,v1:172.21.15.104:6817/941453547] [v2:172.21.15.104:6818/941453547,v1:172.21.15.104:6819/941453547] exists,up fdfb5a42-dca7-4874-b479-176b4a191e21 2024-09-12T01:29:26.327 INFO:teuthology.orchestra.run.smithi001.stdout:osd.9 up in weight 1 up_from 67 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6826/958287586,v1:172.21.15.1:6827/958287586] [v2:172.21.15.1:6828/958287586,v1:172.21.15.1:6829/958287586] exists,up 7f81537a-cae5-4bb8-aec2-4eb07deb7958 2024-09-12T01:29:26.327 INFO:teuthology.orchestra.run.smithi001.stdout:osd.10 up in weight 1 up_from 72 up_thru 95 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6824/3357909498,v1:172.21.15.53:6825/3357909498] [v2:172.21.15.53:6826/3357909498,v1:172.21.15.53:6827/3357909498] exists,up e3c9c37d-46b9-4585-990d-69c447453bd8 2024-09-12T01:29:26.327 INFO:teuthology.orchestra.run.smithi001.stdout:osd.11 up in weight 1 up_from 79 up_thru 97 down_at 0 last_clean_interval [0,0) [v2:172.21.15.104:6824/3381427987,v1:172.21.15.104:6825/3381427987] [v2:172.21.15.104:6826/3381427987,v1:172.21.15.104:6827/3381427987] exists,up 474b5345-f2ae-4e0e-a197-53ab11f58c04 2024-09-12T01:29:26.327 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 2.0 [7,11] 2024-09-12T01:29:26.327 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 2.b [8,11] 2024-09-12T01:29:26.328 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 2.14 [2,4] 2024-09-12T01:29:26.328 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 2.1a [8,1] 2024-09-12T01:29:26.328 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 2.1d [2,4] 2024-09-12T01:29:26.328 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 3.3 [4,9] 2024-09-12T01:29:26.328 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 3.20 [5,9] 2024-09-12T01:29:26.328 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 3.24 [4,9] 2024-09-12T01:29:26.328 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 4.5 [9,10] 2024-09-12T01:29:26.329 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 4.11 [5,8] 2024-09-12T01:29:26.329 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 4.1b [9,8] 2024-09-12T01:29:26.329 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 4.1f [9,8] 2024-09-12T01:29:26.329 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:0/1373984726 expires 2024-09-13T01:19:14.580637+0000 2024-09-12T01:29:26.329 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:0/1429963718 expires 2024-09-13T01:19:14.580637+0000 2024-09-12T01:29:26.329 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:6801/4101281615 expires 2024-09-13T01:19:14.580637+0000 2024-09-12T01:29:26.329 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:6800/4101281615 expires 2024-09-13T01:19:14.580637+0000 2024-09-12T01:29:26.330 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:0/2564759586 expires 2024-09-13T01:19:14.580637+0000 2024-09-12T01:29:26.330 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:6801/2935968453 expires 2024-09-13T01:18:45.882682+0000 2024-09-12T01:29:26.330 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:6800/2935968453 expires 2024-09-13T01:18:45.882682+0000 2024-09-12T01:29:26.330 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:0/1459848813 expires 2024-09-13T01:18:45.882682+0000 2024-09-12T01:29:26.330 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:0/578605922 expires 2024-09-13T01:18:45.882682+0000 2024-09-12T01:29:26.331 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:0/738035035 expires 2024-09-13T01:18:45.882682+0000 2024-09-12T01:29:26.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.325+0000 7f9569000640 1 -- 172.21.15.1:0/2751349351 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f95480a2c30 msgr2=0x7f95480a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:26.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.325+0000 7f9569000640 1 --2- 172.21.15.1:0/2751349351 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f95480a2c30 0x7f95480a50f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f95540058c0 tx=0x7f9554005f90 comp rx=0 tx=0).stop 2024-09-12T01:29:26.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.325+0000 7f9569000640 1 -- 172.21.15.1:0/2751349351 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9564104e50 msgr2=0x7f956419a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:26.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.325+0000 7f9569000640 1 --2- 172.21.15.1:0/2751349351 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9564104e50 0x7f956419a3c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9550005fd0 tx=0x7f955000dde0 comp rx=0 tx=0).stop 2024-09-12T01:29:26.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.325+0000 7f9569000640 1 -- 172.21.15.1:0/2751349351 shutdown_connections 2024-09-12T01:29:26.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.325+0000 7f9569000640 1 --2- 172.21.15.1:0/2751349351 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f95480a2c30 0x7f95480a50f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:26.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.325+0000 7f9569000640 1 --2- 172.21.15.1:0/2751349351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9564105810 0x7f95641a1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:26.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.325+0000 7f9569000640 1 --2- 172.21.15.1:0/2751349351 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9564104e50 0x7f956419a3c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:26.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.325+0000 7f9569000640 1 --2- 172.21.15.1:0/2751349351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9564103c50 0x7f9564199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:26.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.325+0000 7f9569000640 1 -- 172.21.15.1:0/2751349351 >> 172.21.15.1:0/2751349351 conn(0x7f95640ff3e0 msgr2=0x7f95641080f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:26.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.325+0000 7f9569000640 1 -- 172.21.15.1:0/2751349351 shutdown_connections 2024-09-12T01:29:26.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:26.325+0000 7f9569000640 1 -- 172.21.15.1:0/2751349351 wait complete. 2024-09-12T01:29:26.976 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:26 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/2751349351' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-12T01:29:26.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:26 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/2751349351' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-12T01:29:27.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:26 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/2751349351' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-12T01:29:27.244 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-12T01:29:27.257 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:29:27.641 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:29:27.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:27 smithi053 ceph-mon[35158]: pgmap v394: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-12T01:29:28.027 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:27 smithi001 ceph-mon[30387]: pgmap v394: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-12T01:29:28.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:27 smithi104 ceph-mon[35219]: pgmap v394: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-12T01:29:28.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.267+0000 7f9abb200640 1 -- 172.21.15.1:0/714848416 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ab41049a0 msgr2=0x7f9ab41089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:28.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.267+0000 7f9abb200640 1 --2- 172.21.15.1:0/714848416 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ab41049a0 0x7f9ab41089f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0009900 tx=0x7f9aa002a410 comp rx=0 tx=0).stop 2024-09-12T01:29:28.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.267+0000 7f9abb200640 1 -- 172.21.15.1:0/714848416 shutdown_connections 2024-09-12T01:29:28.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.267+0000 7f9abb200640 1 --2- 172.21.15.1:0/714848416 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ab4109120 0x7f9ab4110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:28.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.267+0000 7f9abb200640 1 --2- 172.21.15.1:0/714848416 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ab41049a0 0x7f9ab41089f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:28.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.267+0000 7f9abb200640 1 --2- 172.21.15.1:0/714848416 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ab4103ff0 0x7f9ab41043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:28.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.267+0000 7f9abb200640 1 -- 172.21.15.1:0/714848416 >> 172.21.15.1:0/714848416 conn(0x7f9ab40ffa90 msgr2=0x7f9ab4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:28.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.267+0000 7f9abb200640 1 -- 172.21.15.1:0/714848416 shutdown_connections 2024-09-12T01:29:28.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.267+0000 7f9abb200640 1 -- 172.21.15.1:0/714848416 wait complete. 2024-09-12T01:29:28.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.268+0000 7f9abb200640 1 Processor -- start 2024-09-12T01:29:28.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.269+0000 7f9abb200640 1 -- start start 2024-09-12T01:29:28.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.269+0000 7f9abb200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ab4103ff0 0x7f9ab40739e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:28.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.269+0000 7f9abb200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ab41049a0 0x7f9ab406ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:28.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.269+0000 7f9abb200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ab4109120 0x7f9ab406cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:28.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.269+0000 7f9abb200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab4114120 con 0x7f9ab4103ff0 2024-09-12T01:29:28.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.269+0000 7f9abb200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab4113fa0 con 0x7f9ab4109120 2024-09-12T01:29:28.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.269+0000 7f9abb200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab41142a0 con 0x7f9ab41049a0 2024-09-12T01:29:28.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.269+0000 7f9ab3e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ab4103ff0 0x7f9ab40739e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:28.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.269+0000 7f9ab3e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ab4103ff0 0x7f9ab40739e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55080/0 (socket says 172.21.15.1:55080) 2024-09-12T01:29:28.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.269+0000 7f9ab3e00640 1 -- 172.21.15.1:0/3077586172 learned_addr learned my addr 172.21.15.1:0/3077586172 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:28.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.269+0000 7f9ab9200640 1 --2- 172.21.15.1:0/3077586172 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ab4109120 0x7f9ab406cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:28.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.269+0000 7f9ab3400640 1 --2- 172.21.15.1:0/3077586172 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ab41049a0 0x7f9ab406ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:28.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.270+0000 7f9ab3e00640 1 -- 172.21.15.1:0/3077586172 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ab41049a0 msgr2=0x7f9ab406ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:28.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.270+0000 7f9ab3e00640 1 --2- 172.21.15.1:0/3077586172 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ab41049a0 0x7f9ab406ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:28.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.270+0000 7f9ab3e00640 1 -- 172.21.15.1:0/3077586172 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ab4109120 msgr2=0x7f9ab406cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:28.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.270+0000 7f9ab3e00640 1 --2- 172.21.15.1:0/3077586172 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ab4109120 0x7f9ab406cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:28.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.270+0000 7f9ab3e00640 1 -- 172.21.15.1:0/3077586172 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab406d570 con 0x7f9ab4103ff0 2024-09-12T01:29:28.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.270+0000 7f9ab3e00640 1 --2- 172.21.15.1:0/3077586172 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ab4103ff0 0x7f9ab40739e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9aa400e9f0 tx=0x7f9aa400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:28.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.271+0000 7f9ab0c00640 1 -- 172.21.15.1:0/3077586172 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa400cd80 con 0x7f9ab4103ff0 2024-09-12T01:29:28.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.271+0000 7f9ab0c00640 1 -- 172.21.15.1:0/3077586172 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9aa4010910 con 0x7f9ab4103ff0 2024-09-12T01:29:28.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.271+0000 7f9abb200640 1 -- 172.21.15.1:0/3077586172 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ab406d860 con 0x7f9ab4103ff0 2024-09-12T01:29:28.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.271+0000 7f9ab0c00640 1 -- 172.21.15.1:0/3077586172 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa4018990 con 0x7f9ab4103ff0 2024-09-12T01:29:28.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.271+0000 7f9abb200640 1 -- 172.21.15.1:0/3077586172 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ab4071850 con 0x7f9ab4103ff0 2024-09-12T01:29:28.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.272+0000 7f9abb200640 1 -- 172.21.15.1:0/3077586172 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ab41054d0 con 0x7f9ab4103ff0 2024-09-12T01:29:28.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.274+0000 7f9ab0c00640 1 -- 172.21.15.1:0/3077586172 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f9aa4010ac0 con 0x7f9ab4103ff0 2024-09-12T01:29:28.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.276+0000 7f9ab0c00640 1 --2- 172.21.15.1:0/3077586172 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f9a980a2c00 0x7f9a980a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:28.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.276+0000 7f9ab0c00640 1 -- 172.21.15.1:0/3077586172 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f9aa40cf6a0 con 0x7f9ab4103ff0 2024-09-12T01:29:28.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.276+0000 7f9ab3400640 1 --2- 172.21.15.1:0/3077586172 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f9a980a2c00 0x7f9a980a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:28.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.277+0000 7f9ab3400640 1 --2- 172.21.15.1:0/3077586172 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f9a980a2c00 0x7f9a980a50c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0009230 tx=0x7f9aa00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:28.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.279+0000 7f9ab0c00640 1 -- 172.21.15.1:0/3077586172 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9aa409bbc0 con 0x7f9ab4103ff0 2024-09-12T01:29:28.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.506+0000 7f9abb200640 1 -- 172.21.15.1:0/3077586172 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ab4072240 con 0x7f9ab4103ff0 2024-09-12T01:29:28.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.507+0000 7f9ab0c00640 1 -- 172.21.15.1:0/3077586172 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11962 (secure 0 0 0) 0x7f9aa409fec0 con 0x7f9ab4103ff0 2024-09-12T01:29:28.506 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:29:28.506 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":28,"btime":"2024-09-12T01:27:46:332106+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14430,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/572192306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":572192306},{"type":"v1","addr":"172.21.15.1:6835","nonce":572192306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14442,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/1050734277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":1050734277},{"type":"v1","addr":"172.21.15.1:6837","nonce":1050734277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:26:48.555447+0000","modified":"2024-09-12T01:27:46.206001+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24545,"mds_1":24511,"mds_2":24506,"mds_3":24493,"mds_4":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14481":{"gid":14481,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/3651849993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":3651849993},{"type":"v1","addr":"172.21.15.1:6839","nonce":3651849993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14493":{"gid":14493,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2464781973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2464781973},{"type":"v1","addr":"172.21.15.1:6841","nonce":2464781973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2071825633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2071825633},{"type":"v1","addr":"172.21.15.104:6833","nonce":2071825633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/2930310711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2930310711},{"type":"v1","addr":"172.21.15.53:6833","nonce":2930310711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2032381725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2032381725},{"type":"v1","addr":"172.21.15.53:6835","nonce":2032381725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.104:6835/1547038540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1547038540},{"type":"v1","addr":"172.21.15.104:6835","nonce":1547038540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":23,"addr":"172.21.15.104:6837/464720247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":464720247},{"type":"v1","addr":"172.21.15.104:6837","nonce":464720247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":22,"addr":"172.21.15.53:6837/2848743754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2848743754},{"type":"v1","addr":"172.21.15.53:6837","nonce":2848743754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":21,"addr":"172.21.15.104:6839/1459976041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1459976041},{"type":"v1","addr":"172.21.15.104:6839","nonce":1459976041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.53:6839/4019389301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":4019389301},{"type":"v1","addr":"172.21.15.53:6839","nonce":4019389301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24487,24493,24506,24545,24511]},"id":1}]} 2024-09-12T01:29:28.507 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 28 2024-09-12T01:29:28.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.510+0000 7f9abb200640 1 -- 172.21.15.1:0/3077586172 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f9a980a2c00 msgr2=0x7f9a980a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:28.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.510+0000 7f9abb200640 1 --2- 172.21.15.1:0/3077586172 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f9a980a2c00 0x7f9a980a50c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0009230 tx=0x7f9aa00058d0 comp rx=0 tx=0).stop 2024-09-12T01:29:28.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.510+0000 7f9abb200640 1 -- 172.21.15.1:0/3077586172 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ab4103ff0 msgr2=0x7f9ab40739e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:28.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.510+0000 7f9abb200640 1 --2- 172.21.15.1:0/3077586172 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ab4103ff0 0x7f9ab40739e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9aa400e9f0 tx=0x7f9aa400eeb0 comp rx=0 tx=0).stop 2024-09-12T01:29:28.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.510+0000 7f9abb200640 1 -- 172.21.15.1:0/3077586172 shutdown_connections 2024-09-12T01:29:28.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.510+0000 7f9abb200640 1 --2- 172.21.15.1:0/3077586172 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ab4109120 0x7f9ab406cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:28.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.510+0000 7f9abb200640 1 --2- 172.21.15.1:0/3077586172 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f9a980a2c00 0x7f9a980a50c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:28.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.510+0000 7f9abb200640 1 --2- 172.21.15.1:0/3077586172 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ab41049a0 0x7f9ab406ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:28.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.510+0000 7f9abb200640 1 --2- 172.21.15.1:0/3077586172 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ab4103ff0 0x7f9ab40739e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:28.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.510+0000 7f9abb200640 1 -- 172.21.15.1:0/3077586172 >> 172.21.15.1:0/3077586172 conn(0x7f9ab40ffa90 msgr2=0x7f9ab4100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:28.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.511+0000 7f9abb200640 1 -- 172.21.15.1:0/3077586172 shutdown_connections 2024-09-12T01:29:28.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:28.511+0000 7f9abb200640 1 -- 172.21.15.1:0/3077586172 wait complete. 2024-09-12T01:29:28.900 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:28 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/3077586172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:29:28.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:28 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/3077586172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:29:29.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:28 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/3077586172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:29:29.481 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:29:29.867 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:29:29.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:29 smithi001 ceph-mon[30387]: pgmap v395: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-12T01:29:29.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:29 smithi053 ceph-mon[35158]: pgmap v395: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-12T01:29:30.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:29 smithi104 ceph-mon[35219]: pgmap v395: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-12T01:29:30.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.478+0000 7fc7a6a00640 1 -- 172.21.15.1:0/1434628360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7a0106f10 msgr2=0x7fc7a010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:30.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.478+0000 7fc7a6a00640 1 --2- 172.21.15.1:0/1434628360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7a0106f10 0x7fc7a010ea90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc798009ca0 tx=0x7fc79802a8c0 comp rx=0 tx=0).stop 2024-09-12T01:29:30.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.478+0000 7fc7a6a00640 1 -- 172.21.15.1:0/1434628360 shutdown_connections 2024-09-12T01:29:30.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.478+0000 7fc7a6a00640 1 --2- 172.21.15.1:0/1434628360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7a0106f10 0x7fc7a010ea90 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:30.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.478+0000 7fc7a6a00640 1 --2- 172.21.15.1:0/1434628360 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7a0102790 0x7fc7a01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:30.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.478+0000 7fc7a6a00640 1 --2- 172.21.15.1:0/1434628360 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7a0101de0 0x7fc7a01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:30.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.478+0000 7fc7a6a00640 1 -- 172.21.15.1:0/1434628360 >> 172.21.15.1:0/1434628360 conn(0x7fc7a00fd8c0 msgr2=0x7fc7a00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:30.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.478+0000 7fc7a6a00640 1 -- 172.21.15.1:0/1434628360 shutdown_connections 2024-09-12T01:29:30.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.478+0000 7fc7a6a00640 1 -- 172.21.15.1:0/1434628360 wait complete. 2024-09-12T01:29:30.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.479+0000 7fc7a6a00640 1 Processor -- start 2024-09-12T01:29:30.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.479+0000 7fc7a6a00640 1 -- start start 2024-09-12T01:29:30.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.479+0000 7fc7a6a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7a0101de0 0x7fc7a019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:30.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.479+0000 7fc7a6a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7a0102790 0x7fc7a019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:30.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.480+0000 7fc7a6a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7a0106f10 0x7fc7a019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:30.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.480+0000 7fc7a6a00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a0111f70 con 0x7fc7a0106f10 2024-09-12T01:29:30.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.480+0000 7fc7a6a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a0111df0 con 0x7fc7a0101de0 2024-09-12T01:29:30.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.480+0000 7fc7a6a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a01120f0 con 0x7fc7a0102790 2024-09-12T01:29:30.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.480+0000 7fc797400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7a0102790 0x7fc7a019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:30.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.480+0000 7fc79fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7a0101de0 0x7fc7a019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:30.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.480+0000 7fc7a4a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7a0106f10 0x7fc7a019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:30.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.480+0000 7fc797400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7a0102790 0x7fc7a019ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:34126/0 (socket says 172.21.15.1:34126) 2024-09-12T01:29:30.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.480+0000 7fc797400640 1 -- 172.21.15.1:0/1405098954 learned_addr learned my addr 172.21.15.1:0/1405098954 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:30.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.480+0000 7fc797400640 1 -- 172.21.15.1:0/1405098954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7a0101de0 msgr2=0x7fc7a019a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:30.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.480+0000 7fc797400640 1 --2- 172.21.15.1:0/1405098954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7a0101de0 0x7fc7a019a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:30.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.480+0000 7fc797400640 1 -- 172.21.15.1:0/1405098954 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7a0106f10 msgr2=0x7fc7a019eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:30.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.480+0000 7fc797400640 1 --2- 172.21.15.1:0/1405098954 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7a0106f10 0x7fc7a019eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:30.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.480+0000 7fc797400640 1 -- 172.21.15.1:0/1405098954 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a019f590 con 0x7fc7a0102790 2024-09-12T01:29:30.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.481+0000 7fc797400640 1 --2- 172.21.15.1:0/1405098954 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7a0102790 0x7fc7a019ab20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc78800e9f0 tx=0x7fc78800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:30.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.481+0000 7fc79d600640 1 -- 172.21.15.1:0/1405098954 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78800cd80 con 0x7fc7a0102790 2024-09-12T01:29:30.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.481+0000 7fc7a6a00640 1 -- 172.21.15.1:0/1405098954 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7a019f880 con 0x7fc7a0102790 2024-09-12T01:29:30.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.481+0000 7fc79d600640 1 -- 172.21.15.1:0/1405098954 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc788010910 con 0x7fc7a0102790 2024-09-12T01:29:30.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.481+0000 7fc79d600640 1 -- 172.21.15.1:0/1405098954 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7880188f0 con 0x7fc7a0102790 2024-09-12T01:29:30.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.481+0000 7fc7a6a00640 1 -- 172.21.15.1:0/1405098954 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7a01a7160 con 0x7fc7a0102790 2024-09-12T01:29:30.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.482+0000 7fc7a6a00640 1 -- 172.21.15.1:0/1405098954 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7a01021c0 con 0x7fc7a0102790 2024-09-12T01:29:30.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.484+0000 7fc79d600640 1 -- 172.21.15.1:0/1405098954 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fc788010ac0 con 0x7fc7a0102790 2024-09-12T01:29:30.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.485+0000 7fc79d600640 1 --2- 172.21.15.1:0/1405098954 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fc7840a2b80 0x7fc7840a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:30.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.486+0000 7fc79d600640 1 -- 172.21.15.1:0/1405098954 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fc7880cfb40 con 0x7fc7a0102790 2024-09-12T01:29:30.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.486+0000 7fc79fe00640 1 --2- 172.21.15.1:0/1405098954 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fc7840a2b80 0x7fc7840a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:30.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.486+0000 7fc79fe00640 1 --2- 172.21.15.1:0/1405098954 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fc7840a2b80 0x7fc7840a5040 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc7900095f0 tx=0x7fc790005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:30.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.488+0000 7fc79d600640 1 -- 172.21.15.1:0/1405098954 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc78809c060 con 0x7fc7a0102790 2024-09-12T01:29:30.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.713+0000 7fc7a6a00640 1 -- 172.21.15.1:0/1405098954 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7a01a7490 con 0x7fc7a0102790 2024-09-12T01:29:30.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.714+0000 7fc79d600640 1 -- 172.21.15.1:0/1405098954 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11962 (secure 0 0 0) 0x7fc7880a0360 con 0x7fc7a0102790 2024-09-12T01:29:30.714 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:29:30.714 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":28,"btime":"2024-09-12T01:27:46:332106+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14430,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/572192306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":572192306},{"type":"v1","addr":"172.21.15.1:6835","nonce":572192306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14442,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/1050734277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":1050734277},{"type":"v1","addr":"172.21.15.1:6837","nonce":1050734277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:26:48.555447+0000","modified":"2024-09-12T01:27:46.206001+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24545,"mds_1":24511,"mds_2":24506,"mds_3":24493,"mds_4":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14481":{"gid":14481,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/3651849993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":3651849993},{"type":"v1","addr":"172.21.15.1:6839","nonce":3651849993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14493":{"gid":14493,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2464781973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2464781973},{"type":"v1","addr":"172.21.15.1:6841","nonce":2464781973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2071825633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2071825633},{"type":"v1","addr":"172.21.15.104:6833","nonce":2071825633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/2930310711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2930310711},{"type":"v1","addr":"172.21.15.53:6833","nonce":2930310711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2032381725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2032381725},{"type":"v1","addr":"172.21.15.53:6835","nonce":2032381725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.104:6835/1547038540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1547038540},{"type":"v1","addr":"172.21.15.104:6835","nonce":1547038540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":23,"addr":"172.21.15.104:6837/464720247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":464720247},{"type":"v1","addr":"172.21.15.104:6837","nonce":464720247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":22,"addr":"172.21.15.53:6837/2848743754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2848743754},{"type":"v1","addr":"172.21.15.53:6837","nonce":2848743754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":21,"addr":"172.21.15.104:6839/1459976041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1459976041},{"type":"v1","addr":"172.21.15.104:6839","nonce":1459976041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.53:6839/4019389301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":4019389301},{"type":"v1","addr":"172.21.15.53:6839","nonce":4019389301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24487,24493,24506,24545,24511]},"id":1}]} 2024-09-12T01:29:30.714 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 28 2024-09-12T01:29:30.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.717+0000 7fc7a6a00640 1 -- 172.21.15.1:0/1405098954 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fc7840a2b80 msgr2=0x7fc7840a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:30.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.717+0000 7fc7a6a00640 1 --2- 172.21.15.1:0/1405098954 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fc7840a2b80 0x7fc7840a5040 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc7900095f0 tx=0x7fc790005f90 comp rx=0 tx=0).stop 2024-09-12T01:29:30.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.717+0000 7fc7a6a00640 1 -- 172.21.15.1:0/1405098954 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7a0102790 msgr2=0x7fc7a019ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:30.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.717+0000 7fc7a6a00640 1 --2- 172.21.15.1:0/1405098954 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7a0102790 0x7fc7a019ab20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc78800e9f0 tx=0x7fc78800eeb0 comp rx=0 tx=0).stop 2024-09-12T01:29:30.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.717+0000 7fc7a6a00640 1 -- 172.21.15.1:0/1405098954 shutdown_connections 2024-09-12T01:29:30.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.717+0000 7fc7a6a00640 1 --2- 172.21.15.1:0/1405098954 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7a0106f10 0x7fc7a019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:30.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.717+0000 7fc7a6a00640 1 --2- 172.21.15.1:0/1405098954 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fc7840a2b80 0x7fc7840a5040 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:30.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.717+0000 7fc7a6a00640 1 --2- 172.21.15.1:0/1405098954 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7a0102790 0x7fc7a019ab20 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:30.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.717+0000 7fc7a6a00640 1 --2- 172.21.15.1:0/1405098954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7a0101de0 0x7fc7a019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:30.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.717+0000 7fc7a6a00640 1 -- 172.21.15.1:0/1405098954 >> 172.21.15.1:0/1405098954 conn(0x7fc7a00fd8c0 msgr2=0x7fc7a00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:30.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.717+0000 7fc7a6a00640 1 -- 172.21.15.1:0/1405098954 shutdown_connections 2024-09-12T01:29:30.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:30.717+0000 7fc7a6a00640 1 -- 172.21.15.1:0/1405098954 wait complete. 2024-09-12T01:29:31.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:30 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/1405098954' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:29:31.075 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:30 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/1405098954' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:29:31.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:30 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/1405098954' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:29:31.483 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T01:29:31.867 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:29:32.061 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:31 smithi001 ceph-mon[30387]: pgmap v396: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-12T01:29:32.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:31 smithi104 ceph-mon[35219]: pgmap v396: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-12T01:29:32.141 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:31 smithi053 ceph-mon[35158]: pgmap v396: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-12T01:29:32.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.470+0000 7f2655600640 1 -- 172.21.15.1:0/1979524069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2650109120 msgr2=0x7f2650110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:32.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.470+0000 7f2655600640 1 --2- 172.21.15.1:0/1979524069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2650109120 0x7f2650110ca0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2648009ca0 tx=0x7f264802a8c0 comp rx=0 tx=0).stop 2024-09-12T01:29:32.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.470+0000 7f2655600640 1 -- 172.21.15.1:0/1979524069 shutdown_connections 2024-09-12T01:29:32.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.470+0000 7f2655600640 1 --2- 172.21.15.1:0/1979524069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2650109120 0x7f2650110ca0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:32.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.470+0000 7f2655600640 1 --2- 172.21.15.1:0/1979524069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26501049a0 0x7f26501089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:32.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.470+0000 7f2655600640 1 --2- 172.21.15.1:0/1979524069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2650103ff0 0x7f26501043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:32.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.470+0000 7f2655600640 1 -- 172.21.15.1:0/1979524069 >> 172.21.15.1:0/1979524069 conn(0x7f26500ffa90 msgr2=0x7f2650101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:32.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.470+0000 7f2655600640 1 -- 172.21.15.1:0/1979524069 shutdown_connections 2024-09-12T01:29:32.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.470+0000 7f2655600640 1 -- 172.21.15.1:0/1979524069 wait complete. 2024-09-12T01:29:32.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.471+0000 7f2655600640 1 Processor -- start 2024-09-12T01:29:32.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.471+0000 7f2655600640 1 -- start start 2024-09-12T01:29:32.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.471+0000 7f2655600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2650103ff0 0x7f265019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:32.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.471+0000 7f2655600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26501049a0 0x7f265019aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:32.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.471+0000 7f2655600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2650109120 0x7f265019ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:32.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.471+0000 7f2655600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2650114180 con 0x7f2650109120 2024-09-12T01:29:32.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.471+0000 7f2655600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2650114000 con 0x7f26501049a0 2024-09-12T01:29:32.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.471+0000 7f2655600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2650114300 con 0x7f2650103ff0 2024-09-12T01:29:32.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.472+0000 7f264f400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2650109120 0x7f265019ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:32.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.472+0000 7f2647e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26501049a0 0x7f265019aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:32.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.472+0000 7f264ea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2650103ff0 0x7f265019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:32.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.472+0000 7f2647e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26501049a0 0x7f265019aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:41538/0 (socket says 172.21.15.1:41538) 2024-09-12T01:29:32.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.472+0000 7f264ea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2650103ff0 0x7f265019a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:34142/0 (socket says 172.21.15.1:34142) 2024-09-12T01:29:32.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.472+0000 7f2647e00640 1 -- 172.21.15.1:0/1418049885 learned_addr learned my addr 172.21.15.1:0/1418049885 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:32.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.472+0000 7f264f400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2650109120 0x7f265019ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33352/0 (socket says 172.21.15.1:33352) 2024-09-12T01:29:32.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.472+0000 7f2647e00640 1 -- 172.21.15.1:0/1418049885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2650103ff0 msgr2=0x7f265019a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:32.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.472+0000 7f2647e00640 1 --2- 172.21.15.1:0/1418049885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2650103ff0 0x7f265019a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:32.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.472+0000 7f2647e00640 1 -- 172.21.15.1:0/1418049885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2650109120 msgr2=0x7f265019ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:32.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.472+0000 7f2647e00640 1 --2- 172.21.15.1:0/1418049885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2650109120 0x7f265019ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:32.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.472+0000 7f2647e00640 1 -- 172.21.15.1:0/1418049885 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f265019f550 con 0x7f26501049a0 2024-09-12T01:29:32.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.472+0000 7f264ea00640 1 --2- 172.21.15.1:0/1418049885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2650103ff0 0x7f265019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:29:32.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.472+0000 7f264f400640 1 --2- 172.21.15.1:0/1418049885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2650109120 0x7f265019ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:29:32.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.473+0000 7f2647e00640 1 --2- 172.21.15.1:0/1418049885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26501049a0 0x7f265019aae0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f263800d950 tx=0x7f263800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:32.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.473+0000 7f2647400640 1 -- 172.21.15.1:0/1418049885 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638017070 con 0x7f26501049a0 2024-09-12T01:29:32.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.473+0000 7f2655600640 1 -- 172.21.15.1:0/1418049885 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f265019f840 con 0x7f26501049a0 2024-09-12T01:29:32.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.473+0000 7f2647400640 1 -- 172.21.15.1:0/1418049885 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2638009c30 con 0x7f26501049a0 2024-09-12T01:29:32.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.473+0000 7f2647400640 1 -- 172.21.15.1:0/1418049885 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263800f6b0 con 0x7f26501049a0 2024-09-12T01:29:32.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.473+0000 7f2655600640 1 -- 172.21.15.1:0/1418049885 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26501a7120 con 0x7f26501049a0 2024-09-12T01:29:32.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.474+0000 7f2655600640 1 -- 172.21.15.1:0/1418049885 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26501043d0 con 0x7f26501049a0 2024-09-12T01:29:32.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.476+0000 7f2647400640 1 -- 172.21.15.1:0/1418049885 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f26380109d0 con 0x7f26501049a0 2024-09-12T01:29:32.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.478+0000 7f2647400640 1 --2- 172.21.15.1:0/1418049885 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f26340a2c00 0x7f26340a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:32.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.478+0000 7f2647400640 1 -- 172.21.15.1:0/1418049885 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f26380cee70 con 0x7f26501049a0 2024-09-12T01:29:32.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.478+0000 7f264ea00640 1 --2- 172.21.15.1:0/1418049885 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f26340a2c00 0x7f26340a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:32.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.479+0000 7f264ea00640 1 --2- 172.21.15.1:0/1418049885 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f26340a2c00 0x7f26340a50c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2640005d00 tx=0x7f264000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:32.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.480+0000 7f2647400640 1 -- 172.21.15.1:0/1418049885 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2638009de0 con 0x7f26501049a0 2024-09-12T01:29:32.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.668+0000 7f2655600640 1 -- 172.21.15.1:0/1418049885 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f26501a7450 con 0x7f26501049a0 2024-09-12T01:29:32.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.670+0000 7f2647400640 1 -- 172.21.15.1:0/1418049885 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+24751 (secure 0 0 0) 0x7f263809b410 con 0x7f26501049a0 2024-09-12T01:29:32.671 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:29:32.671 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":98,"fsid":"c0e4f180-70a4-11ef-bcea-c7b262605968","created":"2024-09-12T01:18:19.337406+0000","modified":"2024-09-12T01:28:17.407774+0000","last_up_change":"2024-09-12T01:25:50.601477+0000","last_in_change":"2024-09-12T01:25:27.674600+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:21:41.204242+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:26:43.066475+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:26:46.112771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T01:26:56.040130+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b9a396ae-bf97-462f-9ef9-b10ef6b9d337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6803","nonce":1125335136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6805","nonce":1125335136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6809","nonce":1125335136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6807","nonce":1125335136}]},"public_addr":"172.21.15.1:6803/1125335136","cluster_addr":"172.21.15.1:6805/1125335136","heartbeat_back_addr":"172.21.15.1:6809/1125335136","heartbeat_front_addr":"172.21.15.1:6807/1125335136","state":["exists","up"]},{"osd":1,"uuid":"dfe7a8a3-3700-4789-ac64-88660de9e0d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6801","nonce":3131049165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6803","nonce":3131049165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6807","nonce":3131049165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6805","nonce":3131049165}]},"public_addr":"172.21.15.53:6801/3131049165","cluster_addr":"172.21.15.53:6803/3131049165","heartbeat_back_addr":"172.21.15.53:6807/3131049165","heartbeat_front_addr":"172.21.15.53:6805/3131049165","state":["exists","up"]},{"osd":2,"uuid":"5d9d3603-ba7c-4115-9267-12abcb20d55b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6801","nonce":1133285636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6803","nonce":1133285636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6807","nonce":1133285636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6805","nonce":1133285636}]},"public_addr":"172.21.15.104:6801/1133285636","cluster_addr":"172.21.15.104:6803/1133285636","heartbeat_back_addr":"172.21.15.104:6807/1133285636","heartbeat_front_addr":"172.21.15.104:6805/1133285636","state":["exists","up"]},{"osd":3,"uuid":"1e063c66-41b4-4db1-a964-28e122ce628e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6811","nonce":1247468803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6813","nonce":1247468803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6817","nonce":1247468803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6815","nonce":1247468803}]},"public_addr":"172.21.15.1:6811/1247468803","cluster_addr":"172.21.15.1:6813/1247468803","heartbeat_back_addr":"172.21.15.1:6817/1247468803","heartbeat_front_addr":"172.21.15.1:6815/1247468803","state":["exists","up"]},{"osd":4,"uuid":"fe89ece0-2171-45d5-a06d-e400f77233e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6809","nonce":2673413639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6811","nonce":2673413639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6815","nonce":2673413639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6813","nonce":2673413639}]},"public_addr":"172.21.15.53:6809/2673413639","cluster_addr":"172.21.15.53:6811/2673413639","heartbeat_back_addr":"172.21.15.53:6815/2673413639","heartbeat_front_addr":"172.21.15.53:6813/2673413639","state":["exists","up"]},{"osd":5,"uuid":"daab795c-9942-4d16-aa34-8bba30a4b99b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6809","nonce":3131498790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6811","nonce":3131498790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6815","nonce":3131498790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6813","nonce":3131498790}]},"public_addr":"172.21.15.104:6809/3131498790","cluster_addr":"172.21.15.104:6811/3131498790","heartbeat_back_addr":"172.21.15.104:6815/3131498790","heartbeat_front_addr":"172.21.15.104:6813/3131498790","state":["exists","up"]},{"osd":6,"uuid":"f31d8633-e435-499b-9d46-0aa11a2481cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6819","nonce":3413067918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6821","nonce":3413067918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6825","nonce":3413067918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6823","nonce":3413067918}]},"public_addr":"172.21.15.1:6819/3413067918","cluster_addr":"172.21.15.1:6821/3413067918","heartbeat_back_addr":"172.21.15.1:6825/3413067918","heartbeat_front_addr":"172.21.15.1:6823/3413067918","state":["exists","up"]},{"osd":7,"uuid":"e64da49e-d31e-4801-b6cc-634a233b4761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6817","nonce":2841484136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6819","nonce":2841484136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6823","nonce":2841484136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6821","nonce":2841484136}]},"public_addr":"172.21.15.53:6817/2841484136","cluster_addr":"172.21.15.53:6819/2841484136","heartbeat_back_addr":"172.21.15.53:6823/2841484136","heartbeat_front_addr":"172.21.15.53:6821/2841484136","state":["exists","up"]},{"osd":8,"uuid":"fdfb5a42-dca7-4874-b479-176b4a191e21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6817","nonce":941453547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6819","nonce":941453547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6823","nonce":941453547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6821","nonce":941453547}]},"public_addr":"172.21.15.104:6817/941453547","cluster_addr":"172.21.15.104:6819/941453547","heartbeat_back_addr":"172.21.15.104:6823/941453547","heartbeat_front_addr":"172.21.15.104:6821/941453547","state":["exists","up"]},{"osd":9,"uuid":"7f81537a-cae5-4bb8-aec2-4eb07deb7958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6827","nonce":958287586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6829","nonce":958287586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6833","nonce":958287586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6831","nonce":958287586}]},"public_addr":"172.21.15.1:6827/958287586","cluster_addr":"172.21.15.1:6829/958287586","heartbeat_back_addr":"172.21.15.1:6833/958287586","heartbeat_front_addr":"172.21.15.1:6831/958287586","state":["exists","up"]},{"osd":10,"uuid":"e3c9c37d-46b9-4585-990d-69c447453bd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6825","nonce":3357909498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6827","nonce":3357909498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6831","nonce":3357909498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6829","nonce":3357909498}]},"public_addr":"172.21.15.53:6825/3357909498","cluster_addr":"172.21.15.53:6827/3357909498","heartbeat_back_addr":"172.21.15.53:6831/3357909498","heartbeat_front_addr":"172.21.15.53:6829/3357909498","state":["exists","up"]},{"osd":11,"uuid":"474b5345-f2ae-4e0e-a197-53ab11f58c04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6825","nonce":3381427987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6827","nonce":3381427987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6831","nonce":3381427987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6829","nonce":3381427987}]},"public_addr":"172.21.15.104:6825/3381427987","cluster_addr":"172.21.15.104:6827/3381427987","heartbeat_back_addr":"172.21.15.104:6831/3381427987","heartbeat_front_addr":"172.21.15.104:6829/3381427987","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:20:46.903181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:21:12.735556+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:21:37.868739+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:22:09.141228+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:22:36.438563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:23:02.220948+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:23:33.505881+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:24:00.982176+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:24:26.010444+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:24:56.711121+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:25:23.846495+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:25:48.815427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1373984726":"2024-09-13T01:19:14.580637+0000","172.21.15.1:0/1429963718":"2024-09-13T01:19:14.580637+0000","172.21.15.1:6801/4101281615":"2024-09-13T01:19:14.580637+0000","172.21.15.1:6800/4101281615":"2024-09-13T01:19:14.580637+0000","172.21.15.1:0/2564759586":"2024-09-13T01:19:14.580637+0000","172.21.15.1:6801/2935968453":"2024-09-13T01:18:45.882682+0000","172.21.15.1:6800/2935968453":"2024-09-13T01:18:45.882682+0000","172.21.15.1:0/1459848813":"2024-09-13T01:18:45.882682+0000","172.21.15.1:0/578605922":"2024-09-13T01:18:45.882682+0000","172.21.15.1:0/738035035":"2024-09-13T01:18:45.882682+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:29:32.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.673+0000 7f2655600640 1 -- 172.21.15.1:0/1418049885 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f26340a2c00 msgr2=0x7f26340a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:32.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.673+0000 7f2655600640 1 --2- 172.21.15.1:0/1418049885 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f26340a2c00 0x7f26340a50c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2640005d00 tx=0x7f264000b410 comp rx=0 tx=0).stop 2024-09-12T01:29:32.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.673+0000 7f2655600640 1 -- 172.21.15.1:0/1418049885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26501049a0 msgr2=0x7f265019aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:32.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.673+0000 7f2655600640 1 --2- 172.21.15.1:0/1418049885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26501049a0 0x7f265019aae0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f263800d950 tx=0x7f263800de10 comp rx=0 tx=0).stop 2024-09-12T01:29:32.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.673+0000 7f2655600640 1 -- 172.21.15.1:0/1418049885 shutdown_connections 2024-09-12T01:29:32.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.673+0000 7f2655600640 1 --2- 172.21.15.1:0/1418049885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2650109120 0x7f265019ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:32.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.673+0000 7f2655600640 1 --2- 172.21.15.1:0/1418049885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26501049a0 0x7f265019aae0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:32.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.673+0000 7f2655600640 1 --2- 172.21.15.1:0/1418049885 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f26340a2c00 0x7f26340a50c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:32.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.673+0000 7f2655600640 1 --2- 172.21.15.1:0/1418049885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2650103ff0 0x7f265019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:32.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.673+0000 7f2655600640 1 -- 172.21.15.1:0/1418049885 >> 172.21.15.1:0/1418049885 conn(0x7f26500ffa90 msgr2=0x7f26501005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:32.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.673+0000 7f2655600640 1 -- 172.21.15.1:0/1418049885 shutdown_connections 2024-09-12T01:29:32.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:32.673+0000 7f2655600640 1 -- 172.21.15.1:0/1418049885 wait complete. 2024-09-12T01:29:32.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:32 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/1418049885' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:29:33.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:32 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/1418049885' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:29:33.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:32 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/1418049885' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:29:33.466 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-12T01:26:48.555447+0000', 'modified': '2024-09-12T01:27:46.206001+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24545, 'mds_1': 24511, 'mds_2': 24506, 'mds_3': 24493, 'mds_4': 24487}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14481': {'gid': 14481, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.1:6839/3651849993', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6838', 'nonce': 3651849993}, {'type': 'v1', 'addr': '172.21.15.1:6839', 'nonce': 3651849993}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14493': {'gid': 14493, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.1:6841/2464781973', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6840', 'nonce': 2464781973}, {'type': 'v1', 'addr': '172.21.15.1:6841', 'nonce': 2464781973}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24439': {'gid': 24439, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.104:6833/2071825633', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6832', 'nonce': 2071825633}, {'type': 'v1', 'addr': '172.21.15.104:6833', 'nonce': 2071825633}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24446': {'gid': 24446, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.53:6833/2930310711', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6832', 'nonce': 2930310711}, {'type': 'v1', 'addr': '172.21.15.53:6833', 'nonce': 2930310711}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.53:6835/2032381725', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6834', 'nonce': 2032381725}, {'type': 'v1', 'addr': '172.21.15.53:6835', 'nonce': 2032381725}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 23, 'addr': '172.21.15.104:6835/1547038540', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6834', 'nonce': 1547038540}, {'type': 'v1', 'addr': '172.21.15.104:6835', 'nonce': 1547038540}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 23, 'addr': '172.21.15.104:6837/464720247', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6836', 'nonce': 464720247}, {'type': 'v1', 'addr': '172.21.15.104:6837', 'nonce': 464720247}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24506': {'gid': 24506, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.53:6837/2848743754', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6836', 'nonce': 2848743754}, {'type': 'v1', 'addr': '172.21.15.53:6837', 'nonce': 2848743754}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24511': {'gid': 24511, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.104:6839/1459976041', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6838', 'nonce': 1459976041}, {'type': 'v1', 'addr': '172.21.15.104:6839', 'nonce': 1459976041}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24545': {'gid': 24545, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.53:6839/4019389301', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6838', 'nonce': 4019389301}, {'type': 'v1', 'addr': '172.21.15.53:6839', 'nonce': 4019389301}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24545, 'qdb_cluster': [24487, 24493, 24506, 24545, 24511]} 2024-09-12T01:29:33.466 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-12T01:29:33.466 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph tell mds.f status 2024-09-12T01:29:33.845 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:29:34.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:33 smithi104 ceph-mon[35219]: pgmap v397: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T01:29:34.158 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:33 smithi001 ceph-mon[30387]: pgmap v397: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T01:29:34.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:33 smithi053 ceph-mon[35158]: pgmap v397: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T01:29:34.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.386+0000 7f5391a00640 1 -- 172.21.15.1:0/4022031407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f538c102b90 msgr2=0x7f538c102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:34.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.386+0000 7f5391a00640 1 --2- 172.21.15.1:0/4022031407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f538c102b90 0x7f538c102f70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f537c009900 tx=0x7f537c02a480 comp rx=0 tx=0).stop 2024-09-12T01:29:34.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.386+0000 7f5391a00640 1 -- 172.21.15.1:0/4022031407 shutdown_connections 2024-09-12T01:29:34.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.386+0000 7f5391a00640 1 --2- 172.21.15.1:0/4022031407 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f538c10bfc0 0x7f538c10e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:34.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.386+0000 7f5391a00640 1 --2- 172.21.15.1:0/4022031407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f538c1034b0 0x7f538c10b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:34.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.386+0000 7f5391a00640 1 --2- 172.21.15.1:0/4022031407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f538c102b90 0x7f538c102f70 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:34.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.386+0000 7f5391a00640 1 -- 172.21.15.1:0/4022031407 >> 172.21.15.1:0/4022031407 conn(0x7f538c0fc610 msgr2=0x7f538c0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:34.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.386+0000 7f5391a00640 1 -- 172.21.15.1:0/4022031407 shutdown_connections 2024-09-12T01:29:34.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.386+0000 7f5391a00640 1 -- 172.21.15.1:0/4022031407 wait complete. 2024-09-12T01:29:34.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.388+0000 7f5391a00640 1 Processor -- start 2024-09-12T01:29:34.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.388+0000 7f5391a00640 1 -- start start 2024-09-12T01:29:34.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.388+0000 7f5391a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f538c102b90 0x7f538c1007f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:34.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.388+0000 7f5391a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f538c1034b0 0x7f538c100d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:34.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.388+0000 7f5391a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f538c10bfc0 0x7f538c101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:34.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.388+0000 7f5391a00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f538c111dc0 con 0x7f538c1034b0 2024-09-12T01:29:34.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.388+0000 7f5391a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f538c111c40 con 0x7f538c102b90 2024-09-12T01:29:34.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.388+0000 7f5391a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f538c111f40 con 0x7f538c10bfc0 2024-09-12T01:29:34.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.389+0000 7f5383400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f538c1034b0 0x7f538c100d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:34.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.389+0000 7f5383400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f538c1034b0 0x7f538c100d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33374/0 (socket says 172.21.15.1:33374) 2024-09-12T01:29:34.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.389+0000 7f5383400640 1 -- 172.21.15.1:0/3979543355 learned_addr learned my addr 172.21.15.1:0/3979543355 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:34.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.389+0000 7f538be00640 1 --2- 172.21.15.1:0/3979543355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f538c102b90 0x7f538c1007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:34.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.389+0000 7f5391000640 1 --2- 172.21.15.1:0/3979543355 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f538c10bfc0 0x7f538c101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:34.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.389+0000 7f5383400640 1 -- 172.21.15.1:0/3979543355 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f538c10bfc0 msgr2=0x7f538c101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:34.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.389+0000 7f5383400640 1 --2- 172.21.15.1:0/3979543355 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f538c10bfc0 0x7f538c101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:34.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.389+0000 7f5383400640 1 -- 172.21.15.1:0/3979543355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f538c102b90 msgr2=0x7f538c1007f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:34.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.389+0000 7f5383400640 1 --2- 172.21.15.1:0/3979543355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f538c102b90 0x7f538c1007f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:34.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.389+0000 7f5383400640 1 -- 172.21.15.1:0/3979543355 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f538c1a67d0 con 0x7f538c1034b0 2024-09-12T01:29:34.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.390+0000 7f5383400640 1 --2- 172.21.15.1:0/3979543355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f538c1034b0 0x7f538c100d30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f537400e9f0 tx=0x7f537400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:34.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.390+0000 7f5389600640 1 -- 172.21.15.1:0/3979543355 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537400ccc0 con 0x7f538c1034b0 2024-09-12T01:29:34.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.390+0000 7f5391a00640 1 -- 172.21.15.1:0/3979543355 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f538c1a6ac0 con 0x7f538c1034b0 2024-09-12T01:29:34.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.390+0000 7f5389600640 1 -- 172.21.15.1:0/3979543355 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5374010910 con 0x7f538c1034b0 2024-09-12T01:29:34.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.390+0000 7f5389600640 1 -- 172.21.15.1:0/3979543355 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5374018920 con 0x7f538c1034b0 2024-09-12T01:29:34.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.390+0000 7f5391a00640 1 -- 172.21.15.1:0/3979543355 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f538c1a7000 con 0x7f538c1034b0 2024-09-12T01:29:34.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.393+0000 7f5389600640 1 -- 172.21.15.1:0/3979543355 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f5374010430 con 0x7f538c1034b0 2024-09-12T01:29:34.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.393+0000 7f5395aa5740 1 Processor -- start 2024-09-12T01:29:34.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.393+0000 7f5395aa5740 1 -- start start 2024-09-12T01:29:34.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.393+0000 7f5395aa5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571b0bd74e0 0x5571b0bd78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:34.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.393+0000 7f5395aa5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571b0bd7e00 0x5571b0be2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:34.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.394+0000 7f5395aa5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571b0be26c0 0x5571b0be4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:34.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.394+0000 7f5395aa5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5571b0b3d0b0 con 0x5571b0bd7e00 2024-09-12T01:29:34.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.394+0000 7f5395aa5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5571b0ae4ab0 con 0x5571b0bd74e0 2024-09-12T01:29:34.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.394+0000 7f5395aa5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5571b0b39070 con 0x5571b0be26c0 2024-09-12T01:29:34.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.394+0000 7f5391000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571b0bd74e0 0x5571b0bd78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:34.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.394+0000 7f538be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571b0bd7e00 0x5571b0be2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:34.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.394+0000 7f5391000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571b0bd74e0 0x5571b0bd78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:41556/0 (socket says 172.21.15.1:41556) 2024-09-12T01:29:34.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.394+0000 7f5391000640 1 -- 172.21.15.1:0/164331292 learned_addr learned my addr 172.21.15.1:0/164331292 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:34.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.394+0000 7f5383400640 1 --2- 172.21.15.1:0/164331292 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571b0be26c0 0x5571b0be4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:34.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.394+0000 7f5391000640 1 -- 172.21.15.1:0/164331292 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571b0be26c0 msgr2=0x5571b0be4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:34.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.394+0000 7f5391000640 1 --2- 172.21.15.1:0/164331292 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571b0be26c0 0x5571b0be4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:34.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.394+0000 7f5391000640 1 -- 172.21.15.1:0/164331292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571b0bd7e00 msgr2=0x5571b0be2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:34.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.394+0000 7f5391000640 1 --2- 172.21.15.1:0/164331292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571b0bd7e00 0x5571b0be2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:34.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.394+0000 7f5391000640 1 -- 172.21.15.1:0/164331292 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571b0af14a0 con 0x5571b0bd74e0 2024-09-12T01:29:34.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.395+0000 7f5391000640 1 --2- 172.21.15.1:0/164331292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571b0bd74e0 0x5571b0bd78c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5384009f50 tx=0x7f53840118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc558a9736a96b43 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:34.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.395+0000 7f5389600640 1 --2- 172.21.15.1:0/3979543355 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f53700a2b80 0x7f53700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:34.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.395+0000 7f5389600640 1 -- 172.21.15.1:0/3979543355 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f53740cf650 con 0x7f538c1034b0 2024-09-12T01:29:34.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.395+0000 7f5381600640 1 -- 172.21.15.1:0/164331292 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5384004d20 con 0x5571b0bd74e0 2024-09-12T01:29:34.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.395+0000 7f538be00640 1 --2- 172.21.15.1:0/3979543355 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f53700a2b80 0x7f53700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:34.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.395+0000 7f5381600640 1 -- 172.21.15.1:0/164331292 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5384004050 con 0x5571b0bd74e0 2024-09-12T01:29:34.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.395+0000 7f5381600640 1 -- 172.21.15.1:0/164331292 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5384005280 con 0x5571b0bd74e0 2024-09-12T01:29:34.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.396+0000 7f5395aa5740 1 -- 172.21.15.1:0/164331292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571b0bd74e0 msgr2=0x5571b0bd78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:34.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.396+0000 7f538be00640 1 --2- 172.21.15.1:0/3979543355 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f53700a2b80 0x7f53700a5040 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f537c009230 tx=0x7f537c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:34.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.396+0000 7f5395aa5740 1 --2- 172.21.15.1:0/164331292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571b0bd74e0 0x5571b0bd78c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5384009f50 tx=0x7f53840118d0 comp rx=0 tx=0).stop 2024-09-12T01:29:34.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.396+0000 7f5395aa5740 1 -- 172.21.15.1:0/164331292 shutdown_connections 2024-09-12T01:29:34.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.396+0000 7f5395aa5740 1 --2- 172.21.15.1:0/164331292 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571b0be26c0 0x5571b0be4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:34.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.396+0000 7f5395aa5740 1 --2- 172.21.15.1:0/164331292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571b0bd7e00 0x5571b0be2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:34.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.396+0000 7f5395aa5740 1 --2- 172.21.15.1:0/164331292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571b0bd74e0 0x5571b0bd78c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:34.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.396+0000 7f5395aa5740 1 -- 172.21.15.1:0/164331292 >> 172.21.15.1:0/164331292 conn(0x5571b0bd3a70 msgr2=0x5571b0bd3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:34.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.396+0000 7f5395aa5740 1 -- 172.21.15.1:0/164331292 shutdown_connections 2024-09-12T01:29:34.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.396+0000 7f5395aa5740 1 -- 172.21.15.1:0/164331292 wait complete. 2024-09-12T01:29:34.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.397+0000 7f5395aa5740 1 -- start start 2024-09-12T01:29:34.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.397+0000 7f5395aa5740 1 Processor -- start 2024-09-12T01:29:34.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.397+0000 7f5395aa5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571b0bd74e0 0x5571b0bdf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:34.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.397+0000 7f5395aa5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571b0bd7e00 0x5571b0bdfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:34.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.397+0000 7f5395aa5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571b0be26c0 0x5571b0bdff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:34.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.397+0000 7f5395aa5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5571b0be5330 con 0x5571b0bd74e0 2024-09-12T01:29:34.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.397+0000 7f5395aa5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5571b0ac5920 con 0x5571b0be26c0 2024-09-12T01:29:34.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.397+0000 7f5395aa5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5571b0be54b0 con 0x5571b0bd7e00 2024-09-12T01:29:34.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.397+0000 7f5395aa5740 10 client.0 _ll_register_callbacks cb 0x5571b0ae98a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:29:34.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.398+0000 7f538be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571b0bd74e0 0x5571b0bdf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:34.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.398+0000 7f538be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571b0bd74e0 0x5571b0bdf4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33380/0 (socket says 172.21.15.1:33380) 2024-09-12T01:29:34.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.398+0000 7f538be00640 1 -- 172.21.15.1:0/3292604437 learned_addr learned my addr 172.21.15.1:0/3292604437 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:34.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.398+0000 7f5383400640 1 --2- 172.21.15.1:0/3292604437 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571b0bd7e00 0x5571b0bdfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:34.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.398+0000 7f5391000640 1 --2- 172.21.15.1:0/3292604437 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571b0be26c0 0x5571b0bdff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:34.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.398+0000 7f538be00640 1 -- 172.21.15.1:0/3292604437 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571b0bd7e00 msgr2=0x5571b0bdfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:34.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.398+0000 7f538be00640 1 --2- 172.21.15.1:0/3292604437 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571b0bd7e00 0x5571b0bdfa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:34.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.398+0000 7f538be00640 1 -- 172.21.15.1:0/3292604437 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571b0be26c0 msgr2=0x5571b0bdff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:34.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.398+0000 7f538be00640 1 --2- 172.21.15.1:0/3292604437 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571b0be26c0 0x5571b0bdff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:34.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.398+0000 7f538be00640 1 -- 172.21.15.1:0/3292604437 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571b08ae2b0 con 0x5571b0bd74e0 2024-09-12T01:29:34.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.398+0000 7f538be00640 1 --2- 172.21.15.1:0/3292604437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571b0bd74e0 0x5571b0bdf4f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f537c033000 tx=0x7f537c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56036e3d1e687375 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:34.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.398+0000 7f537a000640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-12T01:29:34.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.399+0000 7f537a000640 1 -- 172.21.15.1:0/3292604437 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537c04e070 con 0x5571b0bd74e0 2024-09-12T01:29:34.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.399+0000 7f537a000640 1 -- 172.21.15.1:0/3292604437 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f537c003b80 con 0x5571b0bd74e0 2024-09-12T01:29:34.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.399+0000 7f5395aa5740 1 -- 172.21.15.1:0/3292604437 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5384004050 con 0x5571b0bd74e0 2024-09-12T01:29:34.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.399+0000 7f537a000640 1 -- 172.21.15.1:0/3292604437 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f537c05a050 con 0x5571b0bd74e0 2024-09-12T01:29:34.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.399+0000 7f5395aa5740 10 client.15096 fetch_fsmap learned FSMap version 28 2024-09-12T01:29:34.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.399+0000 7f5395aa5740 1 -- 172.21.15.1:0/3292604437 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5571b0bf0f30 con 0x5571b0bd74e0 2024-09-12T01:29:34.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.399+0000 7f537a000640 1 -- 172.21.15.1:0/3292604437 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5384004050 con 0x5571b0bd74e0 2024-09-12T01:29:34.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.399+0000 7f537a000640 1 -- 172.21.15.1:0/3292604437 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f537c03d450 con 0x5571b0bd74e0 2024-09-12T01:29:34.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.399+0000 7f5395aa5740 10 client.15096 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:29:34.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.400+0000 7f5395aa5740 10 client.15096 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-12T01:29:34.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.400+0000 7f5395aa5740 20 client.15096 populate_metadata read hostname 'smithi001' 2024-09-12T01:29:34.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.400+0000 7f5395aa5740 1 --2- 172.21.15.1:0/3292604437 >> [v2:172.21.15.104:6834/1547038540,v1:172.21.15.104:6835/1547038540] conn(0x5571b0bf3550 0x5571b0bf49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:34.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.400+0000 7f5395aa5740 4 client.15096 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:29:34.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.400+0000 7f5395aa5740 1 -- 172.21.15.1:0/3292604437 --> [v2:172.21.15.104:6834/1547038540,v1:172.21.15.104:6835/1547038540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571b0b4a620 con 0x5571b0bf3550 2024-09-12T01:29:34.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.400+0000 7f5383400640 1 --2- 172.21.15.1:0/3292604437 >> [v2:172.21.15.104:6834/1547038540,v1:172.21.15.104:6835/1547038540] conn(0x5571b0bf3550 0x5571b0bf49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:34.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.400+0000 7f5383400640 1 --2- 172.21.15.1:0/3292604437 >> [v2:172.21.15.104:6834/1547038540,v1:172.21.15.104:6835/1547038540] conn(0x5571b0bf3550 0x5571b0bf49c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=80098844e37ec02d server_cookie=8823ea47bb46a65b in_seq=0 out_seq=0 2024-09-12T01:29:34.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.400+0000 7f537a000640 10 client.15096 ms_handle_connect on v2:172.21.15.104:6834/1547038540 2024-09-12T01:29:34.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.401+0000 7f537a000640 1 -- 172.21.15.1:0/3292604437 <== mds.4 v2:172.21.15.104:6834/1547038540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5571b0b4a620 con 0x5571b0bf3550 2024-09-12T01:29:34.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.401+0000 7f537a000640 10 client.15096 handle_command_reply: tid=0 2024-09-12T01:29:34.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.401+0000 7f5395aa5740 1 client.15096 shutdown 2024-09-12T01:29:34.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.402+0000 7f5395aa5740 1 -- 172.21.15.1:0/3292604437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571b0bd74e0 msgr2=0x5571b0bdf4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:34.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.402+0000 7f5395aa5740 1 --2- 172.21.15.1:0/3292604437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571b0bd74e0 0x5571b0bdf4f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f537c033000 tx=0x7f537c009260 comp rx=0 tx=0).stop 2024-09-12T01:29:34.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.402+0000 7f5395aa5740 1 -- 172.21.15.1:0/3292604437 shutdown_connections 2024-09-12T01:29:34.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.402+0000 7f5395aa5740 1 --2- 172.21.15.1:0/3292604437 >> [v2:172.21.15.104:6834/1547038540,v1:172.21.15.104:6835/1547038540] conn(0x5571b0bf3550 0x5571b0bf49c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:34.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.402+0000 7f5395aa5740 1 --2- 172.21.15.1:0/3292604437 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571b0be26c0 0x5571b0bdff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:34.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.402+0000 7f5395aa5740 1 --2- 172.21.15.1:0/3292604437 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571b0bd7e00 0x5571b0bdfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:34.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.402+0000 7f5395aa5740 1 --2- 172.21.15.1:0/3292604437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571b0bd74e0 0x5571b0bdf4f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:34.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.402+0000 7f537a000640 0 client.15096 ms_handle_reset on v2:172.21.15.104:6834/1547038540 2024-09-12T01:29:34.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.402+0000 7f5395aa5740 1 -- 172.21.15.1:0/3292604437 >> 172.21.15.1:0/3292604437 conn(0x5571b0bd3a70 msgr2=0x5571b0bd2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:34.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.402+0000 7f5395aa5740 1 -- 172.21.15.1:0/3292604437 shutdown_connections 2024-09-12T01:29:34.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.402+0000 7f5395aa5740 1 -- 172.21.15.1:0/3292604437 wait complete. 2024-09-12T01:29:34.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.402+0000 7f5395aa5740 20 client.15096 trim_cache size 0 max 16384 2024-09-12T01:29:34.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.421+0000 7f5395aa5740 1 Processor -- start 2024-09-12T01:29:34.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.421+0000 7f5395aa5740 1 -- start start 2024-09-12T01:29:34.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.421+0000 7f5395aa5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571b0bd74e0 0x5571b0bf49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:34.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.421+0000 7f5395aa5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571b0bd7e00 0x5571b0bd9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:34.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.421+0000 7f5395aa5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571b0be26c0 0x5571b0bd3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:34.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.421+0000 7f5395aa5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5571b0bf2d00 con 0x5571b0be26c0 2024-09-12T01:29:34.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.421+0000 7f5395aa5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5571b0bf33d0 con 0x5571b0bd74e0 2024-09-12T01:29:34.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.421+0000 7f5395aa5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5571b0bf3aa0 con 0x5571b0bd7e00 2024-09-12T01:29:34.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.422+0000 7f538be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571b0bd74e0 0x5571b0bf49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:34.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.422+0000 7f5391000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571b0be26c0 0x5571b0bd3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:34.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.422+0000 7f538be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571b0bd74e0 0x5571b0bf49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:41574/0 (socket says 172.21.15.1:41574) 2024-09-12T01:29:34.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.422+0000 7f538be00640 1 -- 172.21.15.1:0/410048543 learned_addr learned my addr 172.21.15.1:0/410048543 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:34.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.422+0000 7f5391000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571b0be26c0 0x5571b0bd3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33382/0 (socket says 172.21.15.1:33382) 2024-09-12T01:29:34.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.422+0000 7f5383400640 1 --2- 172.21.15.1:0/410048543 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571b0bd7e00 0x5571b0bd9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:34.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.422+0000 7f538be00640 1 -- 172.21.15.1:0/410048543 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571b0bd7e00 msgr2=0x5571b0bd9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:34.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.422+0000 7f538be00640 1 --2- 172.21.15.1:0/410048543 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571b0bd7e00 0x5571b0bd9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:34.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.422+0000 7f538be00640 1 -- 172.21.15.1:0/410048543 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571b0be26c0 msgr2=0x5571b0bd3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:34.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.422+0000 7f538be00640 1 --2- 172.21.15.1:0/410048543 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571b0be26c0 0x5571b0bd3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:34.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.422+0000 7f538be00640 1 -- 172.21.15.1:0/410048543 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571b0bd38a0 con 0x5571b0bd74e0 2024-09-12T01:29:34.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.422+0000 7f538be00640 1 --2- 172.21.15.1:0/410048543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571b0bd74e0 0x5571b0bf49c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f537c004770 tx=0x7f537c0315b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aadf227e13292ee2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:34.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.423+0000 7f5380c00640 1 -- 172.21.15.1:0/410048543 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537c04e070 con 0x5571b0bd74e0 2024-09-12T01:29:34.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.423+0000 7f5380c00640 1 -- 172.21.15.1:0/410048543 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f537c063070 con 0x5571b0bd74e0 2024-09-12T01:29:34.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.423+0000 7f5380c00640 1 -- 172.21.15.1:0/410048543 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537c035040 con 0x5571b0bd74e0 2024-09-12T01:29:34.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.423+0000 7f5395aa5740 1 -- 172.21.15.1:0/410048543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571b0bd74e0 msgr2=0x5571b0bf49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:34.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.423+0000 7f5395aa5740 1 --2- 172.21.15.1:0/410048543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571b0bd74e0 0x5571b0bf49c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f537c004770 tx=0x7f537c0315b0 comp rx=0 tx=0).stop 2024-09-12T01:29:34.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.424+0000 7f5395aa5740 1 -- 172.21.15.1:0/410048543 shutdown_connections 2024-09-12T01:29:34.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.424+0000 7f5395aa5740 1 --2- 172.21.15.1:0/410048543 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571b0be26c0 0x5571b0bd3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:34.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.424+0000 7f5395aa5740 1 --2- 172.21.15.1:0/410048543 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571b0bd7e00 0x5571b0bd9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:34.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.424+0000 7f5395aa5740 1 --2- 172.21.15.1:0/410048543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571b0bd74e0 0x5571b0bf49c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:34.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.424+0000 7f5395aa5740 1 -- 172.21.15.1:0/410048543 >> 172.21.15.1:0/410048543 conn(0x5571b0bd3a70 msgr2=0x5571b0bed700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:34.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.424+0000 7f5395aa5740 1 -- 172.21.15.1:0/410048543 shutdown_connections 2024-09-12T01:29:34.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.424+0000 7f5395aa5740 1 -- 172.21.15.1:0/410048543 wait complete. 2024-09-12T01:29:34.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.424+0000 7f5395aa5740 1 -- start start 2024-09-12T01:29:34.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.424+0000 7f5395aa5740 1 Processor -- start 2024-09-12T01:29:34.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.425+0000 7f5395aa5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571b0bd74e0 0x5571b0bf8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:34.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.425+0000 7f5395aa5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571b0bd7e00 0x5571b0bf9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:34.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.425+0000 7f5395aa5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571b0be26c0 0x5571b0bf9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:34.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.425+0000 7f5395aa5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5571b0bf4840 con 0x5571b0be26c0 2024-09-12T01:29:34.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.425+0000 7f5395aa5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5571b0bf4170 con 0x5571b0bd74e0 2024-09-12T01:29:34.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.425+0000 7f5395aa5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5571b0bf4f10 con 0x5571b0bd7e00 2024-09-12T01:29:34.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.425+0000 7f5395aa5740 10 client.0 _ll_register_callbacks cb 0x7f537c0318e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:29:34.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.425+0000 7f538be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571b0bd74e0 0x5571b0bf8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:34.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.425+0000 7f5391000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571b0be26c0 0x5571b0bf9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:34.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.425+0000 7f538be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571b0bd74e0 0x5571b0bf8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:41578/0 (socket says 172.21.15.1:41578) 2024-09-12T01:29:34.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.425+0000 7f538be00640 1 -- 172.21.15.1:0/4065327680 learned_addr learned my addr 172.21.15.1:0/4065327680 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:34.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.425+0000 7f5383400640 1 --2- 172.21.15.1:0/4065327680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571b0bd7e00 0x5571b0bf9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:34.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.426+0000 7f538be00640 1 -- 172.21.15.1:0/4065327680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571b0bd7e00 msgr2=0x5571b0bf9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:34.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.426+0000 7f538be00640 1 --2- 172.21.15.1:0/4065327680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571b0bd7e00 0x5571b0bf9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:34.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.426+0000 7f538be00640 1 -- 172.21.15.1:0/4065327680 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571b0be26c0 msgr2=0x5571b0bf9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:34.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.426+0000 7f538be00640 1 --2- 172.21.15.1:0/4065327680 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571b0be26c0 0x5571b0bf9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:34.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.426+0000 7f538be00640 1 -- 172.21.15.1:0/4065327680 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571b0c656c0 con 0x5571b0bd74e0 2024-09-12T01:29:34.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.426+0000 7f5383400640 1 --2- 172.21.15.1:0/4065327680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571b0bd7e00 0x5571b0bf9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:29:34.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.426+0000 7f5391000640 1 --2- 172.21.15.1:0/4065327680 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571b0be26c0 0x5571b0bf9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:29:34.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.426+0000 7f538be00640 1 --2- 172.21.15.1:0/4065327680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571b0bd74e0 0x5571b0bf8cc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f537c02a9a0 tx=0x7f537c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfe47aafa2d6fd06 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:34.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.426+0000 7f537a000640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-12T01:29:34.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.426+0000 7f537a000640 1 -- 172.21.15.1:0/4065327680 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537c04e040 con 0x5571b0bd74e0 2024-09-12T01:29:34.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.426+0000 7f537a000640 1 -- 172.21.15.1:0/4065327680 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f537c003980 con 0x5571b0bd74e0 2024-09-12T01:29:34.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.426+0000 7f5395aa5740 1 -- 172.21.15.1:0/4065327680 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f537c063070 con 0x5571b0bd74e0 2024-09-12T01:29:34.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.427+0000 7f537a000640 1 -- 172.21.15.1:0/4065327680 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f537c074050 con 0x5571b0bd74e0 2024-09-12T01:29:34.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.427+0000 7f5395aa5740 10 client.25136 fetch_fsmap learned FSMap version 28 2024-09-12T01:29:34.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.427+0000 7f5395aa5740 1 -- 172.21.15.1:0/4065327680 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5571b0c669a0 con 0x5571b0bd74e0 2024-09-12T01:29:34.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.427+0000 7f537a000640 1 -- 172.21.15.1:0/4065327680 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f537c063070 con 0x5571b0bd74e0 2024-09-12T01:29:34.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.427+0000 7f537a000640 1 -- 172.21.15.1:0/4065327680 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f537c067450 con 0x5571b0bd74e0 2024-09-12T01:29:34.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.427+0000 7f5395aa5740 10 client.25136 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:29:34.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.427+0000 7f5395aa5740 10 client.25136 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-12T01:29:34.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.427+0000 7f5395aa5740 20 client.25136 populate_metadata read hostname 'smithi001' 2024-09-12T01:29:34.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.427+0000 7f5395aa5740 1 --2- 172.21.15.1:0/4065327680 >> [v2:172.21.15.104:6834/1547038540,v1:172.21.15.104:6835/1547038540] conn(0x5571b0c6b140 0x5571b0c69e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:34.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.427+0000 7f5395aa5740 4 client.25136 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T01:29:34.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.427+0000 7f5395aa5740 1 -- 172.21.15.1:0/4065327680 --> [v2:172.21.15.104:6834/1547038540,v1:172.21.15.104:6835/1547038540] -- command(tid 0: {"prefix": "status"}) -- 0x5571b0b44b30 con 0x5571b0c6b140 2024-09-12T01:29:34.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.428+0000 7f5383400640 1 --2- 172.21.15.1:0/4065327680 >> [v2:172.21.15.104:6834/1547038540,v1:172.21.15.104:6835/1547038540] conn(0x5571b0c6b140 0x5571b0c69e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:34.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.428+0000 7f5383400640 1 --2- 172.21.15.1:0/4065327680 >> [v2:172.21.15.104:6834/1547038540,v1:172.21.15.104:6835/1547038540] conn(0x5571b0c6b140 0x5571b0c69e50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=65d58bc53641e6a5 server_cookie=ca574875a7527281 in_seq=0 out_seq=0 2024-09-12T01:29:34.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.428+0000 7f537a000640 10 client.25136 ms_handle_connect on v2:172.21.15.104:6834/1547038540 2024-09-12T01:29:34.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.429+0000 7f537a000640 1 -- 172.21.15.1:0/4065327680 <== mds.4 v2:172.21.15.104:6834/1547038540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5571b0b44b30 con 0x5571b0c6b140 2024-09-12T01:29:34.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.429+0000 7f537a000640 10 client.25136 handle_command_reply: tid=0 2024-09-12T01:29:34.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.429+0000 7f5395aa5740 1 client.25136 shutdown 2024-09-12T01:29:34.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.429+0000 7f5395aa5740 1 -- 172.21.15.1:0/4065327680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571b0bd74e0 msgr2=0x5571b0bf8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:34.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.429+0000 7f5395aa5740 1 --2- 172.21.15.1:0/4065327680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571b0bd74e0 0x5571b0bf8cc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f537c02a9a0 tx=0x7f537c009260 comp rx=0 tx=0).stop 2024-09-12T01:29:34.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.429+0000 7f5395aa5740 1 -- 172.21.15.1:0/4065327680 shutdown_connections 2024-09-12T01:29:34.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.429+0000 7f5395aa5740 1 --2- 172.21.15.1:0/4065327680 >> [v2:172.21.15.104:6834/1547038540,v1:172.21.15.104:6835/1547038540] conn(0x5571b0c6b140 0x5571b0c69e50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:34.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.429+0000 7f5395aa5740 1 --2- 172.21.15.1:0/4065327680 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571b0be26c0 0x5571b0bf9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:34.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.429+0000 7f5395aa5740 1 --2- 172.21.15.1:0/4065327680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571b0bd7e00 0x5571b0bf9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:34.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.429+0000 7f5395aa5740 1 --2- 172.21.15.1:0/4065327680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571b0bd74e0 0x5571b0bf8cc0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:34.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.429+0000 7f5395aa5740 1 -- 172.21.15.1:0/4065327680 >> 172.21.15.1:0/4065327680 conn(0x5571b0bd3a70 msgr2=0x5571b0bedf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:34.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.429+0000 7f537a000640 0 client.25136 ms_handle_reset on v2:172.21.15.104:6834/1547038540 2024-09-12T01:29:34.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.429+0000 7f5395aa5740 1 -- 172.21.15.1:0/4065327680 shutdown_connections 2024-09-12T01:29:34.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.429+0000 7f5395aa5740 1 -- 172.21.15.1:0/4065327680 wait complete. 2024-09-12T01:29:34.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.429+0000 7f5395aa5740 20 client.25136 trim_cache size 0 max 16384 2024-09-12T01:29:34.432 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-12T01:29:34.432 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "c0e4f180-70a4-11ef-bcea-c7b262605968", 2024-09-12T01:29:34.432 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 4, 2024-09-12T01:29:34.432 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24487, 2024-09-12T01:29:34.432 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-09-12T01:29:34.432 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-09-12T01:29:34.432 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-09-12T01:29:34.432 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 110.113136314, 2024-09-12T01:29:34.433 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 28, 2024-09-12T01:29:34.433 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 98, 2024-09-12T01:29:34.433 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 96, 2024-09-12T01:29:34.433 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 198.05390786699999 2024-09-12T01:29:34.433 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-12T01:29:34.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.430+0000 7f5391a00640 1 -- 172.21.15.1:0/3979543355 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f53700a2b80 msgr2=0x7f53700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:34.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.430+0000 7f5391a00640 1 --2- 172.21.15.1:0/3979543355 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f53700a2b80 0x7f53700a5040 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f537c009230 tx=0x7f537c0058d0 comp rx=0 tx=0).stop 2024-09-12T01:29:34.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.430+0000 7f5391a00640 1 -- 172.21.15.1:0/3979543355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f538c1034b0 msgr2=0x7f538c100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:34.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.430+0000 7f5391a00640 1 --2- 172.21.15.1:0/3979543355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f538c1034b0 0x7f538c100d30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f537400e9f0 tx=0x7f537400eeb0 comp rx=0 tx=0).stop 2024-09-12T01:29:34.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.430+0000 7f5391a00640 1 -- 172.21.15.1:0/3979543355 shutdown_connections 2024-09-12T01:29:34.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.430+0000 7f5391a00640 1 --2- 172.21.15.1:0/3979543355 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f53700a2b80 0x7f53700a5040 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:34.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.430+0000 7f5391a00640 1 --2- 172.21.15.1:0/3979543355 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f538c10bfc0 0x7f538c101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:34.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.430+0000 7f5391a00640 1 --2- 172.21.15.1:0/3979543355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f538c1034b0 0x7f538c100d30 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:34.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.430+0000 7f5391a00640 1 --2- 172.21.15.1:0/3979543355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f538c102b90 0x7f538c1007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:34.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.430+0000 7f5391a00640 1 -- 172.21.15.1:0/3979543355 >> 172.21.15.1:0/3979543355 conn(0x7f538c0fc610 msgr2=0x7f538c0fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:34.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.430+0000 7f5391a00640 1 -- 172.21.15.1:0/3979543355 shutdown_connections 2024-09-12T01:29:34.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:34.430+0000 7f5391a00640 1 -- 172.21.15.1:0/3979543355 wait complete. 2024-09-12T01:29:35.356 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-12T01:29:35.738 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:29:36.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:35 smithi104 ceph-mon[35219]: pgmap v398: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T01:29:36.075 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:35 smithi001 ceph-mon[30387]: pgmap v398: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T01:29:36.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:35 smithi053 ceph-mon[35158]: pgmap v398: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T01:29:36.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.332+0000 7ff2a4c00640 1 -- 172.21.15.1:0/2645506544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2a0103ff0 msgr2=0x7ff2a01043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:36.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.332+0000 7ff2a4c00640 1 --2- 172.21.15.1:0/2645506544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2a0103ff0 0x7ff2a01043d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff290009900 tx=0x7ff29002a3f0 comp rx=0 tx=0).stop 2024-09-12T01:29:36.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.332+0000 7ff2a4c00640 1 -- 172.21.15.1:0/2645506544 shutdown_connections 2024-09-12T01:29:36.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.332+0000 7ff2a4c00640 1 --2- 172.21.15.1:0/2645506544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2a0109120 0x7ff2a0110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:36.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.332+0000 7ff2a4c00640 1 --2- 172.21.15.1:0/2645506544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2a01049a0 0x7ff2a01089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:36.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.332+0000 7ff2a4c00640 1 --2- 172.21.15.1:0/2645506544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2a0103ff0 0x7ff2a01043d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:36.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.332+0000 7ff2a4c00640 1 -- 172.21.15.1:0/2645506544 >> 172.21.15.1:0/2645506544 conn(0x7ff2a00ffa90 msgr2=0x7ff2a0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:36.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.332+0000 7ff2a4c00640 1 -- 172.21.15.1:0/2645506544 shutdown_connections 2024-09-12T01:29:36.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.333+0000 7ff2a4c00640 1 -- 172.21.15.1:0/2645506544 wait complete. 2024-09-12T01:29:36.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.333+0000 7ff2a4c00640 1 Processor -- start 2024-09-12T01:29:36.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.334+0000 7ff2a4c00640 1 -- start start 2024-09-12T01:29:36.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.334+0000 7ff2a4c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2a0103ff0 0x7ff2a0076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:36.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.334+0000 7ff2a4c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2a01049a0 0x7ff2a0076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:36.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.334+0000 7ff2a4c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2a0109120 0x7ff2a0077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:36.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.334+0000 7ff2a4c00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a01141a0 con 0x7ff2a0103ff0 2024-09-12T01:29:36.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.334+0000 7ff2a4c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a0114020 con 0x7ff2a0109120 2024-09-12T01:29:36.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.334+0000 7ff2a4c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a0114320 con 0x7ff2a01049a0 2024-09-12T01:29:36.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.334+0000 7ff29de00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2a0103ff0 0x7ff2a0076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:36.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.334+0000 7ff29de00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2a0103ff0 0x7ff2a0076700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33426/0 (socket says 172.21.15.1:33426) 2024-09-12T01:29:36.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.334+0000 7ff29de00640 1 -- 172.21.15.1:0/1449600705 learned_addr learned my addr 172.21.15.1:0/1449600705 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:36.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.335+0000 7ff295400640 1 --2- 172.21.15.1:0/1449600705 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2a01049a0 0x7ff2a0076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:36.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.335+0000 7ff29e800640 1 --2- 172.21.15.1:0/1449600705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2a0109120 0x7ff2a0077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:36.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.335+0000 7ff29de00640 1 -- 172.21.15.1:0/1449600705 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2a01049a0 msgr2=0x7ff2a0076c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:36.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.335+0000 7ff29de00640 1 --2- 172.21.15.1:0/1449600705 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2a01049a0 0x7ff2a0076c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:36.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.335+0000 7ff29de00640 1 -- 172.21.15.1:0/1449600705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2a0109120 msgr2=0x7ff2a0077180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:36.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.335+0000 7ff29de00640 1 --2- 172.21.15.1:0/1449600705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2a0109120 0x7ff2a0077180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:36.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.335+0000 7ff29de00640 1 -- 172.21.15.1:0/1449600705 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a01aacc0 con 0x7ff2a0103ff0 2024-09-12T01:29:36.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.335+0000 7ff295400640 1 --2- 172.21.15.1:0/1449600705 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2a01049a0 0x7ff2a0076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:29:36.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.335+0000 7ff29de00640 1 --2- 172.21.15.1:0/1449600705 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2a0103ff0 0x7ff2a0076700 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff29002a9e0 tx=0x7ff290003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:36.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.336+0000 7ff297400640 1 -- 172.21.15.1:0/1449600705 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29003f070 con 0x7ff2a0103ff0 2024-09-12T01:29:36.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.336+0000 7ff297400640 1 -- 172.21.15.1:0/1449600705 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff290031c90 con 0x7ff2a0103ff0 2024-09-12T01:29:36.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.336+0000 7ff2a4c00640 1 -- 172.21.15.1:0/1449600705 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2a01aaf50 con 0x7ff2a0103ff0 2024-09-12T01:29:36.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.336+0000 7ff297400640 1 -- 172.21.15.1:0/1449600705 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff290035030 con 0x7ff2a0103ff0 2024-09-12T01:29:36.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.336+0000 7ff2a4c00640 1 -- 172.21.15.1:0/1449600705 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2a01ab3a0 con 0x7ff2a0103ff0 2024-09-12T01:29:36.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.339+0000 7ff297400640 1 -- 172.21.15.1:0/1449600705 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7ff290042070 con 0x7ff2a0103ff0 2024-09-12T01:29:36.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.339+0000 7ff2a7657740 1 Processor -- start 2024-09-12T01:29:36.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.339+0000 7ff2a7657740 1 -- start start 2024-09-12T01:29:36.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.339+0000 7ff2a7657740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644ec056330 0x5644ec056710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:36.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.339+0000 7ff2a7657740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644ec056c50 0x5644ec060fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:36.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.339+0000 7ff2a7657740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644ec061510 0x5644ec063900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:36.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.339+0000 7ff2a7657740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5644ebfbc0b0 con 0x5644ec056330 2024-09-12T01:29:36.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.339+0000 7ff2a7657740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5644ebf63ab0 con 0x5644ec061510 2024-09-12T01:29:36.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.339+0000 7ff2a7657740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5644ebfb8070 con 0x5644ec056c50 2024-09-12T01:29:36.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.340+0000 7ff29e800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644ec056330 0x5644ec056710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:36.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.340+0000 7ff29de00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644ec056c50 0x5644ec060fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:36.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.340+0000 7ff29e800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644ec056330 0x5644ec056710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33432/0 (socket says 172.21.15.1:33432) 2024-09-12T01:29:36.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.340+0000 7ff29e800640 1 -- 172.21.15.1:0/2091533472 learned_addr learned my addr 172.21.15.1:0/2091533472 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:36.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.340+0000 7ff295400640 1 --2- 172.21.15.1:0/2091533472 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644ec061510 0x5644ec063900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:36.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.340+0000 7ff295400640 1 -- 172.21.15.1:0/2091533472 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644ec056c50 msgr2=0x5644ec060fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:36.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.340+0000 7ff295400640 1 --2- 172.21.15.1:0/2091533472 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644ec056c50 0x5644ec060fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:36.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.340+0000 7ff295400640 1 -- 172.21.15.1:0/2091533472 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644ec056330 msgr2=0x5644ec056710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:36.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.340+0000 7ff295400640 1 --2- 172.21.15.1:0/2091533472 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644ec056330 0x5644ec056710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:36.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.340+0000 7ff295400640 1 -- 172.21.15.1:0/2091533472 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644ebf704a0 con 0x5644ec061510 2024-09-12T01:29:36.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.340+0000 7ff29e800640 1 --2- 172.21.15.1:0/2091533472 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644ec056330 0x5644ec056710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:29:36.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.340+0000 7ff297400640 1 --2- 172.21.15.1:0/1449600705 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff2840a2b80 0x7ff2840a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:36.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.340+0000 7ff295400640 1 --2- 172.21.15.1:0/2091533472 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644ec061510 0x5644ec063900 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0077f50 tx=0x7ff288010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19caa7dba87a75b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:36.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.341+0000 7ff297400640 1 -- 172.21.15.1:0/1449600705 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7ff2900efa90 con 0x7ff2a0103ff0 2024-09-12T01:29:36.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.341+0000 7ff29e800640 1 --2- 172.21.15.1:0/1449600705 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff2840a2b80 0x7ff2840a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:36.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.341+0000 7ff28f000640 1 -- 172.21.15.1:0/2091533472 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2880117c0 con 0x5644ec061510 2024-09-12T01:29:36.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.341+0000 7ff28f000640 1 -- 172.21.15.1:0/2091533472 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff288011e40 con 0x5644ec061510 2024-09-12T01:29:36.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.341+0000 7ff28f000640 1 -- 172.21.15.1:0/2091533472 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28801a380 con 0x5644ec061510 2024-09-12T01:29:36.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.341+0000 7ff2a7657740 1 -- 172.21.15.1:0/2091533472 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644ec061510 msgr2=0x5644ec063900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:36.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.341+0000 7ff2a7657740 1 --2- 172.21.15.1:0/2091533472 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644ec061510 0x5644ec063900 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0077f50 tx=0x7ff288010860 comp rx=0 tx=0).stop 2024-09-12T01:29:36.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.341+0000 7ff29e800640 1 --2- 172.21.15.1:0/1449600705 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff2840a2b80 0x7ff2840a5040 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff298009fd0 tx=0x7ff298009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:36.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.342+0000 7ff2a7657740 1 -- 172.21.15.1:0/2091533472 shutdown_connections 2024-09-12T01:29:36.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.342+0000 7ff2a7657740 1 --2- 172.21.15.1:0/2091533472 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644ec061510 0x5644ec063900 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:36.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.342+0000 7ff2a7657740 1 --2- 172.21.15.1:0/2091533472 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644ec056c50 0x5644ec060fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:36.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.342+0000 7ff2a7657740 1 --2- 172.21.15.1:0/2091533472 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644ec056330 0x5644ec056710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:36.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.342+0000 7ff2a7657740 1 -- 172.21.15.1:0/2091533472 >> 172.21.15.1:0/2091533472 conn(0x5644ec0528c0 msgr2=0x5644ec052ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:36.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.342+0000 7ff2a7657740 1 -- 172.21.15.1:0/2091533472 shutdown_connections 2024-09-12T01:29:36.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.342+0000 7ff2a7657740 1 -- 172.21.15.1:0/2091533472 wait complete. 2024-09-12T01:29:36.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.342+0000 7ff2a7657740 1 -- start start 2024-09-12T01:29:36.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.342+0000 7ff2a7657740 1 Processor -- start 2024-09-12T01:29:36.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.343+0000 7ff2a7657740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644ec056330 0x5644ec05e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:36.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.343+0000 7ff2a7657740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644ec056c50 0x5644ec05ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:36.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.343+0000 7ff2a7657740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644ec061510 0x5644ec05efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:36.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.343+0000 7ff2a7657740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5644ec064060 con 0x5644ec056c50 2024-09-12T01:29:36.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.343+0000 7ff2a7657740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5644ebf44920 con 0x5644ec061510 2024-09-12T01:29:36.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.343+0000 7ff2a7657740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5644ec0641e0 con 0x5644ec056330 2024-09-12T01:29:36.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.343+0000 7ff2a7657740 10 client.0 _ll_register_callbacks cb 0x5644ebf688a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:29:36.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.343+0000 7ff295400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644ec056c50 0x5644ec05ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:36.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.343+0000 7ff29de00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644ec056330 0x5644ec05e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:36.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.343+0000 7ff295400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644ec056c50 0x5644ec05ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33446/0 (socket says 172.21.15.1:33446) 2024-09-12T01:29:36.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.343+0000 7ff29de00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644ec056330 0x5644ec05e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:34224/0 (socket says 172.21.15.1:34224) 2024-09-12T01:29:36.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.343+0000 7ff295400640 1 -- 172.21.15.1:0/1461573010 learned_addr learned my addr 172.21.15.1:0/1461573010 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:36.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.343+0000 7ff29e800640 1 --2- 172.21.15.1:0/1461573010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644ec061510 0x5644ec05efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:36.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.344+0000 7ff29de00640 1 -- 172.21.15.1:0/1461573010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644ec061510 msgr2=0x5644ec05efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:36.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.344+0000 7ff29de00640 1 --2- 172.21.15.1:0/1461573010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644ec061510 0x5644ec05efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:36.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.344+0000 7ff29de00640 1 -- 172.21.15.1:0/1461573010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644ec056c50 msgr2=0x5644ec05ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:36.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.344+0000 7ff29de00640 1 --2- 172.21.15.1:0/1461573010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644ec056c50 0x5644ec05ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:36.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.344+0000 7ff29de00640 1 -- 172.21.15.1:0/1461573010 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644ebd2d2b0 con 0x5644ec056330 2024-09-12T01:29:36.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.344+0000 7ff29de00640 1 --2- 172.21.15.1:0/1461573010 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644ec056330 0x5644ec05e540 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x5644ec053720 tx=0x7ff290009620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b02b6ba324a44803 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:36.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.344+0000 7ff283400640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-12T01:29:36.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.344+0000 7ff283400640 1 -- 172.21.15.1:0/1461573010 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff290049070 con 0x5644ec056330 2024-09-12T01:29:36.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.344+0000 7ff283400640 1 -- 172.21.15.1:0/1461573010 <== mon.1 v2:172.21.15.104:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7ff2900b8620 con 0x5644ec056330 2024-09-12T01:29:36.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.344+0000 7ff2a7657740 1 -- 172.21.15.1:0/1461573010 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5644ebfc9620 con 0x5644ec056330 2024-09-12T01:29:36.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.344+0000 7ff283400640 1 -- 172.21.15.1:0/1461573010 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff2900f5020 con 0x5644ec056330 2024-09-12T01:29:36.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.344+0000 7ff2a7657740 10 client.25135 fetch_fsmap learned FSMap version 28 2024-09-12T01:29:36.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.344+0000 7ff2a7657740 1 -- 172.21.15.1:0/1461573010 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5644ec064060 con 0x5644ec056330 2024-09-12T01:29:36.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.345+0000 7ff283400640 1 -- 172.21.15.1:0/1461573010 <== mon.1 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5644ebfc9620 con 0x5644ec056330 2024-09-12T01:29:36.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.345+0000 7ff283400640 1 -- 172.21.15.1:0/1461573010 <== mon.1 v2:172.21.15.104:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7ff2900c46e0 con 0x5644ec056330 2024-09-12T01:29:36.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.345+0000 7ff2a7657740 10 client.25135 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:29:36.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.345+0000 7ff2a7657740 10 client.25135 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T01:29:36.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.345+0000 7ff2a7657740 20 client.25135 populate_metadata read hostname 'smithi001' 2024-09-12T01:29:36.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.345+0000 7ff2a7657740 1 --2- 172.21.15.1:0/1461573010 >> [v2:172.21.15.104:6836/464720247,v1:172.21.15.104:6837/464720247] conn(0x5644ec072410 0x5644ec073880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:36.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.345+0000 7ff2a7657740 4 client.25135 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:29:36.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.345+0000 7ff2a7657740 1 -- 172.21.15.1:0/1461573010 --> [v2:172.21.15.104:6836/464720247,v1:172.21.15.104:6837/464720247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644ebfc3b30 con 0x5644ec072410 2024-09-12T01:29:36.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.345+0000 7ff295400640 1 --2- 172.21.15.1:0/1461573010 >> [v2:172.21.15.104:6836/464720247,v1:172.21.15.104:6837/464720247] conn(0x5644ec072410 0x5644ec073880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:36.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.346+0000 7ff295400640 1 --2- 172.21.15.1:0/1461573010 >> [v2:172.21.15.104:6836/464720247,v1:172.21.15.104:6837/464720247] conn(0x5644ec072410 0x5644ec073880 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=83ba1c9ed073b3fa server_cookie=623f3d3ac29fd96 in_seq=0 out_seq=0 2024-09-12T01:29:36.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.346+0000 7ff283400640 10 client.25135 ms_handle_connect on v2:172.21.15.104:6836/464720247 2024-09-12T01:29:36.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.347+0000 7ff283400640 1 -- 172.21.15.1:0/1461573010 <== mds.3 v2:172.21.15.104:6836/464720247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5644ebfc3b30 con 0x5644ec072410 2024-09-12T01:29:36.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.347+0000 7ff283400640 10 client.25135 handle_command_reply: tid=0 2024-09-12T01:29:36.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.347+0000 7ff2a7657740 1 client.25135 shutdown 2024-09-12T01:29:36.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.347+0000 7ff2a7657740 1 -- 172.21.15.1:0/1461573010 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644ec056330 msgr2=0x5644ec05e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:36.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.347+0000 7ff2a7657740 1 --2- 172.21.15.1:0/1461573010 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644ec056330 0x5644ec05e540 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x5644ec053720 tx=0x7ff290009620 comp rx=0 tx=0).stop 2024-09-12T01:29:36.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.347+0000 7ff2a7657740 1 -- 172.21.15.1:0/1461573010 shutdown_connections 2024-09-12T01:29:36.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.347+0000 7ff2a7657740 1 --2- 172.21.15.1:0/1461573010 >> [v2:172.21.15.104:6836/464720247,v1:172.21.15.104:6837/464720247] conn(0x5644ec072410 0x5644ec073880 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:36.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.347+0000 7ff2a7657740 1 --2- 172.21.15.1:0/1461573010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644ec061510 0x5644ec05efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:36.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.347+0000 7ff2a7657740 1 --2- 172.21.15.1:0/1461573010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644ec056c50 0x5644ec05ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:36.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.347+0000 7ff283400640 0 client.25135 ms_handle_reset on v2:172.21.15.104:6836/464720247 2024-09-12T01:29:36.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.347+0000 7ff2a7657740 1 --2- 172.21.15.1:0/1461573010 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644ec056330 0x5644ec05e540 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:36.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.347+0000 7ff2a7657740 1 -- 172.21.15.1:0/1461573010 >> 172.21.15.1:0/1461573010 conn(0x5644ec0528c0 msgr2=0x5644ec0585f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:36.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.348+0000 7ff2a7657740 1 -- 172.21.15.1:0/1461573010 shutdown_connections 2024-09-12T01:29:36.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.348+0000 7ff2a7657740 1 -- 172.21.15.1:0/1461573010 wait complete. 2024-09-12T01:29:36.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.348+0000 7ff2a7657740 20 client.25135 trim_cache size 0 max 16384 2024-09-12T01:29:36.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.366+0000 7ff2a7657740 1 Processor -- start 2024-09-12T01:29:36.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.367+0000 7ff2a7657740 1 -- start start 2024-09-12T01:29:36.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.367+0000 7ff2a7657740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644ec056330 0x5644ec052210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:36.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.367+0000 7ff2a7657740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644ec056c50 0x5644ec058d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:36.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.367+0000 7ff2a7657740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644ec061510 0x5644ec05b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:36.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.367+0000 7ff2a7657740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5644ec071bc0 con 0x5644ec056330 2024-09-12T01:29:36.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.367+0000 7ff2a7657740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5644ec072290 con 0x5644ec061510 2024-09-12T01:29:36.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.367+0000 7ff2a7657740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5644ec072960 con 0x5644ec056c50 2024-09-12T01:29:36.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.367+0000 7ff29de00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644ec056330 0x5644ec052210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:36.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.367+0000 7ff29de00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644ec056330 0x5644ec052210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33462/0 (socket says 172.21.15.1:33462) 2024-09-12T01:29:36.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.367+0000 7ff29de00640 1 -- 172.21.15.1:0/3862323843 learned_addr learned my addr 172.21.15.1:0/3862323843 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:36.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.367+0000 7ff295400640 1 --2- 172.21.15.1:0/3862323843 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644ec056c50 0x5644ec058d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:36.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.368+0000 7ff29de00640 1 -- 172.21.15.1:0/3862323843 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644ec056c50 msgr2=0x5644ec058d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:36.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.368+0000 7ff29e800640 1 --2- 172.21.15.1:0/3862323843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644ec061510 0x5644ec05b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:36.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.368+0000 7ff29de00640 1 --2- 172.21.15.1:0/3862323843 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644ec056c50 0x5644ec058d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:36.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.368+0000 7ff29de00640 1 -- 172.21.15.1:0/3862323843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644ec061510 msgr2=0x5644ec05b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:36.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.368+0000 7ff29de00640 1 --2- 172.21.15.1:0/3862323843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644ec061510 0x5644ec05b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:36.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.368+0000 7ff29de00640 1 -- 172.21.15.1:0/3862323843 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644ec063c70 con 0x5644ec056330 2024-09-12T01:29:36.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.368+0000 7ff295400640 1 --2- 172.21.15.1:0/3862323843 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644ec056c50 0x5644ec058d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:29:36.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.368+0000 7ff29e800640 1 --2- 172.21.15.1:0/3862323843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644ec061510 0x5644ec05b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:29:36.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.368+0000 7ff29de00640 1 --2- 172.21.15.1:0/3862323843 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644ec056330 0x5644ec052210 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff2900b8980 tx=0x7ff290005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=676418db9ba9343f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:36.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.368+0000 7ff28e600640 1 -- 172.21.15.1:0/3862323843 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff290049070 con 0x5644ec056330 2024-09-12T01:29:36.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.368+0000 7ff28e600640 1 -- 172.21.15.1:0/3862323843 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2900c02b0 con 0x5644ec056330 2024-09-12T01:29:36.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.368+0000 7ff28e600640 1 -- 172.21.15.1:0/3862323843 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2900bb400 con 0x5644ec056330 2024-09-12T01:29:36.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.369+0000 7ff2a7657740 1 -- 172.21.15.1:0/3862323843 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644ec056330 msgr2=0x5644ec052210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:36.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.369+0000 7ff2a7657740 1 --2- 172.21.15.1:0/3862323843 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644ec056330 0x5644ec052210 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff2900b8980 tx=0x7ff290005c00 comp rx=0 tx=0).stop 2024-09-12T01:29:36.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.369+0000 7ff2a7657740 1 -- 172.21.15.1:0/3862323843 shutdown_connections 2024-09-12T01:29:36.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.369+0000 7ff2a7657740 1 --2- 172.21.15.1:0/3862323843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644ec061510 0x5644ec05b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:36.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.369+0000 7ff2a7657740 1 --2- 172.21.15.1:0/3862323843 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644ec056c50 0x5644ec058d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:36.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.369+0000 7ff2a7657740 1 --2- 172.21.15.1:0/3862323843 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644ec056330 0x5644ec052210 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:36.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.369+0000 7ff2a7657740 1 -- 172.21.15.1:0/3862323843 >> 172.21.15.1:0/3862323843 conn(0x5644ec0528c0 msgr2=0x5644ec05ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:36.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.369+0000 7ff2a7657740 1 -- 172.21.15.1:0/3862323843 shutdown_connections 2024-09-12T01:29:36.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.369+0000 7ff2a7657740 1 -- 172.21.15.1:0/3862323843 wait complete. 2024-09-12T01:29:36.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.369+0000 7ff2a7657740 1 -- start start 2024-09-12T01:29:36.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.369+0000 7ff2a7657740 1 Processor -- start 2024-09-12T01:29:36.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.370+0000 7ff2a7657740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644ec056330 0x5644ec05b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:36.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.370+0000 7ff2a7657740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644ec056c50 0x5644ec05bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:36.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.370+0000 7ff2a7657740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644ec061510 0x5644ec05c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:36.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.370+0000 7ff2a7657740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5644ec073700 con 0x5644ec061510 2024-09-12T01:29:36.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.370+0000 7ff2a7657740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5644ec073030 con 0x5644ec056330 2024-09-12T01:29:36.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.370+0000 7ff2a7657740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5644ec073dd0 con 0x5644ec056c50 2024-09-12T01:29:36.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.370+0000 7ff2a7657740 10 client.0 _ll_register_callbacks cb 0x7ff29003b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:29:36.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.370+0000 7ff29e800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644ec061510 0x5644ec05c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:36.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.370+0000 7ff29de00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644ec056330 0x5644ec05b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:36.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.370+0000 7ff29de00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644ec056330 0x5644ec05b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:41646/0 (socket says 172.21.15.1:41646) 2024-09-12T01:29:36.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.370+0000 7ff29de00640 1 -- 172.21.15.1:0/3745877343 learned_addr learned my addr 172.21.15.1:0/3745877343 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:36.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.370+0000 7ff295400640 1 --2- 172.21.15.1:0/3745877343 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644ec056c50 0x5644ec05bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:36.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.371+0000 7ff295400640 1 -- 172.21.15.1:0/3745877343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644ec056330 msgr2=0x5644ec05b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:36.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.371+0000 7ff295400640 1 --2- 172.21.15.1:0/3745877343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644ec056330 0x5644ec05b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:36.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.371+0000 7ff295400640 1 -- 172.21.15.1:0/3745877343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644ec061510 msgr2=0x5644ec05c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:36.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.371+0000 7ff295400640 1 --2- 172.21.15.1:0/3745877343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644ec061510 0x5644ec05c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:36.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.371+0000 7ff295400640 1 -- 172.21.15.1:0/3745877343 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644ec07a160 con 0x5644ec056c50 2024-09-12T01:29:36.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.371+0000 7ff295400640 1 --2- 172.21.15.1:0/3745877343 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644ec056c50 0x5644ec05bc00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff288010d60 tx=0x7ff28801caf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2000db38306daa50 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:36.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.371+0000 7ff283400640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-12T01:29:36.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.371+0000 7ff283400640 1 -- 172.21.15.1:0/3745877343 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff288005bf0 con 0x5644ec056c50 2024-09-12T01:29:36.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.371+0000 7ff283400640 1 -- 172.21.15.1:0/3745877343 <== mon.1 v2:172.21.15.104:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7ff28801c480 con 0x5644ec056c50 2024-09-12T01:29:36.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.371+0000 7ff2a7657740 1 -- 172.21.15.1:0/3745877343 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2900c02b0 con 0x5644ec056c50 2024-09-12T01:29:36.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.372+0000 7ff283400640 1 -- 172.21.15.1:0/3745877343 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff28801c680 con 0x5644ec056c50 2024-09-12T01:29:36.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.372+0000 7ff2a7657740 10 client.25141 fetch_fsmap learned FSMap version 28 2024-09-12T01:29:36.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.372+0000 7ff2a7657740 1 -- 172.21.15.1:0/3745877343 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5644ec0dcda0 con 0x5644ec056c50 2024-09-12T01:29:36.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.372+0000 7ff283400640 1 -- 172.21.15.1:0/3745877343 <== mon.1 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2900c02b0 con 0x5644ec056c50 2024-09-12T01:29:36.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.372+0000 7ff283400640 1 -- 172.21.15.1:0/3745877343 <== mon.1 v2:172.21.15.104:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7ff288005410 con 0x5644ec056c50 2024-09-12T01:29:36.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.372+0000 7ff2a7657740 10 client.25141 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:29:36.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.372+0000 7ff2a7657740 10 client.25141 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T01:29:36.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.372+0000 7ff2a7657740 20 client.25141 populate_metadata read hostname 'smithi001' 2024-09-12T01:29:36.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.372+0000 7ff2a7657740 1 --2- 172.21.15.1:0/3745877343 >> [v2:172.21.15.104:6836/464720247,v1:172.21.15.104:6837/464720247] conn(0x5644ec0e17b0 0x5644ec0e04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:36.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.372+0000 7ff2a7657740 4 client.25141 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T01:29:36.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.372+0000 7ff2a7657740 1 -- 172.21.15.1:0/3745877343 --> [v2:172.21.15.104:6836/464720247,v1:172.21.15.104:6837/464720247] -- command(tid 0: {"prefix": "status"}) -- 0x5644ebf3ff80 con 0x5644ec0e17b0 2024-09-12T01:29:36.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.373+0000 7ff29de00640 1 --2- 172.21.15.1:0/3745877343 >> [v2:172.21.15.104:6836/464720247,v1:172.21.15.104:6837/464720247] conn(0x5644ec0e17b0 0x5644ec0e04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:36.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.373+0000 7ff29de00640 1 --2- 172.21.15.1:0/3745877343 >> [v2:172.21.15.104:6836/464720247,v1:172.21.15.104:6837/464720247] conn(0x5644ec0e17b0 0x5644ec0e04c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=874c403820e98e83 server_cookie=2d7be5430c224052 in_seq=0 out_seq=0 2024-09-12T01:29:36.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.373+0000 7ff283400640 10 client.25141 ms_handle_connect on v2:172.21.15.104:6836/464720247 2024-09-12T01:29:36.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.373+0000 7ff283400640 1 -- 172.21.15.1:0/3745877343 <== mds.3 v2:172.21.15.104:6836/464720247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5644ebf3ff80 con 0x5644ec0e17b0 2024-09-12T01:29:36.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.373+0000 7ff283400640 10 client.25141 handle_command_reply: tid=0 2024-09-12T01:29:36.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.373+0000 7ff2a7657740 1 client.25141 shutdown 2024-09-12T01:29:36.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.374+0000 7ff2a7657740 1 -- 172.21.15.1:0/3745877343 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644ec056c50 msgr2=0x5644ec05bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:36.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.374+0000 7ff2a7657740 1 --2- 172.21.15.1:0/3745877343 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644ec056c50 0x5644ec05bc00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff288010d60 tx=0x7ff28801caf0 comp rx=0 tx=0).stop 2024-09-12T01:29:36.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.374+0000 7ff2a7657740 1 -- 172.21.15.1:0/3745877343 shutdown_connections 2024-09-12T01:29:36.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.374+0000 7ff2a7657740 1 --2- 172.21.15.1:0/3745877343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644ec061510 0x5644ec05c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:36.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.374+0000 7ff2a7657740 1 --2- 172.21.15.1:0/3745877343 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644ec056c50 0x5644ec05bc00 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:36.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.374+0000 7ff2a7657740 1 --2- 172.21.15.1:0/3745877343 >> [v2:172.21.15.104:6836/464720247,v1:172.21.15.104:6837/464720247] conn(0x5644ec0e17b0 0x5644ec0e04c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:36.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.374+0000 7ff2a7657740 1 --2- 172.21.15.1:0/3745877343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644ec056330 0x5644ec05b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:36.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.374+0000 7ff283400640 0 client.25141 ms_handle_reset on v2:172.21.15.104:6836/464720247 2024-09-12T01:29:36.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.374+0000 7ff2a7657740 1 -- 172.21.15.1:0/3745877343 >> 172.21.15.1:0/3745877343 conn(0x5644ec0528c0 msgr2=0x5644ec06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:36.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.374+0000 7ff2a7657740 1 -- 172.21.15.1:0/3745877343 shutdown_connections 2024-09-12T01:29:36.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.374+0000 7ff2a7657740 1 -- 172.21.15.1:0/3745877343 wait complete. 2024-09-12T01:29:36.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.374+0000 7ff2a7657740 20 client.25141 trim_cache size 0 max 16384 2024-09-12T01:29:36.492 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-12T01:29:36.492 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "c0e4f180-70a4-11ef-bcea-c7b262605968", 2024-09-12T01:29:36.492 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 3, 2024-09-12T01:29:36.492 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24493, 2024-09-12T01:29:36.492 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-09-12T01:29:36.492 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-09-12T01:29:36.492 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-09-12T01:29:36.492 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 113.066601429, 2024-09-12T01:29:36.492 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 28, 2024-09-12T01:29:36.493 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 98, 2024-09-12T01:29:36.493 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 96, 2024-09-12T01:29:36.493 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 197.452130783 2024-09-12T01:29:36.493 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-12T01:29:36.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.375+0000 7ff2a4c00640 1 -- 172.21.15.1:0/1449600705 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff2840a2b80 msgr2=0x7ff2840a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:36.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.375+0000 7ff2a4c00640 1 --2- 172.21.15.1:0/1449600705 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff2840a2b80 0x7ff2840a5040 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff298009fd0 tx=0x7ff298009500 comp rx=0 tx=0).stop 2024-09-12T01:29:36.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.375+0000 7ff2a4c00640 1 -- 172.21.15.1:0/1449600705 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2a0103ff0 msgr2=0x7ff2a0076700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:36.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.375+0000 7ff2a4c00640 1 --2- 172.21.15.1:0/1449600705 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2a0103ff0 0x7ff2a0076700 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff29002a9e0 tx=0x7ff290003c10 comp rx=0 tx=0).stop 2024-09-12T01:29:36.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.375+0000 7ff2a4c00640 1 -- 172.21.15.1:0/1449600705 shutdown_connections 2024-09-12T01:29:36.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.375+0000 7ff2a4c00640 1 --2- 172.21.15.1:0/1449600705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2a0109120 0x7ff2a0077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:36.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.375+0000 7ff2a4c00640 1 --2- 172.21.15.1:0/1449600705 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff2840a2b80 0x7ff2840a5040 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:36.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.375+0000 7ff2a4c00640 1 --2- 172.21.15.1:0/1449600705 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2a01049a0 0x7ff2a0076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:36.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.375+0000 7ff2a4c00640 1 --2- 172.21.15.1:0/1449600705 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2a0103ff0 0x7ff2a0076700 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:36.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.375+0000 7ff2a4c00640 1 -- 172.21.15.1:0/1449600705 >> 172.21.15.1:0/1449600705 conn(0x7ff2a00ffa90 msgr2=0x7ff2a0100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:36.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.375+0000 7ff2a4c00640 1 -- 172.21.15.1:0/1449600705 shutdown_connections 2024-09-12T01:29:36.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:36.375+0000 7ff2a4c00640 1 -- 172.21.15.1:0/1449600705 wait complete. 2024-09-12T01:29:37.251 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-12T01:29:37.633 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:29:38.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:37 smithi104 ceph-mon[35219]: pgmap v399: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-12T01:29:38.107 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:37 smithi001 ceph-mon[30387]: pgmap v399: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-12T01:29:38.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.221+0000 7fa8ebe00640 1 -- 172.21.15.1:0/540076455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8ec108f60 msgr2=0x7fa8ec110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:38.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.221+0000 7fa8ebe00640 1 --2- 172.21.15.1:0/540076455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8ec108f60 0x7fa8ec110ae0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4009be0 tx=0x7fa8e402a6c0 comp rx=0 tx=0).stop 2024-09-12T01:29:38.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.221+0000 7fa8ebe00640 1 -- 172.21.15.1:0/540076455 shutdown_connections 2024-09-12T01:29:38.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.221+0000 7fa8ebe00640 1 --2- 172.21.15.1:0/540076455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8ec108f60 0x7fa8ec110ae0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:38.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.221+0000 7fa8ebe00640 1 --2- 172.21.15.1:0/540076455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8ec1047e0 0x7fa8ec108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:38.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.221+0000 7fa8ebe00640 1 --2- 172.21.15.1:0/540076455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8ec103e30 0x7fa8ec104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:38.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.221+0000 7fa8ebe00640 1 -- 172.21.15.1:0/540076455 >> 172.21.15.1:0/540076455 conn(0x7fa8ec0ff8d0 msgr2=0x7fa8ec101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:38.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.222+0000 7fa8ebe00640 1 -- 172.21.15.1:0/540076455 shutdown_connections 2024-09-12T01:29:38.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.222+0000 7fa8ebe00640 1 -- 172.21.15.1:0/540076455 wait complete. 2024-09-12T01:29:38.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.222+0000 7fa8ebe00640 1 Processor -- start 2024-09-12T01:29:38.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.223+0000 7fa8ebe00640 1 -- start start 2024-09-12T01:29:38.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.223+0000 7fa8ebe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8ec103e30 0x7fa8ec19a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:38.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.223+0000 7fa8ebe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8ec1047e0 0x7fa8ec19a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:38.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.223+0000 7fa8ebe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8ec108f60 0x7fa8ec19ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:38.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.223+0000 7fa8ebe00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ec113fc0 con 0x7fa8ec103e30 2024-09-12T01:29:38.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.223+0000 7fa8ebe00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ec113e40 con 0x7fa8ec108f60 2024-09-12T01:29:38.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.223+0000 7fa8ebe00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ec114140 con 0x7fa8ec1047e0 2024-09-12T01:29:38.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.223+0000 7fa8eaa00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8ec103e30 0x7fa8ec19a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:38.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.223+0000 7fa8eaa00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8ec103e30 0x7fa8ec19a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33496/0 (socket says 172.21.15.1:33496) 2024-09-12T01:29:38.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.223+0000 7fa8eaa00640 1 -- 172.21.15.1:0/2746665349 learned_addr learned my addr 172.21.15.1:0/2746665349 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:38.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.223+0000 7fa8eb400640 1 --2- 172.21.15.1:0/2746665349 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8ec108f60 0x7fa8ec19ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:38.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.223+0000 7fa8e2000640 1 --2- 172.21.15.1:0/2746665349 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8ec1047e0 0x7fa8ec19a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:38.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.224+0000 7fa8eaa00640 1 -- 172.21.15.1:0/2746665349 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8ec1047e0 msgr2=0x7fa8ec19a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:38.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.224+0000 7fa8eaa00640 1 --2- 172.21.15.1:0/2746665349 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8ec1047e0 0x7fa8ec19a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:38.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.224+0000 7fa8eaa00640 1 -- 172.21.15.1:0/2746665349 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8ec108f60 msgr2=0x7fa8ec19ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:38.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.224+0000 7fa8eaa00640 1 --2- 172.21.15.1:0/2746665349 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8ec108f60 0x7fa8ec19ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:38.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.224+0000 7fa8eaa00640 1 -- 172.21.15.1:0/2746665349 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8ec19f390 con 0x7fa8ec103e30 2024-09-12T01:29:38.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.224+0000 7fa8e2000640 1 --2- 172.21.15.1:0/2746665349 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8ec1047e0 0x7fa8ec19a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:29:38.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.224+0000 7fa8eb400640 1 --2- 172.21.15.1:0/2746665349 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8ec108f60 0x7fa8ec19ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:29:38.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.224+0000 7fa8eaa00640 1 --2- 172.21.15.1:0/2746665349 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8ec103e30 0x7fa8ec19a3e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc00e9f0 tx=0x7fa8dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:38.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.224+0000 7fa8e3e00640 1 -- 172.21.15.1:0/2746665349 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc00cd80 con 0x7fa8ec103e30 2024-09-12T01:29:38.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.224+0000 7fa8e3e00640 1 -- 172.21.15.1:0/2746665349 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8dc010910 con 0x7fa8ec103e30 2024-09-12T01:29:38.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.224+0000 7fa8e3e00640 1 -- 172.21.15.1:0/2746665349 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc018990 con 0x7fa8ec103e30 2024-09-12T01:29:38.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.224+0000 7fa8ebe00640 1 -- 172.21.15.1:0/2746665349 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8ec19f680 con 0x7fa8ec103e30 2024-09-12T01:29:38.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.225+0000 7fa8ebe00640 1 -- 172.21.15.1:0/2746665349 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8ec1a6f60 con 0x7fa8ec103e30 2024-09-12T01:29:38.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.228+0000 7fa8f4212740 1 Processor -- start 2024-09-12T01:29:38.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.228+0000 7fa8e3e00640 1 -- 172.21.15.1:0/2746665349 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fa8dc010ac0 con 0x7fa8ec103e30 2024-09-12T01:29:38.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.228+0000 7fa8f4212740 1 -- start start 2024-09-12T01:29:38.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.228+0000 7fa8f4212740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557eefc104e0 0x557eefc108c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:38.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.228+0000 7fa8f4212740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557eefc10e00 0x557eefc1b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:38.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.228+0000 7fa8f4212740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eefc1b6c0 0x557eefc1dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:38.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.228+0000 7fa8f4212740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x557eefb760b0 con 0x557eefc10e00 2024-09-12T01:29:38.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.228+0000 7fa8f4212740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x557eefb1dab0 con 0x557eefc1b6c0 2024-09-12T01:29:38.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.228+0000 7fa8f4212740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557eefb72070 con 0x557eefc104e0 2024-09-12T01:29:38.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.228+0000 7fa8eaa00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557eefc10e00 0x557eefc1b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:38.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.228+0000 7fa8e2000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eefc1b6c0 0x557eefc1dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:38.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.228+0000 7fa8eaa00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557eefc10e00 0x557eefc1b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33498/0 (socket says 172.21.15.1:33498) 2024-09-12T01:29:38.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.228+0000 7fa8eaa00640 1 -- 172.21.15.1:0/333237557 learned_addr learned my addr 172.21.15.1:0/333237557 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:38.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.228+0000 7fa8e2000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eefc1b6c0 0x557eefc1dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:41674/0 (socket says 172.21.15.1:41674) 2024-09-12T01:29:38.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.229+0000 7fa8eb400640 1 --2- 172.21.15.1:0/333237557 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557eefc104e0 0x557eefc108c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:38.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.229+0000 7fa8eaa00640 1 -- 172.21.15.1:0/333237557 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557eefc104e0 msgr2=0x557eefc108c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:38.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.229+0000 7fa8eaa00640 1 --2- 172.21.15.1:0/333237557 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557eefc104e0 0x557eefc108c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:38.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.229+0000 7fa8eaa00640 1 -- 172.21.15.1:0/333237557 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eefc1b6c0 msgr2=0x557eefc1dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:38.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.229+0000 7fa8eaa00640 1 --2- 172.21.15.1:0/333237557 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eefc1b6c0 0x557eefc1dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:38.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.229+0000 7fa8eaa00640 1 -- 172.21.15.1:0/333237557 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557eefb2a4a0 con 0x557eefc10e00 2024-09-12T01:29:38.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.229+0000 7fa8e2000640 1 --2- 172.21.15.1:0/333237557 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eefc1b6c0 0x557eefc1dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:29:38.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.229+0000 7fa8eaa00640 1 --2- 172.21.15.1:0/333237557 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557eefc10e00 0x557eefc1b180 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc0d5c20 tx=0x7fa8dc0d7670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e780330d87dc08 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:38.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.230+0000 7fa8dbe00640 1 -- 172.21.15.1:0/333237557 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc0de070 con 0x557eefc10e00 2024-09-12T01:29:38.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.230+0000 7fa8dbe00640 1 -- 172.21.15.1:0/333237557 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8dc0cfcd0 con 0x557eefc10e00 2024-09-12T01:29:38.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.230+0000 7fa8dbe00640 1 -- 172.21.15.1:0/333237557 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc0da800 con 0x557eefc10e00 2024-09-12T01:29:38.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.230+0000 7fa8e3e00640 1 --2- 172.21.15.1:0/2746665349 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fa8d00a2cd0 0x7fa8d00a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:38.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.230+0000 7fa8e3e00640 1 -- 172.21.15.1:0/2746665349 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fa8dc0cf6e0 con 0x7fa8ec103e30 2024-09-12T01:29:38.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.230+0000 7fa8eb400640 1 --2- 172.21.15.1:0/2746665349 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fa8d00a2cd0 0x7fa8d00a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:38.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.230+0000 7fa8f4212740 1 -- 172.21.15.1:0/333237557 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557eefc10e00 msgr2=0x557eefc1b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:38.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.230+0000 7fa8f4212740 1 --2- 172.21.15.1:0/333237557 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557eefc10e00 0x557eefc1b180 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc0d5c20 tx=0x7fa8dc0d7670 comp rx=0 tx=0).stop 2024-09-12T01:29:38.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.231+0000 7fa8f4212740 1 -- 172.21.15.1:0/333237557 shutdown_connections 2024-09-12T01:29:38.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.231+0000 7fa8f4212740 1 --2- 172.21.15.1:0/333237557 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eefc1b6c0 0x557eefc1dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:38.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.231+0000 7fa8f4212740 1 --2- 172.21.15.1:0/333237557 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557eefc10e00 0x557eefc1b180 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:38.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.231+0000 7fa8f4212740 1 --2- 172.21.15.1:0/333237557 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557eefc104e0 0x557eefc108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:38.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.231+0000 7fa8f4212740 1 -- 172.21.15.1:0/333237557 >> 172.21.15.1:0/333237557 conn(0x557eefc0ca70 msgr2=0x557eefc0ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:38.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.231+0000 7fa8f4212740 1 -- 172.21.15.1:0/333237557 shutdown_connections 2024-09-12T01:29:38.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.231+0000 7fa8f4212740 1 -- 172.21.15.1:0/333237557 wait complete. 2024-09-12T01:29:38.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.231+0000 7fa8eb400640 1 --2- 172.21.15.1:0/2746665349 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fa8d00a2cd0 0x7fa8d00a5190 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4004af0 tx=0x7fa8e4005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:38.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.231+0000 7fa8f4212740 1 -- start start 2024-09-12T01:29:38.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.232+0000 7fa8f4212740 1 Processor -- start 2024-09-12T01:29:38.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.232+0000 7fa8f4212740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557eefc104e0 0x557eefc185a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:38.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.232+0000 7fa8f4212740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eefc10e00 0x557eefc18ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:38.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.232+0000 7fa8f4212740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557eefc1b6c0 0x557eefc19020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:38.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.232+0000 7fa8f4212740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x557eefc1e330 con 0x557eefc1b6c0 2024-09-12T01:29:38.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.232+0000 7fa8f4212740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x557eefafe920 con 0x557eefc10e00 2024-09-12T01:29:38.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.232+0000 7fa8f4212740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557eefc1e4b0 con 0x557eefc104e0 2024-09-12T01:29:38.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.232+0000 7fa8f4212740 10 client.0 _ll_register_callbacks cb 0x557eefb228a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:29:38.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.232+0000 7fa8eb400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557eefc1b6c0 0x557eefc19020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:38.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.232+0000 7fa8eb400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557eefc1b6c0 0x557eefc19020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33506/0 (socket says 172.21.15.1:33506) 2024-09-12T01:29:38.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.232+0000 7fa8eb400640 1 -- 172.21.15.1:0/2246259821 learned_addr learned my addr 172.21.15.1:0/2246259821 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:38.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.232+0000 7fa8eaa00640 1 --2- 172.21.15.1:0/2246259821 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557eefc104e0 0x557eefc185a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:38.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.233+0000 7fa8e2000640 1 --2- 172.21.15.1:0/2246259821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eefc10e00 0x557eefc18ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:38.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.233+0000 7fa8eaa00640 1 -- 172.21.15.1:0/2246259821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eefc10e00 msgr2=0x557eefc18ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:38.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.233+0000 7fa8eaa00640 1 --2- 172.21.15.1:0/2246259821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eefc10e00 0x557eefc18ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:38.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.233+0000 7fa8eaa00640 1 -- 172.21.15.1:0/2246259821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557eefc1b6c0 msgr2=0x557eefc19020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:38.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.233+0000 7fa8eaa00640 1 --2- 172.21.15.1:0/2246259821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557eefc1b6c0 0x557eefc19020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:38.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.233+0000 7fa8eaa00640 1 -- 172.21.15.1:0/2246259821 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557eef8e72b0 con 0x557eefc104e0 2024-09-12T01:29:38.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.233+0000 7fa8eaa00640 1 --2- 172.21.15.1:0/2246259821 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557eefc104e0 0x557eefc185a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc00c6a0 tx=0x7fa8dc018420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=defd2e940408a800 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:38.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.233+0000 7fa8d8c00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-12T01:29:38.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.233+0000 7fa8d8c00640 1 -- 172.21.15.1:0/2246259821 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc0de030 con 0x557eefc104e0 2024-09-12T01:29:38.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.233+0000 7fa8f4212740 1 -- 172.21.15.1:0/2246259821 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8dc0cfcd0 con 0x557eefc104e0 2024-09-12T01:29:38.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.233+0000 7fa8d8c00640 1 -- 172.21.15.1:0/2246259821 <== mon.1 v2:172.21.15.104:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fa8dc0cf3f0 con 0x557eefc104e0 2024-09-12T01:29:38.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.234+0000 7fa8d8c00640 1 -- 172.21.15.1:0/2246259821 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa8dc097df0 con 0x557eefc104e0 2024-09-12T01:29:38.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.234+0000 7fa8f4212740 10 client.25153 fetch_fsmap learned FSMap version 28 2024-09-12T01:29:38.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.234+0000 7fa8f4212740 1 -- 172.21.15.1:0/2246259821 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557eefc1e330 con 0x557eefc104e0 2024-09-12T01:29:38.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.234+0000 7fa8d8c00640 1 -- 172.21.15.1:0/2246259821 <== mon.1 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8dc0cfcd0 con 0x557eefc104e0 2024-09-12T01:29:38.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.234+0000 7fa8d8c00640 1 -- 172.21.15.1:0/2246259821 <== mon.1 v2:172.21.15.104:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fa8dc0cce70 con 0x557eefc104e0 2024-09-12T01:29:38.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.234+0000 7fa8f4212740 10 client.25153 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:29:38.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.234+0000 7fa8f4212740 10 client.25153 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-12T01:29:38.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.234+0000 7fa8f4212740 20 client.25153 populate_metadata read hostname 'smithi001' 2024-09-12T01:29:38.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.235+0000 7fa8f4212740 1 --2- 172.21.15.1:0/2246259821 >> [v2:172.21.15.53:6836/2848743754,v1:172.21.15.53:6837/2848743754] conn(0x557eefc2c160 0x557eefc2d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:38.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.235+0000 7fa8f4212740 4 client.25153 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:29:38.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.235+0000 7fa8f4212740 1 -- 172.21.15.1:0/2246259821 --> [v2:172.21.15.53:6836/2848743754,v1:172.21.15.53:6837/2848743754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557eefb83620 con 0x557eefc2c160 2024-09-12T01:29:38.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.235+0000 7fa8e2000640 1 --2- 172.21.15.1:0/2246259821 >> [v2:172.21.15.53:6836/2848743754,v1:172.21.15.53:6837/2848743754] conn(0x557eefc2c160 0x557eefc2d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:38.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.235+0000 7fa8e2000640 1 --2- 172.21.15.1:0/2246259821 >> [v2:172.21.15.53:6836/2848743754,v1:172.21.15.53:6837/2848743754] conn(0x557eefc2c160 0x557eefc2d5d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fda8ba232a09cf17 server_cookie=c714d309b40639f1 in_seq=0 out_seq=0 2024-09-12T01:29:38.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.235+0000 7fa8d8c00640 10 client.25153 ms_handle_connect on v2:172.21.15.53:6836/2848743754 2024-09-12T01:29:38.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.236+0000 7fa8d8c00640 1 -- 172.21.15.1:0/2246259821 <== mds.2 v2:172.21.15.53:6836/2848743754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557eefb83620 con 0x557eefc2c160 2024-09-12T01:29:38.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.236+0000 7fa8d8c00640 10 client.25153 handle_command_reply: tid=0 2024-09-12T01:29:38.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.236+0000 7fa8f4212740 1 client.25153 shutdown 2024-09-12T01:29:38.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.236+0000 7fa8f4212740 1 -- 172.21.15.1:0/2246259821 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557eefc104e0 msgr2=0x557eefc185a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:38.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.236+0000 7fa8f4212740 1 --2- 172.21.15.1:0/2246259821 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557eefc104e0 0x557eefc185a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc00c6a0 tx=0x7fa8dc018420 comp rx=0 tx=0).stop 2024-09-12T01:29:38.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.237+0000 7fa8f4212740 1 -- 172.21.15.1:0/2246259821 shutdown_connections 2024-09-12T01:29:38.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.237+0000 7fa8f4212740 1 --2- 172.21.15.1:0/2246259821 >> [v2:172.21.15.53:6836/2848743754,v1:172.21.15.53:6837/2848743754] conn(0x557eefc2c160 0x557eefc2d5d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:38.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.237+0000 7fa8f4212740 1 --2- 172.21.15.1:0/2246259821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557eefc1b6c0 0x557eefc19020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:38.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.237+0000 7fa8f4212740 1 --2- 172.21.15.1:0/2246259821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eefc10e00 0x557eefc18ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:38.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.237+0000 7fa8f4212740 1 --2- 172.21.15.1:0/2246259821 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557eefc104e0 0x557eefc185a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:38.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.237+0000 7fa8d8c00640 0 client.25153 ms_handle_reset on v2:172.21.15.53:6836/2848743754 2024-09-12T01:29:38.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.237+0000 7fa8f4212740 1 -- 172.21.15.1:0/2246259821 >> 172.21.15.1:0/2246259821 conn(0x557eefc0ca70 msgr2=0x557eefc12750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:38.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.237+0000 7fa8f4212740 1 -- 172.21.15.1:0/2246259821 shutdown_connections 2024-09-12T01:29:38.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.237+0000 7fa8f4212740 1 -- 172.21.15.1:0/2246259821 wait complete. 2024-09-12T01:29:38.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.237+0000 7fa8f4212740 20 client.25153 trim_cache size 0 max 16384 2024-09-12T01:29:38.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:37 smithi053 ceph-mon[35158]: pgmap v399: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-12T01:29:38.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.255+0000 7fa8f4212740 1 Processor -- start 2024-09-12T01:29:38.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.256+0000 7fa8f4212740 1 -- start start 2024-09-12T01:29:38.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.256+0000 7fa8f4212740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557eefc104e0 0x557eefc12eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:38.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.256+0000 7fa8f4212740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eefc10e00 0x557eefc15800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:38.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.256+0000 7fa8f4212740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557eefc1b6c0 0x557eefc15d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:38.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.256+0000 7fa8f4212740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x557eefc2b910 con 0x557eefc1b6c0 2024-09-12T01:29:38.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.256+0000 7fa8f4212740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x557eefc2bfe0 con 0x557eefc10e00 2024-09-12T01:29:38.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.256+0000 7fa8f4212740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557eefc2c6b0 con 0x557eefc104e0 2024-09-12T01:29:38.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.256+0000 7fa8eb400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557eefc1b6c0 0x557eefc15d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:38.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.256+0000 7fa8eb400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557eefc1b6c0 0x557eefc15d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33516/0 (socket says 172.21.15.1:33516) 2024-09-12T01:29:38.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.256+0000 7fa8eb400640 1 -- 172.21.15.1:0/598265886 learned_addr learned my addr 172.21.15.1:0/598265886 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:38.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.256+0000 7fa8eaa00640 1 --2- 172.21.15.1:0/598265886 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557eefc104e0 0x557eefc12eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:38.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.256+0000 7fa8e2000640 1 --2- 172.21.15.1:0/598265886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eefc10e00 0x557eefc15800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:38.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.257+0000 7fa8e2000640 1 -- 172.21.15.1:0/598265886 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557eefc104e0 msgr2=0x557eefc12eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:38.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.257+0000 7fa8e2000640 1 --2- 172.21.15.1:0/598265886 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557eefc104e0 0x557eefc12eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:38.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.257+0000 7fa8e2000640 1 -- 172.21.15.1:0/598265886 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557eefc1b6c0 msgr2=0x557eefc15d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:38.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.257+0000 7fa8e2000640 1 --2- 172.21.15.1:0/598265886 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557eefc1b6c0 0x557eefc15d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:38.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.257+0000 7fa8e2000640 1 -- 172.21.15.1:0/598265886 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557eefc27d20 con 0x557eefc10e00 2024-09-12T01:29:38.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.257+0000 7fa8eb400640 1 --2- 172.21.15.1:0/598265886 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557eefc1b6c0 0x557eefc15d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:29:38.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.257+0000 7fa8e2000640 1 --2- 172.21.15.1:0/598265886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eefc10e00 0x557eefc15800 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x557eefc19be0 tx=0x7fa8d4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14d4666c35cf77e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:38.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.257+0000 7fa8db400640 1 -- 172.21.15.1:0/598265886 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4003ed0 con 0x557eefc10e00 2024-09-12T01:29:38.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.258+0000 7fa8db400640 1 -- 172.21.15.1:0/598265886 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8d4005500 con 0x557eefc10e00 2024-09-12T01:29:38.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.258+0000 7fa8db400640 1 -- 172.21.15.1:0/598265886 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d40116a0 con 0x557eefc10e00 2024-09-12T01:29:38.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.258+0000 7fa8f4212740 1 -- 172.21.15.1:0/598265886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eefc10e00 msgr2=0x557eefc15800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:38.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.258+0000 7fa8f4212740 1 --2- 172.21.15.1:0/598265886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eefc10e00 0x557eefc15800 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x557eefc19be0 tx=0x7fa8d4009260 comp rx=0 tx=0).stop 2024-09-12T01:29:38.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.258+0000 7fa8f4212740 1 -- 172.21.15.1:0/598265886 shutdown_connections 2024-09-12T01:29:38.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.258+0000 7fa8f4212740 1 --2- 172.21.15.1:0/598265886 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557eefc1b6c0 0x557eefc15d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:38.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.258+0000 7fa8f4212740 1 --2- 172.21.15.1:0/598265886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eefc10e00 0x557eefc15800 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:38.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.258+0000 7fa8f4212740 1 --2- 172.21.15.1:0/598265886 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557eefc104e0 0x557eefc12eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:38.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.258+0000 7fa8f4212740 1 -- 172.21.15.1:0/598265886 >> 172.21.15.1:0/598265886 conn(0x557eefc0ca70 msgr2=0x557eefc2b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:38.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.258+0000 7fa8f4212740 1 -- 172.21.15.1:0/598265886 shutdown_connections 2024-09-12T01:29:38.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.258+0000 7fa8f4212740 1 -- 172.21.15.1:0/598265886 wait complete. 2024-09-12T01:29:38.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.259+0000 7fa8f4212740 1 -- start start 2024-09-12T01:29:38.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.259+0000 7fa8f4212740 1 Processor -- start 2024-09-12T01:29:38.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.259+0000 7fa8f4212740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557eefc104e0 0x557eefc35e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:38.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.259+0000 7fa8f4212740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eefc10e00 0x557eefc2f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:38.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.259+0000 7fa8f4212740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557eefc1b6c0 0x557eefc2f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:38.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.259+0000 7fa8f4212740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x557eefc2d450 con 0x557eefc1b6c0 2024-09-12T01:29:38.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.259+0000 7fa8f4212740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x557eefc2cd80 con 0x557eefc10e00 2024-09-12T01:29:38.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.259+0000 7fa8f4212740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557eefc2db20 con 0x557eefc104e0 2024-09-12T01:29:38.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.259+0000 7fa8f4212740 10 client.0 _ll_register_callbacks cb 0x7fa8dc0d5a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:29:38.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.260+0000 7fa8e2000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eefc10e00 0x557eefc2f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:38.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.260+0000 7fa8eb400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557eefc1b6c0 0x557eefc2f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:38.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.260+0000 7fa8e2000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eefc10e00 0x557eefc2f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:41714/0 (socket says 172.21.15.1:41714) 2024-09-12T01:29:38.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.260+0000 7fa8e2000640 1 -- 172.21.15.1:0/2229337944 learned_addr learned my addr 172.21.15.1:0/2229337944 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:38.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.260+0000 7fa8eaa00640 1 --2- 172.21.15.1:0/2229337944 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557eefc104e0 0x557eefc35e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:38.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.260+0000 7fa8e2000640 1 -- 172.21.15.1:0/2229337944 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557eefc104e0 msgr2=0x557eefc35e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:38.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.260+0000 7fa8e2000640 1 --2- 172.21.15.1:0/2229337944 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557eefc104e0 0x557eefc35e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:38.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.260+0000 7fa8e2000640 1 -- 172.21.15.1:0/2229337944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557eefc1b6c0 msgr2=0x557eefc2f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:38.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.260+0000 7fa8e2000640 1 --2- 172.21.15.1:0/2229337944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557eefc1b6c0 0x557eefc2f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:38.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.260+0000 7fa8e2000640 1 -- 172.21.15.1:0/2229337944 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557eefc2fce0 con 0x557eefc10e00 2024-09-12T01:29:38.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.260+0000 7fa8e2000640 1 --2- 172.21.15.1:0/2229337944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eefc10e00 0x557eefc2f130 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x557eefc2ddd0 tx=0x7fa8d4009b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c466b1d752bd027 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:38.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.260+0000 7fa8d8c00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-12T01:29:38.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.261+0000 7fa8d8c00640 1 -- 172.21.15.1:0/2229337944 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4003ed0 con 0x557eefc10e00 2024-09-12T01:29:38.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.261+0000 7fa8f4212740 1 -- 172.21.15.1:0/2229337944 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8d4005500 con 0x557eefc10e00 2024-09-12T01:29:38.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.261+0000 7fa8d8c00640 1 -- 172.21.15.1:0/2229337944 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fa8d4004730 con 0x557eefc10e00 2024-09-12T01:29:38.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.261+0000 7fa8d8c00640 1 -- 172.21.15.1:0/2229337944 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa8d4039050 con 0x557eefc10e00 2024-09-12T01:29:38.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.261+0000 7fa8f4212740 10 client.25169 fetch_fsmap learned FSMap version 28 2024-09-12T01:29:38.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.261+0000 7fa8f4212740 1 -- 172.21.15.1:0/2229337944 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557eefc96870 con 0x557eefc10e00 2024-09-12T01:29:38.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.262+0000 7fa8d8c00640 1 -- 172.21.15.1:0/2229337944 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8d4005500 con 0x557eefc10e00 2024-09-12T01:29:38.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.262+0000 7fa8d8c00640 1 -- 172.21.15.1:0/2229337944 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fa8d4011710 con 0x557eefc10e00 2024-09-12T01:29:38.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.262+0000 7fa8f4212740 10 client.25169 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:29:38.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.262+0000 7fa8f4212740 10 client.25169 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-12T01:29:38.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.262+0000 7fa8f4212740 20 client.25169 populate_metadata read hostname 'smithi001' 2024-09-12T01:29:38.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.262+0000 7fa8f4212740 1 --2- 172.21.15.1:0/2229337944 >> [v2:172.21.15.53:6836/2848743754,v1:172.21.15.53:6837/2848743754] conn(0x557eefc9afb0 0x557eefc99cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:38.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.262+0000 7fa8f4212740 4 client.25169 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T01:29:38.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.262+0000 7fa8f4212740 1 -- 172.21.15.1:0/2229337944 --> [v2:172.21.15.53:6836/2848743754,v1:172.21.15.53:6837/2848743754] -- command(tid 0: {"prefix": "status"}) -- 0x557eefb7db30 con 0x557eefc9afb0 2024-09-12T01:29:38.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.263+0000 7fa8eaa00640 1 --2- 172.21.15.1:0/2229337944 >> [v2:172.21.15.53:6836/2848743754,v1:172.21.15.53:6837/2848743754] conn(0x557eefc9afb0 0x557eefc99cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:38.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.263+0000 7fa8eaa00640 1 --2- 172.21.15.1:0/2229337944 >> [v2:172.21.15.53:6836/2848743754,v1:172.21.15.53:6837/2848743754] conn(0x557eefc9afb0 0x557eefc99cc0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=59a8cc05cb336e4d server_cookie=643f702e70bfe804 in_seq=0 out_seq=0 2024-09-12T01:29:38.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.263+0000 7fa8d8c00640 10 client.25169 ms_handle_connect on v2:172.21.15.53:6836/2848743754 2024-09-12T01:29:38.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.263+0000 7fa8d8c00640 1 -- 172.21.15.1:0/2229337944 <== mds.2 v2:172.21.15.53:6836/2848743754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x557eefb7db30 con 0x557eefc9afb0 2024-09-12T01:29:38.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.263+0000 7fa8d8c00640 10 client.25169 handle_command_reply: tid=0 2024-09-12T01:29:38.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.263+0000 7fa8f4212740 1 client.25169 shutdown 2024-09-12T01:29:38.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.263+0000 7fa8f4212740 1 -- 172.21.15.1:0/2229337944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eefc10e00 msgr2=0x557eefc2f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:38.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.263+0000 7fa8f4212740 1 --2- 172.21.15.1:0/2229337944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eefc10e00 0x557eefc2f130 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x557eefc2ddd0 tx=0x7fa8d4009b20 comp rx=0 tx=0).stop 2024-09-12T01:29:38.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.264+0000 7fa8f4212740 1 -- 172.21.15.1:0/2229337944 shutdown_connections 2024-09-12T01:29:38.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.264+0000 7fa8f4212740 1 --2- 172.21.15.1:0/2229337944 >> [v2:172.21.15.53:6836/2848743754,v1:172.21.15.53:6837/2848743754] conn(0x557eefc9afb0 0x557eefc99cc0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:38.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.264+0000 7fa8f4212740 1 --2- 172.21.15.1:0/2229337944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557eefc1b6c0 0x557eefc2f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:38.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.264+0000 7fa8f4212740 1 --2- 172.21.15.1:0/2229337944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557eefc10e00 0x557eefc2f130 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:38.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.264+0000 7fa8f4212740 1 --2- 172.21.15.1:0/2229337944 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557eefc104e0 0x557eefc35e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:38.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.264+0000 7fa8d8c00640 0 client.25169 ms_handle_reset on v2:172.21.15.53:6836/2848743754 2024-09-12T01:29:38.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.264+0000 7fa8f4212740 1 -- 172.21.15.1:0/2229337944 >> 172.21.15.1:0/2229337944 conn(0x557eefc0ca70 msgr2=0x557eefc16e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:38.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.264+0000 7fa8f4212740 1 -- 172.21.15.1:0/2229337944 shutdown_connections 2024-09-12T01:29:38.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.264+0000 7fa8f4212740 1 -- 172.21.15.1:0/2229337944 wait complete. 2024-09-12T01:29:38.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.264+0000 7fa8f4212740 20 client.25169 trim_cache size 0 max 16384 2024-09-12T01:29:38.266 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-12T01:29:38.267 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "c0e4f180-70a4-11ef-bcea-c7b262605968", 2024-09-12T01:29:38.267 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 2, 2024-09-12T01:29:38.267 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24506, 2024-09-12T01:29:38.267 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-09-12T01:29:38.267 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-09-12T01:29:38.267 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-09-12T01:29:38.267 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 115.96649528099999, 2024-09-12T01:29:38.267 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 28, 2024-09-12T01:29:38.267 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 98, 2024-09-12T01:29:38.267 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 96, 2024-09-12T01:29:38.267 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 196.56928111799999 2024-09-12T01:29:38.267 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-12T01:29:38.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.265+0000 7fa8ebe00640 1 -- 172.21.15.1:0/2746665349 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fa8d00a2cd0 msgr2=0x7fa8d00a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:38.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.265+0000 7fa8ebe00640 1 --2- 172.21.15.1:0/2746665349 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fa8d00a2cd0 0x7fa8d00a5190 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4004af0 tx=0x7fa8e4005fd0 comp rx=0 tx=0).stop 2024-09-12T01:29:38.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.265+0000 7fa8ebe00640 1 -- 172.21.15.1:0/2746665349 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8ec103e30 msgr2=0x7fa8ec19a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:38.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.265+0000 7fa8ebe00640 1 --2- 172.21.15.1:0/2746665349 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8ec103e30 0x7fa8ec19a3e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc00e9f0 tx=0x7fa8dc00eeb0 comp rx=0 tx=0).stop 2024-09-12T01:29:38.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.265+0000 7fa8ebe00640 1 -- 172.21.15.1:0/2746665349 shutdown_connections 2024-09-12T01:29:38.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.265+0000 7fa8ebe00640 1 --2- 172.21.15.1:0/2746665349 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8ec108f60 0x7fa8ec19ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:38.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.265+0000 7fa8ebe00640 1 --2- 172.21.15.1:0/2746665349 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fa8d00a2cd0 0x7fa8d00a5190 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:38.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.265+0000 7fa8ebe00640 1 --2- 172.21.15.1:0/2746665349 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8ec1047e0 0x7fa8ec19a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:38.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.265+0000 7fa8ebe00640 1 --2- 172.21.15.1:0/2746665349 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8ec103e30 0x7fa8ec19a3e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:38.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.265+0000 7fa8ebe00640 1 -- 172.21.15.1:0/2746665349 >> 172.21.15.1:0/2746665349 conn(0x7fa8ec0ff8d0 msgr2=0x7fa8ec100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:38.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.265+0000 7fa8ebe00640 1 -- 172.21.15.1:0/2746665349 shutdown_connections 2024-09-12T01:29:38.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:38.265+0000 7fa8ebe00640 1 -- 172.21.15.1:0/2746665349 wait complete. 2024-09-12T01:29:39.311 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-12T01:29:39.697 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:29:39.993 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:39 smithi001 ceph-mon[30387]: pgmap v400: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T01:29:40.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:39 smithi104 ceph-mon[35219]: pgmap v400: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T01:29:40.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:39 smithi053 ceph-mon[35158]: pgmap v400: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T01:29:40.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.256+0000 7f6bcb400640 1 -- 172.21.15.1:0/2794372140 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bc4077040 msgr2=0x7f6bc41124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:40.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.256+0000 7f6bcb400640 1 --2- 172.21.15.1:0/2794372140 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bc4077040 0x7f6bc41124f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0009900 tx=0x7f6bb002a410 comp rx=0 tx=0).stop 2024-09-12T01:29:40.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.256+0000 7f6bcb400640 1 -- 172.21.15.1:0/2794372140 shutdown_connections 2024-09-12T01:29:40.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.256+0000 7f6bcb400640 1 --2- 172.21.15.1:0/2794372140 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6bc4112a30 0x7f6bc4114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:40.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.256+0000 7f6bcb400640 1 --2- 172.21.15.1:0/2794372140 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bc4077040 0x7f6bc41124f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:40.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.256+0000 7f6bcb400640 1 --2- 172.21.15.1:0/2794372140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6bc4076720 0x7f6bc4076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:40.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.256+0000 7f6bcb400640 1 -- 172.21.15.1:0/2794372140 >> 172.21.15.1:0/2794372140 conn(0x7f6bc40ffa90 msgr2=0x7f6bc4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:40.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.256+0000 7f6bcb400640 1 -- 172.21.15.1:0/2794372140 shutdown_connections 2024-09-12T01:29:40.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.256+0000 7f6bcb400640 1 -- 172.21.15.1:0/2794372140 wait complete. 2024-09-12T01:29:40.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.257+0000 7f6bcb400640 1 Processor -- start 2024-09-12T01:29:40.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.257+0000 7f6bcb400640 1 -- start start 2024-09-12T01:29:40.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.258+0000 7f6bcb400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bc4076720 0x7f6bc41a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:40.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.258+0000 7f6bcb400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6bc4077040 0x7f6bc41a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:40.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.258+0000 7f6bcb400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6bc4112a30 0x7f6bc419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:40.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.258+0000 7f6bcb400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc4118600 con 0x7f6bc4076720 2024-09-12T01:29:40.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.258+0000 7f6bcb400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc4118480 con 0x7f6bc4112a30 2024-09-12T01:29:40.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.258+0000 7f6bcb400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc4118780 con 0x7f6bc4077040 2024-09-12T01:29:40.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.258+0000 7f6bc8a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bc4076720 0x7f6bc41a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:40.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.258+0000 7f6bc8a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bc4076720 0x7f6bc41a46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46682/0 (socket says 172.21.15.1:46682) 2024-09-12T01:29:40.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.258+0000 7f6bc8a00640 1 -- 172.21.15.1:0/3482083053 learned_addr learned my addr 172.21.15.1:0/3482083053 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:40.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.258+0000 7f6bc9400640 1 --2- 172.21.15.1:0/3482083053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6bc4112a30 0x7f6bc419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:40.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.258+0000 7f6bc3e00640 1 --2- 172.21.15.1:0/3482083053 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6bc4077040 0x7f6bc41a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:40.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.258+0000 7f6bc8a00640 1 -- 172.21.15.1:0/3482083053 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6bc4077040 msgr2=0x7f6bc41a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:40.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.258+0000 7f6bc8a00640 1 --2- 172.21.15.1:0/3482083053 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6bc4077040 0x7f6bc41a4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:40.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.258+0000 7f6bc8a00640 1 -- 172.21.15.1:0/3482083053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6bc4112a30 msgr2=0x7f6bc419e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:40.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.258+0000 7f6bc8a00640 1 --2- 172.21.15.1:0/3482083053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6bc4112a30 0x7f6bc419e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:40.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.259+0000 7f6bc8a00640 1 -- 172.21.15.1:0/3482083053 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc419efc0 con 0x7f6bc4076720 2024-09-12T01:29:40.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.259+0000 7f6bc8a00640 1 --2- 172.21.15.1:0/3482083053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bc4076720 0x7f6bc41a46d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6bb800e9f0 tx=0x7f6bb800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:40.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.259+0000 7f6bc1600640 1 -- 172.21.15.1:0/3482083053 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb800cd80 con 0x7f6bc4076720 2024-09-12T01:29:40.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.259+0000 7f6bc1600640 1 -- 172.21.15.1:0/3482083053 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bb8010910 con 0x7f6bc4076720 2024-09-12T01:29:40.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.259+0000 7f6bc1600640 1 -- 172.21.15.1:0/3482083053 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb8018990 con 0x7f6bc4076720 2024-09-12T01:29:40.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.259+0000 7f6bcb400640 1 -- 172.21.15.1:0/3482083053 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bc419f2b0 con 0x7f6bc4076720 2024-09-12T01:29:40.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.259+0000 7f6bcb400640 1 -- 172.21.15.1:0/3482083053 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bc41ab5a0 con 0x7f6bc4076720 2024-09-12T01:29:40.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.262+0000 7f6bc1600640 1 -- 172.21.15.1:0/3482083053 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f6bb8010ac0 con 0x7f6bc4076720 2024-09-12T01:29:40.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.262+0000 7f6bcdfc4740 1 Processor -- start 2024-09-12T01:29:40.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.263+0000 7f6bcdfc4740 1 -- start start 2024-09-12T01:29:40.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.263+0000 7f6bcdfc4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555717800330 0x555717800710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:40.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.263+0000 7f6bcdfc4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555717800c50 0x55571780afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:40.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.263+0000 7f6bcdfc4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55571780b510 0x55571780d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:40.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.263+0000 7f6bcdfc4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5557177660b0 con 0x555717800330 2024-09-12T01:29:40.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.263+0000 7f6bcdfc4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55571770dab0 con 0x555717800c50 2024-09-12T01:29:40.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.263+0000 7f6bcdfc4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x555717762070 con 0x55571780b510 2024-09-12T01:29:40.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.263+0000 7f6bc9400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555717800330 0x555717800710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:40.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.263+0000 7f6bc9400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555717800330 0x555717800710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46690/0 (socket says 172.21.15.1:46690) 2024-09-12T01:29:40.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.263+0000 7f6bc9400640 1 -- 172.21.15.1:0/2876414367 learned_addr learned my addr 172.21.15.1:0/2876414367 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:40.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.263+0000 7f6bc8a00640 1 --2- 172.21.15.1:0/2876414367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555717800c50 0x55571780afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:40.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.263+0000 7f6bc3e00640 1 --2- 172.21.15.1:0/2876414367 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55571780b510 0x55571780d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:40.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.264+0000 7f6bc3e00640 1 -- 172.21.15.1:0/2876414367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555717800c50 msgr2=0x55571780afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:40.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.264+0000 7f6bc3e00640 1 --2- 172.21.15.1:0/2876414367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555717800c50 0x55571780afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:40.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.264+0000 7f6bc3e00640 1 -- 172.21.15.1:0/2876414367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555717800330 msgr2=0x555717800710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:40.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.264+0000 7f6bc3e00640 1 --2- 172.21.15.1:0/2876414367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555717800330 0x555717800710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:40.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.264+0000 7f6bc3e00640 1 -- 172.21.15.1:0/2876414367 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55571771a4a0 con 0x55571780b510 2024-09-12T01:29:40.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.264+0000 7f6bc1600640 1 --2- 172.21.15.1:0/3482083053 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f6bac0a2b80 0x7f6bac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:40.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.264+0000 7f6bc3e00640 1 --2- 172.21.15.1:0/2876414367 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55571780b510 0x55571780d900 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6bb002aa00 tx=0x7f6bb0005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4799da406d296830 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:40.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.264+0000 7f6bc1600640 1 -- 172.21.15.1:0/3482083053 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f6bb80cf6e0 con 0x7f6bc4076720 2024-09-12T01:29:40.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.265+0000 7f6bc9400640 1 --2- 172.21.15.1:0/3482083053 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f6bac0a2b80 0x7f6bac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:40.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.265+0000 7f6bb5c00640 1 -- 172.21.15.1:0/2876414367 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb0035040 con 0x55571780b510 2024-09-12T01:29:40.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.265+0000 7f6bb5c00640 1 -- 172.21.15.1:0/2876414367 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bb0004400 con 0x55571780b510 2024-09-12T01:29:40.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.265+0000 7f6bb5c00640 1 -- 172.21.15.1:0/2876414367 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb00035f0 con 0x55571780b510 2024-09-12T01:29:40.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.265+0000 7f6bcdfc4740 1 -- 172.21.15.1:0/2876414367 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55571780b510 msgr2=0x55571780d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:40.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.265+0000 7f6bc9400640 1 --2- 172.21.15.1:0/3482083053 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f6bac0a2b80 0x7f6bac0a5040 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc011630 tx=0x7f6bbc009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:40.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.265+0000 7f6bcdfc4740 1 --2- 172.21.15.1:0/2876414367 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55571780b510 0x55571780d900 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6bb002aa00 tx=0x7f6bb0005de0 comp rx=0 tx=0).stop 2024-09-12T01:29:40.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.266+0000 7f6bcdfc4740 1 -- 172.21.15.1:0/2876414367 shutdown_connections 2024-09-12T01:29:40.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.266+0000 7f6bcdfc4740 1 --2- 172.21.15.1:0/2876414367 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55571780b510 0x55571780d900 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:40.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.266+0000 7f6bcdfc4740 1 --2- 172.21.15.1:0/2876414367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555717800c50 0x55571780afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:40.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.266+0000 7f6bcdfc4740 1 --2- 172.21.15.1:0/2876414367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555717800330 0x555717800710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:40.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.266+0000 7f6bcdfc4740 1 -- 172.21.15.1:0/2876414367 >> 172.21.15.1:0/2876414367 conn(0x5557177fc8c0 msgr2=0x5557177fcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:40.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.266+0000 7f6bcdfc4740 1 -- 172.21.15.1:0/2876414367 shutdown_connections 2024-09-12T01:29:40.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.266+0000 7f6bcdfc4740 1 -- 172.21.15.1:0/2876414367 wait complete. 2024-09-12T01:29:40.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.266+0000 7f6bcdfc4740 1 -- start start 2024-09-12T01:29:40.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.266+0000 7f6bcdfc4740 1 Processor -- start 2024-09-12T01:29:40.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.267+0000 7f6bcdfc4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555717800330 0x5557178085d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:40.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.267+0000 7f6bcdfc4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555717800c50 0x555717808b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:40.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.267+0000 7f6bcdfc4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55571780b510 0x555717809050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:40.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.267+0000 7f6bcdfc4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55571780e060 con 0x555717800c50 2024-09-12T01:29:40.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.267+0000 7f6bcdfc4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5557176ee920 con 0x555717800330 2024-09-12T01:29:40.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.267+0000 7f6bcdfc4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55571780e1e0 con 0x55571780b510 2024-09-12T01:29:40.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.267+0000 7f6bcdfc4740 10 client.0 _ll_register_callbacks cb 0x5557177128a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:29:40.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.267+0000 7f6bc3e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555717800c50 0x555717808b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:40.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.267+0000 7f6bc8a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555717800330 0x5557178085d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:40.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.267+0000 7f6bc8a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555717800330 0x5557178085d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:51160/0 (socket says 172.21.15.1:51160) 2024-09-12T01:29:40.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.267+0000 7f6bc3e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555717800c50 0x555717808b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46694/0 (socket says 172.21.15.1:46694) 2024-09-12T01:29:40.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.267+0000 7f6bc8a00640 1 -- 172.21.15.1:0/2645268225 learned_addr learned my addr 172.21.15.1:0/2645268225 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:40.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.267+0000 7f6bc9400640 1 --2- 172.21.15.1:0/2645268225 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55571780b510 0x555717809050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:40.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.268+0000 7f6bc3e00640 1 -- 172.21.15.1:0/2645268225 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55571780b510 msgr2=0x555717809050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:40.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.268+0000 7f6bc3e00640 1 --2- 172.21.15.1:0/2645268225 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55571780b510 0x555717809050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:40.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.268+0000 7f6bc3e00640 1 -- 172.21.15.1:0/2645268225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555717800330 msgr2=0x5557178085d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:40.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.268+0000 7f6bc3e00640 1 --2- 172.21.15.1:0/2645268225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555717800330 0x5557178085d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:40.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.268+0000 7f6bc3e00640 1 -- 172.21.15.1:0/2645268225 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557174d72b0 con 0x555717800c50 2024-09-12T01:29:40.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.268+0000 7f6bc3e00640 1 --2- 172.21.15.1:0/2645268225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555717800c50 0x555717808b10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6bb002a930 tx=0x7f6bb00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e74ba2438fe93166 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:40.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.268+0000 7f6baa000640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-12T01:29:40.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.268+0000 7f6baa000640 1 -- 172.21.15.1:0/2645268225 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb0052060 con 0x555717800c50 2024-09-12T01:29:40.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.268+0000 7f6baa000640 1 -- 172.21.15.1:0/2645268225 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f6bb0004200 con 0x555717800c50 2024-09-12T01:29:40.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.268+0000 7f6bcdfc4740 1 -- 172.21.15.1:0/2645268225 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bb0004400 con 0x555717800c50 2024-09-12T01:29:40.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.269+0000 7f6baa000640 1 -- 172.21.15.1:0/2645268225 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6bb0059050 con 0x555717800c50 2024-09-12T01:29:40.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.269+0000 7f6bcdfc4740 10 client.15150 fetch_fsmap learned FSMap version 28 2024-09-12T01:29:40.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.269+0000 7f6bcdfc4740 1 -- 172.21.15.1:0/2645268225 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555717819ea0 con 0x555717800c50 2024-09-12T01:29:40.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.269+0000 7f6baa000640 1 -- 172.21.15.1:0/2645268225 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bb0004400 con 0x555717800c50 2024-09-12T01:29:40.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.269+0000 7f6baa000640 1 -- 172.21.15.1:0/2645268225 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f6bb0031a60 con 0x555717800c50 2024-09-12T01:29:40.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.269+0000 7f6bcdfc4740 10 client.15150 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:29:40.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.269+0000 7f6bcdfc4740 10 client.15150 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T01:29:40.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.269+0000 7f6bcdfc4740 20 client.15150 populate_metadata read hostname 'smithi001' 2024-09-12T01:29:40.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.269+0000 7f6bcdfc4740 1 --2- 172.21.15.1:0/2645268225 >> [v2:172.21.15.104:6838/1459976041,v1:172.21.15.104:6839/1459976041] conn(0x55571781bfa0 0x55571781d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:40.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.269+0000 7f6bcdfc4740 4 client.15150 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:29:40.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.269+0000 7f6bcdfc4740 1 -- 172.21.15.1:0/2645268225 --> [v2:172.21.15.104:6838/1459976041,v1:172.21.15.104:6839/1459976041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555717773620 con 0x55571781bfa0 2024-09-12T01:29:40.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.270+0000 7f6bc8a00640 1 --2- 172.21.15.1:0/2645268225 >> [v2:172.21.15.104:6838/1459976041,v1:172.21.15.104:6839/1459976041] conn(0x55571781bfa0 0x55571781d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:40.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.270+0000 7f6bc8a00640 1 --2- 172.21.15.1:0/2645268225 >> [v2:172.21.15.104:6838/1459976041,v1:172.21.15.104:6839/1459976041] conn(0x55571781bfa0 0x55571781d410 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fef3512c34e31404 server_cookie=d7dd269590af15a0 in_seq=0 out_seq=0 2024-09-12T01:29:40.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.270+0000 7f6baa000640 10 client.15150 ms_handle_connect on v2:172.21.15.104:6838/1459976041 2024-09-12T01:29:40.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.271+0000 7f6baa000640 1 -- 172.21.15.1:0/2645268225 <== mds.1 v2:172.21.15.104:6838/1459976041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555717773620 con 0x55571781bfa0 2024-09-12T01:29:40.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.271+0000 7f6baa000640 10 client.15150 handle_command_reply: tid=0 2024-09-12T01:29:40.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.271+0000 7f6bcdfc4740 1 client.15150 shutdown 2024-09-12T01:29:40.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.271+0000 7f6bcdfc4740 1 -- 172.21.15.1:0/2645268225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555717800c50 msgr2=0x555717808b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:40.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.271+0000 7f6bcdfc4740 1 --2- 172.21.15.1:0/2645268225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555717800c50 0x555717808b10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6bb002a930 tx=0x7f6bb00048f0 comp rx=0 tx=0).stop 2024-09-12T01:29:40.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.271+0000 7f6bcdfc4740 1 -- 172.21.15.1:0/2645268225 shutdown_connections 2024-09-12T01:29:40.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.271+0000 7f6bcdfc4740 1 --2- 172.21.15.1:0/2645268225 >> [v2:172.21.15.104:6838/1459976041,v1:172.21.15.104:6839/1459976041] conn(0x55571781bfa0 0x55571781d410 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:40.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.271+0000 7f6bcdfc4740 1 --2- 172.21.15.1:0/2645268225 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55571780b510 0x555717809050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:40.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.271+0000 7f6bcdfc4740 1 --2- 172.21.15.1:0/2645268225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555717800c50 0x555717808b10 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:40.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.271+0000 7f6bcdfc4740 1 --2- 172.21.15.1:0/2645268225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555717800330 0x5557178085d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:40.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.271+0000 7f6baa000640 0 client.15150 ms_handle_reset on v2:172.21.15.104:6838/1459976041 2024-09-12T01:29:40.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.271+0000 7f6bcdfc4740 1 -- 172.21.15.1:0/2645268225 >> 172.21.15.1:0/2645268225 conn(0x5557177fc8c0 msgr2=0x5557178025f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:40.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.272+0000 7f6bcdfc4740 1 -- 172.21.15.1:0/2645268225 shutdown_connections 2024-09-12T01:29:40.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.272+0000 7f6bcdfc4740 1 -- 172.21.15.1:0/2645268225 wait complete. 2024-09-12T01:29:40.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.272+0000 7f6bcdfc4740 20 client.15150 trim_cache size 0 max 16384 2024-09-12T01:29:40.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.290+0000 7f6bcdfc4740 1 Processor -- start 2024-09-12T01:29:40.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.290+0000 7f6bcdfc4740 1 -- start start 2024-09-12T01:29:40.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.291+0000 7f6bcdfc4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555717800330 0x555717808ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:40.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.291+0000 7f6bcdfc4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555717800c50 0x555717802d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:40.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.291+0000 7f6bcdfc4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55571780b510 0x555717805890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:40.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.291+0000 7f6bcdfc4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55571781b750 con 0x555717800330 2024-09-12T01:29:40.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.291+0000 7f6bcdfc4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55571781be20 con 0x55571780b510 2024-09-12T01:29:40.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.291+0000 7f6bcdfc4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55571781c4f0 con 0x555717800c50 2024-09-12T01:29:40.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.291+0000 7f6bc8a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555717800330 0x555717808ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:40.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.291+0000 7f6bc8a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555717800330 0x555717808ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46696/0 (socket says 172.21.15.1:46696) 2024-09-12T01:29:40.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.291+0000 7f6bc8a00640 1 -- 172.21.15.1:0/4149363628 learned_addr learned my addr 172.21.15.1:0/4149363628 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:40.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.291+0000 7f6bc9400640 1 --2- 172.21.15.1:0/4149363628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55571780b510 0x555717805890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:40.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.291+0000 7f6bc3e00640 1 --2- 172.21.15.1:0/4149363628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555717800c50 0x555717802d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:40.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.291+0000 7f6bc8a00640 1 -- 172.21.15.1:0/4149363628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555717800c50 msgr2=0x555717802d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:40.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.291+0000 7f6bc8a00640 1 --2- 172.21.15.1:0/4149363628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555717800c50 0x555717802d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:40.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.291+0000 7f6bc8a00640 1 -- 172.21.15.1:0/4149363628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55571780b510 msgr2=0x555717805890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:40.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.291+0000 7f6bc8a00640 1 --2- 172.21.15.1:0/4149363628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55571780b510 0x555717805890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:40.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.291+0000 7f6bc8a00640 1 -- 172.21.15.1:0/4149363628 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555717800120 con 0x555717800330 2024-09-12T01:29:40.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.292+0000 7f6bc8a00640 1 --2- 172.21.15.1:0/4149363628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555717800330 0x555717808ea0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6bb80980c0 tx=0x7f6bb80d5550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=383ae3fe63b1e00b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:40.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.292+0000 7f6bb5200640 1 -- 172.21.15.1:0/4149363628 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb809b5c0 con 0x555717800330 2024-09-12T01:29:40.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.292+0000 7f6bb5200640 1 -- 172.21.15.1:0/4149363628 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bb809bc60 con 0x555717800330 2024-09-12T01:29:40.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.292+0000 7f6bb5200640 1 -- 172.21.15.1:0/4149363628 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb80cf390 con 0x555717800330 2024-09-12T01:29:40.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.292+0000 7f6bcdfc4740 1 -- 172.21.15.1:0/4149363628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555717800330 msgr2=0x555717808ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:40.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.292+0000 7f6bcdfc4740 1 --2- 172.21.15.1:0/4149363628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555717800330 0x555717808ea0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6bb80980c0 tx=0x7f6bb80d5550 comp rx=0 tx=0).stop 2024-09-12T01:29:40.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.293+0000 7f6bcdfc4740 1 -- 172.21.15.1:0/4149363628 shutdown_connections 2024-09-12T01:29:40.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.293+0000 7f6bcdfc4740 1 --2- 172.21.15.1:0/4149363628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55571780b510 0x555717805890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:40.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.293+0000 7f6bcdfc4740 1 --2- 172.21.15.1:0/4149363628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555717800c50 0x555717802d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:40.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.293+0000 7f6bcdfc4740 1 --2- 172.21.15.1:0/4149363628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555717800330 0x555717808ea0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:40.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.293+0000 7f6bcdfc4740 1 -- 172.21.15.1:0/4149363628 >> 172.21.15.1:0/4149363628 conn(0x5557177fc8c0 msgr2=0x55571780a650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:40.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.293+0000 7f6bcdfc4740 1 -- 172.21.15.1:0/4149363628 shutdown_connections 2024-09-12T01:29:40.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.293+0000 7f6bcdfc4740 1 -- 172.21.15.1:0/4149363628 wait complete. 2024-09-12T01:29:40.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.293+0000 7f6bcdfc4740 1 -- start start 2024-09-12T01:29:40.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.293+0000 7f6bcdfc4740 1 Processor -- start 2024-09-12T01:29:40.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.294+0000 7f6bcdfc4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555717800c50 0x555717825670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:40.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.294+0000 7f6bcdfc4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55571780b510 0x555717825bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:40.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.294+0000 7f6bcdfc4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555717820620 0x555717820aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:40.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.294+0000 7f6bcdfc4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55571781d290 con 0x55571780b510 2024-09-12T01:29:40.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.294+0000 7f6bcdfc4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55571781cbc0 con 0x555717800c50 2024-09-12T01:29:40.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.294+0000 7f6bcdfc4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55571781d960 con 0x555717820620 2024-09-12T01:29:40.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.294+0000 7f6bcdfc4740 10 client.0 _ll_register_callbacks cb 0x7f6bb0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:29:40.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.294+0000 7f6bc3e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55571780b510 0x555717825bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:40.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.294+0000 7f6bc8a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555717800c50 0x555717825670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:40.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.294+0000 7f6bc9400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555717820620 0x555717820aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:40.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.294+0000 7f6bc3e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55571780b510 0x555717825bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46702/0 (socket says 172.21.15.1:46702) 2024-09-12T01:29:40.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.294+0000 7f6bc3e00640 1 -- 172.21.15.1:0/3710064600 learned_addr learned my addr 172.21.15.1:0/3710064600 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:40.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.295+0000 7f6bc3e00640 1 -- 172.21.15.1:0/3710064600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555717820620 msgr2=0x555717820aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:40.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.295+0000 7f6bc3e00640 1 --2- 172.21.15.1:0/3710064600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555717820620 0x555717820aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:40.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.295+0000 7f6bc3e00640 1 -- 172.21.15.1:0/3710064600 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555717800c50 msgr2=0x555717825670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:40.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.295+0000 7f6bc3e00640 1 --2- 172.21.15.1:0/3710064600 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555717800c50 0x555717825670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:40.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.295+0000 7f6bc3e00640 1 -- 172.21.15.1:0/3710064600 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555717820fe0 con 0x55571780b510 2024-09-12T01:29:40.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.295+0000 7f6bc9400640 1 --2- 172.21.15.1:0/3710064600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555717820620 0x555717820aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:29:40.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.295+0000 7f6bc8a00640 1 --2- 172.21.15.1:0/3710064600 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555717800c50 0x555717825670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:29:40.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.295+0000 7f6bc3e00640 1 --2- 172.21.15.1:0/3710064600 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55571780b510 0x555717825bb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0009900 tx=0x7f6bb0004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd7db2c198d2c83c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:40.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.295+0000 7f6baa000640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-12T01:29:40.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.295+0000 7f6baa000640 1 -- 172.21.15.1:0/3710064600 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb0062060 con 0x55571780b510 2024-09-12T01:29:40.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.295+0000 7f6baa000640 1 -- 172.21.15.1:0/3710064600 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f6bb0004b70 con 0x55571780b510 2024-09-12T01:29:40.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.295+0000 7f6bcdfc4740 1 -- 172.21.15.1:0/3710064600 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bb809bc60 con 0x55571780b510 2024-09-12T01:29:40.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.296+0000 7f6baa000640 1 -- 172.21.15.1:0/3710064600 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6bb0069050 con 0x55571780b510 2024-09-12T01:29:40.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.296+0000 7f6bcdfc4740 10 client.15156 fetch_fsmap learned FSMap version 28 2024-09-12T01:29:40.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.296+0000 7f6bcdfc4740 1 -- 172.21.15.1:0/3710064600 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555717887290 con 0x55571780b510 2024-09-12T01:29:40.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.296+0000 7f6baa000640 1 -- 172.21.15.1:0/3710064600 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bb809bc60 con 0x55571780b510 2024-09-12T01:29:40.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.296+0000 7f6baa000640 1 -- 172.21.15.1:0/3710064600 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f6bb00318c0 con 0x55571780b510 2024-09-12T01:29:40.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.296+0000 7f6bcdfc4740 10 client.15156 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:29:40.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.296+0000 7f6bcdfc4740 10 client.15156 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T01:29:40.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.296+0000 7f6bcdfc4740 20 client.15156 populate_metadata read hostname 'smithi001' 2024-09-12T01:29:40.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.296+0000 7f6bcdfc4740 1 --2- 172.21.15.1:0/3710064600 >> [v2:172.21.15.104:6838/1459976041,v1:172.21.15.104:6839/1459976041] conn(0x55571788bd60 0x55571788aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:40.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.296+0000 7f6bcdfc4740 4 client.15156 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T01:29:40.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.296+0000 7f6bcdfc4740 1 -- 172.21.15.1:0/3710064600 --> [v2:172.21.15.104:6838/1459976041,v1:172.21.15.104:6839/1459976041] -- command(tid 0: {"prefix": "status"}) -- 0x55571776db30 con 0x55571788bd60 2024-09-12T01:29:40.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.297+0000 7f6bc8a00640 1 --2- 172.21.15.1:0/3710064600 >> [v2:172.21.15.104:6838/1459976041,v1:172.21.15.104:6839/1459976041] conn(0x55571788bd60 0x55571788aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:40.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.298+0000 7f6bc8a00640 1 --2- 172.21.15.1:0/3710064600 >> [v2:172.21.15.104:6838/1459976041,v1:172.21.15.104:6839/1459976041] conn(0x55571788bd60 0x55571788aa70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2a76975975bb3b3a server_cookie=bbf2a5927ee0a1d3 in_seq=0 out_seq=0 2024-09-12T01:29:40.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.298+0000 7f6baa000640 10 client.15156 ms_handle_connect on v2:172.21.15.104:6838/1459976041 2024-09-12T01:29:40.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.298+0000 7f6baa000640 1 -- 172.21.15.1:0/3710064600 <== mds.1 v2:172.21.15.104:6838/1459976041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55571776db30 con 0x55571788bd60 2024-09-12T01:29:40.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.298+0000 7f6baa000640 10 client.15156 handle_command_reply: tid=0 2024-09-12T01:29:40.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.298+0000 7f6bcdfc4740 1 client.15156 shutdown 2024-09-12T01:29:40.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.298+0000 7f6bcdfc4740 1 -- 172.21.15.1:0/3710064600 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55571780b510 msgr2=0x555717825bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:40.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.298+0000 7f6bcdfc4740 1 --2- 172.21.15.1:0/3710064600 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55571780b510 0x555717825bb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0009900 tx=0x7f6bb0004410 comp rx=0 tx=0).stop 2024-09-12T01:29:40.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.298+0000 7f6bcdfc4740 1 -- 172.21.15.1:0/3710064600 shutdown_connections 2024-09-12T01:29:40.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.298+0000 7f6bcdfc4740 1 --2- 172.21.15.1:0/3710064600 >> [v2:172.21.15.104:6838/1459976041,v1:172.21.15.104:6839/1459976041] conn(0x55571788bd60 0x55571788aa70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:40.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.298+0000 7f6bcdfc4740 1 --2- 172.21.15.1:0/3710064600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555717820620 0x555717820aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:40.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.298+0000 7f6baa000640 0 client.15156 ms_handle_reset on v2:172.21.15.104:6838/1459976041 2024-09-12T01:29:40.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.298+0000 7f6bcdfc4740 1 --2- 172.21.15.1:0/3710064600 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55571780b510 0x555717825bb0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:40.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.298+0000 7f6bcdfc4740 1 --2- 172.21.15.1:0/3710064600 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555717800c50 0x555717825670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:40.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.298+0000 7f6bcdfc4740 1 -- 172.21.15.1:0/3710064600 >> 172.21.15.1:0/3710064600 conn(0x5557177fc8c0 msgr2=0x555717816430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:40.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.298+0000 7f6bcdfc4740 1 -- 172.21.15.1:0/3710064600 shutdown_connections 2024-09-12T01:29:40.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.299+0000 7f6bcdfc4740 1 -- 172.21.15.1:0/3710064600 wait complete. 2024-09-12T01:29:40.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.299+0000 7f6bcdfc4740 20 client.15156 trim_cache size 0 max 16384 2024-09-12T01:29:40.302 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-12T01:29:40.302 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "c0e4f180-70a4-11ef-bcea-c7b262605968", 2024-09-12T01:29:40.302 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 1, 2024-09-12T01:29:40.303 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24511, 2024-09-12T01:29:40.303 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-09-12T01:29:40.303 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-09-12T01:29:40.303 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-09-12T01:29:40.303 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 119.006643954, 2024-09-12T01:29:40.303 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 28, 2024-09-12T01:29:40.303 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 98, 2024-09-12T01:29:40.303 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 96, 2024-09-12T01:29:40.303 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 196.01075744100001 2024-09-12T01:29:40.303 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-12T01:29:40.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.299+0000 7f6bcb400640 1 -- 172.21.15.1:0/3482083053 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f6bac0a2b80 msgr2=0x7f6bac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:40.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.299+0000 7f6bcb400640 1 --2- 172.21.15.1:0/3482083053 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f6bac0a2b80 0x7f6bac0a5040 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc011630 tx=0x7f6bbc009500 comp rx=0 tx=0).stop 2024-09-12T01:29:40.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.299+0000 7f6bcb400640 1 -- 172.21.15.1:0/3482083053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bc4076720 msgr2=0x7f6bc41a46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:40.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.299+0000 7f6bcb400640 1 --2- 172.21.15.1:0/3482083053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bc4076720 0x7f6bc41a46d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6bb800e9f0 tx=0x7f6bb800eeb0 comp rx=0 tx=0).stop 2024-09-12T01:29:40.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.299+0000 7f6bcb400640 1 -- 172.21.15.1:0/3482083053 shutdown_connections 2024-09-12T01:29:40.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.299+0000 7f6bcb400640 1 --2- 172.21.15.1:0/3482083053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6bc4112a30 0x7f6bc419e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:40.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.299+0000 7f6bcb400640 1 --2- 172.21.15.1:0/3482083053 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f6bac0a2b80 0x7f6bac0a5040 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:40.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.299+0000 7f6bcb400640 1 --2- 172.21.15.1:0/3482083053 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6bc4077040 0x7f6bc41a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:40.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.299+0000 7f6bcb400640 1 --2- 172.21.15.1:0/3482083053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bc4076720 0x7f6bc41a46d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:40.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.299+0000 7f6bcb400640 1 -- 172.21.15.1:0/3482083053 >> 172.21.15.1:0/3482083053 conn(0x7f6bc40ffa90 msgr2=0x7f6bc4101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:40.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.300+0000 7f6bcb400640 1 -- 172.21.15.1:0/3482083053 shutdown_connections 2024-09-12T01:29:40.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:40.300+0000 7f6bcb400640 1 -- 172.21.15.1:0/3482083053 wait complete. 2024-09-12T01:29:41.261 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-12T01:29:41.639 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:29:42.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:41 smithi104 ceph-mon[35219]: pgmap v401: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T01:29:42.109 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:41 smithi001 ceph-mon[30387]: pgmap v401: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T01:29:42.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:41 smithi053 ceph-mon[35158]: pgmap v401: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T01:29:42.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.347+0000 7f5b46400640 1 -- 172.21.15.1:0/3072529388 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b40077040 msgr2=0x7f5b401124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:42.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.347+0000 7f5b46400640 1 --2- 172.21.15.1:0/3072529388 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b40077040 0x7f5b401124f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c009900 tx=0x7f5b2c02a3f0 comp rx=0 tx=0).stop 2024-09-12T01:29:42.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.348+0000 7f5b46400640 1 -- 172.21.15.1:0/3072529388 shutdown_connections 2024-09-12T01:29:42.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.348+0000 7f5b46400640 1 --2- 172.21.15.1:0/3072529388 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b40112a30 0x7f5b40114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:42.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.348+0000 7f5b46400640 1 --2- 172.21.15.1:0/3072529388 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b40077040 0x7f5b401124f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:42.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.348+0000 7f5b46400640 1 --2- 172.21.15.1:0/3072529388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b40076720 0x7f5b40076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:42.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.348+0000 7f5b46400640 1 -- 172.21.15.1:0/3072529388 >> 172.21.15.1:0/3072529388 conn(0x7f5b400ffa90 msgr2=0x7f5b40101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:42.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.348+0000 7f5b46400640 1 -- 172.21.15.1:0/3072529388 shutdown_connections 2024-09-12T01:29:42.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.348+0000 7f5b46400640 1 -- 172.21.15.1:0/3072529388 wait complete. 2024-09-12T01:29:42.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.349+0000 7f5b46400640 1 Processor -- start 2024-09-12T01:29:42.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.349+0000 7f5b46400640 1 -- start start 2024-09-12T01:29:42.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.349+0000 7f5b46400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b40076720 0x7f5b401a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:42.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.349+0000 7f5b46400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b40077040 0x7f5b401a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:42.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.349+0000 7f5b46400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b40112a30 0x7f5b4019e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:42.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.349+0000 7f5b46400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5b40118600 con 0x7f5b40077040 2024-09-12T01:29:42.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.349+0000 7f5b46400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5b40118480 con 0x7f5b40112a30 2024-09-12T01:29:42.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.349+0000 7f5b46400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5b40118780 con 0x7f5b40076720 2024-09-12T01:29:42.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.349+0000 7f5b3ea00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b40077040 0x7f5b401a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:42.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.350+0000 7f5b3f400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b40076720 0x7f5b401a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:42.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.350+0000 7f5b3ea00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b40077040 0x7f5b401a4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46728/0 (socket says 172.21.15.1:46728) 2024-09-12T01:29:42.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.350+0000 7f5b3ea00640 1 -- 172.21.15.1:0/3998162981 learned_addr learned my addr 172.21.15.1:0/3998162981 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:42.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.350+0000 7f5b3fe00640 1 --2- 172.21.15.1:0/3998162981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b40112a30 0x7f5b4019e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:42.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.350+0000 7f5b3ea00640 1 -- 172.21.15.1:0/3998162981 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b40076720 msgr2=0x7f5b401a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:42.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.350+0000 7f5b3ea00640 1 --2- 172.21.15.1:0/3998162981 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b40076720 0x7f5b401a46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:42.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.350+0000 7f5b3ea00640 1 -- 172.21.15.1:0/3998162981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b40112a30 msgr2=0x7f5b4019e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:42.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.350+0000 7f5b3ea00640 1 --2- 172.21.15.1:0/3998162981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b40112a30 0x7f5b4019e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:42.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.350+0000 7f5b3ea00640 1 -- 172.21.15.1:0/3998162981 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b4019efc0 con 0x7f5b40077040 2024-09-12T01:29:42.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.350+0000 7f5b3f400640 1 --2- 172.21.15.1:0/3998162981 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b40076720 0x7f5b401a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:29:42.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.350+0000 7f5b3fe00640 1 --2- 172.21.15.1:0/3998162981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b40112a30 0x7f5b4019e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:29:42.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.350+0000 7f5b3ea00640 1 --2- 172.21.15.1:0/3998162981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b40077040 0x7f5b401a4c10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c004770 tx=0x7f5b2c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:42.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.351+0000 7f5b37e00640 1 -- 172.21.15.1:0/3998162981 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c03f070 con 0x7f5b40077040 2024-09-12T01:29:42.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.351+0000 7f5b37e00640 1 -- 172.21.15.1:0/3998162981 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b2c031ca0 con 0x7f5b40077040 2024-09-12T01:29:42.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.351+0000 7f5b37e00640 1 -- 172.21.15.1:0/3998162981 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c035030 con 0x7f5b40077040 2024-09-12T01:29:42.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.351+0000 7f5b46400640 1 -- 172.21.15.1:0/3998162981 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b4019f250 con 0x7f5b40077040 2024-09-12T01:29:42.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.351+0000 7f5b46400640 1 -- 172.21.15.1:0/3998162981 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b401ab5a0 con 0x7f5b40077040 2024-09-12T01:29:42.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.354+0000 7f5b48fd0740 1 Processor -- start 2024-09-12T01:29:42.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.354+0000 7f5b48fd0740 1 -- start start 2024-09-12T01:29:42.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.355+0000 7f5b48fd0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e4a0b330 0x5602e4a0b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:42.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.355+0000 7f5b48fd0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602e4a0bc50 0x5602e4a15fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:42.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.355+0000 7f5b37e00640 1 -- 172.21.15.1:0/3998162981 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f5b2c042070 con 0x7f5b40077040 2024-09-12T01:29:42.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.355+0000 7f5b48fd0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602e4a16510 0x5602e4a18900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:42.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.355+0000 7f5b48fd0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5602e49710b0 con 0x5602e4a16510 2024-09-12T01:29:42.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.355+0000 7f5b48fd0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5602e4918ab0 con 0x5602e4a0b330 2024-09-12T01:29:42.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.355+0000 7f5b48fd0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5602e496d070 con 0x5602e4a0bc50 2024-09-12T01:29:42.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.355+0000 7f5b3ea00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602e4a16510 0x5602e4a18900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:42.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.355+0000 7f5b3fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e4a0b330 0x5602e4a0b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:42.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.355+0000 7f5b3ea00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602e4a16510 0x5602e4a18900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46744/0 (socket says 172.21.15.1:46744) 2024-09-12T01:29:42.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.355+0000 7f5b3ea00640 1 -- 172.21.15.1:0/2403335562 learned_addr learned my addr 172.21.15.1:0/2403335562 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:42.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.355+0000 7f5b3fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e4a0b330 0x5602e4a0b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:51208/0 (socket says 172.21.15.1:51208) 2024-09-12T01:29:42.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.355+0000 7f5b3f400640 1 --2- 172.21.15.1:0/2403335562 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602e4a0bc50 0x5602e4a15fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:42.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.356+0000 7f5b3fe00640 1 -- 172.21.15.1:0/2403335562 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602e4a0bc50 msgr2=0x5602e4a15fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:42.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.356+0000 7f5b3fe00640 1 --2- 172.21.15.1:0/2403335562 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602e4a0bc50 0x5602e4a15fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:42.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.356+0000 7f5b3fe00640 1 -- 172.21.15.1:0/2403335562 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602e4a16510 msgr2=0x5602e4a18900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:42.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.356+0000 7f5b3fe00640 1 --2- 172.21.15.1:0/2403335562 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602e4a16510 0x5602e4a18900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:42.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.356+0000 7f5b3fe00640 1 -- 172.21.15.1:0/2403335562 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602e49254a0 con 0x5602e4a0b330 2024-09-12T01:29:42.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.356+0000 7f5b3fe00640 1 --2- 172.21.15.1:0/2403335562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e4a0b330 0x5602e4a0b710 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5b4006b460 tx=0x7f5b38011b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f518e0af142e4d21 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:42.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.356+0000 7f5b34c00640 1 -- 172.21.15.1:0/2403335562 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b38004d20 con 0x5602e4a0b330 2024-09-12T01:29:42.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.356+0000 7f5b34c00640 1 -- 172.21.15.1:0/2403335562 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b3800f970 con 0x5602e4a0b330 2024-09-12T01:29:42.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.356+0000 7f5b34c00640 1 -- 172.21.15.1:0/2403335562 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b38004470 con 0x5602e4a0b330 2024-09-12T01:29:42.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.356+0000 7f5b37e00640 1 --2- 172.21.15.1:0/3998162981 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f5b240a2c00 0x7f5b240a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:42.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.357+0000 7f5b37e00640 1 -- 172.21.15.1:0/3998162981 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f5b2c0ef5a0 con 0x7f5b40077040 2024-09-12T01:29:42.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.357+0000 7f5b3f400640 1 --2- 172.21.15.1:0/3998162981 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f5b240a2c00 0x7f5b240a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:42.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.357+0000 7f5b48fd0740 1 -- 172.21.15.1:0/2403335562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e4a0b330 msgr2=0x5602e4a0b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:42.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.357+0000 7f5b48fd0740 1 --2- 172.21.15.1:0/2403335562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e4a0b330 0x5602e4a0b710 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5b4006b460 tx=0x7f5b38011b30 comp rx=0 tx=0).stop 2024-09-12T01:29:42.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.357+0000 7f5b48fd0740 1 -- 172.21.15.1:0/2403335562 shutdown_connections 2024-09-12T01:29:42.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.357+0000 7f5b48fd0740 1 --2- 172.21.15.1:0/2403335562 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602e4a16510 0x5602e4a18900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:42.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.357+0000 7f5b48fd0740 1 --2- 172.21.15.1:0/2403335562 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602e4a0bc50 0x5602e4a15fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:42.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.357+0000 7f5b48fd0740 1 --2- 172.21.15.1:0/2403335562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e4a0b330 0x5602e4a0b710 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:42.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.357+0000 7f5b48fd0740 1 -- 172.21.15.1:0/2403335562 >> 172.21.15.1:0/2403335562 conn(0x5602e4a078c0 msgr2=0x5602e4a07ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:42.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.357+0000 7f5b48fd0740 1 -- 172.21.15.1:0/2403335562 shutdown_connections 2024-09-12T01:29:42.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.357+0000 7f5b48fd0740 1 -- 172.21.15.1:0/2403335562 wait complete. 2024-09-12T01:29:42.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.358+0000 7f5b3f400640 1 --2- 172.21.15.1:0/3998162981 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f5b240a2c00 0x7f5b240a50c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x5602e4a08720 tx=0x7f5b30005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:42.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.358+0000 7f5b48fd0740 1 -- start start 2024-09-12T01:29:42.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.358+0000 7f5b48fd0740 1 Processor -- start 2024-09-12T01:29:42.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.359+0000 7f5b48fd0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602e4a0b330 0x5602e4a134e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:42.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.359+0000 7f5b48fd0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602e4a0bc50 0x5602e4a13a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:42.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.359+0000 7f5b48fd0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e4a16510 0x5602e4a13f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:42.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.359+0000 7f5b48fd0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5602e4a19060 con 0x5602e4a0b330 2024-09-12T01:29:42.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.359+0000 7f5b48fd0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5602e48f9920 con 0x5602e4a16510 2024-09-12T01:29:42.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.359+0000 7f5b48fd0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5602e4a191e0 con 0x5602e4a0bc50 2024-09-12T01:29:42.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.359+0000 7f5b48fd0740 10 client.0 _ll_register_callbacks cb 0x5602e491d8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:29:42.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.359+0000 7f5b3f400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602e4a0b330 0x5602e4a134e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:42.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.359+0000 7f5b3ea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602e4a0bc50 0x5602e4a13a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:42.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.359+0000 7f5b3fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e4a16510 0x5602e4a13f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:42.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.359+0000 7f5b3ea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602e4a0bc50 0x5602e4a13a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:37568/0 (socket says 172.21.15.1:37568) 2024-09-12T01:29:42.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.359+0000 7f5b3ea00640 1 -- 172.21.15.1:0/861087879 learned_addr learned my addr 172.21.15.1:0/861087879 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:42.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.359+0000 7f5b3ea00640 1 -- 172.21.15.1:0/861087879 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e4a16510 msgr2=0x5602e4a13f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:42.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.359+0000 7f5b3ea00640 1 --2- 172.21.15.1:0/861087879 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e4a16510 0x5602e4a13f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:42.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.360+0000 7f5b3ea00640 1 -- 172.21.15.1:0/861087879 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602e4a0b330 msgr2=0x5602e4a134e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:42.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.360+0000 7f5b3ea00640 1 --2- 172.21.15.1:0/861087879 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602e4a0b330 0x5602e4a134e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:42.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.360+0000 7f5b3ea00640 1 -- 172.21.15.1:0/861087879 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602e46e22b0 con 0x5602e4a0bc50 2024-09-12T01:29:42.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.360+0000 7f5b3ea00640 1 --2- 172.21.15.1:0/861087879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602e4a0bc50 0x5602e4a13a20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c0b85b0 tx=0x7f5b2c0f2c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d0c855d8dde6918 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:42.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.360+0000 7f5b29600640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-12T01:29:42.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.360+0000 7f5b29600640 1 -- 172.21.15.1:0/861087879 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c0bce60 con 0x5602e4a0bc50 2024-09-12T01:29:42.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.360+0000 7f5b48fd0740 1 -- 172.21.15.1:0/861087879 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b3800f970 con 0x5602e4a0bc50 2024-09-12T01:29:42.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.360+0000 7f5b29600640 1 -- 172.21.15.1:0/861087879 <== mon.1 v2:172.21.15.104:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f5b2c0ece20 con 0x5602e4a0bc50 2024-09-12T01:29:42.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.360+0000 7f5b29600640 1 -- 172.21.15.1:0/861087879 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5b2c049070 con 0x5602e4a0bc50 2024-09-12T01:29:42.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.360+0000 7f5b48fd0740 10 client.25189 fetch_fsmap learned FSMap version 28 2024-09-12T01:29:42.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.360+0000 7f5b48fd0740 1 -- 172.21.15.1:0/861087879 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5602e4a190c0 con 0x5602e4a0bc50 2024-09-12T01:29:42.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.361+0000 7f5b29600640 1 -- 172.21.15.1:0/861087879 <== mon.1 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b3800f970 con 0x5602e4a0bc50 2024-09-12T01:29:42.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.361+0000 7f5b29600640 1 -- 172.21.15.1:0/861087879 <== mon.1 v2:172.21.15.104:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f5b2c04c060 con 0x5602e4a0bc50 2024-09-12T01:29:42.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.361+0000 7f5b48fd0740 10 client.25189 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:29:42.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.361+0000 7f5b48fd0740 10 client.25189 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T01:29:42.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.361+0000 7f5b48fd0740 20 client.25189 populate_metadata read hostname 'smithi001' 2024-09-12T01:29:42.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.361+0000 7f5b48fd0740 1 --2- 172.21.15.1:0/861087879 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5602e4a27440 0x5602e4a288b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:42.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.361+0000 7f5b48fd0740 4 client.25189 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:29:42.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.361+0000 7f5b48fd0740 1 -- 172.21.15.1:0/861087879 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602e497e620 con 0x5602e4a27440 2024-09-12T01:29:42.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.362+0000 7f5b3f400640 1 --2- 172.21.15.1:0/861087879 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5602e4a27440 0x5602e4a288b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:42.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.362+0000 7f5b3f400640 1 --2- 172.21.15.1:0/861087879 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5602e4a27440 0x5602e4a288b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9628b9e63f13241 server_cookie=f5ac2f7e2019517 in_seq=0 out_seq=0 2024-09-12T01:29:42.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.362+0000 7f5b29600640 10 client.25189 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:29:42.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.363+0000 7f5b29600640 1 -- 172.21.15.1:0/861087879 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5602e497e620 con 0x5602e4a27440 2024-09-12T01:29:42.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.363+0000 7f5b29600640 10 client.25189 handle_command_reply: tid=0 2024-09-12T01:29:42.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.363+0000 7f5b48fd0740 1 client.25189 shutdown 2024-09-12T01:29:42.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.364+0000 7f5b48fd0740 1 -- 172.21.15.1:0/861087879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602e4a0bc50 msgr2=0x5602e4a13a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:42.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.364+0000 7f5b48fd0740 1 --2- 172.21.15.1:0/861087879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602e4a0bc50 0x5602e4a13a20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c0b85b0 tx=0x7f5b2c0f2c60 comp rx=0 tx=0).stop 2024-09-12T01:29:42.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.364+0000 7f5b48fd0740 1 -- 172.21.15.1:0/861087879 shutdown_connections 2024-09-12T01:29:42.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.364+0000 7f5b48fd0740 1 --2- 172.21.15.1:0/861087879 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5602e4a27440 0x5602e4a288b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:42.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.364+0000 7f5b48fd0740 1 --2- 172.21.15.1:0/861087879 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e4a16510 0x5602e4a13f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:42.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.364+0000 7f5b48fd0740 1 --2- 172.21.15.1:0/861087879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602e4a0bc50 0x5602e4a13a20 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:42.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.364+0000 7f5b29600640 0 client.25189 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:29:42.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.364+0000 7f5b48fd0740 1 --2- 172.21.15.1:0/861087879 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602e4a0b330 0x5602e4a134e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:42.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.364+0000 7f5b48fd0740 1 -- 172.21.15.1:0/861087879 >> 172.21.15.1:0/861087879 conn(0x5602e4a078c0 msgr2=0x5602e4a0d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:42.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.364+0000 7f5b48fd0740 1 -- 172.21.15.1:0/861087879 shutdown_connections 2024-09-12T01:29:42.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.364+0000 7f5b48fd0740 1 -- 172.21.15.1:0/861087879 wait complete. 2024-09-12T01:29:42.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.364+0000 7f5b48fd0740 20 client.25189 trim_cache size 0 max 16384 2024-09-12T01:29:42.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.383+0000 7f5b48fd0740 1 Processor -- start 2024-09-12T01:29:42.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.383+0000 7f5b48fd0740 1 -- start start 2024-09-12T01:29:42.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.383+0000 7f5b48fd0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602e4a0b330 0x5602e4a12da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:42.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.383+0000 7f5b48fd0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e4a0bc50 0x5602e4a25f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:42.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.383+0000 7f5b48fd0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602e4a16510 0x5602e4a264d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:42.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.383+0000 7f5b48fd0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5602e4a26bf0 con 0x5602e4a16510 2024-09-12T01:29:42.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.383+0000 7f5b48fd0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5602e4a272c0 con 0x5602e4a0bc50 2024-09-12T01:29:42.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.383+0000 7f5b48fd0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5602e4a27990 con 0x5602e4a0b330 2024-09-12T01:29:42.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.384+0000 7f5b3ea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e4a0bc50 0x5602e4a25f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:42.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.384+0000 7f5b3fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602e4a16510 0x5602e4a264d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:42.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.384+0000 7f5b3ea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e4a0bc50 0x5602e4a25f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:51230/0 (socket says 172.21.15.1:51230) 2024-09-12T01:29:42.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.384+0000 7f5b3ea00640 1 -- 172.21.15.1:0/3161109200 learned_addr learned my addr 172.21.15.1:0/3161109200 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:42.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.384+0000 7f5b3f400640 1 --2- 172.21.15.1:0/3161109200 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602e4a0b330 0x5602e4a12da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:42.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.384+0000 7f5b3ea00640 1 -- 172.21.15.1:0/3161109200 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602e4a0b330 msgr2=0x5602e4a12da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:42.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.384+0000 7f5b3ea00640 1 --2- 172.21.15.1:0/3161109200 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602e4a0b330 0x5602e4a12da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:42.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.384+0000 7f5b3ea00640 1 -- 172.21.15.1:0/3161109200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602e4a16510 msgr2=0x5602e4a264d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:42.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.384+0000 7f5b3ea00640 1 --2- 172.21.15.1:0/3161109200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602e4a16510 0x5602e4a264d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:42.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.384+0000 7f5b3ea00640 1 -- 172.21.15.1:0/3161109200 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602e4a26a10 con 0x5602e4a0bc50 2024-09-12T01:29:42.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.384+0000 7f5b3ea00640 1 --2- 172.21.15.1:0/3161109200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e4a0bc50 0x5602e4a25f90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c0f2cd0 tx=0x7f5b2c02a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cf31a5064be84c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:42.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.384+0000 7f5b2be00640 1 -- 172.21.15.1:0/3161109200 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c0d8070 con 0x5602e4a0bc50 2024-09-12T01:29:42.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.384+0000 7f5b2be00640 1 -- 172.21.15.1:0/3161109200 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b2c0b8250 con 0x5602e4a0bc50 2024-09-12T01:29:42.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.384+0000 7f5b2be00640 1 -- 172.21.15.1:0/3161109200 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c0bce60 con 0x5602e4a0bc50 2024-09-12T01:29:42.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.385+0000 7f5b48fd0740 1 -- 172.21.15.1:0/3161109200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e4a0bc50 msgr2=0x5602e4a25f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:42.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.385+0000 7f5b48fd0740 1 --2- 172.21.15.1:0/3161109200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e4a0bc50 0x5602e4a25f90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c0f2cd0 tx=0x7f5b2c02a3f0 comp rx=0 tx=0).stop 2024-09-12T01:29:42.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.385+0000 7f5b48fd0740 1 -- 172.21.15.1:0/3161109200 shutdown_connections 2024-09-12T01:29:42.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.385+0000 7f5b48fd0740 1 --2- 172.21.15.1:0/3161109200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602e4a16510 0x5602e4a264d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:42.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.385+0000 7f5b48fd0740 1 --2- 172.21.15.1:0/3161109200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e4a0bc50 0x5602e4a25f90 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:42.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.385+0000 7f5b48fd0740 1 --2- 172.21.15.1:0/3161109200 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602e4a0b330 0x5602e4a12da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:42.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.385+0000 7f5b48fd0740 1 -- 172.21.15.1:0/3161109200 >> 172.21.15.1:0/3161109200 conn(0x5602e4a078c0 msgr2=0x5602e4a11db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:42.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.385+0000 7f5b48fd0740 1 -- 172.21.15.1:0/3161109200 shutdown_connections 2024-09-12T01:29:42.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.385+0000 7f5b48fd0740 1 -- 172.21.15.1:0/3161109200 wait complete. 2024-09-12T01:29:42.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.386+0000 7f5b48fd0740 1 -- start start 2024-09-12T01:29:42.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.386+0000 7f5b48fd0740 1 Processor -- start 2024-09-12T01:29:42.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.386+0000 7f5b48fd0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602e4a0b330 0x5602e4a312d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:42.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.386+0000 7f5b48fd0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e4a0bc50 0x5602e4a2a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:42.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.386+0000 7f5b48fd0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602e4a16510 0x5602e4a2abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:42.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.386+0000 7f5b48fd0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5602e4a28730 con 0x5602e4a0b330 2024-09-12T01:29:42.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.386+0000 7f5b48fd0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5602e4a28060 con 0x5602e4a0bc50 2024-09-12T01:29:42.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.386+0000 7f5b48fd0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5602e4a28e00 con 0x5602e4a16510 2024-09-12T01:29:42.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.386+0000 7f5b48fd0740 10 client.0 _ll_register_callbacks cb 0x7f5b2c0f2d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:29:42.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.387+0000 7f5b3f400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602e4a0b330 0x5602e4a312d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:42.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.387+0000 7f5b3fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602e4a16510 0x5602e4a2abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:42.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.387+0000 7f5b3ea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e4a0bc50 0x5602e4a2a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:42.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.387+0000 7f5b3fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602e4a16510 0x5602e4a2abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:37576/0 (socket says 172.21.15.1:37576) 2024-09-12T01:29:42.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.387+0000 7f5b3fe00640 1 -- 172.21.15.1:0/532630034 learned_addr learned my addr 172.21.15.1:0/532630034 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:42.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.387+0000 7f5b3ea00640 1 -- 172.21.15.1:0/532630034 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602e4a16510 msgr2=0x5602e4a2abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:42.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.387+0000 7f5b3ea00640 1 --2- 172.21.15.1:0/532630034 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602e4a16510 0x5602e4a2abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:42.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.387+0000 7f5b3ea00640 1 -- 172.21.15.1:0/532630034 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602e4a0b330 msgr2=0x5602e4a312d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:29:42.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.387+0000 7f5b3ea00640 1 --2- 172.21.15.1:0/532630034 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602e4a0b330 0x5602e4a312d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:42.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.387+0000 7f5b3ea00640 1 -- 172.21.15.1:0/532630034 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602e4a2b400 con 0x5602e4a0bc50 2024-09-12T01:29:42.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.387+0000 7f5b3ea00640 1 --2- 172.21.15.1:0/532630034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e4a0bc50 0x5602e4a2a650 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x5602e4a02ea0 tx=0x7f5b2c0bb9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec244230505db7e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:42.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.387+0000 7f5b29600640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-12T01:29:42.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.388+0000 7f5b29600640 1 -- 172.21.15.1:0/532630034 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c0d8070 con 0x5602e4a0bc50 2024-09-12T01:29:42.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.388+0000 7f5b29600640 1 -- 172.21.15.1:0/532630034 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f5b2c0c4520 con 0x5602e4a0bc50 2024-09-12T01:29:42.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.388+0000 7f5b48fd0740 1 -- 172.21.15.1:0/532630034 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b2c0b8250 con 0x5602e4a0bc50 2024-09-12T01:29:42.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.388+0000 7f5b29600640 1 -- 172.21.15.1:0/532630034 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5b2c0bce60 con 0x5602e4a0bc50 2024-09-12T01:29:42.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.388+0000 7f5b48fd0740 10 client.25205 fetch_fsmap learned FSMap version 28 2024-09-12T01:29:42.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.388+0000 7f5b48fd0740 1 -- 172.21.15.1:0/532630034 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5602e4a91c10 con 0x5602e4a0bc50 2024-09-12T01:29:42.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.388+0000 7f5b29600640 1 -- 172.21.15.1:0/532630034 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b2c0b8250 con 0x5602e4a0bc50 2024-09-12T01:29:42.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.388+0000 7f5b29600640 1 -- 172.21.15.1:0/532630034 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f5b2c0b7930 con 0x5602e4a0bc50 2024-09-12T01:29:42.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.388+0000 7f5b48fd0740 10 client.25205 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:29:42.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.389+0000 7f5b48fd0740 10 client.25205 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T01:29:42.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.389+0000 7f5b48fd0740 20 client.25205 populate_metadata read hostname 'smithi001' 2024-09-12T01:29:42.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.389+0000 7f5b48fd0740 1 --2- 172.21.15.1:0/532630034 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5602e4a96230 0x5602e4a94f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:42.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.389+0000 7f5b48fd0740 4 client.25205 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T01:29:42.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.389+0000 7f5b48fd0740 1 -- 172.21.15.1:0/532630034 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "status"}) -- 0x5602e4978b30 con 0x5602e4a96230 2024-09-12T01:29:42.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.389+0000 7f5b3f400640 1 --2- 172.21.15.1:0/532630034 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5602e4a96230 0x5602e4a94f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:42.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.389+0000 7f5b3f400640 1 --2- 172.21.15.1:0/532630034 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5602e4a96230 0x5602e4a94f40 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70d804cba8e74151 server_cookie=1a2bd55cfedaa061 in_seq=0 out_seq=0 2024-09-12T01:29:42.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.389+0000 7f5b29600640 10 client.25205 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:29:42.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.390+0000 7f5b29600640 1 -- 172.21.15.1:0/532630034 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5602e4978b30 con 0x5602e4a96230 2024-09-12T01:29:42.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.390+0000 7f5b29600640 10 client.25205 handle_command_reply: tid=0 2024-09-12T01:29:42.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.390+0000 7f5b48fd0740 1 client.25205 shutdown 2024-09-12T01:29:42.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.390+0000 7f5b48fd0740 1 -- 172.21.15.1:0/532630034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e4a0bc50 msgr2=0x5602e4a2a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:42.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.390+0000 7f5b48fd0740 1 --2- 172.21.15.1:0/532630034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e4a0bc50 0x5602e4a2a650 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x5602e4a02ea0 tx=0x7f5b2c0bb9b0 comp rx=0 tx=0).stop 2024-09-12T01:29:42.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.390+0000 7f5b48fd0740 1 -- 172.21.15.1:0/532630034 shutdown_connections 2024-09-12T01:29:42.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.390+0000 7f5b48fd0740 1 --2- 172.21.15.1:0/532630034 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5602e4a96230 0x5602e4a94f40 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:42.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.390+0000 7f5b48fd0740 1 --2- 172.21.15.1:0/532630034 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602e4a16510 0x5602e4a2abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:42.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.390+0000 7f5b48fd0740 1 --2- 172.21.15.1:0/532630034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602e4a0bc50 0x5602e4a2a650 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:42.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.390+0000 7f5b48fd0740 1 --2- 172.21.15.1:0/532630034 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602e4a0b330 0x5602e4a312d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:42.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.390+0000 7f5b29600640 0 client.25205 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:29:42.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.390+0000 7f5b48fd0740 1 -- 172.21.15.1:0/532630034 >> 172.21.15.1:0/532630034 conn(0x5602e4a078c0 msgr2=0x5602e4a0e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:42.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.390+0000 7f5b48fd0740 1 -- 172.21.15.1:0/532630034 shutdown_connections 2024-09-12T01:29:42.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.390+0000 7f5b48fd0740 1 -- 172.21.15.1:0/532630034 wait complete. 2024-09-12T01:29:42.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.390+0000 7f5b48fd0740 20 client.25205 trim_cache size 0 max 16384 2024-09-12T01:29:42.393 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-12T01:29:42.393 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "c0e4f180-70a4-11ef-bcea-c7b262605968", 2024-09-12T01:29:42.393 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 0, 2024-09-12T01:29:42.394 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24545, 2024-09-12T01:29:42.394 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-09-12T01:29:42.394 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-09-12T01:29:42.394 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-09-12T01:29:42.394 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 172.91910596299999, 2024-09-12T01:29:42.394 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 28, 2024-09-12T01:29:42.394 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 98, 2024-09-12T01:29:42.394 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 96, 2024-09-12T01:29:42.394 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 187.52367643299999 2024-09-12T01:29:42.394 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-12T01:29:42.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.391+0000 7f5b46400640 1 -- 172.21.15.1:0/3998162981 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f5b240a2c00 msgr2=0x7f5b240a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:42.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.391+0000 7f5b46400640 1 --2- 172.21.15.1:0/3998162981 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f5b240a2c00 0x7f5b240a50c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x5602e4a08720 tx=0x7f5b30005f90 comp rx=0 tx=0).stop 2024-09-12T01:29:42.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.391+0000 7f5b46400640 1 -- 172.21.15.1:0/3998162981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b40077040 msgr2=0x7f5b401a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:42.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.391+0000 7f5b46400640 1 --2- 172.21.15.1:0/3998162981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b40077040 0x7f5b401a4c10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c004770 tx=0x7f5b2c003c10 comp rx=0 tx=0).stop 2024-09-12T01:29:42.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.391+0000 7f5b46400640 1 -- 172.21.15.1:0/3998162981 shutdown_connections 2024-09-12T01:29:42.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.391+0000 7f5b46400640 1 --2- 172.21.15.1:0/3998162981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b40112a30 0x7f5b4019e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:42.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.391+0000 7f5b46400640 1 --2- 172.21.15.1:0/3998162981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b40077040 0x7f5b401a4c10 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:42.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.391+0000 7f5b46400640 1 --2- 172.21.15.1:0/3998162981 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f5b240a2c00 0x7f5b240a50c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:42.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.391+0000 7f5b46400640 1 --2- 172.21.15.1:0/3998162981 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b40076720 0x7f5b401a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:42.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.391+0000 7f5b46400640 1 -- 172.21.15.1:0/3998162981 >> 172.21.15.1:0/3998162981 conn(0x7f5b400ffa90 msgr2=0x7f5b40101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:42.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.391+0000 7f5b46400640 1 -- 172.21.15.1:0/3998162981 shutdown_connections 2024-09-12T01:29:42.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:42.392+0000 7f5b46400640 1 -- 172.21.15.1:0/3998162981 wait complete. 2024-09-12T01:29:43.304 INFO:teuthology.run_tasks:Running task sequential... 2024-09-12T01:29:43.317 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-12T01:29:43.317 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-12T01:29:43.318 INFO:teuthology.task.print:Enabling mgr modules 2024-09-12T01:29:43.318 INFO:teuthology.run_tasks:Running task kclient... 2024-09-12T01:29:43.331 INFO:tasks.kclient:Mounting kernel clients... 2024-09-12T01:29:43.331 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-12T01:29:43.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:29:43.332 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-09-12T01:29:43.368 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-12T01:29:43.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:29:43.369 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-12T01:29:43.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:29:43.466 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link delete ceph-brx 2024-09-12T01:29:43.802 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-09-12T01:29:43.803 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-12T01:29:43.803 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T01:29:43.803 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-12T01:29:44.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:43 smithi104 ceph-mon[35219]: pgmap v402: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T01:29:44.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:43 smithi001 ceph-mon[30387]: pgmap v402: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T01:29:44.185 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:29:44.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:43 smithi053 ceph-mon[35158]: pgmap v402: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T01:29:44.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:44.769+0000 7f5689000640 1 -- 172.21.15.1:0/4031318513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5684109120 msgr2=0x7f5684110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:44.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:44.769+0000 7f5689000640 1 --2- 172.21.15.1:0/4031318513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5684109120 0x7f5684110ca0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f567c009ca0 tx=0x7f567c02a8c0 comp rx=0 tx=0).stop 2024-09-12T01:29:44.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:44.770+0000 7f5689000640 1 -- 172.21.15.1:0/4031318513 shutdown_connections 2024-09-12T01:29:44.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:44.770+0000 7f5689000640 1 --2- 172.21.15.1:0/4031318513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5684109120 0x7f5684110ca0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:44.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:44.770+0000 7f5689000640 1 --2- 172.21.15.1:0/4031318513 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56841049a0 0x7f56841089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:44.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:44.770+0000 7f5689000640 1 --2- 172.21.15.1:0/4031318513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5684103ff0 0x7f56841043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:44.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:44.770+0000 7f5689000640 1 -- 172.21.15.1:0/4031318513 >> 172.21.15.1:0/4031318513 conn(0x7f56840ffa90 msgr2=0x7f5684101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:44.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:44.770+0000 7f5689000640 1 -- 172.21.15.1:0/4031318513 shutdown_connections 2024-09-12T01:29:44.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:44.770+0000 7f5689000640 1 -- 172.21.15.1:0/4031318513 wait complete. 2024-09-12T01:29:44.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:44.771+0000 7f5689000640 1 Processor -- start 2024-09-12T01:29:44.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:44.771+0000 7f5689000640 1 -- start start 2024-09-12T01:29:44.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:44.771+0000 7f5689000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5684103ff0 0x7f5684076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:44.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:44.771+0000 7f5689000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f56841049a0 0x7f5684076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:44.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:44.771+0000 7f5689000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5684109120 0x7f5684077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:44.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:44.771+0000 7f5689000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5684114180 con 0x7f56841049a0 2024-09-12T01:29:44.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:44.771+0000 7f5689000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5684114000 con 0x7f5684109120 2024-09-12T01:29:44.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:44.771+0000 7f5689000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5684114300 con 0x7f5684103ff0 2024-09-12T01:29:44.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:44.772+0000 7f567be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f56841049a0 0x7f5684076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:44.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:44.772+0000 7f5681e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5684103ff0 0x7f5684076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:44.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:44.772+0000 7f567be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f56841049a0 0x7f5684076c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46776/0 (socket says 172.21.15.1:46776) 2024-09-12T01:29:44.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:44.772+0000 7f567be00640 1 -- 172.21.15.1:0/3906065274 learned_addr learned my addr 172.21.15.1:0/3906065274 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:44.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:44.772+0000 7f5682800640 1 --2- 172.21.15.1:0/3906065274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5684109120 0x7f5684077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:44.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:44.772+0000 7f5681e00640 1 -- 172.21.15.1:0/3906065274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5684109120 msgr2=0x7f5684077180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:44.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:44.772+0000 7f5681e00640 1 --2- 172.21.15.1:0/3906065274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5684109120 0x7f5684077180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:44.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:44.772+0000 7f5681e00640 1 -- 172.21.15.1:0/3906065274 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f56841049a0 msgr2=0x7f5684076c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:44.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:44.772+0000 7f5681e00640 1 --2- 172.21.15.1:0/3906065274 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f56841049a0 0x7f5684076c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:44.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:44.772+0000 7f5681e00640 1 -- 172.21.15.1:0/3906065274 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56841aac60 con 0x7f5684103ff0 2024-09-12T01:29:44.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:44.772+0000 7f567be00640 1 --2- 172.21.15.1:0/3906065274 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f56841049a0 0x7f5684076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:29:44.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:44.773+0000 7f5681e00640 1 --2- 172.21.15.1:0/3906065274 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5684103ff0 0x7f5684076700 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f567400ea20 tx=0x7f567400eee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:44.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:44.773+0000 7f567aa00640 1 -- 172.21.15.1:0/3906065274 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567400cd50 con 0x7f5684103ff0 2024-09-12T01:29:44.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:44.773+0000 7f567aa00640 1 -- 172.21.15.1:0/3906065274 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56740035f0 con 0x7f5684103ff0 2024-09-12T01:29:44.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:44.773+0000 7f5689000640 1 -- 172.21.15.1:0/3906065274 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56841aaf50 con 0x7f5684103ff0 2024-09-12T01:29:44.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:44.773+0000 7f567aa00640 1 -- 172.21.15.1:0/3906065274 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5674010690 con 0x7f5684103ff0 2024-09-12T01:29:44.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:44.773+0000 7f5689000640 1 -- 172.21.15.1:0/3906065274 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56841ab450 con 0x7f5684103ff0 2024-09-12T01:29:44.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:44.774+0000 7f5689000640 1 -- 172.21.15.1:0/3906065274 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56841054d0 con 0x7f5684103ff0 2024-09-12T01:29:44.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:44.776+0000 7f567aa00640 1 -- 172.21.15.1:0/3906065274 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f56740037a0 con 0x7f5684103ff0 2024-09-12T01:29:44.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:44.777+0000 7f567aa00640 1 --2- 172.21.15.1:0/3906065274 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f56680a2c00 0x7f56680a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:44.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:44.778+0000 7f567aa00640 1 -- 172.21.15.1:0/3906065274 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f56740cf5c0 con 0x7f5684103ff0 2024-09-12T01:29:44.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:44.778+0000 7f567be00640 1 --2- 172.21.15.1:0/3906065274 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f56680a2c00 0x7f56680a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:44.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:44.778+0000 7f567be00640 1 --2- 172.21.15.1:0/3906065274 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f56680a2c00 0x7f56680a50c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5684077ec0 tx=0x7f566c00b480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:44.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:44.779+0000 7f567aa00640 1 -- 172.21.15.1:0/3906065274 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f567409c130 con 0x7f5684103ff0 2024-09-12T01:29:45.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:45.006+0000 7f5689000640 1 -- 172.21.15.1:0/3906065274 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f56841043d0 con 0x7f5684103ff0 2024-09-12T01:29:45.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:45.006+0000 7f567aa00640 1 -- 172.21.15.1:0/3906065274 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) ==== 53+0+88 (secure 0 0 0) 0x7f567409ba30 con 0x7f5684103ff0 2024-09-12T01:29:45.006 INFO:teuthology.orchestra.run.smithi001.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-12T01:29:45.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:45.008+0000 7f5689000640 1 -- 172.21.15.1:0/3906065274 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f56680a2c00 msgr2=0x7f56680a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:45.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:45.008+0000 7f5689000640 1 --2- 172.21.15.1:0/3906065274 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f56680a2c00 0x7f56680a50c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5684077ec0 tx=0x7f566c00b480 comp rx=0 tx=0).stop 2024-09-12T01:29:45.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:45.009+0000 7f5689000640 1 -- 172.21.15.1:0/3906065274 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5684103ff0 msgr2=0x7f5684076700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:45.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:45.009+0000 7f5689000640 1 --2- 172.21.15.1:0/3906065274 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5684103ff0 0x7f5684076700 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f567400ea20 tx=0x7f567400eee0 comp rx=0 tx=0).stop 2024-09-12T01:29:45.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:45.009+0000 7f5689000640 1 -- 172.21.15.1:0/3906065274 shutdown_connections 2024-09-12T01:29:45.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:45.009+0000 7f5689000640 1 --2- 172.21.15.1:0/3906065274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5684109120 0x7f5684077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:45.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:45.009+0000 7f5689000640 1 --2- 172.21.15.1:0/3906065274 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f56841049a0 0x7f5684076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:45.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:45.009+0000 7f5689000640 1 --2- 172.21.15.1:0/3906065274 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f56680a2c00 0x7f56680a50c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:45.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:45.009+0000 7f5689000640 1 --2- 172.21.15.1:0/3906065274 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5684103ff0 0x7f5684076700 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:45.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:45.009+0000 7f5689000640 1 -- 172.21.15.1:0/3906065274 >> 172.21.15.1:0/3906065274 conn(0x7f56840ffa90 msgr2=0x7f56841004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:45.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:45.009+0000 7f5689000640 1 -- 172.21.15.1:0/3906065274 shutdown_connections 2024-09-12T01:29:45.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:45.009+0000 7f5689000640 1 -- 172.21.15.1:0/3906065274 wait complete. 2024-09-12T01:29:45.914 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T01:29:45.914 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T01:29:45.914 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi001.front.sepia.ceph.com 2024-09-12T01:29:45.914 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-12T01:29:45.914 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T01:29:45.914 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T01:29:45.914 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T01:29:45.914 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T01:29:45.914 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T01:29:45.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:29:45.915 DEBUG:teuthology.orchestra.run.smithi001:> ip addr 2024-09-12T01:29:45.951 INFO:teuthology.orchestra.run.smithi001.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T01:29:45.951 INFO:teuthology.orchestra.run.smithi001.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T01:29:45.952 INFO:teuthology.orchestra.run.smithi001.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T01:29:45.952 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:29:45.952 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 ::1/128 scope host 2024-09-12T01:29:45.952 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:29:45.952 INFO:teuthology.orchestra.run.smithi001.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T01:29:45.952 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7a brd ff:ff:ff:ff:ff:ff 2024-09-12T01:29:45.952 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f0 2024-09-12T01:29:45.952 INFO:teuthology.orchestra.run.smithi001.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T01:29:45.952 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7b brd ff:ff:ff:ff:ff:ff 2024-09-12T01:29:45.952 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f1 2024-09-12T01:29:45.952 INFO:teuthology.orchestra.run.smithi001.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T01:29:45.953 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e8 brd ff:ff:ff:ff:ff:ff 2024-09-12T01:29:45.953 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f0 2024-09-12T01:29:45.953 INFO:teuthology.orchestra.run.smithi001.stdout: inet 172.21.15.1/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T01:29:45.953 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft 41994sec preferred_lft 41994sec 2024-09-12T01:29:45.953 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::ec4:7aff:febd:15e8/64 scope link 2024-09-12T01:29:45.953 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:29:45.953 INFO:teuthology.orchestra.run.smithi001.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T01:29:45.953 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e9 brd ff:ff:ff:ff:ff:ff 2024-09-12T01:29:45.953 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f1 2024-09-12T01:29:45.953 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-12T01:29:45.954 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T01:29:45.954 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-09-12T01:29:45.954 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link add name ceph-brx type bridge 2024-09-12T01:29:45.954 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip addr flush dev ceph-brx 2024-09-12T01:29:45.954 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set ceph-brx up 2024-09-12T01:29:45.954 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-12T01:29:45.954 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-09-12T01:29:46.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T01:29:46.075 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:45 smithi001 ceph-mon[30387]: pgmap v403: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T01:29:46.075 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:45 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/3906065274' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T01:29:46.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:45 smithi104 ceph-mon[35219]: pgmap v403: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T01:29:46.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:45 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/3906065274' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T01:29:46.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:29:46.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:29:46.181 DEBUG:teuthology.orchestra.run.smithi001:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T01:29:46.235 INFO:teuthology.orchestra.run.smithi001.stdout:1 2024-09-12T01:29:46.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:29:46.238 DEBUG:teuthology.orchestra.run.smithi001:> route 2024-09-12T01:29:46.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:45 smithi053 ceph-mon[35158]: pgmap v403: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T01:29:46.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:45 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/3906065274' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T01:29:46.275 INFO:teuthology.orchestra.run.smithi001.stdout:Kernel IP routing table 2024-09-12T01:29:46.275 INFO:teuthology.orchestra.run.smithi001.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T01:29:46.275 INFO:teuthology.orchestra.run.smithi001.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T01:29:46.275 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T01:29:46.275 INFO:teuthology.orchestra.run.smithi001.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T01:29:46.276 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T01:29:46.276 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-09-12T01:29:46.276 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T01:29:46.276 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T01:29:46.276 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T01:29:46.276 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-09-12T01:29:46.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T01:29:46.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:29:46.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:29:46.476 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-09-12T01:29:46.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:29:46.512 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list-id 2024-09-12T01:29:46.588 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T01:29:46.588 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-09-12T01:29:46.588 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T01:29:46.589 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-12T01:29:46.589 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-09-12T01:29:46.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T01:29:46.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:29:46.765 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-12T01:29:46.766 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T01:29:46.766 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-09-12T01:29:46.766 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-12T01:29:46.766 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-12T01:29:46.766 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-12T01:29:46.766 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-12T01:29:46.766 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-12T01:29:46.766 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-09-12T01:29:46.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T01:29:46.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:29:46.988 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T01:29:46.988 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-09-12T01:29:46.988 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set brx.0 up 2024-09-12T01:29:46.988 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set dev brx.0 master ceph-brx 2024-09-12T01:29:46.989 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-09-12T01:29:47.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T01:29:47.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:29:47.156 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T01:29:47.156 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-12T01:29:47.191 INFO:teuthology.orchestra.run.smithi001.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-12T01:29:47.191 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T01:29:47.192 DEBUG:teuthology.orchestra.run.smithi001:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-12T01:29:47.266 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-12T01:29:47.266 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-12T01:29:47.266 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/68dfb7ec-f69e-4d16-ba25-71857c337627 2024-09-12T01:29:47.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:29:47.266 DEBUG:teuthology.orchestra.run.smithi001:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/68dfb7ec-f69e-4d16-ba25-71857c337627 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-09-12T01:29:47.396 INFO:teuthology.orchestra.run.smithi001.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-09-12T01:29:47.396 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-09-12T01:29:47.396 INFO:teuthology.orchestra.run.smithi001.stdout:invalid new device string format 2024-09-12T01:29:48.024 INFO:teuthology.orchestra.run.smithi001.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-09-12T01:29:48.024 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-09-12T01:29:48.024 INFO:teuthology.orchestra.run.smithi001.stdout:invalid new device string format 2024-09-12T01:29:48.024 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: resolved to: "172.21.15.1:6789,172.21.15.53:6789,172.21.15.104:6789" 2024-09-12T01:29:48.024 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.1:6789,172.21.15.53:6789,172.21.15.104:6789:/volumes/qa/sv_1/68dfb7ec-f69e-4d16-ba25-71857c337627 2024-09-12T01:29:48.024 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync,key=0,fsid=c0e4f180-70a4-11ef-bcea-c7b262605968" will pass to kernel 2024-09-12T01:29:48.026 INFO:teuthology.orchestra.run.smithi001.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-12T01:29:48.026 INFO:teuthology.orchestra.run.smithi001.stdout: You just mounted a file system that supports labels which does not 2024-09-12T01:29:48.026 INFO:teuthology.orchestra.run.smithi001.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-12T01:29:48.026 INFO:teuthology.orchestra.run.smithi001.stdout: applications will generate AVC messages and not be allowed access to 2024-09-12T01:29:48.026 INFO:teuthology.orchestra.run.smithi001.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-12T01:29:48.027 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:47 smithi001 ceph-mon[30387]: pgmap v404: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-12T01:29:48.029 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-12T01:29:48.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:29:48.030 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-12T01:29:48.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:47 smithi104 ceph-mon[35219]: pgmap v404: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-12T01:29:48.092 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T01:29:48.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T01:29:48.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:29:48.185 INFO:teuthology.orchestra.run.smithi001.stdout:client25214 2024-09-12T01:29:48.186 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-12T01:29:48.186 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T01:29:48.222 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T01:29:48.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:47 smithi053 ceph-mon[35158]: pgmap v404: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-12T01:29:48.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T01:29:48.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:29:48.352 INFO:teuthology.orchestra.run.smithi001.stdout:client25214 2024-09-12T01:29:48.353 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T01:29:48.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T01:29:48.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:29:48.444 INFO:teuthology.orchestra.run.smithi001.stdout:client25214 2024-09-12T01:29:48.444 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968.client25214/status') 2024-09-12T01:29:48.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T01:29:48.546 INFO:teuthology.orchestra.run.smithi001.stdout:instance: client.25214 (3)172.21.15.1:0/1839769767 2024-09-12T01:29:48.546 INFO:teuthology.orchestra.run.smithi001.stdout:blocklisted: false 2024-09-12T01:29:48.546 INFO:teuthology.orchestra.run.smithi001.stderr:0+1 records in 2024-09-12T01:29:48.546 INFO:teuthology.orchestra.run.smithi001.stderr:0+1 records out 2024-09-12T01:29:48.546 INFO:teuthology.orchestra.run.smithi001.stderr:70 bytes copied, 5.6047e-05 s, 1.2 MB/s 2024-09-12T01:29:48.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:29:48.556 INFO:teuthology.run_tasks:Running task exec... 2024-09-12T01:29:48.567 INFO:teuthology.task.exec:Executing custom commands... 2024-09-12T01:29:48.567 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi001.front.sepia.ceph.com 2024-09-12T01:29:48.567 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-09-12T01:29:48.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48.724+0000 7f04eae00640 1 -- 172.21.15.1:0/1575086087 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04e4103d50 msgr2=0x7f04e4104150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:48.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48.724+0000 7f04eae00640 1 --2- 172.21.15.1:0/1575086087 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04e4103d50 0x7f04e4104150 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f04d4009900 tx=0x7f04d402a3f0 comp rx=0 tx=0).stop 2024-09-12T01:29:48.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48.724+0000 7f04eae00640 1 -- 172.21.15.1:0/1575086087 shutdown_connections 2024-09-12T01:29:48.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48.724+0000 7f04eae00640 1 --2- 172.21.15.1:0/1575086087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04e4105900 0x7f04e410c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:48.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48.724+0000 7f04eae00640 1 --2- 172.21.15.1:0/1575086087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04e4104f40 0x7f04e41053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:48.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48.724+0000 7f04eae00640 1 --2- 172.21.15.1:0/1575086087 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04e4103d50 0x7f04e4104150 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:48.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48.724+0000 7f04eae00640 1 -- 172.21.15.1:0/1575086087 >> 172.21.15.1:0/1575086087 conn(0x7f04e40ff4b0 msgr2=0x7f04e41018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:48.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48.724+0000 7f04eae00640 1 -- 172.21.15.1:0/1575086087 shutdown_connections 2024-09-12T01:29:48.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48.724+0000 7f04eae00640 1 -- 172.21.15.1:0/1575086087 wait complete. 2024-09-12T01:29:48.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48.725+0000 7f04eae00640 1 Processor -- start 2024-09-12T01:29:48.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48.725+0000 7f04eae00640 1 -- start start 2024-09-12T01:29:48.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48.725+0000 7f04eae00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04e4103d50 0x7f04e419a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:48.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48.725+0000 7f04eae00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04e4104f40 0x7f04e419a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:48.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48.726+0000 7f04eae00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04e4105900 0x7f04e41a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:48.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48.726+0000 7f04eae00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f04e410f9b0 con 0x7f04e4104f40 2024-09-12T01:29:48.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48.726+0000 7f04eae00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f04e410f830 con 0x7f04e4103d50 2024-09-12T01:29:48.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48.726+0000 7f04eae00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f04e410fb30 con 0x7f04e4105900 2024-09-12T01:29:48.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48.726+0000 7f04db400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04e4104f40 0x7f04e419a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:48.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48.726+0000 7f04e3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04e4103d50 0x7f04e419a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:48.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48.726+0000 7f04e3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04e4103d50 0x7f04e419a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:51280/0 (socket says 172.21.15.1:51280) 2024-09-12T01:29:48.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48.726+0000 7f04db400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04e4104f40 0x7f04e419a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46784/0 (socket says 172.21.15.1:46784) 2024-09-12T01:29:48.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48.726+0000 7f04e3e00640 1 -- 172.21.15.1:0/4168030815 learned_addr learned my addr 172.21.15.1:0/4168030815 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:48.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48.726+0000 7f04e8e00640 1 --2- 172.21.15.1:0/4168030815 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04e4105900 0x7f04e41a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:48.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48.726+0000 7f04db400640 1 -- 172.21.15.1:0/4168030815 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04e4105900 msgr2=0x7f04e41a1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:48.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48.726+0000 7f04db400640 1 --2- 172.21.15.1:0/4168030815 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04e4105900 0x7f04e41a1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:48.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48.726+0000 7f04db400640 1 -- 172.21.15.1:0/4168030815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04e4103d50 msgr2=0x7f04e419a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:48.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48.726+0000 7f04db400640 1 --2- 172.21.15.1:0/4168030815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04e4103d50 0x7f04e419a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:48.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48.726+0000 7f04db400640 1 -- 172.21.15.1:0/4168030815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e41a1cb0 con 0x7f04e4104f40 2024-09-12T01:29:48.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48.726+0000 7f04e3e00640 1 --2- 172.21.15.1:0/4168030815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04e4103d50 0x7f04e419a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:29:48.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48.727+0000 7f04db400640 1 --2- 172.21.15.1:0/4168030815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04e4104f40 0x7f04e419a5d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f04cc00e9f0 tx=0x7f04cc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:48.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48.727+0000 7f04e1600640 1 -- 172.21.15.1:0/4168030815 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc00ccc0 con 0x7f04e4104f40 2024-09-12T01:29:48.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48.727+0000 7f04e1600640 1 -- 172.21.15.1:0/4168030815 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04cc010910 con 0x7f04e4104f40 2024-09-12T01:29:48.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48.727+0000 7f04e1600640 1 -- 172.21.15.1:0/4168030815 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc018800 con 0x7f04e4104f40 2024-09-12T01:29:48.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48.727+0000 7f04eae00640 1 -- 172.21.15.1:0/4168030815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e41a1fa0 con 0x7f04e4104f40 2024-09-12T01:29:48.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48.727+0000 7f04eae00640 1 -- 172.21.15.1:0/4168030815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e41a2410 con 0x7f04e4104f40 2024-09-12T01:29:48.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48.728+0000 7f04eae00640 1 -- 172.21.15.1:0/4168030815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04e410b1e0 con 0x7f04e4104f40 2024-09-12T01:29:48.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48.730+0000 7f04e1600640 1 -- 172.21.15.1:0/4168030815 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f04cc010ac0 con 0x7f04e4104f40 2024-09-12T01:29:48.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48.731+0000 7f04e1600640 1 --2- 172.21.15.1:0/4168030815 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f04c80a2c00 0x7f04c80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:48.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48.731+0000 7f04e1600640 1 -- 172.21.15.1:0/4168030815 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f04cc0cf680 con 0x7f04e4104f40 2024-09-12T01:29:48.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48.732+0000 7f04e3e00640 1 --2- 172.21.15.1:0/4168030815 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f04c80a2c00 0x7f04c80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:48.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48.732+0000 7f04e3e00640 1 --2- 172.21.15.1:0/4168030815 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f04c80a2c00 0x7f04c80a50c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f04d4000c00 tx=0x7f04d40046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:48.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48.735+0000 7f04e1600640 1 -- 172.21.15.1:0/4168030815 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f04cc09c1f0 con 0x7f04e4104f40 2024-09-12T01:29:48.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:48.962+0000 7f04eae00640 1 -- 172.21.15.1:0/4168030815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f04e4062e90 con 0x7f04e4104f40 2024-09-12T01:29:49.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.810+0000 7f04e1600640 1 -- 172.21.15.1:0/4168030815 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v29) ==== 118+0+0 (secure 0 0 0) 0x7f04cc09baf0 con 0x7f04e4104f40 2024-09-12T01:29:49.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.813+0000 7f04eae00640 1 -- 172.21.15.1:0/4168030815 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f04c80a2c00 msgr2=0x7f04c80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:49.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.813+0000 7f04eae00640 1 --2- 172.21.15.1:0/4168030815 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f04c80a2c00 0x7f04c80a50c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f04d4000c00 tx=0x7f04d40046c0 comp rx=0 tx=0).stop 2024-09-12T01:29:49.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.813+0000 7f04eae00640 1 -- 172.21.15.1:0/4168030815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04e4104f40 msgr2=0x7f04e419a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:49.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.813+0000 7f04eae00640 1 --2- 172.21.15.1:0/4168030815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04e4104f40 0x7f04e419a5d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f04cc00e9f0 tx=0x7f04cc00edf0 comp rx=0 tx=0).stop 2024-09-12T01:29:49.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.813+0000 7f04eae00640 1 -- 172.21.15.1:0/4168030815 shutdown_connections 2024-09-12T01:29:49.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.813+0000 7f04eae00640 1 --2- 172.21.15.1:0/4168030815 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f04c80a2c00 0x7f04c80a50c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:49.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.813+0000 7f04eae00640 1 --2- 172.21.15.1:0/4168030815 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04e4105900 0x7f04e41a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:49.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.813+0000 7f04eae00640 1 --2- 172.21.15.1:0/4168030815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04e4104f40 0x7f04e419a5d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:49.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.813+0000 7f04eae00640 1 --2- 172.21.15.1:0/4168030815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04e4103d50 0x7f04e419a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:49.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.813+0000 7f04eae00640 1 -- 172.21.15.1:0/4168030815 >> 172.21.15.1:0/4168030815 conn(0x7f04e40ff4b0 msgr2=0x7f04e41083b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:49.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.813+0000 7f04eae00640 1 -- 172.21.15.1:0/4168030815 shutdown_connections 2024-09-12T01:29:49.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.813+0000 7f04eae00640 1 -- 172.21.15.1:0/4168030815 wait complete. 2024-09-12T01:29:49.832 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-09-12T01:29:49.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.995+0000 7f16dc600640 1 -- 172.21.15.1:0/266526392 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16d4104ee0 msgr2=0x7f16d41072d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:49.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.995+0000 7f16dc600640 1 --2- 172.21.15.1:0/266526392 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16d4104ee0 0x7f16d41072d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f16c0009900 tx=0x7f16c002a410 comp rx=0 tx=0).stop 2024-09-12T01:29:49.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.995+0000 7f16dc600640 1 -- 172.21.15.1:0/266526392 shutdown_connections 2024-09-12T01:29:49.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.995+0000 7f16dc600640 1 --2- 172.21.15.1:0/266526392 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16d4107810 0x7f16d4109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:49.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.995+0000 7f16dc600640 1 --2- 172.21.15.1:0/266526392 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16d4104ee0 0x7f16d41072d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:49.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.995+0000 7f16dc600640 1 --2- 172.21.15.1:0/266526392 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16d41025b0 0x7f16d41049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:49.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.995+0000 7f16dc600640 1 -- 172.21.15.1:0/266526392 >> 172.21.15.1:0/266526392 conn(0x7f16d40fc190 msgr2=0x7f16d40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:49.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.995+0000 7f16dc600640 1 -- 172.21.15.1:0/266526392 shutdown_connections 2024-09-12T01:29:49.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.995+0000 7f16dc600640 1 -- 172.21.15.1:0/266526392 wait complete. 2024-09-12T01:29:49.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.996+0000 7f16dc600640 1 Processor -- start 2024-09-12T01:29:49.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.996+0000 7f16dc600640 1 -- start start 2024-09-12T01:29:49.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.996+0000 7f16dc600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16d41025b0 0x7f16d419a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:49.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.996+0000 7f16dc600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16d4104ee0 0x7f16d419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:49.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.996+0000 7f16dc600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16d4107810 0x7f16d41a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:49.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.996+0000 7f16dc600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f16d410d7f0 con 0x7f16d41025b0 2024-09-12T01:29:49.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.996+0000 7f16dc600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f16d410d670 con 0x7f16d4107810 2024-09-12T01:29:49.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.996+0000 7f16dc600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f16d410d970 con 0x7f16d4104ee0 2024-09-12T01:29:49.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.996+0000 7f16d9c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16d41025b0 0x7f16d419a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:49.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.997+0000 7f16d9c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16d41025b0 0x7f16d419a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50866/0 (socket says 172.21.15.1:50866) 2024-09-12T01:29:49.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.997+0000 7f16d9c00640 1 -- 172.21.15.1:0/1026004427 learned_addr learned my addr 172.21.15.1:0/1026004427 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:29:49.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.997+0000 7f16d9200640 1 --2- 172.21.15.1:0/1026004427 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16d4104ee0 0x7f16d419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:49.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.997+0000 7f16da600640 1 --2- 172.21.15.1:0/1026004427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16d4107810 0x7f16d41a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:49.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.997+0000 7f16d9c00640 1 -- 172.21.15.1:0/1026004427 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16d4104ee0 msgr2=0x7f16d419a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:49.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.997+0000 7f16d9c00640 1 --2- 172.21.15.1:0/1026004427 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16d4104ee0 0x7f16d419a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:49.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.997+0000 7f16d9c00640 1 -- 172.21.15.1:0/1026004427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16d4107810 msgr2=0x7f16d41a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:49.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.997+0000 7f16d9c00640 1 --2- 172.21.15.1:0/1026004427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16d4107810 0x7f16d41a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:49.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.997+0000 7f16d9c00640 1 -- 172.21.15.1:0/1026004427 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16d41a1d50 con 0x7f16d41025b0 2024-09-12T01:29:49.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.997+0000 7f16da600640 1 --2- 172.21.15.1:0/1026004427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16d4107810 0x7f16d41a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:29:49.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.997+0000 7f16d9c00640 1 --2- 172.21.15.1:0/1026004427 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16d41025b0 0x7f16d419a130 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f16c800e9f0 tx=0x7f16c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:49.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.997+0000 7f16d2000640 1 -- 172.21.15.1:0/1026004427 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c800cd80 con 0x7f16d41025b0 2024-09-12T01:29:49.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.998+0000 7f16d2000640 1 -- 172.21.15.1:0/1026004427 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16c8010910 con 0x7f16d41025b0 2024-09-12T01:29:49.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.998+0000 7f16dc600640 1 -- 172.21.15.1:0/1026004427 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16d41a1fe0 con 0x7f16d41025b0 2024-09-12T01:29:49.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.998+0000 7f16d2000640 1 -- 172.21.15.1:0/1026004427 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c8018990 con 0x7f16d41025b0 2024-09-12T01:29:49.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.998+0000 7f16dc600640 1 -- 172.21.15.1:0/1026004427 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16d41a24a0 con 0x7f16d41025b0 2024-09-12T01:29:49.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:49.999+0000 7f16dc600640 1 -- 172.21.15.1:0/1026004427 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16d4109380 con 0x7f16d41025b0 2024-09-12T01:29:50.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:50.001+0000 7f16d2000640 1 -- 172.21.15.1:0/1026004427 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f16c8010ac0 con 0x7f16d41025b0 2024-09-12T01:29:50.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:50.002+0000 7f16d2000640 1 --2- 172.21.15.1:0/1026004427 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f16bc0a2b80 0x7f16bc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:29:50.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:50.002+0000 7f16d2000640 1 -- 172.21.15.1:0/1026004427 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f16c80cf770 con 0x7f16d41025b0 2024-09-12T01:29:50.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:50.002+0000 7f16d9200640 1 --2- 172.21.15.1:0/1026004427 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f16bc0a2b80 0x7f16bc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:29:50.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:50.003+0000 7f16d9200640 1 --2- 172.21.15.1:0/1026004427 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f16bc0a2b80 0x7f16bc0a5040 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f16c0005ab0 tx=0x7f16c0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:29:50.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:50.006+0000 7f16d2000640 1 -- 172.21.15.1:0/1026004427 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f16c809c340 con 0x7f16d41025b0 2024-09-12T01:29:50.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:49 smithi104 ceph-mon[35219]: pgmap v405: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T01:29:50.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:49 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/4168030815' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-12T01:29:50.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:49 smithi001 ceph-mon[30387]: pgmap v405: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T01:29:50.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:49 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/4168030815' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-12T01:29:50.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:50.220+0000 7f16dc600640 1 -- 172.21.15.1:0/1026004427 --> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f16d4062790 con 0x7f16bc0a2b80 2024-09-12T01:29:50.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:49 smithi053 ceph-mon[35158]: pgmap v405: 129 pgs: 129 active+clean; 646 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T01:29:50.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:49 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/4168030815' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-12T01:29:50.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:50.240+0000 7f16d2000640 1 -- 172.21.15.1:0/1026004427 <== mgr.14150 v2:172.21.15.1:6800/1928662456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f16d4062790 con 0x7f16bc0a2b80 2024-09-12T01:29:50.240 INFO:teuthology.orchestra.run.smithi001.stdout:{} 2024-09-12T01:29:50.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:50.242+0000 7f16dc600640 1 -- 172.21.15.1:0/1026004427 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f16bc0a2b80 msgr2=0x7f16bc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:50.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:50.242+0000 7f16dc600640 1 --2- 172.21.15.1:0/1026004427 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f16bc0a2b80 0x7f16bc0a5040 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f16c0005ab0 tx=0x7f16c0000bc0 comp rx=0 tx=0).stop 2024-09-12T01:29:50.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:50.242+0000 7f16dc600640 1 -- 172.21.15.1:0/1026004427 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16d41025b0 msgr2=0x7f16d419a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:29:50.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:50.242+0000 7f16dc600640 1 --2- 172.21.15.1:0/1026004427 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16d41025b0 0x7f16d419a130 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f16c800e9f0 tx=0x7f16c800eeb0 comp rx=0 tx=0).stop 2024-09-12T01:29:50.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:50.243+0000 7f16dc600640 1 -- 172.21.15.1:0/1026004427 shutdown_connections 2024-09-12T01:29:50.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:50.243+0000 7f16dc600640 1 --2- 172.21.15.1:0/1026004427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16d4107810 0x7f16d41a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:50.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:50.243+0000 7f16dc600640 1 --2- 172.21.15.1:0/1026004427 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f16bc0a2b80 0x7f16bc0a5040 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:50.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:50.243+0000 7f16dc600640 1 --2- 172.21.15.1:0/1026004427 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16d4104ee0 0x7f16d419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:50.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:50.243+0000 7f16dc600640 1 --2- 172.21.15.1:0/1026004427 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16d41025b0 0x7f16d419a130 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:29:50.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:50.243+0000 7f16dc600640 1 -- 172.21.15.1:0/1026004427 >> 172.21.15.1:0/1026004427 conn(0x7f16d40fc190 msgr2=0x7f16d4101d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:29:50.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:50.243+0000 7f16dc600640 1 -- 172.21.15.1:0/1026004427 shutdown_connections 2024-09-12T01:29:50.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:29:50.243+0000 7f16dc600640 1 -- 172.21.15.1:0/1026004427 wait complete. 2024-09-12T01:29:50.258 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-12T01:29:50.269 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-12T01:29:50.291 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi001.front.sepia.ceph.com 2024-09-12T01:29:50.291 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-12T01:29:50.295 INFO:teuthology.run_tasks:Running task workunit... 2024-09-12T01:29:50.308 INFO:tasks.workunit:Pulling workunits from ref 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-12T01:29:50.308 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-12T01:29:50.308 INFO:tasks.workunit:timeout=3h 2024-09-12T01:29:50.308 INFO:tasks.workunit:cleanup=True 2024-09-12T01:29:50.309 DEBUG:teuthology.orchestra.run.smithi001:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-12T01:29:50.345 INFO:teuthology.orchestra.run.smithi001.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-12T01:29:50.345 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-12T01:29:50.345 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-12T01:29:50.345 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:29:50.345 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-12T01:29:50.345 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-09-12 01:27:54.654293289 +0000 2024-09-12T01:29:50.345 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-09-12 01:27:54.654293289 +0000 2024-09-12T01:29:50.346 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-09-12 01:29:48.083424985 +0000 2024-09-12T01:29:50.346 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: 2024-09-12 01:27:54.654293289 +0000 2024-09-12T01:29:50.346 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-12T01:29:50.346 DEBUG:teuthology.orchestra.run.smithi001:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-12T01:29:50.453 DEBUG:teuthology.orchestra.run.smithi001:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-12T01:29:50.491 INFO:tasks.workunit.client.0.smithi001.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-12T01:29:50.678 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:29:50 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:29:50.226+0000 7feafdc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T01:29:50.678 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:29:50 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:29:50.226+0000 7feafdc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T01:29:50.678 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:29:50 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:29:50.226+0000 7feafdc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T01:29:50.678 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:29:50 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:29:50.226+0000 7feafdc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T01:29:50.678 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:29:50 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:29:50.226+0000 7feafdc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T01:29:51.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:50 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/4168030815' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-12T01:29:51.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:50 smithi104 ceph-mon[35219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T01:29:51.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:50 smithi104 ceph-mon[35219]: from='client.15192 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:29:51.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:50 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/4168030815' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-12T01:29:51.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:50 smithi001 ceph-mon[30387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T01:29:51.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:50 smithi001 ceph-mon[30387]: from='client.15192 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:29:51.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:50 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/4168030815' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-12T01:29:51.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:50 smithi053 ceph-mon[35158]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T01:29:51.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:50 smithi053 ceph-mon[35158]: from='client.15192 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:29:52.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:51 smithi104 ceph-mon[35219]: pgmap v406: 129 pgs: 129 active+clean; 647 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-12T01:29:52.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:51 smithi104 ceph-mon[35219]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-09-12T01:29:52.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:51 smithi001 ceph-mon[30387]: pgmap v406: 129 pgs: 129 active+clean; 647 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-12T01:29:52.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:51 smithi001 ceph-mon[30387]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-09-12T01:29:52.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:51 smithi053 ceph-mon[35158]: pgmap v406: 129 pgs: 129 active+clean; 647 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-12T01:29:52.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:51 smithi053 ceph-mon[35158]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-09-12T01:29:54.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:53 smithi001 ceph-mon[30387]: pgmap v407: 129 pgs: 129 active+clean; 658 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-09-12T01:29:54.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:53 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:29:54.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:53 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:29:54.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:53 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:29:54.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:53 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:29:54.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:53 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:29:54.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:53 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:29:54.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:53 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:29:54.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:53 smithi053 ceph-mon[35158]: pgmap v407: 129 pgs: 129 active+clean; 658 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-09-12T01:29:54.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:53 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:29:54.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:53 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:29:54.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:53 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:29:54.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:53 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:29:54.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:53 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:29:54.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:53 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:29:54.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:53 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:29:54.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:53 smithi104 ceph-mon[35219]: pgmap v407: 129 pgs: 129 active+clean; 658 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-09-12T01:29:54.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:53 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:29:54.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:53 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:29:54.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:53 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:29:54.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:53 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:29:54.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:53 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:29:54.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:53 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:29:54.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:53 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:29:55.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:54 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:29:55.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:54 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:29:55.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:54 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:29:55.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:54 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:29:55.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:54 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:29:55.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:54 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:29:55.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:54 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:29:55.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:54 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:29:55.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:54 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:29:56.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:56 smithi001 ceph-mon[30387]: pgmap v408: 129 pgs: 129 active+clean; 659 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-09-12T01:29:56.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:56 smithi104 ceph-mon[35219]: pgmap v408: 129 pgs: 129 active+clean; 659 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-09-12T01:29:56.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:56 smithi053 ceph-mon[35158]: pgmap v408: 129 pgs: 129 active+clean; 659 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-09-12T01:29:58.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:29:58 smithi104 ceph-mon[35219]: pgmap v409: 129 pgs: 129 active+clean; 659 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-09-12T01:29:58.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:29:58 smithi001 ceph-mon[30387]: pgmap v409: 129 pgs: 129 active+clean; 659 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-09-12T01:29:58.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:29:58 smithi053 ceph-mon[35158]: pgmap v409: 129 pgs: 129 active+clean; 659 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-09-12T01:30:00.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:30:00 smithi104 ceph-mon[35219]: pgmap v410: 129 pgs: 129 active+clean; 659 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-12T01:30:00.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:30:00 smithi104 ceph-mon[35219]: overall HEALTH_OK 2024-09-12T01:30:00.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:30:00 smithi001 ceph-mon[30387]: pgmap v410: 129 pgs: 129 active+clean; 659 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-12T01:30:00.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:30:00 smithi001 ceph-mon[30387]: overall HEALTH_OK 2024-09-12T01:30:00.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:30:00 smithi053 ceph-mon[35158]: pgmap v410: 129 pgs: 129 active+clean; 659 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-12T01:30:00.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:30:00 smithi053 ceph-mon[35158]: overall HEALTH_OK 2024-09-12T01:30:02.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:30:02 smithi104 ceph-mon[35219]: pgmap v411: 129 pgs: 129 active+clean; 659 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-09-12T01:30:02.641 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:30:02 smithi053 ceph-mon[35158]: pgmap v411: 129 pgs: 129 active+clean; 659 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-09-12T01:30:02.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:30:02 smithi001 ceph-mon[30387]: pgmap v411: 129 pgs: 129 active+clean; 659 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-09-12T01:30:03.499 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.499+0000 7f1debe00640 1 -- 172.21.15.1:0/2839297093 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1dec104d50 msgr2=0x7f1dec107140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:03.499 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.499+0000 7f1debe00640 1 --2- 172.21.15.1:0/2839297093 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1dec104d50 0x7f1dec107140 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8009900 tx=0x7f1dd802a3f0 comp rx=0 tx=0).stop 2024-09-12T01:30:03.499 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.499+0000 7f1debe00640 1 -- 172.21.15.1:0/2839297093 shutdown_connections 2024-09-12T01:30:03.499 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.499+0000 7f1debe00640 1 --2- 172.21.15.1:0/2839297093 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1dec107680 0x7f1dec109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.499+0000 7f1debe00640 1 --2- 172.21.15.1:0/2839297093 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1dec104d50 0x7f1dec107140 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.499+0000 7f1debe00640 1 --2- 172.21.15.1:0/2839297093 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dec102420 0x7f1dec104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.499+0000 7f1debe00640 1 -- 172.21.15.1:0/2839297093 >> 172.21.15.1:0/2839297093 conn(0x7f1dec0fc040 msgr2=0x7f1dec0fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:30:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.499+0000 7f1debe00640 1 -- 172.21.15.1:0/2839297093 shutdown_connections 2024-09-12T01:30:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.499+0000 7f1debe00640 1 -- 172.21.15.1:0/2839297093 wait complete. 2024-09-12T01:30:03.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.500+0000 7f1debe00640 1 Processor -- start 2024-09-12T01:30:03.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.501+0000 7f1debe00640 1 -- start start 2024-09-12T01:30:03.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.501+0000 7f1debe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1dec102420 0x7f1dec199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:03.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.501+0000 7f1debe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1dec104d50 0x7f1dec19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:03.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.501+0000 7f1debe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dec107680 0x7f1dec1a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:03.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.501+0000 7f1debe00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1dec10d6b0 con 0x7f1dec107680 2024-09-12T01:30:03.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.501+0000 7f1debe00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1dec10d530 con 0x7f1dec104d50 2024-09-12T01:30:03.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.501+0000 7f1debe00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1dec10d830 con 0x7f1dec102420 2024-09-12T01:30:03.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.501+0000 7f1deb400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dec107680 0x7f1dec1a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:03.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.501+0000 7f1deb400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dec107680 0x7f1dec1a1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59810/0 (socket says 172.21.15.1:59810) 2024-09-12T01:30:03.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.501+0000 7f1deb400640 1 -- 172.21.15.1:0/3739337979 learned_addr learned my addr 172.21.15.1:0/3739337979 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:30:03.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.501+0000 7f1deaa00640 1 --2- 172.21.15.1:0/3739337979 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1dec102420 0x7f1dec199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:03.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.501+0000 7f1dea000640 1 --2- 172.21.15.1:0/3739337979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1dec104d50 0x7f1dec19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:03.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.502+0000 7f1deb400640 1 -- 172.21.15.1:0/3739337979 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1dec102420 msgr2=0x7f1dec199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:03.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.502+0000 7f1deb400640 1 --2- 172.21.15.1:0/3739337979 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1dec102420 0x7f1dec199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:03.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.502+0000 7f1deb400640 1 -- 172.21.15.1:0/3739337979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1dec104d50 msgr2=0x7f1dec19a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:03.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.502+0000 7f1deb400640 1 --2- 172.21.15.1:0/3739337979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1dec104d50 0x7f1dec19a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:03.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.502+0000 7f1deb400640 1 -- 172.21.15.1:0/3739337979 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dec1a1b90 con 0x7f1dec107680 2024-09-12T01:30:03.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.502+0000 7f1deb400640 1 --2- 172.21.15.1:0/3739337979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dec107680 0x7f1dec1a1520 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1de400eb40 tx=0x7f1de400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:30:03.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.502+0000 7f1de3400640 1 -- 172.21.15.1:0/3739337979 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de4004b20 con 0x7f1dec107680 2024-09-12T01:30:03.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.502+0000 7f1de3400640 1 -- 172.21.15.1:0/3739337979 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1de4004cc0 con 0x7f1dec107680 2024-09-12T01:30:03.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.502+0000 7f1debe00640 1 -- 172.21.15.1:0/3739337979 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dec1a1e80 con 0x7f1dec107680 2024-09-12T01:30:03.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.502+0000 7f1de3400640 1 -- 172.21.15.1:0/3739337979 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de4010850 con 0x7f1dec107680 2024-09-12T01:30:03.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.503+0000 7f1debe00640 1 -- 172.21.15.1:0/3739337979 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dec1a2340 con 0x7f1dec107680 2024-09-12T01:30:03.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.505+0000 7f1df4a16740 1 Processor -- start 2024-09-12T01:30:03.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.506+0000 7f1de3400640 1 -- 172.21.15.1:0/3739337979 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f1de4005000 con 0x7f1dec107680 2024-09-12T01:30:03.506 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.506+0000 7f1df4a16740 1 -- start start 2024-09-12T01:30:03.506 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.506+0000 7f1df4a16740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b09badf260 0x55b09badf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:03.506 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.506+0000 7f1df4a16740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b09badfba0 0x55b09bae4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:03.506 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.506+0000 7f1df4a16740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b09bae4c30 0x55b09bae70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:03.506 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.506+0000 7f1df4a16740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b09ba44100 con 0x55b09badf260 2024-09-12T01:30:03.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.506+0000 7f1df4a16740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b09b9ec1b0 con 0x55b09bae4c30 2024-09-12T01:30:03.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.506+0000 7f1df4a16740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b09ba400c0 con 0x55b09badfba0 2024-09-12T01:30:03.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.506+0000 7f1dea000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b09badf260 0x55b09badf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:03.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.507+0000 7f1deaa00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b09badfba0 0x55b09bae4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:03.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.507+0000 7f1deaa00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b09badfba0 0x55b09bae4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:52352/0 (socket says 172.21.15.1:52352) 2024-09-12T01:30:03.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.507+0000 7f1dea000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b09badf260 0x55b09badf660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59824/0 (socket says 172.21.15.1:59824) 2024-09-12T01:30:03.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.507+0000 7f1deaa00640 1 -- 172.21.15.1:0/206634939 learned_addr learned my addr 172.21.15.1:0/206634939 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:30:03.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.507+0000 7f1dea000640 1 --2- 172.21.15.1:0/206634939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b09bae4c30 0x55b09bae70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:03.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.507+0000 7f1deaa00640 1 -- 172.21.15.1:0/206634939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b09bae4c30 msgr2=0x55b09bae70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:30:03.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.507+0000 7f1deaa00640 1 --2- 172.21.15.1:0/206634939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b09bae4c30 0x55b09bae70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:03.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.507+0000 7f1deaa00640 1 -- 172.21.15.1:0/206634939 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b09badf260 msgr2=0x55b09badf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:30:03.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.507+0000 7f1deaa00640 1 --2- 172.21.15.1:0/206634939 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b09badf260 0x55b09badf660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:03.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.507+0000 7f1deaa00640 1 -- 172.21.15.1:0/206634939 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b09b757210 con 0x55b09badfba0 2024-09-12T01:30:03.514 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.507+0000 7f1de3400640 1 --2- 172.21.15.1:0/3739337979 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1dd00a2c00 0x7f1dd00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:03.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.507+0000 7f1deaa00640 1 --2- 172.21.15.1:0/206634939 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b09badfba0 0x55b09bae4660 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc009cf0 tx=0x7f1ddc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bde4952cdc9582 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:30:03.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.508+0000 7f1de3400640 1 -- 172.21.15.1:0/3739337979 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f1de40cf980 con 0x7f1dec107680 2024-09-12T01:30:03.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.508+0000 7f1dea000640 1 --2- 172.21.15.1:0/3739337979 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1dd00a2c00 0x7f1dd00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:03.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.508+0000 7f1dd7e00640 1 -- 172.21.15.1:0/206634939 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc004230 con 0x55b09badfba0 2024-09-12T01:30:03.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.508+0000 7f1dd7e00640 1 -- 172.21.15.1:0/206634939 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ddc0043d0 con 0x55b09badfba0 2024-09-12T01:30:03.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.508+0000 7f1dd7e00640 1 -- 172.21.15.1:0/206634939 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc019840 con 0x55b09badfba0 2024-09-12T01:30:03.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.508+0000 7f1df4a16740 1 -- 172.21.15.1:0/206634939 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b09badfba0 msgr2=0x55b09bae4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:03.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.508+0000 7f1dea000640 1 --2- 172.21.15.1:0/3739337979 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1dd00a2c00 0x7f1dd00a50c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1dd802a9e0 tx=0x7f1dd8005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:30:03.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.508+0000 7f1df4a16740 1 --2- 172.21.15.1:0/206634939 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b09badfba0 0x55b09bae4660 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc009cf0 tx=0x7f1ddc011810 comp rx=0 tx=0).stop 2024-09-12T01:30:03.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.509+0000 7f1df4a16740 1 -- 172.21.15.1:0/206634939 shutdown_connections 2024-09-12T01:30:03.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.509+0000 7f1df4a16740 1 --2- 172.21.15.1:0/206634939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b09bae4c30 0x55b09bae70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:03.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.509+0000 7f1df4a16740 1 --2- 172.21.15.1:0/206634939 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b09badfba0 0x55b09bae4660 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:03.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.509+0000 7f1df4a16740 1 --2- 172.21.15.1:0/206634939 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b09badf260 0x55b09badf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:03.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.509+0000 7f1df4a16740 1 -- 172.21.15.1:0/206634939 >> 172.21.15.1:0/206634939 conn(0x55b09bada8d0 msgr2=0x55b09badacd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:30:03.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.509+0000 7f1df4a16740 1 -- 172.21.15.1:0/206634939 shutdown_connections 2024-09-12T01:30:03.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.509+0000 7f1df4a16740 1 -- 172.21.15.1:0/206634939 wait complete. 2024-09-12T01:30:03.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.509+0000 7f1df4a16740 1 -- start start 2024-09-12T01:30:03.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.509+0000 7f1df4a16740 1 Processor -- start 2024-09-12T01:30:03.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.510+0000 7f1df4a16740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b09badf260 0x55b09bae93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:03.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.510+0000 7f1df4a16740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b09badfba0 0x55b09bae9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:03.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.510+0000 7f1df4a16740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b09bae4c30 0x55b09baf0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:03.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.510+0000 7f1df4a16740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b09bae7c00 con 0x55b09bae4c30 2024-09-12T01:30:03.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.510+0000 7f1df4a16740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b09b9cd090 con 0x55b09badf260 2024-09-12T01:30:03.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.510+0000 7f1df4a16740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b09bae7d80 con 0x55b09badfba0 2024-09-12T01:30:03.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.510+0000 7f1df4a16740 10 client.0 _ll_register_callbacks cb 0x55b09ba21f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:30:03.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.510+0000 7f1deb400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b09bae4c30 0x55b09baf0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:03.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.510+0000 7f1deb400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b09bae4c30 0x55b09baf0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59834/0 (socket says 172.21.15.1:59834) 2024-09-12T01:30:03.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.510+0000 7f1deb400640 1 -- 172.21.15.1:0/844025654 learned_addr learned my addr 172.21.15.1:0/844025654 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:30:03.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.510+0000 7f1dea000640 1 --2- 172.21.15.1:0/844025654 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b09badfba0 0x55b09bae9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:03.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.510+0000 7f1deaa00640 1 --2- 172.21.15.1:0/844025654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b09badf260 0x55b09bae93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:03.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.511+0000 7f1deb400640 1 -- 172.21.15.1:0/844025654 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b09badfba0 msgr2=0x55b09bae9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:30:03.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.511+0000 7f1deb400640 1 --2- 172.21.15.1:0/844025654 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b09badfba0 0x55b09bae9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:03.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.511+0000 7f1deb400640 1 -- 172.21.15.1:0/844025654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b09badf260 msgr2=0x55b09bae93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:30:03.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.511+0000 7f1deb400640 1 --2- 172.21.15.1:0/844025654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b09badf260 0x55b09bae93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:03.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.511+0000 7f1deb400640 1 -- 172.21.15.1:0/844025654 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b09b823620 con 0x55b09bae4c30 2024-09-12T01:30:03.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.511+0000 7f1deb400640 1 --2- 172.21.15.1:0/844025654 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b09bae4c30 0x55b09baf0970 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1de40980f0 tx=0x7f1de409b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf80dc16ec1c3a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:30:03.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.511+0000 7f1dd4c00640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-12T01:30:03.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.511+0000 7f1dd4c00640 1 -- 172.21.15.1:0/844025654 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de40a0040 con 0x55b09bae4c30 2024-09-12T01:30:03.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.511+0000 7f1dd4c00640 1 -- 172.21.15.1:0/844025654 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f1de409b0e0 con 0x55b09bae4c30 2024-09-12T01:30:03.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.511+0000 7f1df4a16740 1 -- 172.21.15.1:0/844025654 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ddc0043d0 con 0x55b09bae4c30 2024-09-12T01:30:03.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.511+0000 7f1dd4c00640 1 -- 172.21.15.1:0/844025654 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1de40a4900 con 0x55b09bae4c30 2024-09-12T01:30:03.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.511+0000 7f1df4a16740 10 client.15210 fetch_fsmap learned FSMap version 29 2024-09-12T01:30:03.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.511+0000 7f1df4a16740 1 -- 172.21.15.1:0/844025654 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b09baf37d0 con 0x55b09bae4c30 2024-09-12T01:30:03.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.512+0000 7f1dd4c00640 1 -- 172.21.15.1:0/844025654 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ddc0043d0 con 0x55b09bae4c30 2024-09-12T01:30:03.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.512+0000 7f1dd4c00640 1 -- 172.21.15.1:0/844025654 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f1de401f020 con 0x55b09bae4c30 2024-09-12T01:30:03.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.512+0000 7f1df4a16740 10 client.15210 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T01:30:03.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.512+0000 7f1df4a16740 10 client.15210 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:30:03.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.512+0000 7f1df4a16740 20 client.15210 populate_metadata read hostname 'smithi001' 2024-09-12T01:30:03.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.512+0000 7f1df4a16740 1 --2- 172.21.15.1:0/844025654 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55b09baf4ea0 0x55b09baf7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:03.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.512+0000 7f1df4a16740 4 client.15210 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:30:03.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.512+0000 7f1df4a16740 1 -- 172.21.15.1:0/844025654 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b09ba51830 con 0x55b09baf4ea0 2024-09-12T01:30:03.519 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.512+0000 7f1deaa00640 1 --2- 172.21.15.1:0/844025654 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55b09baf4ea0 0x55b09baf7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:03.519 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.513+0000 7f1deaa00640 1 --2- 172.21.15.1:0/844025654 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55b09baf4ea0 0x55b09baf7310 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46e485aaee7673bf server_cookie=c1f9c49ae1d17d5 in_seq=0 out_seq=0 2024-09-12T01:30:03.519 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.513+0000 7f1dd4c00640 10 client.15210 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:30:03.519 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.514+0000 7f1dd4c00640 1 -- 172.21.15.1:0/844025654 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b09ba51830 con 0x55b09baf4ea0 2024-09-12T01:30:03.519 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.514+0000 7f1dd4c00640 10 client.15210 handle_command_reply: tid=0 2024-09-12T01:30:03.519 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.514+0000 7f1df4a16740 1 client.15210 shutdown 2024-09-12T01:30:03.519 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.514+0000 7f1df4a16740 1 -- 172.21.15.1:0/844025654 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b09bae4c30 msgr2=0x55b09baf0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:03.519 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.514+0000 7f1df4a16740 1 --2- 172.21.15.1:0/844025654 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b09bae4c30 0x55b09baf0970 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1de40980f0 tx=0x7f1de409b6b0 comp rx=0 tx=0).stop 2024-09-12T01:30:03.519 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.514+0000 7f1df4a16740 1 -- 172.21.15.1:0/844025654 shutdown_connections 2024-09-12T01:30:03.519 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.514+0000 7f1df4a16740 1 --2- 172.21.15.1:0/844025654 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55b09baf4ea0 0x55b09baf7310 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:03.519 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.514+0000 7f1df4a16740 1 --2- 172.21.15.1:0/844025654 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b09bae4c30 0x55b09baf0970 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:03.519 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.514+0000 7f1df4a16740 1 --2- 172.21.15.1:0/844025654 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b09badfba0 0x55b09bae9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:03.519 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.514+0000 7f1dd4c00640 0 client.15210 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:30:03.520 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.514+0000 7f1df4a16740 1 --2- 172.21.15.1:0/844025654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b09badf260 0x55b09bae93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:03.520 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.514+0000 7f1df4a16740 1 -- 172.21.15.1:0/844025654 >> 172.21.15.1:0/844025654 conn(0x55b09bada8d0 msgr2=0x55b09badc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:30:03.520 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.514+0000 7f1df4a16740 1 -- 172.21.15.1:0/844025654 shutdown_connections 2024-09-12T01:30:03.520 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.514+0000 7f1df4a16740 1 -- 172.21.15.1:0/844025654 wait complete. 2024-09-12T01:30:03.520 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.514+0000 7f1df4a16740 20 client.15210 trim_cache size 0 max 16384 2024-09-12T01:30:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.530+0000 7f1df4a16740 1 Processor -- start 2024-09-12T01:30:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.530+0000 7f1df4a16740 1 -- start start 2024-09-12T01:30:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.530+0000 7f1df4a16740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b09badf260 0x55b09baf1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.530+0000 7f1df4a16740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b09badfba0 0x55b09baf0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.530+0000 7f1df4a16740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b09bae4c30 0x55b09badb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.530+0000 7f1df4a16740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1dec1ac7d0 con 0x55b09bae4c30 2024-09-12T01:30:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.530+0000 7f1df4a16740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1dec1ae310 con 0x55b09badf260 2024-09-12T01:30:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.530+0000 7f1df4a16740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1dec1acea0 con 0x55b09badfba0 2024-09-12T01:30:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.530+0000 7f1deb400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b09bae4c30 0x55b09badb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.530+0000 7f1deb400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b09bae4c30 0x55b09badb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59842/0 (socket says 172.21.15.1:59842) 2024-09-12T01:30:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.530+0000 7f1deb400640 1 -- 172.21.15.1:0/3121404042 learned_addr learned my addr 172.21.15.1:0/3121404042 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:30:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.530+0000 7f1dea000640 1 --2- 172.21.15.1:0/3121404042 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b09badfba0 0x55b09baf0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.530+0000 7f1deaa00640 1 --2- 172.21.15.1:0/3121404042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b09badf260 0x55b09baf1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.531+0000 7f1dea000640 1 -- 172.21.15.1:0/3121404042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b09badf260 msgr2=0x55b09baf1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:30:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.531+0000 7f1dea000640 1 --2- 172.21.15.1:0/3121404042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b09badf260 0x55b09baf1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.531+0000 7f1dea000640 1 -- 172.21.15.1:0/3121404042 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b09bae4c30 msgr2=0x55b09badb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:30:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.531+0000 7f1dea000640 1 --2- 172.21.15.1:0/3121404042 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b09bae4c30 0x55b09badb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.531+0000 7f1dea000640 1 -- 172.21.15.1:0/3121404042 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b09bae8480 con 0x55b09badfba0 2024-09-12T01:30:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.531+0000 7f1deb400640 1 --2- 172.21.15.1:0/3121404042 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b09bae4c30 0x55b09badb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:30:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.531+0000 7f1dea000640 1 --2- 172.21.15.1:0/3121404042 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b09badfba0 0x55b09baf0930 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8009230 tx=0x7f1dd8009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc156f0376044357 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:30:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.531+0000 7f1dd7400640 1 -- 172.21.15.1:0/3121404042 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd8035040 con 0x55b09badfba0 2024-09-12T01:30:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.531+0000 7f1dd7400640 1 -- 172.21.15.1:0/3121404042 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1dd8050070 con 0x55b09badfba0 2024-09-12T01:30:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.531+0000 7f1dd7400640 1 -- 172.21.15.1:0/3121404042 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd8003850 con 0x55b09badfba0 2024-09-12T01:30:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.532+0000 7f1df4a16740 1 -- 172.21.15.1:0/3121404042 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b09badfba0 msgr2=0x55b09baf0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.532+0000 7f1df4a16740 1 --2- 172.21.15.1:0/3121404042 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b09badfba0 0x55b09baf0930 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8009230 tx=0x7f1dd8009260 comp rx=0 tx=0).stop 2024-09-12T01:30:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.532+0000 7f1df4a16740 1 -- 172.21.15.1:0/3121404042 shutdown_connections 2024-09-12T01:30:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.532+0000 7f1df4a16740 1 --2- 172.21.15.1:0/3121404042 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b09bae4c30 0x55b09badb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.532+0000 7f1df4a16740 1 --2- 172.21.15.1:0/3121404042 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b09badfba0 0x55b09baf0930 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.532+0000 7f1df4a16740 1 --2- 172.21.15.1:0/3121404042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b09badf260 0x55b09baf1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.532+0000 7f1df4a16740 1 -- 172.21.15.1:0/3121404042 >> 172.21.15.1:0/3121404042 conn(0x55b09bada8d0 msgr2=0x55b09baf1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:30:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.532+0000 7f1df4a16740 1 -- 172.21.15.1:0/3121404042 shutdown_connections 2024-09-12T01:30:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.532+0000 7f1df4a16740 1 -- 172.21.15.1:0/3121404042 wait complete. 2024-09-12T01:30:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.533+0000 7f1df4a16740 1 -- start start 2024-09-12T01:30:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.533+0000 7f1df4a16740 1 Processor -- start 2024-09-12T01:30:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.533+0000 7f1df4a16740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b09badf260 0x55b09baedc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.533+0000 7f1df4a16740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b09badfba0 0x55b09baee190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.533+0000 7f1df4a16740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b09bae4c30 0x55b09baef150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.533+0000 7f1df4a16740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1dec1adc40 con 0x55b09badf260 2024-09-12T01:30:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.533+0000 7f1df4a16740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1dec1ad570 con 0x55b09badfba0 2024-09-12T01:30:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.533+0000 7f1df4a16740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1dec1aa7b0 con 0x55b09bae4c30 2024-09-12T01:30:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.533+0000 7f1df4a16740 10 client.0 _ll_register_callbacks cb 0x7f1de4004e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:30:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.533+0000 7f1deaa00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b09badf260 0x55b09baedc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.533+0000 7f1deb400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b09bae4c30 0x55b09baef150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.533+0000 7f1dea000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b09badfba0 0x55b09baee190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.533+0000 7f1deb400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b09bae4c30 0x55b09baef150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:52382/0 (socket says 172.21.15.1:52382) 2024-09-12T01:30:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.534+0000 7f1deb400640 1 -- 172.21.15.1:0/2658550729 learned_addr learned my addr 172.21.15.1:0/2658550729 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:30:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.534+0000 7f1dea000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b09badfba0 0x55b09baee190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:46064/0 (socket says 172.21.15.1:46064) 2024-09-12T01:30:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.534+0000 7f1deb400640 1 -- 172.21.15.1:0/2658550729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b09badfba0 msgr2=0x55b09baee190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:30:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.534+0000 7f1deb400640 1 --2- 172.21.15.1:0/2658550729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b09badfba0 0x55b09baee190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.534+0000 7f1deb400640 1 -- 172.21.15.1:0/2658550729 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b09badf260 msgr2=0x55b09baedc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:30:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.534+0000 7f1deb400640 1 --2- 172.21.15.1:0/2658550729 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b09badf260 0x55b09baedc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.534+0000 7f1deb400640 1 -- 172.21.15.1:0/2658550729 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b09baef760 con 0x55b09bae4c30 2024-09-12T01:30:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.534+0000 7f1dea000640 1 --2- 172.21.15.1:0/2658550729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b09badfba0 0x55b09baee190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:30:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.534+0000 7f1deaa00640 1 --2- 172.21.15.1:0/2658550729 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b09badf260 0x55b09baedc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:30:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.534+0000 7f1deb400640 1 --2- 172.21.15.1:0/2658550729 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b09bae4c30 0x55b09baef150 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1de400c7d0 tx=0x7f1de409c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e5103bc3856f7ed server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:30:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.534+0000 7f1dd4c00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-12T01:30:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.534+0000 7f1dd4c00640 1 -- 172.21.15.1:0/2658550729 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de409b2e0 con 0x55b09bae4c30 2024-09-12T01:30:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.534+0000 7f1df4a16740 1 -- 172.21.15.1:0/2658550729 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1dd8050070 con 0x55b09bae4c30 2024-09-12T01:30:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.534+0000 7f1dd4c00640 1 -- 172.21.15.1:0/2658550729 <== mon.1 v2:172.21.15.104:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f1de407ab00 con 0x55b09bae4c30 2024-09-12T01:30:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.535+0000 7f1dd4c00640 1 -- 172.21.15.1:0/2658550729 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1de409b7d0 con 0x55b09bae4c30 2024-09-12T01:30:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.535+0000 7f1df4a16740 10 client.25237 fetch_fsmap learned FSMap version 29 2024-09-12T01:30:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.535+0000 7f1df4a16740 1 -- 172.21.15.1:0/2658550729 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b09baf0ae0 con 0x55b09bae4c30 2024-09-12T01:30:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.535+0000 7f1dd4c00640 1 -- 172.21.15.1:0/2658550729 <== mon.1 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1dd8050070 con 0x55b09bae4c30 2024-09-12T01:30:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.535+0000 7f1dd4c00640 1 -- 172.21.15.1:0/2658550729 <== mon.1 v2:172.21.15.104:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f1de401f070 con 0x55b09bae4c30 2024-09-12T01:30:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.535+0000 7f1df4a16740 10 client.25237 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T01:30:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.535+0000 7f1df4a16740 10 client.25237 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:30:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.535+0000 7f1df4a16740 20 client.25237 populate_metadata read hostname 'smithi001' 2024-09-12T01:30:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.535+0000 7f1df4a16740 1 --2- 172.21.15.1:0/2658550729 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55b09bb5ff60 0x55b09bb62380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.535+0000 7f1df4a16740 4 client.25237 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:30:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.535+0000 7f1df4a16740 1 -- 172.21.15.1:0/2658550729 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b09ba4bb80 con 0x55b09bb5ff60 2024-09-12T01:30:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.536+0000 7f1deaa00640 1 --2- 172.21.15.1:0/2658550729 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55b09bb5ff60 0x55b09bb62380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:03.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.536+0000 7f1deaa00640 1 --2- 172.21.15.1:0/2658550729 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55b09bb5ff60 0x55b09bb62380 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33b8f9191af08494 server_cookie=85dc4d7d6d57f3a1 in_seq=0 out_seq=0 2024-09-12T01:30:03.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.536+0000 7f1dd4c00640 10 client.25237 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:30:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.550+0000 7f1dd4c00640 1 -- 172.21.15.1:0/2658550729 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b09ba4bb80 con 0x55b09bb5ff60 2024-09-12T01:30:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.550+0000 7f1dd4c00640 10 client.25237 handle_command_reply: tid=0 2024-09-12T01:30:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.550+0000 7f1df4a16740 1 client.25237 shutdown 2024-09-12T01:30:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.550+0000 7f1df4a16740 1 -- 172.21.15.1:0/2658550729 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b09bae4c30 msgr2=0x55b09baef150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.550+0000 7f1df4a16740 1 --2- 172.21.15.1:0/2658550729 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b09bae4c30 0x55b09baef150 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1de400c7d0 tx=0x7f1de409c4d0 comp rx=0 tx=0).stop 2024-09-12T01:30:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.550+0000 7f1df4a16740 1 -- 172.21.15.1:0/2658550729 shutdown_connections 2024-09-12T01:30:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.550+0000 7f1df4a16740 1 --2- 172.21.15.1:0/2658550729 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55b09bb5ff60 0x55b09bb62380 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.550+0000 7f1df4a16740 1 --2- 172.21.15.1:0/2658550729 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b09bae4c30 0x55b09baef150 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.550+0000 7f1dd4c00640 0 client.25237 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:30:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.550+0000 7f1df4a16740 1 --2- 172.21.15.1:0/2658550729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b09badfba0 0x55b09baee190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.550+0000 7f1df4a16740 1 --2- 172.21.15.1:0/2658550729 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b09badf260 0x55b09baedc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.550+0000 7f1df4a16740 1 -- 172.21.15.1:0/2658550729 >> 172.21.15.1:0/2658550729 conn(0x55b09bada8d0 msgr2=0x55b09badb780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:30:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.550+0000 7f1df4a16740 1 -- 172.21.15.1:0/2658550729 shutdown_connections 2024-09-12T01:30:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.550+0000 7f1df4a16740 1 -- 172.21.15.1:0/2658550729 wait complete. 2024-09-12T01:30:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.550+0000 7f1df4a16740 20 client.25237 trim_cache size 0 max 16384 2024-09-12T01:30:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-12T01:30:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-12T01:30:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-12T01:30:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-12T01:30:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.551+0000 7f1debe00640 1 -- 172.21.15.1:0/3739337979 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1dd00a2c00 msgr2=0x7f1dd00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.551+0000 7f1debe00640 1 --2- 172.21.15.1:0/3739337979 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1dd00a2c00 0x7f1dd00a50c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1dd802a9e0 tx=0x7f1dd8005d50 comp rx=0 tx=0).stop 2024-09-12T01:30:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.551+0000 7f1debe00640 1 -- 172.21.15.1:0/3739337979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dec107680 msgr2=0x7f1dec1a1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.551+0000 7f1debe00640 1 --2- 172.21.15.1:0/3739337979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dec107680 0x7f1dec1a1520 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1de400eb40 tx=0x7f1de400c2d0 comp rx=0 tx=0).stop 2024-09-12T01:30:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.551+0000 7f1debe00640 1 -- 172.21.15.1:0/3739337979 shutdown_connections 2024-09-12T01:30:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.551+0000 7f1debe00640 1 --2- 172.21.15.1:0/3739337979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dec107680 0x7f1dec1a1520 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.551+0000 7f1debe00640 1 --2- 172.21.15.1:0/3739337979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1dec104d50 0x7f1dec19a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.551+0000 7f1debe00640 1 --2- 172.21.15.1:0/3739337979 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1dd00a2c00 0x7f1dd00a50c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.551+0000 7f1debe00640 1 --2- 172.21.15.1:0/3739337979 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1dec102420 0x7f1dec199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.551+0000 7f1debe00640 1 -- 172.21.15.1:0/3739337979 >> 172.21.15.1:0/3739337979 conn(0x7f1dec0fc040 msgr2=0x7f1dec101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:30:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.551+0000 7f1debe00640 1 -- 172.21.15.1:0/3739337979 shutdown_connections 2024-09-12T01:30:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:03.551+0000 7f1debe00640 1 -- 172.21.15.1:0/3739337979 wait complete. 2024-09-12T01:30:04.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:30:04 smithi104 ceph-mon[35219]: pgmap v412: 129 pgs: 129 active+clean; 661 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-09-12T01:30:04.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:30:04 smithi001 ceph-mon[30387]: pgmap v412: 129 pgs: 129 active+clean; 661 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-09-12T01:30:04.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:30:04 smithi053 ceph-mon[35158]: pgmap v412: 129 pgs: 129 active+clean; 661 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-09-12T01:30:06.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:30:06 smithi104 ceph-mon[35219]: pgmap v413: 129 pgs: 129 active+clean; 661 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-12T01:30:06.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:30:06 smithi001 ceph-mon[30387]: pgmap v413: 129 pgs: 129 active+clean; 661 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-12T01:30:06.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:30:06 smithi053 ceph-mon[35158]: pgmap v413: 129 pgs: 129 active+clean; 661 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-12T01:30:08.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:30:08 smithi104 ceph-mon[35219]: pgmap v414: 129 pgs: 129 active+clean; 663 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 15 op/s 2024-09-12T01:30:08.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:30:08 smithi001 ceph-mon[30387]: pgmap v414: 129 pgs: 129 active+clean; 663 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 15 op/s 2024-09-12T01:30:08.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:30:08 smithi053 ceph-mon[35158]: pgmap v414: 129 pgs: 129 active+clean; 663 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 15 op/s 2024-09-12T01:30:10.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:30:10 smithi104 ceph-mon[35219]: pgmap v415: 129 pgs: 129 active+clean; 663 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-09-12T01:30:10.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:30:10 smithi001 ceph-mon[30387]: pgmap v415: 129 pgs: 129 active+clean; 663 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-09-12T01:30:10.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:30:10 smithi053 ceph-mon[35158]: pgmap v415: 129 pgs: 129 active+clean; 663 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-09-12T01:30:12.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:30:12 smithi104 ceph-mon[35219]: pgmap v416: 129 pgs: 129 active+clean; 663 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.8 KiB/s wr, 14 op/s 2024-09-12T01:30:12.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:30:12 smithi001 ceph-mon[30387]: pgmap v416: 129 pgs: 129 active+clean; 663 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.8 KiB/s wr, 14 op/s 2024-09-12T01:30:12.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:30:12 smithi053 ceph-mon[35158]: pgmap v416: 129 pgs: 129 active+clean; 663 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.8 KiB/s wr, 14 op/s 2024-09-12T01:30:14.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:30:14 smithi104 ceph-mon[35219]: pgmap v417: 129 pgs: 129 active+clean; 663 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.8 KiB/s wr, 15 op/s 2024-09-12T01:30:14.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:30:14 smithi001 ceph-mon[30387]: pgmap v417: 129 pgs: 129 active+clean; 663 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.8 KiB/s wr, 15 op/s 2024-09-12T01:30:14.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:30:14 smithi053 ceph-mon[35158]: pgmap v417: 129 pgs: 129 active+clean; 663 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.8 KiB/s wr, 15 op/s 2024-09-12T01:30:16.429 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:30:16 smithi001 ceph-mon[30387]: pgmap v418: 129 pgs: 129 active+clean; 663 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-09-12T01:30:16.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:30:16 smithi104 ceph-mon[35219]: pgmap v418: 129 pgs: 129 active+clean; 663 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-09-12T01:30:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.679+0000 7f0a21c00640 1 -- 172.21.15.1:0/2960684530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a1c1019b0 msgr2=0x7f0a1c101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.679+0000 7f0a21c00640 1 --2- 172.21.15.1:0/2960684530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a1c1019b0 0x7f0a1c101db0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c009860 tx=0x7f0a0c02a340 comp rx=0 tx=0).stop 2024-09-12T01:30:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.679+0000 7f0a21c00640 1 -- 172.21.15.1:0/2960684530 shutdown_connections 2024-09-12T01:30:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.679+0000 7f0a21c00640 1 --2- 172.21.15.1:0/2960684530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a1c103560 0x7f0a1c109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.679+0000 7f0a21c00640 1 --2- 172.21.15.1:0/2960684530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a1c102ba0 0x7f0a1c103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.679+0000 7f0a21c00640 1 --2- 172.21.15.1:0/2960684530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a1c1019b0 0x7f0a1c101db0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.679+0000 7f0a21c00640 1 -- 172.21.15.1:0/2960684530 >> 172.21.15.1:0/2960684530 conn(0x7f0a1c0fd130 msgr2=0x7f0a1c0ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:30:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.679+0000 7f0a21c00640 1 -- 172.21.15.1:0/2960684530 shutdown_connections 2024-09-12T01:30:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.679+0000 7f0a21c00640 1 -- 172.21.15.1:0/2960684530 wait complete. 2024-09-12T01:30:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.680+0000 7f0a21c00640 1 Processor -- start 2024-09-12T01:30:16.683 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.680+0000 7f0a21c00640 1 -- start start 2024-09-12T01:30:16.683 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.681+0000 7f0a21c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a1c1019b0 0x7f0a1c197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:16.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.681+0000 7f0a21c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a1c102ba0 0x7f0a1c198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:16.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.681+0000 7f0a21c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a1c103560 0x7f0a1c19f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:16.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.681+0000 7f0a21c00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c10d630 con 0x7f0a1c102ba0 2024-09-12T01:30:16.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.681+0000 7f0a21c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c10d4b0 con 0x7f0a1c1019b0 2024-09-12T01:30:16.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.681+0000 7f0a21c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c10d7b0 con 0x7f0a1c103560 2024-09-12T01:30:16.685 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.681+0000 7f0a1b400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a1c102ba0 0x7f0a1c198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:16.685 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.681+0000 7f0a1b400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a1c102ba0 0x7f0a1c198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37530/0 (socket says 172.21.15.1:37530) 2024-09-12T01:30:16.685 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.681+0000 7f0a1b400640 1 -- 172.21.15.1:0/3644902257 learned_addr learned my addr 172.21.15.1:0/3644902257 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:30:16.685 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.681+0000 7f0a1be00640 1 --2- 172.21.15.1:0/3644902257 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a1c1019b0 0x7f0a1c197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:16.686 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.681+0000 7f0a21200640 1 --2- 172.21.15.1:0/3644902257 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a1c103560 0x7f0a1c19f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:16.686 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.682+0000 7f0a1be00640 1 -- 172.21.15.1:0/3644902257 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a1c103560 msgr2=0x7f0a1c19f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:16.686 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.682+0000 7f0a1be00640 1 --2- 172.21.15.1:0/3644902257 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a1c103560 0x7f0a1c19f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:16.686 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.682+0000 7f0a1be00640 1 -- 172.21.15.1:0/3644902257 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a1c102ba0 msgr2=0x7f0a1c198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:16.686 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.682+0000 7f0a1be00640 1 --2- 172.21.15.1:0/3644902257 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a1c102ba0 0x7f0a1c198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:16.686 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.682+0000 7f0a1be00640 1 -- 172.21.15.1:0/3644902257 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a1c19f910 con 0x7f0a1c1019b0 2024-09-12T01:30:16.686 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.682+0000 7f0a21200640 1 --2- 172.21.15.1:0/3644902257 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a1c103560 0x7f0a1c19f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:30:16.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.682+0000 7f0a1b400640 1 --2- 172.21.15.1:0/3644902257 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a1c102ba0 0x7f0a1c198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:30:16.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.682+0000 7f0a1be00640 1 --2- 172.21.15.1:0/3644902257 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a1c1019b0 0x7f0a1c197d00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c02a860 tx=0x7f0a0c033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:30:16.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.682+0000 7f0a18c00640 1 -- 172.21.15.1:0/3644902257 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c037040 con 0x7f0a1c1019b0 2024-09-12T01:30:16.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.682+0000 7f0a18c00640 1 -- 172.21.15.1:0/3644902257 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a0c030850 con 0x7f0a1c1019b0 2024-09-12T01:30:16.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.682+0000 7f0a21c00640 1 -- 172.21.15.1:0/3644902257 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a1c19fba0 con 0x7f0a1c1019b0 2024-09-12T01:30:16.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.682+0000 7f0a18c00640 1 -- 172.21.15.1:0/3644902257 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c004920 con 0x7f0a1c1019b0 2024-09-12T01:30:16.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.683+0000 7f0a21c00640 1 -- 172.21.15.1:0/3644902257 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a1c1a00b0 con 0x7f0a1c1019b0 2024-09-12T01:30:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.685+0000 7f0a26033740 1 Processor -- start 2024-09-12T01:30:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.686+0000 7f0a18c00640 1 -- 172.21.15.1:0/3644902257 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f0a0c004050 con 0x7f0a1c1019b0 2024-09-12T01:30:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.686+0000 7f0a26033740 1 -- start start 2024-09-12T01:30:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.686+0000 7f0a26033740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a43c27670 0x557a43c27a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.686+0000 7f0a26033740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a43c27fb0 0x557a43c2ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.686+0000 7f0a26033740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a43c2d040 0x557a43c2f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.686+0000 7f0a26033740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x557a43b88100 con 0x557a43c27670 2024-09-12T01:30:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.686+0000 7f0a26033740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x557a43b301b0 con 0x557a43c27fb0 2024-09-12T01:30:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.686+0000 7f0a26033740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557a43b840c0 con 0x557a43c2d040 2024-09-12T01:30:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.686+0000 7f0a21200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a43c27670 0x557a43c27a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.686+0000 7f0a21200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a43c27670 0x557a43c27a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37544/0 (socket says 172.21.15.1:37544) 2024-09-12T01:30:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.686+0000 7f0a21200640 1 -- 172.21.15.1:0/2715290502 learned_addr learned my addr 172.21.15.1:0/2715290502 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:30:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.686+0000 7f0a1be00640 1 --2- 172.21.15.1:0/2715290502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a43c27fb0 0x557a43c2ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.687+0000 7f0a1b400640 1 --2- 172.21.15.1:0/2715290502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a43c2d040 0x557a43c2f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.687+0000 7f0a1b400640 1 -- 172.21.15.1:0/2715290502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a43c27fb0 msgr2=0x557a43c2ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:30:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.687+0000 7f0a1b400640 1 --2- 172.21.15.1:0/2715290502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a43c27fb0 0x557a43c2ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.687+0000 7f0a1b400640 1 -- 172.21.15.1:0/2715290502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a43c27670 msgr2=0x557a43c27a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:30:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.687+0000 7f0a1b400640 1 --2- 172.21.15.1:0/2715290502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a43c27670 0x557a43c27a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.687+0000 7f0a1b400640 1 -- 172.21.15.1:0/2715290502 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a4389b210 con 0x557a43c2d040 2024-09-12T01:30:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.687+0000 7f0a21200640 1 --2- 172.21.15.1:0/2715290502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a43c27670 0x557a43c27a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:30:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.687+0000 7f0a18c00640 1 --2- 172.21.15.1:0/3644902257 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f0a000a2c80 0x7f0a000a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.687+0000 7f0a1b400640 1 --2- 172.21.15.1:0/2715290502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a43c2d040 0x557a43c2f500 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0a08011510 tx=0x7f0a080119d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a678c80fed6f9cd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:30:16.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.687+0000 7f0a18c00640 1 -- 172.21.15.1:0/3644902257 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f0a0c0ef260 con 0x7f0a1c1019b0 2024-09-12T01:30:16.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.688+0000 7f0a21200640 1 --2- 172.21.15.1:0/3644902257 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f0a000a2c80 0x7f0a000a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:16.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.688+0000 7f0a11600640 1 -- 172.21.15.1:0/2715290502 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0801e070 con 0x557a43c2d040 2024-09-12T01:30:16.697 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.688+0000 7f0a11600640 1 -- 172.21.15.1:0/2715290502 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a0800faa0 con 0x557a43c2d040 2024-09-12T01:30:16.697 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.688+0000 7f0a11600640 1 -- 172.21.15.1:0/2715290502 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a08019c30 con 0x557a43c2d040 2024-09-12T01:30:16.697 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.688+0000 7f0a21200640 1 --2- 172.21.15.1:0/3644902257 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f0a000a2c80 0x7f0a000a5140 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0a14005c40 tx=0x7f0a140095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:30:16.697 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.688+0000 7f0a26033740 1 -- 172.21.15.1:0/2715290502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a43c2d040 msgr2=0x557a43c2f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:16.697 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.688+0000 7f0a26033740 1 --2- 172.21.15.1:0/2715290502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a43c2d040 0x557a43c2f500 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0a08011510 tx=0x7f0a080119d0 comp rx=0 tx=0).stop 2024-09-12T01:30:16.697 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.689+0000 7f0a26033740 1 -- 172.21.15.1:0/2715290502 shutdown_connections 2024-09-12T01:30:16.697 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.689+0000 7f0a26033740 1 --2- 172.21.15.1:0/2715290502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a43c2d040 0x557a43c2f500 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:16.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.689+0000 7f0a26033740 1 --2- 172.21.15.1:0/2715290502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a43c27fb0 0x557a43c2ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:16.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.689+0000 7f0a26033740 1 --2- 172.21.15.1:0/2715290502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a43c27670 0x557a43c27a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:16.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.689+0000 7f0a26033740 1 -- 172.21.15.1:0/2715290502 >> 172.21.15.1:0/2715290502 conn(0x557a43c22ce0 msgr2=0x557a43c230e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:30:16.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.689+0000 7f0a26033740 1 -- 172.21.15.1:0/2715290502 shutdown_connections 2024-09-12T01:30:16.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.689+0000 7f0a26033740 1 -- 172.21.15.1:0/2715290502 wait complete. 2024-09-12T01:30:16.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.689+0000 7f0a26033740 1 -- start start 2024-09-12T01:30:16.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.689+0000 7f0a26033740 1 Processor -- start 2024-09-12T01:30:16.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.690+0000 7f0a26033740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a43c27670 0x557a43c31880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.690+0000 7f0a26033740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a43c27fb0 0x557a43c31dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.690+0000 7f0a26033740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a43c2d040 0x557a43c38d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.690+0000 7f0a26033740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x557a43c2ffb0 con 0x557a43c27fb0 2024-09-12T01:30:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.690+0000 7f0a26033740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x557a43b11090 con 0x557a43c27670 2024-09-12T01:30:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.690+0000 7f0a26033740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557a43c30130 con 0x557a43c2d040 2024-09-12T01:30:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.690+0000 7f0a26033740 10 client.0 _ll_register_callbacks cb 0x557a43b65f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:30:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.690+0000 7f0a1b400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a43c27fb0 0x557a43c31dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.690+0000 7f0a21200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a43c2d040 0x557a43c38d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.690+0000 7f0a1be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a43c27670 0x557a43c31880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.690+0000 7f0a1b400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a43c27fb0 0x557a43c31dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37552/0 (socket says 172.21.15.1:37552) 2024-09-12T01:30:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.690+0000 7f0a1b400640 1 -- 172.21.15.1:0/1132021009 learned_addr learned my addr 172.21.15.1:0/1132021009 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:30:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.690+0000 7f0a1be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a43c27670 0x557a43c31880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:53872/0 (socket says 172.21.15.1:53872) 2024-09-12T01:30:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.691+0000 7f0a1be00640 1 -- 172.21.15.1:0/1132021009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a43c2d040 msgr2=0x557a43c38d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:30:16.701 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.691+0000 7f0a1be00640 1 --2- 172.21.15.1:0/1132021009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a43c2d040 0x557a43c38d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:16.701 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.691+0000 7f0a1be00640 1 -- 172.21.15.1:0/1132021009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a43c27fb0 msgr2=0x557a43c31dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:30:16.701 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.691+0000 7f0a1be00640 1 --2- 172.21.15.1:0/1132021009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a43c27fb0 0x557a43c31dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:16.701 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.691+0000 7f0a1be00640 1 -- 172.21.15.1:0/1132021009 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a43967620 con 0x557a43c27670 2024-09-12T01:30:16.701 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.691+0000 7f0a1be00640 1 --2- 172.21.15.1:0/1132021009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a43c27670 0x557a43c31880 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c0ec930 tx=0x7f0a0c02a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=429ceb980dec2cdd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:30:16.701 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.691+0000 7f0a06000640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-12T01:30:16.702 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.691+0000 7f0a06000640 1 -- 172.21.15.1:0/1132021009 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c0b7890 con 0x557a43c27670 2024-09-12T01:30:16.702 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.691+0000 7f0a26033740 1 -- 172.21.15.1:0/1132021009 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a0800faa0 con 0x557a43c27670 2024-09-12T01:30:16.702 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.691+0000 7f0a06000640 1 -- 172.21.15.1:0/1132021009 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f0a0c0c3e60 con 0x557a43c27670 2024-09-12T01:30:16.702 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.692+0000 7f0a06000640 1 -- 172.21.15.1:0/1132021009 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0a0c0b97c0 con 0x557a43c27670 2024-09-12T01:30:16.702 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.692+0000 7f0a26033740 10 client.25262 fetch_fsmap learned FSMap version 29 2024-09-12T01:30:16.702 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.692+0000 7f0a26033740 1 -- 172.21.15.1:0/1132021009 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557a43c3bb80 con 0x557a43c27670 2024-09-12T01:30:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.692+0000 7f0a06000640 1 -- 172.21.15.1:0/1132021009 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a0800faa0 con 0x557a43c27670 2024-09-12T01:30:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.692+0000 7f0a06000640 1 -- 172.21.15.1:0/1132021009 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f0a0c03a070 con 0x557a43c27670 2024-09-12T01:30:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.692+0000 7f0a26033740 10 client.25262 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T01:30:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.692+0000 7f0a26033740 10 client.25262 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:30:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.692+0000 7f0a26033740 20 client.25262 populate_metadata read hostname 'smithi001' 2024-09-12T01:30:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.692+0000 7f0a26033740 1 --2- 172.21.15.1:0/1132021009 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x557a43c3d260 0x557a43c3f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.692+0000 7f0a26033740 4 client.25262 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:30:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.692+0000 7f0a26033740 1 -- 172.21.15.1:0/1132021009 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a43b95830 con 0x557a43c3d260 2024-09-12T01:30:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.693+0000 7f0a1b400640 1 --2- 172.21.15.1:0/1132021009 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x557a43c3d260 0x557a43c3f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.693+0000 7f0a1b400640 1 --2- 172.21.15.1:0/1132021009 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x557a43c3d260 0x557a43c3f690 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5f7aca4d6f42522 server_cookie=682cae30fa0f2044 in_seq=0 out_seq=0 2024-09-12T01:30:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.693+0000 7f0a06000640 10 client.25262 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:30:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.694+0000 7f0a06000640 1 -- 172.21.15.1:0/1132021009 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557a43b95830 con 0x557a43c3d260 2024-09-12T01:30:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.694+0000 7f0a06000640 10 client.25262 handle_command_reply: tid=0 2024-09-12T01:30:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.694+0000 7f0a26033740 1 client.25262 shutdown 2024-09-12T01:30:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.694+0000 7f0a26033740 1 -- 172.21.15.1:0/1132021009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a43c27670 msgr2=0x557a43c31880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.694+0000 7f0a26033740 1 --2- 172.21.15.1:0/1132021009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a43c27670 0x557a43c31880 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c0ec930 tx=0x7f0a0c02a340 comp rx=0 tx=0).stop 2024-09-12T01:30:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.694+0000 7f0a26033740 1 -- 172.21.15.1:0/1132021009 shutdown_connections 2024-09-12T01:30:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.694+0000 7f0a26033740 1 --2- 172.21.15.1:0/1132021009 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x557a43c3d260 0x557a43c3f690 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.695+0000 7f0a26033740 1 --2- 172.21.15.1:0/1132021009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a43c2d040 0x557a43c38d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.695+0000 7f0a26033740 1 --2- 172.21.15.1:0/1132021009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a43c27fb0 0x557a43c31dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.695+0000 7f0a06000640 0 client.25262 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:30:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.695+0000 7f0a26033740 1 --2- 172.21.15.1:0/1132021009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a43c27670 0x557a43c31880 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.695+0000 7f0a26033740 1 -- 172.21.15.1:0/1132021009 >> 172.21.15.1:0/1132021009 conn(0x557a43c22ce0 msgr2=0x557a43c244f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:30:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.695+0000 7f0a26033740 1 -- 172.21.15.1:0/1132021009 shutdown_connections 2024-09-12T01:30:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.695+0000 7f0a26033740 1 -- 172.21.15.1:0/1132021009 wait complete. 2024-09-12T01:30:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.695+0000 7f0a26033740 20 client.25262 trim_cache size 0 max 16384 2024-09-12T01:30:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.710+0000 7f0a26033740 1 Processor -- start 2024-09-12T01:30:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.710+0000 7f0a26033740 1 -- start start 2024-09-12T01:30:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.710+0000 7f0a26033740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a43c27670 0x557a43c31e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.710+0000 7f0a26033740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a43c27fb0 0x557a43c38d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.710+0000 7f0a26033740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a43c2d040 0x557a43c1d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.710+0000 7f0a26033740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c1a8310 con 0x557a43c2d040 2024-09-12T01:30:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.710+0000 7f0a26033740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c1a9e50 con 0x557a43c27670 2024-09-12T01:30:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.710+0000 7f0a26033740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c1a89e0 con 0x557a43c27fb0 2024-09-12T01:30:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.711+0000 7f0a21200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a43c2d040 0x557a43c1d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.711+0000 7f0a1be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a43c27670 0x557a43c31e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.711+0000 7f0a1be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a43c27670 0x557a43c31e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:53888/0 (socket says 172.21.15.1:53888) 2024-09-12T01:30:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.711+0000 7f0a1be00640 1 -- 172.21.15.1:0/3133103481 learned_addr learned my addr 172.21.15.1:0/3133103481 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:30:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.711+0000 7f0a1b400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a43c27fb0 0x557a43c38d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.711+0000 7f0a21200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a43c2d040 0x557a43c1d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37560/0 (socket says 172.21.15.1:37560) 2024-09-12T01:30:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.711+0000 7f0a1be00640 1 -- 172.21.15.1:0/3133103481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a43c27fb0 msgr2=0x557a43c38d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:30:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.711+0000 7f0a1be00640 1 --2- 172.21.15.1:0/3133103481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a43c27fb0 0x557a43c38d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.711+0000 7f0a1be00640 1 -- 172.21.15.1:0/3133103481 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a43c2d040 msgr2=0x557a43c1d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:30:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.711+0000 7f0a1be00640 1 --2- 172.21.15.1:0/3133103481 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a43c2d040 0x557a43c1d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.711+0000 7f0a1be00640 1 -- 172.21.15.1:0/3133103481 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a43c3fce0 con 0x557a43c27670 2024-09-12T01:30:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.711+0000 7f0a1b400640 1 --2- 172.21.15.1:0/3133103481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a43c27fb0 0x557a43c38d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:30:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.711+0000 7f0a1be00640 1 --2- 172.21.15.1:0/3133103481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a43c27670 0x557a43c31e50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c0b7f70 tx=0x7f0a0c0b71d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=976b190b92c886af server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:30:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.711+0000 7f0a10c00640 1 -- 172.21.15.1:0/3133103481 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c0bafe0 con 0x557a43c27670 2024-09-12T01:30:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.712+0000 7f0a10c00640 1 -- 172.21.15.1:0/3133103481 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a0c0bbd80 con 0x557a43c27670 2024-09-12T01:30:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.712+0000 7f0a10c00640 1 -- 172.21.15.1:0/3133103481 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c0bb1c0 con 0x557a43c27670 2024-09-12T01:30:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.712+0000 7f0a26033740 1 -- 172.21.15.1:0/3133103481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a43c27670 msgr2=0x557a43c31e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.712+0000 7f0a26033740 1 --2- 172.21.15.1:0/3133103481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a43c27670 0x557a43c31e50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c0b7f70 tx=0x7f0a0c0b71d0 comp rx=0 tx=0).stop 2024-09-12T01:30:16.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.712+0000 7f0a26033740 1 -- 172.21.15.1:0/3133103481 shutdown_connections 2024-09-12T01:30:16.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.712+0000 7f0a26033740 1 --2- 172.21.15.1:0/3133103481 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a43c2d040 0x557a43c1d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:16.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.712+0000 7f0a26033740 1 --2- 172.21.15.1:0/3133103481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a43c27fb0 0x557a43c38d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:16.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.712+0000 7f0a26033740 1 --2- 172.21.15.1:0/3133103481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a43c27670 0x557a43c31e50 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:16.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.712+0000 7f0a26033740 1 -- 172.21.15.1:0/3133103481 >> 172.21.15.1:0/3133103481 conn(0x557a43c22ce0 msgr2=0x557a43c1d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:30:16.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.712+0000 7f0a26033740 1 -- 172.21.15.1:0/3133103481 shutdown_connections 2024-09-12T01:30:16.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.712+0000 7f0a26033740 1 -- 172.21.15.1:0/3133103481 wait complete. 2024-09-12T01:30:16.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.713+0000 7f0a26033740 1 -- start start 2024-09-12T01:30:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.713+0000 7f0a26033740 1 Processor -- start 2024-09-12T01:30:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.713+0000 7f0a26033740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a43c27670 0x557a43c35e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.713+0000 7f0a26033740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a43c27fb0 0x557a43c36360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.713+0000 7f0a26033740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a43c2d040 0x557a43c37370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.713+0000 7f0a26033740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c1a9780 con 0x557a43c27fb0 2024-09-12T01:30:16.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.713+0000 7f0a26033740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c1a90b0 con 0x557a43c27670 2024-09-12T01:30:16.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.713+0000 7f0a26033740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c1a62f0 con 0x557a43c2d040 2024-09-12T01:30:16.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.713+0000 7f0a26033740 10 client.0 _ll_register_callbacks cb 0x7f0a0c0f2a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:30:16.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.714+0000 7f0a1b400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a43c27fb0 0x557a43c36360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:16.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.714+0000 7f0a1be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a43c27670 0x557a43c35e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:16.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.714+0000 7f0a1b400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a43c27fb0 0x557a43c36360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37576/0 (socket says 172.21.15.1:37576) 2024-09-12T01:30:16.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.714+0000 7f0a1b400640 1 -- 172.21.15.1:0/2023247239 learned_addr learned my addr 172.21.15.1:0/2023247239 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:30:16.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.714+0000 7f0a21200640 1 --2- 172.21.15.1:0/2023247239 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a43c2d040 0x557a43c37370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:16.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.714+0000 7f0a1b400640 1 -- 172.21.15.1:0/2023247239 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a43c2d040 msgr2=0x557a43c37370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:30:16.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.714+0000 7f0a1b400640 1 --2- 172.21.15.1:0/2023247239 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a43c2d040 0x557a43c37370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:16.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.714+0000 7f0a1b400640 1 -- 172.21.15.1:0/2023247239 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a43c27670 msgr2=0x557a43c35e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:30:16.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.714+0000 7f0a1b400640 1 --2- 172.21.15.1:0/2023247239 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a43c27670 0x557a43c35e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:16.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.714+0000 7f0a1b400640 1 -- 172.21.15.1:0/2023247239 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a43c37980 con 0x557a43c27fb0 2024-09-12T01:30:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.715+0000 7f0a1b400640 1 --2- 172.21.15.1:0/2023247239 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a43c27fb0 0x557a43c36360 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0a08011510 tx=0x7f0a08005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac72b5218bdb3e43 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:30:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.715+0000 7f0a06000640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-12T01:30:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.715+0000 7f0a06000640 1 -- 172.21.15.1:0/2023247239 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0801e040 con 0x557a43c27fb0 2024-09-12T01:30:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.715+0000 7f0a26033740 1 -- 172.21.15.1:0/2023247239 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a0c0bbd80 con 0x557a43c27fb0 2024-09-12T01:30:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.715+0000 7f0a06000640 1 -- 172.21.15.1:0/2023247239 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f0a0800f5c0 con 0x557a43c27fb0 2024-09-12T01:30:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.715+0000 7f0a06000640 1 -- 172.21.15.1:0/2023247239 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0a08038050 con 0x557a43c27fb0 2024-09-12T01:30:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.715+0000 7f0a26033740 10 client.15234 fetch_fsmap learned FSMap version 29 2024-09-12T01:30:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.715+0000 7f0a26033740 1 -- 172.21.15.1:0/2023247239 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557a43c38d00 con 0x557a43c27fb0 2024-09-12T01:30:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.715+0000 7f0a06000640 1 -- 172.21.15.1:0/2023247239 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a0c0bbd80 con 0x557a43c27fb0 2024-09-12T01:30:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.716+0000 7f0a06000640 1 -- 172.21.15.1:0/2023247239 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f0a08019700 con 0x557a43c27fb0 2024-09-12T01:30:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.716+0000 7f0a26033740 10 client.15234 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T01:30:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.716+0000 7f0a26033740 10 client.15234 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:30:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.716+0000 7f0a26033740 20 client.15234 populate_metadata read hostname 'smithi001' 2024-09-12T01:30:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.716+0000 7f0a26033740 1 --2- 172.21.15.1:0/2023247239 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x557a43ca81b0 0x557a43caa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.716+0000 7f0a26033740 4 client.15234 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:30:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.716+0000 7f0a26033740 1 -- 172.21.15.1:0/2023247239 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557a43b8fb80 con 0x557a43ca81b0 2024-09-12T01:30:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.716+0000 7f0a1be00640 1 --2- 172.21.15.1:0/2023247239 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x557a43ca81b0 0x557a43caa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.717+0000 7f0a06000640 10 client.15234 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:30:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.717+0000 7f0a1be00640 1 --2- 172.21.15.1:0/2023247239 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x557a43ca81b0 0x557a43caa5a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a83e306a74c91e0 server_cookie=7871c2d907992184 in_seq=0 out_seq=0 2024-09-12T01:30:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.723+0000 7f0a06000640 1 -- 172.21.15.1:0/2023247239 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557a43b8fb80 con 0x557a43ca81b0 2024-09-12T01:30:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.723+0000 7f0a06000640 10 client.15234 handle_command_reply: tid=0 2024-09-12T01:30:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.723+0000 7f0a26033740 1 client.15234 shutdown 2024-09-12T01:30:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.723+0000 7f0a26033740 1 -- 172.21.15.1:0/2023247239 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a43c27fb0 msgr2=0x557a43c36360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.723+0000 7f0a26033740 1 --2- 172.21.15.1:0/2023247239 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a43c27fb0 0x557a43c36360 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0a08011510 tx=0x7f0a08005d90 comp rx=0 tx=0).stop 2024-09-12T01:30:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.723+0000 7f0a26033740 1 -- 172.21.15.1:0/2023247239 shutdown_connections 2024-09-12T01:30:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.723+0000 7f0a26033740 1 --2- 172.21.15.1:0/2023247239 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x557a43ca81b0 0x557a43caa5a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.723+0000 7f0a26033740 1 --2- 172.21.15.1:0/2023247239 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a43c2d040 0x557a43c37370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.723+0000 7f0a26033740 1 --2- 172.21.15.1:0/2023247239 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a43c27fb0 0x557a43c36360 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.723+0000 7f0a26033740 1 --2- 172.21.15.1:0/2023247239 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a43c27670 0x557a43c35e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.723+0000 7f0a26033740 1 -- 172.21.15.1:0/2023247239 >> 172.21.15.1:0/2023247239 conn(0x557a43c22ce0 msgr2=0x557a43c31c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:30:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.723+0000 7f0a06000640 0 client.15234 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:30:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.723+0000 7f0a26033740 1 -- 172.21.15.1:0/2023247239 shutdown_connections 2024-09-12T01:30:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.723+0000 7f0a26033740 1 -- 172.21.15.1:0/2023247239 wait complete. 2024-09-12T01:30:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.724+0000 7f0a26033740 20 client.15234 trim_cache size 0 max 16384 2024-09-12T01:30:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-12T01:30:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-12T01:30:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-12T01:30:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-12T01:30:16.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.724+0000 7f0a21c00640 1 -- 172.21.15.1:0/3644902257 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f0a000a2c80 msgr2=0x7f0a000a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:16.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.724+0000 7f0a21c00640 1 --2- 172.21.15.1:0/3644902257 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f0a000a2c80 0x7f0a000a5140 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0a14005c40 tx=0x7f0a140095c0 comp rx=0 tx=0).stop 2024-09-12T01:30:16.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.724+0000 7f0a21c00640 1 -- 172.21.15.1:0/3644902257 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a1c1019b0 msgr2=0x7f0a1c197d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:16.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.724+0000 7f0a21c00640 1 --2- 172.21.15.1:0/3644902257 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a1c1019b0 0x7f0a1c197d00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c02a860 tx=0x7f0a0c033d90 comp rx=0 tx=0).stop 2024-09-12T01:30:16.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.724+0000 7f0a21c00640 1 -- 172.21.15.1:0/3644902257 shutdown_connections 2024-09-12T01:30:16.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.724+0000 7f0a21c00640 1 --2- 172.21.15.1:0/3644902257 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f0a000a2c80 0x7f0a000a5140 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:16.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.724+0000 7f0a21c00640 1 --2- 172.21.15.1:0/3644902257 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a1c103560 0x7f0a1c19f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:16.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.724+0000 7f0a21c00640 1 --2- 172.21.15.1:0/3644902257 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a1c102ba0 0x7f0a1c198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:16.727 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.724+0000 7f0a21c00640 1 --2- 172.21.15.1:0/3644902257 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a1c1019b0 0x7f0a1c197d00 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:16.727 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.724+0000 7f0a21c00640 1 -- 172.21.15.1:0/3644902257 >> 172.21.15.1:0/3644902257 conn(0x7f0a1c0fd130 msgr2=0x7f0a1c106010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:30:16.727 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.725+0000 7f0a21c00640 1 -- 172.21.15.1:0/3644902257 shutdown_connections 2024-09-12T01:30:16.727 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:16.725+0000 7f0a21c00640 1 -- 172.21.15.1:0/3644902257 wait complete. 2024-09-12T01:30:16.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:30:16 smithi053 ceph-mon[35158]: pgmap v418: 129 pgs: 129 active+clean; 663 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-09-12T01:30:18.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:30:18 smithi104 ceph-mon[35219]: pgmap v419: 129 pgs: 129 active+clean; 663 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-09-12T01:30:18.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:30:18 smithi001 ceph-mon[30387]: pgmap v419: 129 pgs: 129 active+clean; 663 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-09-12T01:30:18.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:30:18 smithi053 ceph-mon[35158]: pgmap v419: 129 pgs: 129 active+clean; 663 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-09-12T01:30:20.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:30:20 smithi104 ceph-mon[35219]: pgmap v420: 129 pgs: 129 active+clean; 663 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-12T01:30:20.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:30:20 smithi001 ceph-mon[30387]: pgmap v420: 129 pgs: 129 active+clean; 663 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-12T01:30:20.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:30:20 smithi053 ceph-mon[35158]: pgmap v420: 129 pgs: 129 active+clean; 663 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-12T01:30:22.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:30:22 smithi001 ceph-mon[30387]: pgmap v421: 129 pgs: 129 active+clean; 663 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-12T01:30:22.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:30:22 smithi053 ceph-mon[35158]: pgmap v421: 129 pgs: 129 active+clean; 663 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-12T01:30:22.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:30:22 smithi104 ceph-mon[35219]: pgmap v421: 129 pgs: 129 active+clean; 663 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-12T01:30:24.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:30:24 smithi001 ceph-mon[30387]: pgmap v422: 129 pgs: 129 active+clean; 673 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-12T01:30:24.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:30:24 smithi053 ceph-mon[35158]: pgmap v422: 129 pgs: 129 active+clean; 673 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-12T01:30:24.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:30:24 smithi104 ceph-mon[35219]: pgmap v422: 129 pgs: 129 active+clean; 673 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-12T01:30:26.627 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:30:26 smithi104 ceph-mon[35219]: pgmap v423: 129 pgs: 129 active+clean; 673 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-12T01:30:26.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:30:26 smithi001 ceph-mon[30387]: pgmap v423: 129 pgs: 129 active+clean; 673 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-12T01:30:26.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:30:26 smithi053 ceph-mon[35158]: pgmap v423: 129 pgs: 129 active+clean; 673 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-12T01:30:28.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:30:28 smithi001 ceph-mon[30387]: pgmap v424: 129 pgs: 129 active+clean; 673 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-12T01:30:28.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:30:28 smithi053 ceph-mon[35158]: pgmap v424: 129 pgs: 129 active+clean; 673 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-12T01:30:28.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:30:28 smithi104 ceph-mon[35219]: pgmap v424: 129 pgs: 129 active+clean; 673 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-12T01:30:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.906+0000 7f8c95200640 1 -- 172.21.15.1:0/3908788069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c90103bb0 msgr2=0x7f8c90103fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.906+0000 7f8c95200640 1 --2- 172.21.15.1:0/3908788069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c90103bb0 0x7f8c90103fb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8c80009900 tx=0x7f8c8002a3f0 comp rx=0 tx=0).stop 2024-09-12T01:30:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.906+0000 7f8c95200640 1 -- 172.21.15.1:0/3908788069 shutdown_connections 2024-09-12T01:30:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.906+0000 7f8c95200640 1 --2- 172.21.15.1:0/3908788069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c90105760 0x7f8c9010bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.906+0000 7f8c95200640 1 --2- 172.21.15.1:0/3908788069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c90104da0 0x7f8c90105220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.906+0000 7f8c95200640 1 --2- 172.21.15.1:0/3908788069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c90103bb0 0x7f8c90103fb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.906+0000 7f8c95200640 1 -- 172.21.15.1:0/3908788069 >> 172.21.15.1:0/3908788069 conn(0x7f8c900ff310 msgr2=0x7f8c90101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:30:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.906+0000 7f8c95200640 1 -- 172.21.15.1:0/3908788069 shutdown_connections 2024-09-12T01:30:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.906+0000 7f8c95200640 1 -- 172.21.15.1:0/3908788069 wait complete. 2024-09-12T01:30:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.907+0000 7f8c95200640 1 Processor -- start 2024-09-12T01:30:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.907+0000 7f8c95200640 1 -- start start 2024-09-12T01:30:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.908+0000 7f8c95200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c90103bb0 0x7f8c90199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.908+0000 7f8c95200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c90104da0 0x7f8c9019a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.908+0000 7f8c95200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c90105760 0x7f8c901a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.908+0000 7f8c95200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8c9010f860 con 0x7f8c90104da0 2024-09-12T01:30:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.908+0000 7f8c95200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8c9010f6e0 con 0x7f8c90103bb0 2024-09-12T01:30:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.908+0000 7f8c95200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8c9010f9e0 con 0x7f8c90105760 2024-09-12T01:30:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.908+0000 7f8c8f400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c90103bb0 0x7f8c90199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.908+0000 7f8c8f400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c90103bb0 0x7f8c90199e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:39798/0 (socket says 172.21.15.1:39798) 2024-09-12T01:30:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.908+0000 7f8c8f400640 1 -- 172.21.15.1:0/2007396717 learned_addr learned my addr 172.21.15.1:0/2007396717 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:30:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.908+0000 7f8c8fe00640 1 --2- 172.21.15.1:0/2007396717 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c90105760 0x7f8c901a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:29.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.908+0000 7f8c8ea00640 1 --2- 172.21.15.1:0/2007396717 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c90104da0 0x7f8c9019a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:29.910 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.909+0000 7f8c8f400640 1 -- 172.21.15.1:0/2007396717 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c90105760 msgr2=0x7f8c901a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:29.910 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.909+0000 7f8c8f400640 1 --2- 172.21.15.1:0/2007396717 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c90105760 0x7f8c901a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:29.910 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.909+0000 7f8c8f400640 1 -- 172.21.15.1:0/2007396717 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c90104da0 msgr2=0x7f8c9019a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:29.910 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.909+0000 7f8c8f400640 1 --2- 172.21.15.1:0/2007396717 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c90104da0 0x7f8c9019a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:29.910 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.909+0000 7f8c8f400640 1 -- 172.21.15.1:0/2007396717 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c901a1a80 con 0x7f8c90103bb0 2024-09-12T01:30:29.910 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.909+0000 7f8c8fe00640 1 --2- 172.21.15.1:0/2007396717 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c90105760 0x7f8c901a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:30:29.910 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.909+0000 7f8c8ea00640 1 --2- 172.21.15.1:0/2007396717 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c90104da0 0x7f8c9019a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:30:29.910 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.909+0000 7f8c8f400640 1 --2- 172.21.15.1:0/2007396717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c90103bb0 0x7f8c90199e60 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8c80031440 tx=0x7f8c80032e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:30:29.910 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.909+0000 7f8c87e00640 1 -- 172.21.15.1:0/2007396717 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8003f030 con 0x7f8c90103bb0 2024-09-12T01:30:29.910 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.909+0000 7f8c95200640 1 -- 172.21.15.1:0/2007396717 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c901a1d10 con 0x7f8c90103bb0 2024-09-12T01:30:29.910 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.909+0000 7f8c87e00640 1 -- 172.21.15.1:0/2007396717 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c80003ce0 con 0x7f8c90103bb0 2024-09-12T01:30:29.910 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.909+0000 7f8c87e00640 1 -- 172.21.15.1:0/2007396717 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c80035070 con 0x7f8c90103bb0 2024-09-12T01:30:29.911 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.910+0000 7f8c95200640 1 -- 172.21.15.1:0/2007396717 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c901a21a0 con 0x7f8c90103bb0 2024-09-12T01:30:29.912 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.913+0000 7f8c997be740 1 Processor -- start 2024-09-12T01:30:29.912 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.913+0000 7f8c997be740 1 -- start start 2024-09-12T01:30:29.913 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.913+0000 7f8c997be740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623833d6260 0x5623833d6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:29.913 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.913+0000 7f8c997be740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5623833d6ba0 0x5623833db660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:29.913 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.913+0000 7f8c997be740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5623833dbc30 0x5623833de0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:29.913 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.913+0000 7f8c997be740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56238333b100 con 0x5623833dbc30 2024-09-12T01:30:29.913 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.913+0000 7f8c997be740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5623832e31b0 con 0x5623833d6ba0 2024-09-12T01:30:29.913 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.913+0000 7f8c997be740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5623833370c0 con 0x5623833d6260 2024-09-12T01:30:29.914 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.914+0000 7f8c8fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623833d6260 0x5623833d6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:29.914 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.914+0000 7f8c8fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623833d6260 0x5623833d6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:60654/0 (socket says 172.21.15.1:60654) 2024-09-12T01:30:29.914 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.914+0000 7f8c8fe00640 1 -- 172.21.15.1:0/3261367719 learned_addr learned my addr 172.21.15.1:0/3261367719 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:30:29.914 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.914+0000 7f8c8ea00640 1 --2- 172.21.15.1:0/3261367719 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5623833dbc30 0x5623833de0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:29.914 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.914+0000 7f8c8fe00640 1 -- 172.21.15.1:0/3261367719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5623833d6ba0 msgr2=0x5623833db660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T01:30:29.914 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.914+0000 7f8c8fe00640 1 --2- 172.21.15.1:0/3261367719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5623833d6ba0 0x5623833db660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:29.914 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.914+0000 7f8c8fe00640 1 -- 172.21.15.1:0/3261367719 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5623833dbc30 msgr2=0x5623833de0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:30:29.915 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.914+0000 7f8c8fe00640 1 --2- 172.21.15.1:0/3261367719 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5623833dbc30 0x5623833de0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:29.915 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.914+0000 7f8c8fe00640 1 -- 172.21.15.1:0/3261367719 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56238304e210 con 0x5623833d6260 2024-09-12T01:30:29.915 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.914+0000 7f8c87e00640 1 -- 172.21.15.1:0/2007396717 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f8c80042030 con 0x7f8c90103bb0 2024-09-12T01:30:29.915 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.914+0000 7f8c8fe00640 1 --2- 172.21.15.1:0/3261367719 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623833d6260 0x5623833d6660 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8c8800f790 tx=0x7f8c8800fc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14754fdaf1c67cf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:30:29.915 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.915+0000 7f8c84c00640 1 -- 172.21.15.1:0/3261367719 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c88004d20 con 0x5623833d6260 2024-09-12T01:30:29.915 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.915+0000 7f8c84c00640 1 -- 172.21.15.1:0/3261367719 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c8800de00 con 0x5623833d6260 2024-09-12T01:30:29.915 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.915+0000 7f8c84c00640 1 -- 172.21.15.1:0/3261367719 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c88010430 con 0x5623833d6260 2024-09-12T01:30:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.915+0000 7f8c997be740 1 -- 172.21.15.1:0/3261367719 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623833d6260 msgr2=0x5623833d6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.915+0000 7f8c997be740 1 --2- 172.21.15.1:0/3261367719 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623833d6260 0x5623833d6660 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8c8800f790 tx=0x7f8c8800fc50 comp rx=0 tx=0).stop 2024-09-12T01:30:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.916+0000 7f8c997be740 1 -- 172.21.15.1:0/3261367719 shutdown_connections 2024-09-12T01:30:29.917 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.916+0000 7f8c997be740 1 --2- 172.21.15.1:0/3261367719 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5623833dbc30 0x5623833de0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:29.917 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.916+0000 7f8c997be740 1 --2- 172.21.15.1:0/3261367719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5623833d6ba0 0x5623833db660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:29.917 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.916+0000 7f8c997be740 1 --2- 172.21.15.1:0/3261367719 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623833d6260 0x5623833d6660 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:29.917 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.916+0000 7f8c997be740 1 -- 172.21.15.1:0/3261367719 >> 172.21.15.1:0/3261367719 conn(0x5623833d18d0 msgr2=0x5623833d1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:30:29.917 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.916+0000 7f8c997be740 1 -- 172.21.15.1:0/3261367719 shutdown_connections 2024-09-12T01:30:29.917 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.916+0000 7f8c997be740 1 -- 172.21.15.1:0/3261367719 wait complete. 2024-09-12T01:30:29.917 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.916+0000 7f8c997be740 1 -- start start 2024-09-12T01:30:29.917 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.917+0000 7f8c997be740 1 Processor -- start 2024-09-12T01:30:29.917 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.917+0000 7f8c87e00640 1 --2- 172.21.15.1:0/2007396717 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f8c740a2b10 0x7f8c740a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:29.917 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.917+0000 7f8c87e00640 1 -- 172.21.15.1:0/2007396717 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f8c800efbc0 con 0x7f8c90103bb0 2024-09-12T01:30:29.918 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.917+0000 7f8c997be740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5623833d6260 0x5623833e8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.917+0000 7f8c997be740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623833d6ba0 0x5623833e9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.917+0000 7f8c997be740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5623833dbc30 0x5623833f0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.917+0000 7f8c997be740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5623833dec00 con 0x5623833dbc30 2024-09-12T01:30:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.917+0000 7f8c997be740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5623832c4090 con 0x5623833d6260 2024-09-12T01:30:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.917+0000 7f8c997be740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5623833ded80 con 0x5623833d6ba0 2024-09-12T01:30:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.917+0000 7f8c8fe00640 1 --2- 172.21.15.1:0/2007396717 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f8c740a2b10 0x7f8c740a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.917+0000 7f8c997be740 10 client.0 _ll_register_callbacks cb 0x562383318f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:30:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.918+0000 7f8c8ea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623833d6ba0 0x5623833e9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.918+0000 7f8c8ea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623833d6ba0 0x5623833e9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:60658/0 (socket says 172.21.15.1:60658) 2024-09-12T01:30:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.918+0000 7f8c8fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5623833dbc30 0x5623833f0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.918+0000 7f8c8f400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5623833d6260 0x5623833e8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.918+0000 7f8c8fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5623833dbc30 0x5623833f0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58230/0 (socket says 172.21.15.1:58230) 2024-09-12T01:30:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.918+0000 7f8c8fe00640 1 -- 172.21.15.1:0/2338726316 learned_addr learned my addr 172.21.15.1:0/2338726316 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:30:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.919+0000 7f8c8fe00640 1 --2- 172.21.15.1:0/2007396717 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f8c740a2b10 0x7f8c740a4fd0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8c88018000 tx=0x7f8c88005a00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:30:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.919+0000 7f8c8ea00640 1 -- 172.21.15.1:0/2338726316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5623833d6260 msgr2=0x5623833e8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:30:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.919+0000 7f8c8ea00640 1 --2- 172.21.15.1:0/2338726316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5623833d6260 0x5623833e8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.919+0000 7f8c8ea00640 1 -- 172.21.15.1:0/2338726316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5623833dbc30 msgr2=0x5623833f0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:30:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.919+0000 7f8c8ea00640 1 --2- 172.21.15.1:0/2338726316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5623833dbc30 0x5623833f0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.919+0000 7f8c8ea00640 1 -- 172.21.15.1:0/2338726316 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56238311a620 con 0x5623833d6ba0 2024-09-12T01:30:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.919+0000 7f8c8fe00640 1 --2- 172.21.15.1:0/2338726316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5623833dbc30 0x5623833f0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:30:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.919+0000 7f8c8ea00640 1 --2- 172.21.15.1:0/2338726316 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623833d6ba0 0x5623833e9280 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8c9019b370 tx=0x7f8c7c0045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=298399fb57d6a16f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:30:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.919+0000 7f8c79600640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-12T01:30:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.920+0000 7f8c79600640 1 -- 172.21.15.1:0/2338726316 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c011d70 con 0x5623833d6ba0 2024-09-12T01:30:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.920+0000 7f8c79600640 1 -- 172.21.15.1:0/2338726316 <== mon.1 v2:172.21.15.104:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f8c7c005020 con 0x5623833d6ba0 2024-09-12T01:30:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.920+0000 7f8c997be740 1 -- 172.21.15.1:0/2338726316 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c8800de00 con 0x5623833d6ba0 2024-09-12T01:30:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.920+0000 7f8c79600640 1 -- 172.21.15.1:0/2338726316 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8c7c005220 con 0x5623833d6ba0 2024-09-12T01:30:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.920+0000 7f8c997be740 10 client.25267 fetch_fsmap learned FSMap version 29 2024-09-12T01:30:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.920+0000 7f8c997be740 1 -- 172.21.15.1:0/2338726316 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5623833f3110 con 0x5623833d6ba0 2024-09-12T01:30:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.920+0000 7f8c79600640 1 -- 172.21.15.1:0/2338726316 <== mon.1 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c8800de00 con 0x5623833d6ba0 2024-09-12T01:30:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.920+0000 7f8c79600640 1 -- 172.21.15.1:0/2338726316 <== mon.1 v2:172.21.15.104:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f8c7c01b610 con 0x5623833d6ba0 2024-09-12T01:30:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.921+0000 7f8c997be740 10 client.25267 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T01:30:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.921+0000 7f8c997be740 10 client.25267 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:30:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.921+0000 7f8c997be740 20 client.25267 populate_metadata read hostname 'smithi001' 2024-09-12T01:30:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.921+0000 7f8c997be740 1 --2- 172.21.15.1:0/2338726316 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5623833f4830 0x5623833f6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.921+0000 7f8c997be740 4 client.25267 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:30:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.921+0000 7f8c997be740 1 -- 172.21.15.1:0/2338726316 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562383348830 con 0x5623833f4830 2024-09-12T01:30:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.921+0000 7f8c8f400640 1 --2- 172.21.15.1:0/2338726316 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5623833f4830 0x5623833f6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.922+0000 7f8c8f400640 1 --2- 172.21.15.1:0/2338726316 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5623833f4830 0x5623833f6c20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1c27c1ae7bc7455 server_cookie=7d444d7aa9661f4b in_seq=0 out_seq=0 2024-09-12T01:30:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.922+0000 7f8c79600640 10 client.25267 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:30:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.923+0000 7f8c79600640 1 -- 172.21.15.1:0/2338726316 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562383348830 con 0x5623833f4830 2024-09-12T01:30:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.923+0000 7f8c79600640 10 client.25267 handle_command_reply: tid=0 2024-09-12T01:30:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.923+0000 7f8c997be740 1 client.25267 shutdown 2024-09-12T01:30:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.923+0000 7f8c997be740 1 -- 172.21.15.1:0/2338726316 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623833d6ba0 msgr2=0x5623833e9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.923+0000 7f8c997be740 1 --2- 172.21.15.1:0/2338726316 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623833d6ba0 0x5623833e9280 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8c9019b370 tx=0x7f8c7c0045e0 comp rx=0 tx=0).stop 2024-09-12T01:30:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.923+0000 7f8c997be740 1 -- 172.21.15.1:0/2338726316 shutdown_connections 2024-09-12T01:30:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.923+0000 7f8c997be740 1 --2- 172.21.15.1:0/2338726316 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5623833f4830 0x5623833f6c20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.923+0000 7f8c997be740 1 --2- 172.21.15.1:0/2338726316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5623833dbc30 0x5623833f0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.923+0000 7f8c997be740 1 --2- 172.21.15.1:0/2338726316 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623833d6ba0 0x5623833e9280 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.923+0000 7f8c997be740 1 --2- 172.21.15.1:0/2338726316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5623833d6260 0x5623833e8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.923+0000 7f8c997be740 1 -- 172.21.15.1:0/2338726316 >> 172.21.15.1:0/2338726316 conn(0x5623833d18d0 msgr2=0x5623833d30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:30:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.923+0000 7f8c79600640 0 client.25267 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:30:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.923+0000 7f8c997be740 1 -- 172.21.15.1:0/2338726316 shutdown_connections 2024-09-12T01:30:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.923+0000 7f8c997be740 1 -- 172.21.15.1:0/2338726316 wait complete. 2024-09-12T01:30:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.924+0000 7f8c997be740 20 client.25267 trim_cache size 0 max 16384 2024-09-12T01:30:29.953 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.954+0000 7f8c997be740 1 Processor -- start 2024-09-12T01:30:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.954+0000 7f8c997be740 1 -- start start 2024-09-12T01:30:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.954+0000 7f8c997be740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5623833d6260 0x5623833d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.954+0000 7f8c997be740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623833d6ba0 0x5623833f0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.954+0000 7f8c997be740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5623833dbc30 0x5623833d21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.954+0000 7f8c997be740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8c901a8350 con 0x5623833dbc30 2024-09-12T01:30:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.954+0000 7f8c997be740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8c901a9e90 con 0x5623833d6260 2024-09-12T01:30:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.954+0000 7f8c997be740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8c901a8a20 con 0x5623833d6ba0 2024-09-12T01:30:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.955+0000 7f8c8f400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5623833d6260 0x5623833d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.955+0000 7f8c8f400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5623833d6260 0x5623833d1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:39820/0 (socket says 172.21.15.1:39820) 2024-09-12T01:30:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.955+0000 7f8c8f400640 1 -- 172.21.15.1:0/2562169756 learned_addr learned my addr 172.21.15.1:0/2562169756 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:30:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.955+0000 7f8c8fe00640 1 --2- 172.21.15.1:0/2562169756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5623833dbc30 0x5623833d21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.955+0000 7f8c8ea00640 1 --2- 172.21.15.1:0/2562169756 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623833d6ba0 0x5623833f0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.955+0000 7f8c8f400640 1 -- 172.21.15.1:0/2562169756 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623833d6ba0 msgr2=0x5623833f0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:30:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.955+0000 7f8c8f400640 1 --2- 172.21.15.1:0/2562169756 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623833d6ba0 0x5623833f0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.955+0000 7f8c8f400640 1 -- 172.21.15.1:0/2562169756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5623833dbc30 msgr2=0x5623833d21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:30:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.955+0000 7f8c8f400640 1 --2- 172.21.15.1:0/2562169756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5623833dbc30 0x5623833d21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.955+0000 7f8c8f400640 1 -- 172.21.15.1:0/2562169756 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623833d16f0 con 0x5623833d6260 2024-09-12T01:30:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.955+0000 7f8c8ea00640 1 --2- 172.21.15.1:0/2562169756 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623833d6ba0 0x5623833f0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:30:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.955+0000 7f8c8f400640 1 --2- 172.21.15.1:0/2562169756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5623833d6260 0x5623833d1cb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8c800b8390 tx=0x7f8c800dc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1496e4a52adcaac server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:30:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.956+0000 7f8c7be00640 1 -- 172.21.15.1:0/2562169756 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c80049040 con 0x5623833d6260 2024-09-12T01:30:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.956+0000 7f8c7be00640 1 -- 172.21.15.1:0/2562169756 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c800bbf30 con 0x5623833d6260 2024-09-12T01:30:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.956+0000 7f8c7be00640 1 -- 172.21.15.1:0/2562169756 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c800ed430 con 0x5623833d6260 2024-09-12T01:30:29.957 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.957+0000 7f8c997be740 1 -- 172.21.15.1:0/2562169756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5623833d6260 msgr2=0x5623833d1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:29.957 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.957+0000 7f8c997be740 1 --2- 172.21.15.1:0/2562169756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5623833d6260 0x5623833d1cb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8c800b8390 tx=0x7f8c800dc040 comp rx=0 tx=0).stop 2024-09-12T01:30:29.957 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.957+0000 7f8c997be740 1 -- 172.21.15.1:0/2562169756 shutdown_connections 2024-09-12T01:30:29.957 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.957+0000 7f8c997be740 1 --2- 172.21.15.1:0/2562169756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5623833dbc30 0x5623833d21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:29.957 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.957+0000 7f8c997be740 1 --2- 172.21.15.1:0/2562169756 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623833d6ba0 0x5623833f0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:29.957 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.957+0000 7f8c997be740 1 --2- 172.21.15.1:0/2562169756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5623833d6260 0x5623833d1cb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:29.957 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.957+0000 7f8c997be740 1 -- 172.21.15.1:0/2562169756 >> 172.21.15.1:0/2562169756 conn(0x5623833d18d0 msgr2=0x5623833d7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:30:29.957 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.957+0000 7f8c997be740 1 -- 172.21.15.1:0/2562169756 shutdown_connections 2024-09-12T01:30:29.958 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.957+0000 7f8c997be740 1 -- 172.21.15.1:0/2562169756 wait complete. 2024-09-12T01:30:29.958 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.957+0000 7f8c997be740 1 -- start start 2024-09-12T01:30:29.958 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.958+0000 7f8c997be740 1 Processor -- start 2024-09-12T01:30:29.958 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.958+0000 7f8c997be740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5623833d6260 0x5623833ed200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:29.959 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.958+0000 7f8c997be740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5623833d6ba0 0x5623833ed740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:29.959 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.958+0000 7f8c997be740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623833dbc30 0x56238345d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:29.959 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.958+0000 7f8c997be740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8c901a97c0 con 0x5623833d6ba0 2024-09-12T01:30:29.959 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.958+0000 7f8c997be740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8c901a90f0 con 0x5623833d6260 2024-09-12T01:30:29.959 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.958+0000 7f8c997be740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8c901a6330 con 0x5623833dbc30 2024-09-12T01:30:29.959 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.958+0000 7f8c997be740 10 client.0 _ll_register_callbacks cb 0x7f8c7c0040a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:30:29.959 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.959+0000 7f8c8f400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5623833d6260 0x5623833ed200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:29.959 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.959+0000 7f8c8f400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5623833d6260 0x5623833ed200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:39830/0 (socket says 172.21.15.1:39830) 2024-09-12T01:30:29.959 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.959+0000 7f8c8f400640 1 -- 172.21.15.1:0/4103979804 learned_addr learned my addr 172.21.15.1:0/4103979804 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:30:29.959 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.959+0000 7f8c8ea00640 1 --2- 172.21.15.1:0/4103979804 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5623833d6ba0 0x5623833ed740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:29.959 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.959+0000 7f8c8fe00640 1 --2- 172.21.15.1:0/4103979804 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623833dbc30 0x56238345d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:29.961 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.959+0000 7f8c8f400640 1 -- 172.21.15.1:0/4103979804 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623833dbc30 msgr2=0x56238345d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:30:29.961 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.959+0000 7f8c8f400640 1 --2- 172.21.15.1:0/4103979804 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623833dbc30 0x56238345d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:29.961 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.959+0000 7f8c8f400640 1 -- 172.21.15.1:0/4103979804 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5623833d6ba0 msgr2=0x5623833ed740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:30:29.961 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.959+0000 7f8c8f400640 1 --2- 172.21.15.1:0/4103979804 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5623833d6ba0 0x5623833ed740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:29.961 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.959+0000 7f8c8f400640 1 -- 172.21.15.1:0/4103979804 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56238345da10 con 0x5623833d6260 2024-09-12T01:30:29.961 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.959+0000 7f8c8ea00640 1 --2- 172.21.15.1:0/4103979804 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5623833d6ba0 0x5623833ed740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:30:29.961 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.960+0000 7f8c8f400640 1 --2- 172.21.15.1:0/4103979804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5623833d6260 0x5623833ed200 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8c800bbec0 tx=0x7f8c800092a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94034a78781193b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:30:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.960+0000 7f8c79600640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-12T01:30:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.960+0000 7f8c79600640 1 -- 172.21.15.1:0/4103979804 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c80049070 con 0x5623833d6260 2024-09-12T01:30:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.960+0000 7f8c79600640 1 -- 172.21.15.1:0/4103979804 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f8c800ed430 con 0x5623833d6260 2024-09-12T01:30:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.960+0000 7f8c997be740 1 -- 172.21.15.1:0/4103979804 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c800bbf30 con 0x5623833d6260 2024-09-12T01:30:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.960+0000 7f8c79600640 1 -- 172.21.15.1:0/4103979804 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8c800c4ba0 con 0x5623833d6260 2024-09-12T01:30:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.960+0000 7f8c997be740 10 client.25280 fetch_fsmap learned FSMap version 29 2024-09-12T01:30:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.961+0000 7f8c997be740 1 -- 172.21.15.1:0/4103979804 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56238345ecf0 con 0x5623833d6260 2024-09-12T01:30:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.961+0000 7f8c79600640 1 -- 172.21.15.1:0/4103979804 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c800bbf30 con 0x5623833d6260 2024-09-12T01:30:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.961+0000 7f8c79600640 1 -- 172.21.15.1:0/4103979804 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f8c800a2030 con 0x5623833d6260 2024-09-12T01:30:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.961+0000 7f8c997be740 10 client.25280 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T01:30:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.961+0000 7f8c997be740 10 client.25280 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:30:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.961+0000 7f8c997be740 20 client.25280 populate_metadata read hostname 'smithi001' 2024-09-12T01:30:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.961+0000 7f8c997be740 1 --2- 172.21.15.1:0/4103979804 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x56238345f670 0x562383461a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.961+0000 7f8c997be740 4 client.25280 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:30:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.961+0000 7f8c997be740 1 -- 172.21.15.1:0/4103979804 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562383342b80 con 0x56238345f670 2024-09-12T01:30:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.961+0000 7f8c8ea00640 1 --2- 172.21.15.1:0/4103979804 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x56238345f670 0x562383461a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.962+0000 7f8c8ea00640 1 --2- 172.21.15.1:0/4103979804 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x56238345f670 0x562383461a90 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e851c49cff66920 server_cookie=66537c4c4cbe832c in_seq=0 out_seq=0 2024-09-12T01:30:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.962+0000 7f8c79600640 10 client.25280 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:30:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.970+0000 7f8c79600640 1 -- 172.21.15.1:0/4103979804 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562383342b80 con 0x56238345f670 2024-09-12T01:30:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.970+0000 7f8c79600640 10 client.25280 handle_command_reply: tid=0 2024-09-12T01:30:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.970+0000 7f8c997be740 1 client.25280 shutdown 2024-09-12T01:30:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.971+0000 7f8c997be740 1 -- 172.21.15.1:0/4103979804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5623833d6260 msgr2=0x5623833ed200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.971+0000 7f8c997be740 1 --2- 172.21.15.1:0/4103979804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5623833d6260 0x5623833ed200 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8c800bbec0 tx=0x7f8c800092a0 comp rx=0 tx=0).stop 2024-09-12T01:30:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.971+0000 7f8c997be740 1 -- 172.21.15.1:0/4103979804 shutdown_connections 2024-09-12T01:30:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.971+0000 7f8c997be740 1 --2- 172.21.15.1:0/4103979804 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x56238345f670 0x562383461a90 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.971+0000 7f8c997be740 1 --2- 172.21.15.1:0/4103979804 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623833dbc30 0x56238345d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.971+0000 7f8c997be740 1 --2- 172.21.15.1:0/4103979804 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5623833d6ba0 0x5623833ed740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.971+0000 7f8c997be740 1 --2- 172.21.15.1:0/4103979804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5623833d6260 0x5623833ed200 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.971+0000 7f8c997be740 1 -- 172.21.15.1:0/4103979804 >> 172.21.15.1:0/4103979804 conn(0x5623833d18d0 msgr2=0x5623833d01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:30:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.971+0000 7f8c79600640 0 client.25280 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:30:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.971+0000 7f8c997be740 1 -- 172.21.15.1:0/4103979804 shutdown_connections 2024-09-12T01:30:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.971+0000 7f8c997be740 1 -- 172.21.15.1:0/4103979804 wait complete. 2024-09-12T01:30:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.971+0000 7f8c997be740 20 client.25280 trim_cache size 0 max 16384 2024-09-12T01:30:29.972 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-12T01:30:29.972 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-12T01:30:29.972 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-12T01:30:29.972 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-12T01:30:29.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.972+0000 7f8c95200640 1 -- 172.21.15.1:0/2007396717 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f8c740a2b10 msgr2=0x7f8c740a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:29.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.972+0000 7f8c95200640 1 --2- 172.21.15.1:0/2007396717 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f8c740a2b10 0x7f8c740a4fd0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8c88018000 tx=0x7f8c88005a00 comp rx=0 tx=0).stop 2024-09-12T01:30:29.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.972+0000 7f8c95200640 1 -- 172.21.15.1:0/2007396717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c90103bb0 msgr2=0x7f8c90199e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:29.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.972+0000 7f8c95200640 1 --2- 172.21.15.1:0/2007396717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c90103bb0 0x7f8c90199e60 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8c80031440 tx=0x7f8c80032e20 comp rx=0 tx=0).stop 2024-09-12T01:30:29.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.972+0000 7f8c95200640 1 -- 172.21.15.1:0/2007396717 shutdown_connections 2024-09-12T01:30:29.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.972+0000 7f8c95200640 1 --2- 172.21.15.1:0/2007396717 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f8c740a2b10 0x7f8c740a4fd0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:29.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.972+0000 7f8c95200640 1 --2- 172.21.15.1:0/2007396717 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c90105760 0x7f8c901a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:29.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.972+0000 7f8c95200640 1 --2- 172.21.15.1:0/2007396717 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c90104da0 0x7f8c9019a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:29.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.972+0000 7f8c95200640 1 --2- 172.21.15.1:0/2007396717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c90103bb0 0x7f8c90199e60 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:29.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.972+0000 7f8c95200640 1 -- 172.21.15.1:0/2007396717 >> 172.21.15.1:0/2007396717 conn(0x7f8c900ff310 msgr2=0x7f8c901080a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:30:29.974 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.973+0000 7f8c95200640 1 -- 172.21.15.1:0/2007396717 shutdown_connections 2024-09-12T01:30:29.974 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:29.973+0000 7f8c95200640 1 -- 172.21.15.1:0/2007396717 wait complete. 2024-09-12T01:30:30.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:30:30 smithi001 ceph-mon[30387]: pgmap v425: 129 pgs: 129 active+clean; 673 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-12T01:30:30.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:30:30 smithi053 ceph-mon[35158]: pgmap v425: 129 pgs: 129 active+clean; 673 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-12T01:30:30.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:30:30 smithi104 ceph-mon[35219]: pgmap v425: 129 pgs: 129 active+clean; 673 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-12T01:30:32.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:30:32 smithi001 ceph-mon[30387]: pgmap v426: 129 pgs: 129 active+clean; 673 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-09-12T01:30:32.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:30:32 smithi053 ceph-mon[35158]: pgmap v426: 129 pgs: 129 active+clean; 673 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-09-12T01:30:32.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:30:32 smithi104 ceph-mon[35219]: pgmap v426: 129 pgs: 129 active+clean; 673 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-09-12T01:30:34.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:30:34 smithi001 ceph-mon[30387]: pgmap v427: 129 pgs: 129 active+clean; 674 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-12T01:30:34.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:30:34 smithi053 ceph-mon[35158]: pgmap v427: 129 pgs: 129 active+clean; 674 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-12T01:30:34.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:30:34 smithi104 ceph-mon[35219]: pgmap v427: 129 pgs: 129 active+clean; 674 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-12T01:30:36.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:30:36 smithi001 ceph-mon[30387]: pgmap v428: 129 pgs: 129 active+clean; 674 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-12T01:30:36.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:30:36 smithi053 ceph-mon[35158]: pgmap v428: 129 pgs: 129 active+clean; 674 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-12T01:30:36.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:30:36 smithi104 ceph-mon[35219]: pgmap v428: 129 pgs: 129 active+clean; 674 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-12T01:30:38.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:30:38 smithi001 ceph-mon[30387]: pgmap v429: 129 pgs: 129 active+clean; 674 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-12T01:30:38.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:30:38 smithi053 ceph-mon[35158]: pgmap v429: 129 pgs: 129 active+clean; 674 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-12T01:30:38.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:30:38 smithi104 ceph-mon[35219]: pgmap v429: 129 pgs: 129 active+clean; 674 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-12T01:30:40.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:30:40 smithi001 ceph-mon[30387]: pgmap v430: 129 pgs: 129 active+clean; 674 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-12T01:30:40.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:30:40 smithi053 ceph-mon[35158]: pgmap v430: 129 pgs: 129 active+clean; 674 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-12T01:30:40.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:30:40 smithi104 ceph-mon[35219]: pgmap v430: 129 pgs: 129 active+clean; 674 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-12T01:30:42.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:30:42 smithi001 ceph-mon[30387]: pgmap v431: 129 pgs: 129 active+clean; 674 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-12T01:30:42.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:30:42 smithi053 ceph-mon[35158]: pgmap v431: 129 pgs: 129 active+clean; 674 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-12T01:30:42.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:30:42 smithi104 ceph-mon[35219]: pgmap v431: 129 pgs: 129 active+clean; 674 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-12T01:30:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.164+0000 7fd3a4e00640 1 -- 172.21.15.1:0/798252124 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3a01019b0 msgr2=0x7fd3a0101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.164+0000 7fd3a4e00640 1 --2- 172.21.15.1:0/798252124 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3a01019b0 0x7fd3a0101db0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd390009900 tx=0x7fd39002a3f0 comp rx=0 tx=0).stop 2024-09-12T01:30:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.164+0000 7fd3a4e00640 1 -- 172.21.15.1:0/798252124 shutdown_connections 2024-09-12T01:30:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.164+0000 7fd3a4e00640 1 --2- 172.21.15.1:0/798252124 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3a0103560 0x7fd3a0109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.164+0000 7fd3a4e00640 1 --2- 172.21.15.1:0/798252124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3a0102ba0 0x7fd3a0103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.164+0000 7fd3a4e00640 1 --2- 172.21.15.1:0/798252124 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3a01019b0 0x7fd3a0101db0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.164+0000 7fd3a4e00640 1 -- 172.21.15.1:0/798252124 >> 172.21.15.1:0/798252124 conn(0x7fd3a00fd130 msgr2=0x7fd3a00ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:30:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.164+0000 7fd3a4e00640 1 -- 172.21.15.1:0/798252124 shutdown_connections 2024-09-12T01:30:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.165+0000 7fd3a4e00640 1 -- 172.21.15.1:0/798252124 wait complete. 2024-09-12T01:30:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.166+0000 7fd3a4e00640 1 Processor -- start 2024-09-12T01:30:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.166+0000 7fd3a4e00640 1 -- start start 2024-09-12T01:30:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.167+0000 7fd3a4e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3a01019b0 0x7fd3a019e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.167+0000 7fd3a4e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3a0102ba0 0x7fd3a019e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.167+0000 7fd3a4e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3a0103560 0x7fd3a01a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.167+0000 7fd3a4e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a010b4b0 con 0x7fd3a0102ba0 2024-09-12T01:30:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.167+0000 7fd3a4e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a010b330 con 0x7fd3a0103560 2024-09-12T01:30:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.167+0000 7fd3a4e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a010b630 con 0x7fd3a01019b0 2024-09-12T01:30:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.167+0000 7fd39ea00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3a0102ba0 0x7fd3a019e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.167+0000 7fd39f400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3a01019b0 0x7fd3a019e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.167+0000 7fd39ea00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3a0102ba0 0x7fd3a019e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53906/0 (socket says 172.21.15.1:53906) 2024-09-12T01:30:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.167+0000 7fd39ea00640 1 -- 172.21.15.1:0/174104346 learned_addr learned my addr 172.21.15.1:0/174104346 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:30:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.167+0000 7fd39fe00640 1 --2- 172.21.15.1:0/174104346 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3a0103560 0x7fd3a01a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.168+0000 7fd39f400640 1 -- 172.21.15.1:0/174104346 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3a0103560 msgr2=0x7fd3a01a5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.168+0000 7fd39f400640 1 --2- 172.21.15.1:0/174104346 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3a0103560 0x7fd3a01a5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.168+0000 7fd39f400640 1 -- 172.21.15.1:0/174104346 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3a0102ba0 msgr2=0x7fd3a019e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.168+0000 7fd39f400640 1 --2- 172.21.15.1:0/174104346 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3a0102ba0 0x7fd3a019e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.168+0000 7fd39f400640 1 -- 172.21.15.1:0/174104346 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a01a5ec0 con 0x7fd3a01019b0 2024-09-12T01:30:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.168+0000 7fd39f400640 1 --2- 172.21.15.1:0/174104346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3a01019b0 0x7fd3a019e2a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd390009230 tx=0x7fd390032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:30:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.168+0000 7fd397e00640 1 -- 172.21.15.1:0/174104346 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39003f030 con 0x7fd3a01019b0 2024-09-12T01:30:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.169+0000 7fd397e00640 1 -- 172.21.15.1:0/174104346 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd390003ce0 con 0x7fd3a01019b0 2024-09-12T01:30:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.169+0000 7fd3a4e00640 1 -- 172.21.15.1:0/174104346 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3a01a6150 con 0x7fd3a01019b0 2024-09-12T01:30:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.169+0000 7fd397e00640 1 -- 172.21.15.1:0/174104346 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd390035070 con 0x7fd3a01019b0 2024-09-12T01:30:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.169+0000 7fd3a4e00640 1 -- 172.21.15.1:0/174104346 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3a01a6660 con 0x7fd3a01019b0 2024-09-12T01:30:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.173+0000 7fd3a91e6740 1 Processor -- start 2024-09-12T01:30:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.173+0000 7fd3a91e6740 1 -- start start 2024-09-12T01:30:43.175 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.174+0000 7fd3a91e6740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559811f2c260 0x559811f2c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:43.175 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.174+0000 7fd3a91e6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559811f2cba0 0x559811f31660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:43.175 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.174+0000 7fd3a91e6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559811f31c30 0x559811f340f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:43.175 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.174+0000 7fd3a91e6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559811e91100 con 0x559811f31c30 2024-09-12T01:30:43.175 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.174+0000 7fd3a91e6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x559811e391b0 con 0x559811f2cba0 2024-09-12T01:30:43.175 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.174+0000 7fd3a91e6740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559811e8d0c0 con 0x559811f2c260 2024-09-12T01:30:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.174+0000 7fd397e00640 1 -- 172.21.15.1:0/174104346 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fd390042030 con 0x7fd3a01019b0 2024-09-12T01:30:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.174+0000 7fd39fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559811f2c260 0x559811f2c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.174+0000 7fd39ea00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559811f31c30 0x559811f340f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.174+0000 7fd39fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559811f2c260 0x559811f2c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:48966/0 (socket says 172.21.15.1:48966) 2024-09-12T01:30:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.174+0000 7fd39ea00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559811f31c30 0x559811f340f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53922/0 (socket says 172.21.15.1:53922) 2024-09-12T01:30:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.174+0000 7fd39fe00640 1 -- 172.21.15.1:0/57718101 learned_addr learned my addr 172.21.15.1:0/57718101 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:30:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.175+0000 7fd39f400640 1 --2- 172.21.15.1:0/57718101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559811f2cba0 0x559811f31660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.175+0000 7fd39fe00640 1 -- 172.21.15.1:0/57718101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559811f2cba0 msgr2=0x559811f31660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:30:43.183 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.175+0000 7fd39fe00640 1 --2- 172.21.15.1:0/57718101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559811f2cba0 0x559811f31660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:43.184 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.175+0000 7fd39fe00640 1 -- 172.21.15.1:0/57718101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559811f31c30 msgr2=0x559811f340f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:30:43.184 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.175+0000 7fd39fe00640 1 --2- 172.21.15.1:0/57718101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559811f31c30 0x559811f340f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:43.184 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.175+0000 7fd39fe00640 1 -- 172.21.15.1:0/57718101 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559811ba4210 con 0x559811f2c260 2024-09-12T01:30:43.184 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.175+0000 7fd39ea00640 1 --2- 172.21.15.1:0/57718101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559811f31c30 0x559811f340f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:30:43.184 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.175+0000 7fd39f400640 1 --2- 172.21.15.1:0/57718101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559811f2cba0 0x559811f31660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:30:43.185 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.176+0000 7fd39fe00640 1 --2- 172.21.15.1:0/57718101 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559811f2c260 0x559811f2c660 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd398009fc0 tx=0x7fd398009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4e4d19a616a737f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:30:43.185 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.176+0000 7fd394c00640 1 -- 172.21.15.1:0/57718101 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd398004b20 con 0x559811f2c260 2024-09-12T01:30:43.185 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.176+0000 7fd394c00640 1 -- 172.21.15.1:0/57718101 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd398003e50 con 0x559811f2c260 2024-09-12T01:30:43.185 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.176+0000 7fd394c00640 1 -- 172.21.15.1:0/57718101 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd398005020 con 0x559811f2c260 2024-09-12T01:30:43.185 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.176+0000 7fd397e00640 1 --2- 172.21.15.1:0/174104346 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fd3840a2c80 0x7fd3840a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:43.185 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.177+0000 7fd397e00640 1 -- 172.21.15.1:0/174104346 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fd3900ef920 con 0x7fd3a01019b0 2024-09-12T01:30:43.186 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.177+0000 7fd39f400640 1 --2- 172.21.15.1:0/174104346 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fd3840a2c80 0x7fd3840a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:43.186 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.177+0000 7fd3a91e6740 1 -- 172.21.15.1:0/57718101 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559811f2c260 msgr2=0x559811f2c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:43.186 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.177+0000 7fd3a91e6740 1 --2- 172.21.15.1:0/57718101 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559811f2c260 0x559811f2c660 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd398009fc0 tx=0x7fd398009500 comp rx=0 tx=0).stop 2024-09-12T01:30:43.186 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.177+0000 7fd3a91e6740 1 -- 172.21.15.1:0/57718101 shutdown_connections 2024-09-12T01:30:43.186 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.177+0000 7fd3a91e6740 1 --2- 172.21.15.1:0/57718101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559811f31c30 0x559811f340f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:43.187 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.177+0000 7fd3a91e6740 1 --2- 172.21.15.1:0/57718101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559811f2cba0 0x559811f31660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:43.187 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.177+0000 7fd3a91e6740 1 --2- 172.21.15.1:0/57718101 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559811f2c260 0x559811f2c660 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:43.187 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.177+0000 7fd3a91e6740 1 -- 172.21.15.1:0/57718101 >> 172.21.15.1:0/57718101 conn(0x559811f278d0 msgr2=0x559811f27cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:30:43.187 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.178+0000 7fd39f400640 1 --2- 172.21.15.1:0/174104346 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fd3840a2c80 0x7fd3840a5140 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x559811f2d5d0 tx=0x7fd390009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:30:43.187 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.178+0000 7fd3a91e6740 1 -- 172.21.15.1:0/57718101 shutdown_connections 2024-09-12T01:30:43.187 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.178+0000 7fd3a91e6740 1 -- 172.21.15.1:0/57718101 wait complete. 2024-09-12T01:30:43.187 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.179+0000 7fd3a91e6740 1 -- start start 2024-09-12T01:30:43.188 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.179+0000 7fd3a91e6740 1 Processor -- start 2024-09-12T01:30:43.188 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.180+0000 7fd3a91e6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559811f2c260 0x559811f36440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:43.188 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.180+0000 7fd3a91e6740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559811f2cba0 0x559811f36980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:43.188 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.180+0000 7fd3a91e6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559811f31c30 0x559811f3d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:43.188 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.180+0000 7fd3a91e6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559811f34c00 con 0x559811f31c30 2024-09-12T01:30:43.188 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.180+0000 7fd3a91e6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x559811e1a090 con 0x559811f2c260 2024-09-12T01:30:43.188 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.180+0000 7fd3a91e6740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559811f34d80 con 0x559811f2cba0 2024-09-12T01:30:43.189 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.180+0000 7fd3a91e6740 10 client.0 _ll_register_callbacks cb 0x559811e6ef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:30:43.189 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.180+0000 7fd39fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559811f2c260 0x559811f36440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:43.189 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.180+0000 7fd39ea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559811f2cba0 0x559811f36980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:43.189 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.180+0000 7fd39ea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559811f2cba0 0x559811f36980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:48980/0 (socket says 172.21.15.1:48980) 2024-09-12T01:30:43.189 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.181+0000 7fd39ea00640 1 -- 172.21.15.1:0/3702510834 learned_addr learned my addr 172.21.15.1:0/3702510834 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:30:43.189 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.181+0000 7fd39fe00640 1 --2- 172.21.15.1:0/3702510834 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559811f31c30 0x559811f3d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.181+0000 7fd39fe00640 1 -- 172.21.15.1:0/3702510834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559811f2cba0 msgr2=0x559811f36980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:30:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.181+0000 7fd39fe00640 1 --2- 172.21.15.1:0/3702510834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559811f2cba0 0x559811f36980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.181+0000 7fd39fe00640 1 -- 172.21.15.1:0/3702510834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559811f2c260 msgr2=0x559811f36440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:30:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.181+0000 7fd39fe00640 1 --2- 172.21.15.1:0/3702510834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559811f2c260 0x559811f36440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.181+0000 7fd39fe00640 1 -- 172.21.15.1:0/3702510834 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559811c70620 con 0x559811f31c30 2024-09-12T01:30:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.182+0000 7fd39fe00640 1 --2- 172.21.15.1:0/3702510834 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559811f31c30 0x559811f3d9c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd398009f90 tx=0x7fd398005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe4a865ba46488 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:30:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.182+0000 7fd389600640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-12T01:30:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.182+0000 7fd389600640 1 -- 172.21.15.1:0/3702510834 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd398004a40 con 0x559811f31c30 2024-09-12T01:30:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.182+0000 7fd3a91e6740 1 -- 172.21.15.1:0/3702510834 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd398003e50 con 0x559811f31c30 2024-09-12T01:30:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.182+0000 7fd389600640 1 -- 172.21.15.1:0/3702510834 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fd398004be0 con 0x559811f31c30 2024-09-12T01:30:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.183+0000 7fd389600640 1 -- 172.21.15.1:0/3702510834 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd398004de0 con 0x559811f31c30 2024-09-12T01:30:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.183+0000 7fd3a91e6740 10 client.15264 fetch_fsmap learned FSMap version 29 2024-09-12T01:30:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.183+0000 7fd3a91e6740 1 -- 172.21.15.1:0/3702510834 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559811f40770 con 0x559811f31c30 2024-09-12T01:30:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.183+0000 7fd389600640 1 -- 172.21.15.1:0/3702510834 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd398003e50 con 0x559811f31c30 2024-09-12T01:30:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.184+0000 7fd389600640 1 -- 172.21.15.1:0/3702510834 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fd3980194a0 con 0x559811f31c30 2024-09-12T01:30:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.184+0000 7fd3a91e6740 10 client.15264 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T01:30:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.184+0000 7fd3a91e6740 10 client.15264 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:30:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.184+0000 7fd3a91e6740 20 client.15264 populate_metadata read hostname 'smithi001' 2024-09-12T01:30:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.184+0000 7fd3a91e6740 1 --2- 172.21.15.1:0/3702510834 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x559811f41b80 0x559811f43f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.184+0000 7fd3a91e6740 4 client.15264 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:30:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.184+0000 7fd3a91e6740 1 -- 172.21.15.1:0/3702510834 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559811e9e830 con 0x559811f41b80 2024-09-12T01:30:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.185+0000 7fd39ea00640 1 --2- 172.21.15.1:0/3702510834 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x559811f41b80 0x559811f43f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.185+0000 7fd39ea00640 1 --2- 172.21.15.1:0/3702510834 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x559811f41b80 0x559811f43f70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3eb6d22faea4174 server_cookie=adb80f49e70a08c6 in_seq=0 out_seq=0 2024-09-12T01:30:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.185+0000 7fd389600640 10 client.15264 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:30:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.186+0000 7fd389600640 1 -- 172.21.15.1:0/3702510834 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559811e9e830 con 0x559811f41b80 2024-09-12T01:30:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.186+0000 7fd389600640 10 client.15264 handle_command_reply: tid=0 2024-09-12T01:30:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.186+0000 7fd3a91e6740 1 client.15264 shutdown 2024-09-12T01:30:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.187+0000 7fd3a91e6740 1 -- 172.21.15.1:0/3702510834 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559811f31c30 msgr2=0x559811f3d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.187+0000 7fd3a91e6740 1 --2- 172.21.15.1:0/3702510834 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559811f31c30 0x559811f3d9c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd398009f90 tx=0x7fd398005fd0 comp rx=0 tx=0).stop 2024-09-12T01:30:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.187+0000 7fd3a91e6740 1 -- 172.21.15.1:0/3702510834 shutdown_connections 2024-09-12T01:30:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.187+0000 7fd3a91e6740 1 --2- 172.21.15.1:0/3702510834 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x559811f41b80 0x559811f43f70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.187+0000 7fd3a91e6740 1 --2- 172.21.15.1:0/3702510834 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559811f31c30 0x559811f3d9c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.187+0000 7fd3a91e6740 1 --2- 172.21.15.1:0/3702510834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559811f2cba0 0x559811f36980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.187+0000 7fd3a91e6740 1 --2- 172.21.15.1:0/3702510834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559811f2c260 0x559811f36440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.187+0000 7fd389600640 0 client.15264 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:30:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.187+0000 7fd3a91e6740 1 -- 172.21.15.1:0/3702510834 >> 172.21.15.1:0/3702510834 conn(0x559811f278d0 msgr2=0x559811f290e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:30:43.194 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.187+0000 7fd3a91e6740 1 -- 172.21.15.1:0/3702510834 shutdown_connections 2024-09-12T01:30:43.194 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.187+0000 7fd3a91e6740 1 -- 172.21.15.1:0/3702510834 wait complete. 2024-09-12T01:30:43.194 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.187+0000 7fd3a91e6740 20 client.15264 trim_cache size 0 max 16384 2024-09-12T01:30:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.217+0000 7fd3a91e6740 1 Processor -- start 2024-09-12T01:30:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.218+0000 7fd3a91e6740 1 -- start start 2024-09-12T01:30:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.218+0000 7fd3a91e6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559811f2c260 0x559811f27cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.218+0000 7fd3a91e6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559811f2cba0 0x559811f3d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.218+0000 7fd3a91e6740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559811f31c30 0x559811f281f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.218+0000 7fd3a91e6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a01b0b80 con 0x559811f2cba0 2024-09-12T01:30:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.218+0000 7fd3a91e6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a01b26c0 con 0x559811f2c260 2024-09-12T01:30:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.218+0000 7fd3a91e6740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a01b1250 con 0x559811f31c30 2024-09-12T01:30:43.219 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.218+0000 7fd39ea00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559811f2cba0 0x559811f3d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:43.219 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.218+0000 7fd39fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559811f2c260 0x559811f27cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:43.219 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.218+0000 7fd39fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559811f2c260 0x559811f27cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:38852/0 (socket says 172.21.15.1:38852) 2024-09-12T01:30:43.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.218+0000 7fd39ea00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559811f2cba0 0x559811f3d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53936/0 (socket says 172.21.15.1:53936) 2024-09-12T01:30:43.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.218+0000 7fd39fe00640 1 -- 172.21.15.1:0/2933320782 learned_addr learned my addr 172.21.15.1:0/2933320782 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:30:43.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.218+0000 7fd39fe00640 1 --2- 172.21.15.1:0/2933320782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559811f31c30 0x559811f281f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:43.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.219+0000 7fd39ea00640 1 -- 172.21.15.1:0/2933320782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559811f31c30 msgr2=0x559811f281f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:30:43.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.219+0000 7fd39ea00640 1 --2- 172.21.15.1:0/2933320782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559811f31c30 0x559811f281f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:43.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.219+0000 7fd39ea00640 1 -- 172.21.15.1:0/2933320782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559811f2c260 msgr2=0x559811f27cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:30:43.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.219+0000 7fd39ea00640 1 --2- 172.21.15.1:0/2933320782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559811f2c260 0x559811f27cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:43.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.219+0000 7fd39ea00640 1 -- 172.21.15.1:0/2933320782 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559811f442e0 con 0x559811f2cba0 2024-09-12T01:30:43.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.219+0000 7fd39ea00640 1 --2- 172.21.15.1:0/2933320782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559811f2cba0 0x559811f3d980 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x559811f37920 tx=0x7fd38c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d3d547495f41188 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:30:43.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.220+0000 7fd38be00640 1 -- 172.21.15.1:0/2933320782 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38c003c20 con 0x559811f2cba0 2024-09-12T01:30:43.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.220+0000 7fd38be00640 1 -- 172.21.15.1:0/2933320782 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd38c003dc0 con 0x559811f2cba0 2024-09-12T01:30:43.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.220+0000 7fd38be00640 1 -- 172.21.15.1:0/2933320782 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38c004dd0 con 0x559811f2cba0 2024-09-12T01:30:43.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.221+0000 7fd3a91e6740 1 -- 172.21.15.1:0/2933320782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559811f2cba0 msgr2=0x559811f3d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:43.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.221+0000 7fd3a91e6740 1 --2- 172.21.15.1:0/2933320782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559811f2cba0 0x559811f3d980 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x559811f37920 tx=0x7fd38c005f90 comp rx=0 tx=0).stop 2024-09-12T01:30:43.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.221+0000 7fd3a91e6740 1 -- 172.21.15.1:0/2933320782 shutdown_connections 2024-09-12T01:30:43.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.221+0000 7fd3a91e6740 1 --2- 172.21.15.1:0/2933320782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559811f31c30 0x559811f281f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.221+0000 7fd3a91e6740 1 --2- 172.21.15.1:0/2933320782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559811f2cba0 0x559811f3d980 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.221+0000 7fd3a91e6740 1 --2- 172.21.15.1:0/2933320782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559811f2c260 0x559811f27cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.221+0000 7fd3a91e6740 1 -- 172.21.15.1:0/2933320782 >> 172.21.15.1:0/2933320782 conn(0x559811f278d0 msgr2=0x559811f36420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:30:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.221+0000 7fd3a91e6740 1 -- 172.21.15.1:0/2933320782 shutdown_connections 2024-09-12T01:30:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.221+0000 7fd3a91e6740 1 -- 172.21.15.1:0/2933320782 wait complete. 2024-09-12T01:30:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.222+0000 7fd3a91e6740 1 -- start start 2024-09-12T01:30:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.222+0000 7fd3a91e6740 1 Processor -- start 2024-09-12T01:30:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.223+0000 7fd3a91e6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559811f2c260 0x559811f3a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.223+0000 7fd3a91e6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559811f2cba0 0x559811f3ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.223+0000 7fd3a91e6740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559811f31c30 0x559811faa730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.223+0000 7fd3a91e6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a01b1ff0 con 0x559811f2c260 2024-09-12T01:30:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.223+0000 7fd3a91e6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a01b1920 con 0x559811f2cba0 2024-09-12T01:30:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.223+0000 7fd3a91e6740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a01aeb60 con 0x559811f31c30 2024-09-12T01:30:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.223+0000 7fd3a91e6740 10 client.0 _ll_register_callbacks cb 0x7fd38c009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:30:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.223+0000 7fd39fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559811f2c260 0x559811f3a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.223+0000 7fd39fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559811f2c260 0x559811f3a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53948/0 (socket says 172.21.15.1:53948) 2024-09-12T01:30:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.223+0000 7fd39fe00640 1 -- 172.21.15.1:0/109231519 learned_addr learned my addr 172.21.15.1:0/109231519 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:30:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.223+0000 7fd39ea00640 1 --2- 172.21.15.1:0/109231519 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559811f2cba0 0x559811f3ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.223+0000 7fd39fe00640 1 --2- 172.21.15.1:0/109231519 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559811f31c30 0x559811faa730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.223+0000 7fd39fe00640 1 -- 172.21.15.1:0/109231519 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559811f31c30 msgr2=0x559811faa730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:30:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.223+0000 7fd39fe00640 1 --2- 172.21.15.1:0/109231519 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559811f31c30 0x559811faa730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.223+0000 7fd39fe00640 1 -- 172.21.15.1:0/109231519 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559811f2cba0 msgr2=0x559811f3ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:30:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.223+0000 7fd39fe00640 1 --2- 172.21.15.1:0/109231519 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559811f2cba0 0x559811f3ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.224+0000 7fd39fe00640 1 -- 172.21.15.1:0/109231519 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559811f3b330 con 0x559811f2c260 2024-09-12T01:30:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.224+0000 7fd39fe00640 1 --2- 172.21.15.1:0/109231519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559811f2c260 0x559811f3a750 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd3980198f0 tx=0x7fd3980099c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2feac7a87e09a4a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:30:43.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.224+0000 7fd389600640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-12T01:30:43.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.224+0000 7fd389600640 1 -- 172.21.15.1:0/109231519 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd398003e20 con 0x559811f2c260 2024-09-12T01:30:43.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.224+0000 7fd389600640 1 -- 172.21.15.1:0/109231519 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fd39801e040 con 0x559811f2c260 2024-09-12T01:30:43.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.224+0000 7fd3a91e6740 1 -- 172.21.15.1:0/109231519 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd38c003dc0 con 0x559811f2c260 2024-09-12T01:30:43.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.225+0000 7fd389600640 1 -- 172.21.15.1:0/109231519 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd398019620 con 0x559811f2c260 2024-09-12T01:30:43.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.225+0000 7fd3a91e6740 10 client.15270 fetch_fsmap learned FSMap version 29 2024-09-12T01:30:43.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.225+0000 7fd3a91e6740 1 -- 172.21.15.1:0/109231519 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559811fabdb0 con 0x559811f2c260 2024-09-12T01:30:43.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.225+0000 7fd389600640 1 -- 172.21.15.1:0/109231519 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd38c003dc0 con 0x559811f2c260 2024-09-12T01:30:43.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.225+0000 7fd389600640 1 -- 172.21.15.1:0/109231519 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fd398004f80 con 0x559811f2c260 2024-09-12T01:30:43.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.225+0000 7fd3a91e6740 10 client.15270 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T01:30:43.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.225+0000 7fd3a91e6740 10 client.15270 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:30:43.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.226+0000 7fd3a91e6740 20 client.15270 populate_metadata read hostname 'smithi001' 2024-09-12T01:30:43.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.226+0000 7fd3a91e6740 1 --2- 172.21.15.1:0/109231519 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x559811fac6a0 0x559811faeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:43.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.226+0000 7fd3a91e6740 4 client.15270 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:30:43.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.226+0000 7fd3a91e6740 1 -- 172.21.15.1:0/109231519 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559811e98b80 con 0x559811fac6a0 2024-09-12T01:30:43.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.226+0000 7fd39ea00640 1 --2- 172.21.15.1:0/109231519 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x559811fac6a0 0x559811faeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:43.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.227+0000 7fd39ea00640 1 --2- 172.21.15.1:0/109231519 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x559811fac6a0 0x559811faeac0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba10dfc0eb633cd6 server_cookie=cce3c93b49d4beb5 in_seq=0 out_seq=0 2024-09-12T01:30:43.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.227+0000 7fd389600640 10 client.15270 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:30:43.230 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.230+0000 7fd389600640 1 -- 172.21.15.1:0/109231519 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559811e98b80 con 0x559811fac6a0 2024-09-12T01:30:43.230 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.230+0000 7fd389600640 10 client.15270 handle_command_reply: tid=0 2024-09-12T01:30:43.230 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.230+0000 7fd3a91e6740 1 client.15270 shutdown 2024-09-12T01:30:43.230 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.231+0000 7fd3a91e6740 1 -- 172.21.15.1:0/109231519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559811f2c260 msgr2=0x559811f3a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:43.230 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.231+0000 7fd3a91e6740 1 --2- 172.21.15.1:0/109231519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559811f2c260 0x559811f3a750 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd3980198f0 tx=0x7fd3980099c0 comp rx=0 tx=0).stop 2024-09-12T01:30:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.231+0000 7fd3a91e6740 1 -- 172.21.15.1:0/109231519 shutdown_connections 2024-09-12T01:30:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.231+0000 7fd3a91e6740 1 --2- 172.21.15.1:0/109231519 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x559811fac6a0 0x559811faeac0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.231+0000 7fd389600640 0 client.15270 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:30:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.231+0000 7fd3a91e6740 1 --2- 172.21.15.1:0/109231519 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559811f31c30 0x559811faa730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.231+0000 7fd3a91e6740 1 --2- 172.21.15.1:0/109231519 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559811f2cba0 0x559811f3ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.231+0000 7fd3a91e6740 1 --2- 172.21.15.1:0/109231519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559811f2c260 0x559811f3a750 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.231+0000 7fd3a91e6740 1 -- 172.21.15.1:0/109231519 >> 172.21.15.1:0/109231519 conn(0x559811f278d0 msgr2=0x559811f261f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:30:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.231+0000 7fd3a91e6740 1 -- 172.21.15.1:0/109231519 shutdown_connections 2024-09-12T01:30:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.231+0000 7fd3a91e6740 1 -- 172.21.15.1:0/109231519 wait complete. 2024-09-12T01:30:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.232+0000 7fd3a91e6740 20 client.15270 trim_cache size 0 max 16384 2024-09-12T01:30:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-12T01:30:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-12T01:30:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-12T01:30:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-12T01:30:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.232+0000 7fd3a4e00640 1 -- 172.21.15.1:0/174104346 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fd3840a2c80 msgr2=0x7fd3840a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.232+0000 7fd3a4e00640 1 --2- 172.21.15.1:0/174104346 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fd3840a2c80 0x7fd3840a5140 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x559811f2d5d0 tx=0x7fd390009260 comp rx=0 tx=0).stop 2024-09-12T01:30:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.232+0000 7fd3a4e00640 1 -- 172.21.15.1:0/174104346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3a01019b0 msgr2=0x7fd3a019e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.232+0000 7fd3a4e00640 1 --2- 172.21.15.1:0/174104346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3a01019b0 0x7fd3a019e2a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd390009230 tx=0x7fd390032ce0 comp rx=0 tx=0).stop 2024-09-12T01:30:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.233+0000 7fd3a4e00640 1 -- 172.21.15.1:0/174104346 shutdown_connections 2024-09-12T01:30:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.233+0000 7fd3a4e00640 1 --2- 172.21.15.1:0/174104346 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3a0103560 0x7fd3a01a5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.233+0000 7fd3a4e00640 1 --2- 172.21.15.1:0/174104346 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3a0102ba0 0x7fd3a019e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.233+0000 7fd3a4e00640 1 --2- 172.21.15.1:0/174104346 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fd3840a2c80 0x7fd3840a5140 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.233+0000 7fd3a4e00640 1 --2- 172.21.15.1:0/174104346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3a01019b0 0x7fd3a019e2a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.233+0000 7fd3a4e00640 1 -- 172.21.15.1:0/174104346 >> 172.21.15.1:0/174104346 conn(0x7fd3a00fd130 msgr2=0x7fd3a0105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:30:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.233+0000 7fd3a4e00640 1 -- 172.21.15.1:0/174104346 shutdown_connections 2024-09-12T01:30:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:43.233+0000 7fd3a4e00640 1 -- 172.21.15.1:0/174104346 wait complete. 2024-09-12T01:30:44.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:30:44 smithi001 ceph-mon[30387]: pgmap v432: 129 pgs: 129 active+clean; 674 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-12T01:30:44.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:30:44 smithi053 ceph-mon[35158]: pgmap v432: 129 pgs: 129 active+clean; 674 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-12T01:30:44.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:30:44 smithi104 ceph-mon[35219]: pgmap v432: 129 pgs: 129 active+clean; 674 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-12T01:30:46.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:30:46 smithi001 ceph-mon[30387]: pgmap v433: 129 pgs: 129 active+clean; 674 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-12T01:30:46.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:30:46 smithi053 ceph-mon[35158]: pgmap v433: 129 pgs: 129 active+clean; 674 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-12T01:30:46.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:30:46 smithi104 ceph-mon[35219]: pgmap v433: 129 pgs: 129 active+clean; 674 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-12T01:30:48.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:30:48 smithi001 ceph-mon[30387]: pgmap v434: 129 pgs: 129 active+clean; 675 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-12T01:30:48.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:30:48 smithi053 ceph-mon[35158]: pgmap v434: 129 pgs: 129 active+clean; 675 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-12T01:30:48.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:30:48 smithi104 ceph-mon[35219]: pgmap v434: 129 pgs: 129 active+clean; 675 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-12T01:30:50.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:30:50 smithi001 ceph-mon[30387]: pgmap v435: 129 pgs: 129 active+clean; 675 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-12T01:30:50.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:30:50 smithi053 ceph-mon[35158]: pgmap v435: 129 pgs: 129 active+clean; 675 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-12T01:30:50.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:30:50 smithi104 ceph-mon[35219]: pgmap v435: 129 pgs: 129 active+clean; 675 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-12T01:30:52.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:30:52 smithi001 ceph-mon[30387]: pgmap v436: 129 pgs: 129 active+clean; 675 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-12T01:30:52.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:30:52 smithi053 ceph-mon[35158]: pgmap v436: 129 pgs: 129 active+clean; 675 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-12T01:30:52.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:30:52 smithi104 ceph-mon[35219]: pgmap v436: 129 pgs: 129 active+clean; 675 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-12T01:30:54.589 INFO:tasks.workunit.client.0.smithi001.stderr:Updating files: 62% (8091/12903) Updating files: 63% (8129/12903) Updating files: 64% (8258/12903) Updating files: 65% (8387/12903) Updating files: 66% (8516/12903) Updating files: 67% (8646/12903) Updating files: 68% (8775/12903) Updating files: 69% (8904/12903) Updating files: 70% (9033/12903) Updating files: 71% (9162/12903) Updating files: 72% (9291/12903) Updating files: 73% (9420/12903) Updating files: 74% (9549/12903) Updating files: 75% (9678/12903) Updating files: 76% (9807/12903) Updating files: 77% (9936/12903) Updating files: 78% (10065/12903) Updating files: 79% (10194/12903) Updating files: 80% (10323/12903) Updating files: 81% (10452/12903) Updating files: 82% (10581/12903) Updating files: 83% (10710/12903) Updating files: 84% (10839/12903) Updating files: 85% (10968/12903) Updating files: 86% (11097/12903) Updating files: 87% (11226/12903) Updating files: 88% (11355/12903) Updating files: 89% (11484/12903) Updating files: 90% (11613/12903) Updating files: 91% (11742/12903) Updating files: 92% (11871/12903) Updating files: 93% (12000/12903) Updating files: 94% (12129/12903) Updating files: 95% (12258/12903) Updating files: 96% (12387/12903) Updating files: 97% (12516/12903) Updating files: 97% (12605/12903) Updating files: 98% (12645/12903) Updating files: 99% (12774/12903) Updating files: 100% (12903/12903) Updating files: 100% (12903/12903), done. 2024-09-12T01:30:54.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:30:54 smithi001 ceph-mon[30387]: pgmap v437: 129 pgs: 129 active+clean; 675 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-12T01:30:54.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:30:54 smithi053 ceph-mon[35158]: pgmap v437: 129 pgs: 129 active+clean; 675 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-12T01:30:54.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:30:54 smithi104 ceph-mon[35219]: pgmap v437: 129 pgs: 129 active+clean; 675 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-12T01:30:55.560 INFO:tasks.workunit.client.0.smithi001.stderr:Note: switching to '086c737fa75c22d521a3932c59b31c5e1d1c9ef4'. 2024-09-12T01:30:55.560 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-09-12T01:30:55.560 INFO:tasks.workunit.client.0.smithi001.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-12T01:30:55.560 INFO:tasks.workunit.client.0.smithi001.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-12T01:30:55.560 INFO:tasks.workunit.client.0.smithi001.stderr:state without impacting any branches by switching back to a branch. 2024-09-12T01:30:55.560 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-09-12T01:30:55.560 INFO:tasks.workunit.client.0.smithi001.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-12T01:30:55.560 INFO:tasks.workunit.client.0.smithi001.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-12T01:30:55.560 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-09-12T01:30:55.560 INFO:tasks.workunit.client.0.smithi001.stderr: git switch -c 2024-09-12T01:30:55.560 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-09-12T01:30:55.561 INFO:tasks.workunit.client.0.smithi001.stderr:Or undo this operation with: 2024-09-12T01:30:55.561 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-09-12T01:30:55.561 INFO:tasks.workunit.client.0.smithi001.stderr: git switch - 2024-09-12T01:30:55.561 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-09-12T01:30:55.561 INFO:tasks.workunit.client.0.smithi001.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-12T01:30:55.561 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-09-12T01:30:55.561 INFO:tasks.workunit.client.0.smithi001.stderr:HEAD is now at 086c737fa75 Merge pull request #59733 from zdover23/wip-doc-2024-09-11-backport-59635-to-squid 2024-09-12T01:30:55.576 DEBUG:teuthology.orchestra.run.smithi001:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-12T01:30:55.664 INFO:tasks.workunit.client.0.smithi001.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-12T01:30:55.669 INFO:tasks.workunit.client.0.smithi001.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-12T01:30:55.669 INFO:tasks.workunit.client.0.smithi001.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-12T01:30:55.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:30:55 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:30:55.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:30:55 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:30:55.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:30:55 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:30:55.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:30:55 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:30:55.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:30:55 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:30:55.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:30:55 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:30:55.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:30:55 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:30:55.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:30:55 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:30:55.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:30:55 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:30:55.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:30:55 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:30:55.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:30:55 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:30:55.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:30:55 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:30:56.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.359+0000 7f8855000640 1 -- 172.21.15.1:0/4044682298 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8850105760 msgr2=0x7f885010bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:56.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.359+0000 7f8855000640 1 --2- 172.21.15.1:0/4044682298 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8850105760 0x7f885010bff0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8848009ca0 tx=0x7f884802a8c0 comp rx=0 tx=0).stop 2024-09-12T01:30:56.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.359+0000 7f8855000640 1 -- 172.21.15.1:0/4044682298 shutdown_connections 2024-09-12T01:30:56.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.359+0000 7f8855000640 1 --2- 172.21.15.1:0/4044682298 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8850105760 0x7f885010bff0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:56.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.359+0000 7f8855000640 1 --2- 172.21.15.1:0/4044682298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8850104da0 0x7f8850105220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:56.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.359+0000 7f8855000640 1 --2- 172.21.15.1:0/4044682298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8850103bb0 0x7f8850103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:56.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.359+0000 7f8855000640 1 -- 172.21.15.1:0/4044682298 >> 172.21.15.1:0/4044682298 conn(0x7f88500ff310 msgr2=0x7f8850101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:30:56.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.359+0000 7f8855000640 1 -- 172.21.15.1:0/4044682298 shutdown_connections 2024-09-12T01:30:56.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.359+0000 7f8855000640 1 -- 172.21.15.1:0/4044682298 wait complete. 2024-09-12T01:30:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.360+0000 7f8855000640 1 Processor -- start 2024-09-12T01:30:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.360+0000 7f8855000640 1 -- start start 2024-09-12T01:30:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.361+0000 7f8855000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8850103bb0 0x7f8850199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.361+0000 7f8855000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8850104da0 0x7f885019a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.361+0000 7f8855000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8850105760 0x7f88501a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.361+0000 7f8855000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f885010f830 con 0x7f8850104da0 2024-09-12T01:30:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.361+0000 7f8855000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f885010f6b0 con 0x7f8850103bb0 2024-09-12T01:30:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.361+0000 7f8855000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f885010f9b0 con 0x7f8850105760 2024-09-12T01:30:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.361+0000 7f884ea00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8850104da0 0x7f885019a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.361+0000 7f884f400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8850103bb0 0x7f8850199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.361+0000 7f884ea00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8850104da0 0x7f885019a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58272/0 (socket says 172.21.15.1:58272) 2024-09-12T01:30:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.361+0000 7f884ea00640 1 -- 172.21.15.1:0/1862195267 learned_addr learned my addr 172.21.15.1:0/1862195267 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:30:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.361+0000 7f884fe00640 1 --2- 172.21.15.1:0/1862195267 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8850105760 0x7f88501a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.361+0000 7f884ea00640 1 -- 172.21.15.1:0/1862195267 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8850105760 msgr2=0x7f88501a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.361+0000 7f884ea00640 1 --2- 172.21.15.1:0/1862195267 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8850105760 0x7f88501a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.361+0000 7f884ea00640 1 -- 172.21.15.1:0/1862195267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8850103bb0 msgr2=0x7f8850199e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.361+0000 7f884ea00640 1 --2- 172.21.15.1:0/1862195267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8850103bb0 0x7f8850199e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.361+0000 7f884ea00640 1 -- 172.21.15.1:0/1862195267 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88501a1a80 con 0x7f8850104da0 2024-09-12T01:30:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.361+0000 7f884fe00640 1 --2- 172.21.15.1:0/1862195267 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8850105760 0x7f88501a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:30:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.362+0000 7f884ea00640 1 --2- 172.21.15.1:0/1862195267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8850104da0 0x7f885019a3a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f883c00e9f0 tx=0x7f883c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:30:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.362+0000 7f8847e00640 1 -- 172.21.15.1:0/1862195267 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883c00cd80 con 0x7f8850104da0 2024-09-12T01:30:56.364 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.362+0000 7f8847e00640 1 -- 172.21.15.1:0/1862195267 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f883c005500 con 0x7f8850104da0 2024-09-12T01:30:56.364 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.362+0000 7f8855000640 1 -- 172.21.15.1:0/1862195267 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88501a1d70 con 0x7f8850104da0 2024-09-12T01:30:56.364 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.362+0000 7f8847e00640 1 -- 172.21.15.1:0/1862195267 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883c010690 con 0x7f8850104da0 2024-09-12T01:30:56.364 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.362+0000 7f8855000640 1 -- 172.21.15.1:0/1862195267 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88501a2210 con 0x7f8850104da0 2024-09-12T01:30:56.364 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.365+0000 7f88594c7740 1 Processor -- start 2024-09-12T01:30:56.364 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.365+0000 7f88594c7740 1 -- start start 2024-09-12T01:30:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.365+0000 7f88594c7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc76542260 0x55bc76542660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.365+0000 7f88594c7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc76542ba0 0x55bc76547660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.365+0000 7f88594c7740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc76547c30 0x55bc7654a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.365+0000 7f88594c7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55bc764a7100 con 0x55bc76542ba0 2024-09-12T01:30:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.365+0000 7f88594c7740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55bc7644f1b0 con 0x55bc76547c30 2024-09-12T01:30:56.367 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.365+0000 7f88594c7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55bc764a30c0 con 0x55bc76542260 2024-09-12T01:30:56.367 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.365+0000 7f8847e00640 1 -- 172.21.15.1:0/1862195267 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f883c0046c0 con 0x7f8850104da0 2024-09-12T01:30:56.367 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.366+0000 7f884fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc76542260 0x55bc76542660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:56.367 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.366+0000 7f884fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc76542260 0x55bc76542660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:35494/0 (socket says 172.21.15.1:35494) 2024-09-12T01:30:56.367 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.366+0000 7f884f400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc76542ba0 0x55bc76547660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:56.367 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.366+0000 7f884fe00640 1 -- 172.21.15.1:0/290301440 learned_addr learned my addr 172.21.15.1:0/290301440 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:30:56.367 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.366+0000 7f884ea00640 1 --2- 172.21.15.1:0/290301440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc76547c30 0x55bc7654a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:56.367 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.366+0000 7f884fe00640 1 -- 172.21.15.1:0/290301440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc76547c30 msgr2=0x55bc7654a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:30:56.367 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.366+0000 7f884fe00640 1 --2- 172.21.15.1:0/290301440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc76547c30 0x55bc7654a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:56.367 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.366+0000 7f884fe00640 1 -- 172.21.15.1:0/290301440 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc76542ba0 msgr2=0x55bc76547660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:30:56.367 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.366+0000 7f884fe00640 1 --2- 172.21.15.1:0/290301440 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc76542ba0 0x55bc76547660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:56.367 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.366+0000 7f884fe00640 1 -- 172.21.15.1:0/290301440 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc761ba210 con 0x55bc76542260 2024-09-12T01:30:56.367 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.366+0000 7f884fe00640 1 --2- 172.21.15.1:0/290301440 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc76542260 0x55bc76542660 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8848035000 tx=0x7f88480093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efff4bd0039b2270 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:30:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.367+0000 7f8844c00640 1 -- 172.21.15.1:0/290301440 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8848031cb0 con 0x55bc76542260 2024-09-12T01:30:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.367+0000 7f8844c00640 1 -- 172.21.15.1:0/290301440 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8848036070 con 0x55bc76542260 2024-09-12T01:30:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.367+0000 7f8844c00640 1 -- 172.21.15.1:0/290301440 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8848004390 con 0x55bc76542260 2024-09-12T01:30:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.367+0000 7f8847e00640 1 --2- 172.21.15.1:0/1862195267 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f88340a2c00 0x7f88340a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.367+0000 7f8847e00640 1 -- 172.21.15.1:0/1862195267 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f883c0cfa90 con 0x7f8850104da0 2024-09-12T01:30:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.367+0000 7f884f400640 1 --2- 172.21.15.1:0/1862195267 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f88340a2c00 0x7f88340a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.368+0000 7f88594c7740 1 -- 172.21.15.1:0/290301440 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc76542260 msgr2=0x55bc76542660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.368+0000 7f88594c7740 1 --2- 172.21.15.1:0/290301440 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc76542260 0x55bc76542660 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8848035000 tx=0x7f88480093e0 comp rx=0 tx=0).stop 2024-09-12T01:30:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.368+0000 7f88594c7740 1 -- 172.21.15.1:0/290301440 shutdown_connections 2024-09-12T01:30:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.368+0000 7f88594c7740 1 --2- 172.21.15.1:0/290301440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc76547c30 0x55bc7654a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.368+0000 7f88594c7740 1 --2- 172.21.15.1:0/290301440 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc76542ba0 0x55bc76547660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.368+0000 7f88594c7740 1 --2- 172.21.15.1:0/290301440 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc76542260 0x55bc76542660 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.368+0000 7f88594c7740 1 -- 172.21.15.1:0/290301440 >> 172.21.15.1:0/290301440 conn(0x55bc7653d8d0 msgr2=0x55bc7653dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:30:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.368+0000 7f88594c7740 1 -- 172.21.15.1:0/290301440 shutdown_connections 2024-09-12T01:30:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.368+0000 7f88594c7740 1 -- 172.21.15.1:0/290301440 wait complete. 2024-09-12T01:30:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.368+0000 7f884f400640 1 --2- 172.21.15.1:0/1862195267 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f88340a2c00 0x7f88340a50c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x55bc765435d0 tx=0x7f8840005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:30:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.368+0000 7f88594c7740 1 -- start start 2024-09-12T01:30:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.369+0000 7f88594c7740 1 Processor -- start 2024-09-12T01:30:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.369+0000 7f88594c7740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc76542260 0x55bc7654c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.369+0000 7f88594c7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc76542ba0 0x55bc7654c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.369+0000 7f88594c7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc76547c30 0x55bc765537e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.369+0000 7f88594c7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55bc7654aa20 con 0x55bc76547c30 2024-09-12T01:30:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.369+0000 7f88594c7740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55bc76430090 con 0x55bc76542260 2024-09-12T01:30:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.369+0000 7f88594c7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55bc7654aba0 con 0x55bc76542ba0 2024-09-12T01:30:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.369+0000 7f88594c7740 10 client.0 _ll_register_callbacks cb 0x55bc76484f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:30:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.369+0000 7f884fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc76547c30 0x55bc765537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.369+0000 7f884fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc76547c30 0x55bc765537e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58286/0 (socket says 172.21.15.1:58286) 2024-09-12T01:30:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.369+0000 7f884fe00640 1 -- 172.21.15.1:0/1130451714 learned_addr learned my addr 172.21.15.1:0/1130451714 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:30:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.369+0000 7f884ea00640 1 --2- 172.21.15.1:0/1130451714 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc76542ba0 0x55bc7654c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.369+0000 7f884f400640 1 --2- 172.21.15.1:0/1130451714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc76542260 0x55bc7654c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.370+0000 7f884fe00640 1 -- 172.21.15.1:0/1130451714 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc76542ba0 msgr2=0x55bc7654c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:30:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.370+0000 7f884fe00640 1 --2- 172.21.15.1:0/1130451714 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc76542ba0 0x55bc7654c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.370+0000 7f884fe00640 1 -- 172.21.15.1:0/1130451714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc76542260 msgr2=0x55bc7654c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:30:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.370+0000 7f884fe00640 1 --2- 172.21.15.1:0/1130451714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc76542260 0x55bc7654c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.370+0000 7f884fe00640 1 -- 172.21.15.1:0/1130451714 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc76286620 con 0x55bc76547c30 2024-09-12T01:30:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.370+0000 7f884ea00640 1 --2- 172.21.15.1:0/1130451714 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc76542ba0 0x55bc7654c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:30:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.370+0000 7f884fe00640 1 --2- 172.21.15.1:0/1130451714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc76547c30 0x55bc765537e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8848031ae0 tx=0x7f8848031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b94c08cd78996c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:30:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.370+0000 7f8839600640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-12T01:30:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.370+0000 7f8839600640 1 -- 172.21.15.1:0/1130451714 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8848004330 con 0x55bc76547c30 2024-09-12T01:30:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.370+0000 7f8839600640 1 -- 172.21.15.1:0/1130451714 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f88480044d0 con 0x55bc76547c30 2024-09-12T01:30:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.370+0000 7f88594c7740 1 -- 172.21.15.1:0/1130451714 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8848036070 con 0x55bc76547c30 2024-09-12T01:30:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.370+0000 7f8839600640 1 -- 172.21.15.1:0/1130451714 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8848058050 con 0x55bc76547c30 2024-09-12T01:30:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.370+0000 7f88594c7740 10 client.15282 fetch_fsmap learned FSMap version 29 2024-09-12T01:30:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.371+0000 7f88594c7740 1 -- 172.21.15.1:0/1130451714 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bc7654aa20 con 0x55bc76547c30 2024-09-12T01:30:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.371+0000 7f8839600640 1 -- 172.21.15.1:0/1130451714 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8848036070 con 0x55bc76547c30 2024-09-12T01:30:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.371+0000 7f8839600640 1 -- 172.21.15.1:0/1130451714 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f8848042020 con 0x55bc76547c30 2024-09-12T01:30:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.371+0000 7f88594c7740 10 client.15282 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T01:30:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.371+0000 7f88594c7740 10 client.15282 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:30:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.371+0000 7f88594c7740 20 client.15282 populate_metadata read hostname 'smithi001' 2024-09-12T01:30:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.371+0000 7f88594c7740 1 --2- 172.21.15.1:0/1130451714 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55bc76557a20 0x55bc76559e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.371+0000 7f88594c7740 4 client.15282 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:30:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.371+0000 7f88594c7740 1 -- 172.21.15.1:0/1130451714 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc764b4830 con 0x55bc76557a20 2024-09-12T01:30:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.372+0000 7f884f400640 1 --2- 172.21.15.1:0/1130451714 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55bc76557a20 0x55bc76559e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:56.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.372+0000 7f884f400640 1 --2- 172.21.15.1:0/1130451714 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55bc76557a20 0x55bc76559e10 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a838aedfa205cd5 server_cookie=6fdd70852698506f in_seq=0 out_seq=0 2024-09-12T01:30:56.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.372+0000 7f8839600640 10 client.15282 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:30:56.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.373+0000 7f8839600640 1 -- 172.21.15.1:0/1130451714 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bc764b4830 con 0x55bc76557a20 2024-09-12T01:30:56.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.373+0000 7f8839600640 10 client.15282 handle_command_reply: tid=0 2024-09-12T01:30:56.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.373+0000 7f88594c7740 1 client.15282 shutdown 2024-09-12T01:30:56.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.373+0000 7f88594c7740 1 -- 172.21.15.1:0/1130451714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc76547c30 msgr2=0x55bc765537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:56.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.373+0000 7f88594c7740 1 --2- 172.21.15.1:0/1130451714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc76547c30 0x55bc765537e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8848031ae0 tx=0x7f8848031920 comp rx=0 tx=0).stop 2024-09-12T01:30:56.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.373+0000 7f88594c7740 1 -- 172.21.15.1:0/1130451714 shutdown_connections 2024-09-12T01:30:56.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.373+0000 7f88594c7740 1 --2- 172.21.15.1:0/1130451714 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55bc76557a20 0x55bc76559e10 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:56.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.373+0000 7f88594c7740 1 --2- 172.21.15.1:0/1130451714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc76547c30 0x55bc765537e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:56.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.373+0000 7f88594c7740 1 --2- 172.21.15.1:0/1130451714 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc76542ba0 0x55bc7654c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:56.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.373+0000 7f88594c7740 1 --2- 172.21.15.1:0/1130451714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc76542260 0x55bc7654c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:56.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.374+0000 7f88594c7740 1 -- 172.21.15.1:0/1130451714 >> 172.21.15.1:0/1130451714 conn(0x55bc7653d8d0 msgr2=0x55bc7653f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:30:56.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.374+0000 7f8839600640 0 client.15282 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:30:56.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.374+0000 7f88594c7740 1 -- 172.21.15.1:0/1130451714 shutdown_connections 2024-09-12T01:30:56.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.374+0000 7f88594c7740 1 -- 172.21.15.1:0/1130451714 wait complete. 2024-09-12T01:30:56.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.374+0000 7f88594c7740 20 client.15282 trim_cache size 0 max 16384 2024-09-12T01:30:56.388 INFO:tasks.workunit.client.0.smithi001.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-12T01:30:56.390 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.390+0000 7f88594c7740 1 Processor -- start 2024-09-12T01:30:56.390 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.390+0000 7f88594c7740 1 -- start start 2024-09-12T01:30:56.390 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.390+0000 7f88594c7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc7653d8d0 0x55bc7654c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:56.390 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.390+0000 7f88594c7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc76542260 0x55bc765537a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:56.390 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.390+0000 7f88594c7740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc76542ba0 0x55bc7654c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:56.390 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.390+0000 7f88594c7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f88501ac710 con 0x55bc76542260 2024-09-12T01:30:56.390 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.390+0000 7f88594c7740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f88501ae250 con 0x55bc76542ba0 2024-09-12T01:30:56.390 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.390+0000 7f88594c7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f88501acde0 con 0x55bc7653d8d0 2024-09-12T01:30:56.391 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.391+0000 7f884f400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc7653d8d0 0x55bc7654c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:56.391 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.391+0000 7f884fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc76542ba0 0x55bc7654c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:56.392 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.391+0000 7f884fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc76542ba0 0x55bc7654c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:50176/0 (socket says 172.21.15.1:50176) 2024-09-12T01:30:56.392 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.391+0000 7f884f400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc7653d8d0 0x55bc7654c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:35522/0 (socket says 172.21.15.1:35522) 2024-09-12T01:30:56.392 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.391+0000 7f884fe00640 1 -- 172.21.15.1:0/2205131425 learned_addr learned my addr 172.21.15.1:0/2205131425 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:30:56.392 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.391+0000 7f884ea00640 1 --2- 172.21.15.1:0/2205131425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc76542260 0x55bc765537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:56.392 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.391+0000 7f884fe00640 1 -- 172.21.15.1:0/2205131425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc7653d8d0 msgr2=0x55bc7654c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:30:56.392 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.391+0000 7f884fe00640 1 --2- 172.21.15.1:0/2205131425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc7653d8d0 0x55bc7654c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:56.392 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.391+0000 7f884fe00640 1 -- 172.21.15.1:0/2205131425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc76542260 msgr2=0x55bc765537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:30:56.392 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.391+0000 7f884fe00640 1 --2- 172.21.15.1:0/2205131425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc76542260 0x55bc765537a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:56.392 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.391+0000 7f884fe00640 1 -- 172.21.15.1:0/2205131425 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc76543080 con 0x55bc76542ba0 2024-09-12T01:30:56.392 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.391+0000 7f884ea00640 1 --2- 172.21.15.1:0/2205131425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc76542260 0x55bc765537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:30:56.392 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.391+0000 7f884fe00640 1 --2- 172.21.15.1:0/2205131425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc76542ba0 0x55bc7654c7a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8848000f80 tx=0x7f8848009b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=489ccb6cb96ca3f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:30:56.392 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.391+0000 7f883be00640 1 -- 172.21.15.1:0/2205131425 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8848030a80 con 0x55bc76542ba0 2024-09-12T01:30:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.392+0000 7f883be00640 1 -- 172.21.15.1:0/2205131425 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8848036070 con 0x55bc76542ba0 2024-09-12T01:30:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.392+0000 7f883be00640 1 -- 172.21.15.1:0/2205131425 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88480446a0 con 0x55bc76542ba0 2024-09-12T01:30:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.393+0000 7f88594c7740 1 -- 172.21.15.1:0/2205131425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc76542ba0 msgr2=0x55bc7654c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.393+0000 7f88594c7740 1 --2- 172.21.15.1:0/2205131425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc76542ba0 0x55bc7654c7a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8848000f80 tx=0x7f8848009b20 comp rx=0 tx=0).stop 2024-09-12T01:30:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.393+0000 7f88594c7740 1 -- 172.21.15.1:0/2205131425 shutdown_connections 2024-09-12T01:30:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.393+0000 7f88594c7740 1 --2- 172.21.15.1:0/2205131425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc76542ba0 0x55bc7654c7a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.393+0000 7f88594c7740 1 --2- 172.21.15.1:0/2205131425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc76542260 0x55bc765537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.393+0000 7f88594c7740 1 --2- 172.21.15.1:0/2205131425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc7653d8d0 0x55bc7654c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.393+0000 7f88594c7740 1 -- 172.21.15.1:0/2205131425 >> 172.21.15.1:0/2205131425 conn(0x55bc76547750 msgr2=0x55bc76543d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:30:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.393+0000 7f88594c7740 1 -- 172.21.15.1:0/2205131425 shutdown_connections 2024-09-12T01:30:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.393+0000 7f88594c7740 1 -- 172.21.15.1:0/2205131425 wait complete. 2024-09-12T01:30:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.393+0000 7f88594c7740 1 -- start start 2024-09-12T01:30:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.394+0000 7f88594c7740 1 Processor -- start 2024-09-12T01:30:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.394+0000 7f88594c7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc7653d8d0 0x55bc76550430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.394+0000 7f88594c7740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc76542260 0x55bc76550970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.394+0000 7f88594c7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc76542ba0 0x55bc765c06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.394+0000 7f88594c7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f88501adb80 con 0x55bc76542ba0 2024-09-12T01:30:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.394+0000 7f88594c7740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f88501ad4b0 con 0x55bc76542260 2024-09-12T01:30:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.394+0000 7f88594c7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f88501aa6f0 con 0x55bc7653d8d0 2024-09-12T01:30:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.394+0000 7f88594c7740 10 client.0 _ll_register_callbacks cb 0x7f8840009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:30:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.394+0000 7f884f400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc7653d8d0 0x55bc76550430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.394+0000 7f884fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc76542ba0 0x55bc765c06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.394+0000 7f884f400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc7653d8d0 0x55bc76550430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:35538/0 (socket says 172.21.15.1:35538) 2024-09-12T01:30:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.394+0000 7f884f400640 1 -- 172.21.15.1:0/2772494243 learned_addr learned my addr 172.21.15.1:0/2772494243 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:30:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.394+0000 7f884ea00640 1 --2- 172.21.15.1:0/2772494243 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc76542260 0x55bc76550970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.395+0000 7f884f400640 1 -- 172.21.15.1:0/2772494243 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc76542260 msgr2=0x55bc76550970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:30:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.395+0000 7f884f400640 1 --2- 172.21.15.1:0/2772494243 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc76542260 0x55bc76550970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.395+0000 7f884f400640 1 -- 172.21.15.1:0/2772494243 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc76542ba0 msgr2=0x55bc765c06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:30:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.395+0000 7f884f400640 1 --2- 172.21.15.1:0/2772494243 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc76542ba0 0x55bc765c06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.395+0000 7f884f400640 1 -- 172.21.15.1:0/2772494243 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc76553a60 con 0x55bc7653d8d0 2024-09-12T01:30:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.395+0000 7f884f400640 1 --2- 172.21.15.1:0/2772494243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc7653d8d0 0x55bc76550430 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x55bc7653e750 tx=0x7f88400046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42223d62a3523c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:30:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.395+0000 7f8839600640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-12T01:30:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.395+0000 7f8839600640 1 -- 172.21.15.1:0/2772494243 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884001dd30 con 0x55bc7653d8d0 2024-09-12T01:30:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.395+0000 7f88594c7740 1 -- 172.21.15.1:0/2772494243 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8848036070 con 0x55bc7653d8d0 2024-09-12T01:30:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.395+0000 7f8839600640 1 -- 172.21.15.1:0/2772494243 <== mon.1 v2:172.21.15.104:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f884002a420 con 0x55bc7653d8d0 2024-09-12T01:30:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.396+0000 7f8839600640 1 -- 172.21.15.1:0/2772494243 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f884002a660 con 0x55bc7653d8d0 2024-09-12T01:30:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.396+0000 7f88594c7740 10 client.25309 fetch_fsmap learned FSMap version 29 2024-09-12T01:30:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.396+0000 7f88594c7740 1 -- 172.21.15.1:0/2772494243 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bc765c1d70 con 0x55bc7653d8d0 2024-09-12T01:30:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.396+0000 7f8839600640 1 -- 172.21.15.1:0/2772494243 <== mon.1 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8848036070 con 0x55bc7653d8d0 2024-09-12T01:30:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.396+0000 7f8839600640 1 -- 172.21.15.1:0/2772494243 <== mon.1 v2:172.21.15.104:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f8840033920 con 0x55bc7653d8d0 2024-09-12T01:30:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.396+0000 7f88594c7740 10 client.25309 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T01:30:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.396+0000 7f88594c7740 10 client.25309 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:30:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.396+0000 7f88594c7740 20 client.25309 populate_metadata read hostname 'smithi001' 2024-09-12T01:30:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.396+0000 7f88594c7740 1 --2- 172.21.15.1:0/2772494243 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55bc76547c30 0x55bc765c47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:30:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.396+0000 7f88594c7740 4 client.25309 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:30:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.396+0000 7f88594c7740 1 -- 172.21.15.1:0/2772494243 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bc764aeb80 con 0x55bc76547c30 2024-09-12T01:30:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.397+0000 7f884ea00640 1 --2- 172.21.15.1:0/2772494243 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55bc76547c30 0x55bc765c47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:30:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.397+0000 7f884ea00640 1 --2- 172.21.15.1:0/2772494243 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55bc76547c30 0x55bc765c47c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3859af932ec5f83d server_cookie=6fb286e4a4a3a65e in_seq=0 out_seq=0 2024-09-12T01:30:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.397+0000 7f8839600640 10 client.25309 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:30:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.401+0000 7f8839600640 1 -- 172.21.15.1:0/2772494243 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bc764aeb80 con 0x55bc76547c30 2024-09-12T01:30:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.401+0000 7f8839600640 10 client.25309 handle_command_reply: tid=0 2024-09-12T01:30:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.401+0000 7f88594c7740 1 client.25309 shutdown 2024-09-12T01:30:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.401+0000 7f88594c7740 1 -- 172.21.15.1:0/2772494243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc7653d8d0 msgr2=0x55bc76550430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.401+0000 7f88594c7740 1 --2- 172.21.15.1:0/2772494243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc7653d8d0 0x55bc76550430 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x55bc7653e750 tx=0x7f88400046c0 comp rx=0 tx=0).stop 2024-09-12T01:30:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.401+0000 7f88594c7740 1 -- 172.21.15.1:0/2772494243 shutdown_connections 2024-09-12T01:30:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.401+0000 7f88594c7740 1 --2- 172.21.15.1:0/2772494243 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55bc76547c30 0x55bc765c47c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.401+0000 7f88594c7740 1 --2- 172.21.15.1:0/2772494243 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc76542ba0 0x55bc765c06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.401+0000 7f88594c7740 1 --2- 172.21.15.1:0/2772494243 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc76542260 0x55bc76550970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.401+0000 7f88594c7740 1 --2- 172.21.15.1:0/2772494243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc7653d8d0 0x55bc76550430 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.401+0000 7f8839600640 0 client.25309 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:30:56.402 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.401+0000 7f88594c7740 1 -- 172.21.15.1:0/2772494243 >> 172.21.15.1:0/2772494243 conn(0x55bc76547750 msgr2=0x55bc7654c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:30:56.402 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.401+0000 7f88594c7740 1 -- 172.21.15.1:0/2772494243 shutdown_connections 2024-09-12T01:30:56.402 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.401+0000 7f88594c7740 1 -- 172.21.15.1:0/2772494243 wait complete. 2024-09-12T01:30:56.402 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.401+0000 7f88594c7740 20 client.25309 trim_cache size 0 max 16384 2024-09-12T01:30:56.402 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-12T01:30:56.402 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-12T01:30:56.402 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-12T01:30:56.402 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-12T01:30:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.402+0000 7f8855000640 1 -- 172.21.15.1:0/1862195267 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f88340a2c00 msgr2=0x7f88340a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.402+0000 7f8855000640 1 --2- 172.21.15.1:0/1862195267 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f88340a2c00 0x7f88340a50c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x55bc765435d0 tx=0x7f8840005f90 comp rx=0 tx=0).stop 2024-09-12T01:30:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.402+0000 7f8855000640 1 -- 172.21.15.1:0/1862195267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8850104da0 msgr2=0x7f885019a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:30:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.402+0000 7f8855000640 1 --2- 172.21.15.1:0/1862195267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8850104da0 0x7f885019a3a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f883c00e9f0 tx=0x7f883c00edf0 comp rx=0 tx=0).stop 2024-09-12T01:30:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.402+0000 7f8855000640 1 -- 172.21.15.1:0/1862195267 shutdown_connections 2024-09-12T01:30:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.402+0000 7f8855000640 1 --2- 172.21.15.1:0/1862195267 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f88340a2c00 0x7f88340a50c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.402+0000 7f8855000640 1 --2- 172.21.15.1:0/1862195267 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8850105760 0x7f88501a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.402+0000 7f8855000640 1 --2- 172.21.15.1:0/1862195267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8850104da0 0x7f885019a3a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.402+0000 7f8855000640 1 --2- 172.21.15.1:0/1862195267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8850103bb0 0x7f8850199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:30:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.402+0000 7f8855000640 1 -- 172.21.15.1:0/1862195267 >> 172.21.15.1:0/1862195267 conn(0x7f88500ff310 msgr2=0x7f8850108120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:30:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.402+0000 7f8855000640 1 -- 172.21.15.1:0/1862195267 shutdown_connections 2024-09-12T01:30:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:30:56.402+0000 7f8855000640 1 -- 172.21.15.1:0/1862195267 wait complete. 2024-09-12T01:30:56.490 INFO:tasks.workunit.client.0.smithi001.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-12T01:30:56.557 INFO:tasks.workunit.client.0.smithi001.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-12T01:30:56.560 INFO:tasks.workunit.client.0.smithi001.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-12T01:30:56.561 INFO:tasks.workunit.client.0.smithi001.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-12T01:30:56.626 INFO:tasks.workunit.client.0.smithi001.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-12T01:30:56.633 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-12T01:30:56.633 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-12T01:30:56.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:30:56 smithi001 ceph-mon[30387]: pgmap v438: 129 pgs: 129 active+clean; 675 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-12T01:30:56.709 INFO:tasks.workunit:Running workunits matching suites/fsx.sh on client.0... 2024-09-12T01:30:56.711 INFO:tasks.workunit:Running workunit suites/fsx.sh... 2024-09-12T01:30:56.711 DEBUG:teuthology.orchestra.run.smithi001:workunit test suites/fsx.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh 2024-09-12T01:30:56.726 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:30:56 smithi104 ceph-mon[35219]: pgmap v438: 129 pgs: 129 active+clean; 675 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-12T01:30:56.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:30:56 smithi053 ceph-mon[35158]: pgmap v438: 129 pgs: 129 active+clean; 675 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-12T01:30:56.797 INFO:tasks.workunit.client.0.smithi001.stderr:+ set -e 2024-09-12T01:30:56.797 INFO:tasks.workunit.client.0.smithi001.stderr:+ git clone https://git.ceph.com/xfstests-dev.git 2024-09-12T01:30:56.801 INFO:tasks.workunit.client.0.smithi001.stderr:Cloning into 'xfstests-dev'... 2024-09-12T01:30:58.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:30:58 smithi001 ceph-mon[30387]: pgmap v439: 129 pgs: 129 active+clean; 676 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-12T01:30:58.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:30:58 smithi053 ceph-mon[35158]: pgmap v439: 129 pgs: 129 active+clean; 676 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-12T01:30:58.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:30:58 smithi104 ceph-mon[35219]: pgmap v439: 129 pgs: 129 active+clean; 676 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-12T01:31:00.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:31:00 smithi001 ceph-mon[30387]: pgmap v440: 129 pgs: 129 active+clean; 676 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-12T01:31:00.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:31:00 smithi053 ceph-mon[35158]: pgmap v440: 129 pgs: 129 active+clean; 676 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-12T01:31:00.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:31:00 smithi104 ceph-mon[35219]: pgmap v440: 129 pgs: 129 active+clean; 676 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-12T01:31:01.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:31:01 smithi001 ceph-mon[30387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T01:31:01.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:31:01 smithi053 ceph-mon[35158]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T01:31:01.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:31:01 smithi104 ceph-mon[35219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T01:31:02.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:31:02 smithi001 ceph-mon[30387]: pgmap v441: 129 pgs: 129 active+clean; 10 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 818 KiB/s wr, 11 op/s 2024-09-12T01:31:02.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:31:02 smithi053 ceph-mon[35158]: pgmap v441: 129 pgs: 129 active+clean; 10 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 818 KiB/s wr, 11 op/s 2024-09-12T01:31:02.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:31:02 smithi104 ceph-mon[35219]: pgmap v441: 129 pgs: 129 active+clean; 10 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 818 KiB/s wr, 11 op/s 2024-09-12T01:31:04.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:31:04 smithi001 ceph-mon[30387]: pgmap v442: 129 pgs: 129 active+clean; 15 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 15 op/s 2024-09-12T01:31:04.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:31:04 smithi053 ceph-mon[35158]: pgmap v442: 129 pgs: 129 active+clean; 15 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 15 op/s 2024-09-12T01:31:04.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:31:04 smithi104 ceph-mon[35219]: pgmap v442: 129 pgs: 129 active+clean; 15 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 15 op/s 2024-09-12T01:31:06.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:31:06 smithi001 ceph-mon[30387]: pgmap v443: 129 pgs: 129 active+clean; 15 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 15 op/s 2024-09-12T01:31:06.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:31:06 smithi053 ceph-mon[35158]: pgmap v443: 129 pgs: 129 active+clean; 15 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 15 op/s 2024-09-12T01:31:06.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:31:06 smithi104 ceph-mon[35219]: pgmap v443: 129 pgs: 129 active+clean; 15 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 15 op/s 2024-09-12T01:31:08.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:31:08 smithi001 ceph-mon[30387]: pgmap v444: 129 pgs: 129 active+clean; 16 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 19 op/s 2024-09-12T01:31:08.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:31:08 smithi001 ceph-mon[30387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T01:31:08.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:31:08 smithi053 ceph-mon[35158]: pgmap v444: 129 pgs: 129 active+clean; 16 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 19 op/s 2024-09-12T01:31:08.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:31:08 smithi053 ceph-mon[35158]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T01:31:08.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:31:08 smithi104 ceph-mon[35219]: pgmap v444: 129 pgs: 129 active+clean; 16 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 19 op/s 2024-09-12T01:31:08.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:31:08 smithi104 ceph-mon[35219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T01:31:09.532 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.532+0000 7f7e90a00640 1 -- 172.21.15.1:0/428773291 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7e8c0761d0 msgr2=0x7f7e8c074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:31:09.532 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.532+0000 7f7e90a00640 1 --2- 172.21.15.1:0/428773291 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7e8c0761d0 0x7f7e8c074620 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c009900 tx=0x7f7e7c02a480 comp rx=0 tx=0).stop 2024-09-12T01:31:09.533 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.533+0000 7f7e90a00640 1 -- 172.21.15.1:0/428773291 shutdown_connections 2024-09-12T01:31:09.533 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.533+0000 7f7e90a00640 1 --2- 172.21.15.1:0/428773291 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e8c1099c0 0x7f7e8c10be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:09.533 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.533+0000 7f7e90a00640 1 --2- 172.21.15.1:0/428773291 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e8c074bf0 0x7f7e8c075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:09.533 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.533+0000 7f7e90a00640 1 --2- 172.21.15.1:0/428773291 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7e8c0761d0 0x7f7e8c074620 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:09.533 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.533+0000 7f7e90a00640 1 -- 172.21.15.1:0/428773291 >> 172.21.15.1:0/428773291 conn(0x7f7e8c0fd2b0 msgr2=0x7f7e8c0ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:31:09.533 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.533+0000 7f7e90a00640 1 -- 172.21.15.1:0/428773291 shutdown_connections 2024-09-12T01:31:09.533 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.533+0000 7f7e90a00640 1 -- 172.21.15.1:0/428773291 wait complete. 2024-09-12T01:31:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.534+0000 7f7e90a00640 1 Processor -- start 2024-09-12T01:31:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.534+0000 7f7e90a00640 1 -- start start 2024-09-12T01:31:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.534+0000 7f7e90a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e8c074bf0 0x7f7e8c19a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.534+0000 7f7e90a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e8c0761d0 0x7f7e8c19a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.534+0000 7f7e90a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7e8c1099c0 0x7f7e8c1a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.534+0000 7f7e90a00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c10fa10 con 0x7f7e8c1099c0 2024-09-12T01:31:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.534+0000 7f7e90a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c10f890 con 0x7f7e8c074bf0 2024-09-12T01:31:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.534+0000 7f7e90a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c10fb90 con 0x7f7e8c0761d0 2024-09-12T01:31:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.534+0000 7f7e89e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e8c074bf0 0x7f7e8c19a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.534+0000 7f7e89e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e8c074bf0 0x7f7e8c19a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:60650/0 (socket says 172.21.15.1:60650) 2024-09-12T01:31:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.534+0000 7f7e89e00640 1 -- 172.21.15.1:0/4115981265 learned_addr learned my addr 172.21.15.1:0/4115981265 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:31:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.534+0000 7f7e8a800640 1 --2- 172.21.15.1:0/4115981265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7e8c1099c0 0x7f7e8c1a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.535+0000 7f7e89400640 1 --2- 172.21.15.1:0/4115981265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e8c0761d0 0x7f7e8c19a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.535+0000 7f7e89e00640 1 -- 172.21.15.1:0/4115981265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e8c0761d0 msgr2=0x7f7e8c19a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:31:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.535+0000 7f7e89e00640 1 --2- 172.21.15.1:0/4115981265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e8c0761d0 0x7f7e8c19a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.535+0000 7f7e89e00640 1 -- 172.21.15.1:0/4115981265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7e8c1099c0 msgr2=0x7f7e8c1a1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:31:09.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.535+0000 7f7e89e00640 1 --2- 172.21.15.1:0/4115981265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7e8c1099c0 0x7f7e8c1a1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:09.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.535+0000 7f7e89e00640 1 -- 172.21.15.1:0/4115981265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e8c1a1cb0 con 0x7f7e8c074bf0 2024-09-12T01:31:09.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.535+0000 7f7e8a800640 1 --2- 172.21.15.1:0/4115981265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7e8c1099c0 0x7f7e8c1a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:31:09.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.535+0000 7f7e89400640 1 --2- 172.21.15.1:0/4115981265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e8c0761d0 0x7f7e8c19a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:31:09.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.535+0000 7f7e89e00640 1 --2- 172.21.15.1:0/4115981265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e8c074bf0 0x7f7e8c19a090 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c031440 tx=0x7f7e7c032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:31:09.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.535+0000 7f7e82a00640 1 -- 172.21.15.1:0/4115981265 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c03f030 con 0x7f7e8c074bf0 2024-09-12T01:31:09.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.535+0000 7f7e82a00640 1 -- 172.21.15.1:0/4115981265 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e7c003ce0 con 0x7f7e8c074bf0 2024-09-12T01:31:09.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.535+0000 7f7e90a00640 1 -- 172.21.15.1:0/4115981265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e8c1a1f40 con 0x7f7e8c074bf0 2024-09-12T01:31:09.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.535+0000 7f7e82a00640 1 -- 172.21.15.1:0/4115981265 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c035070 con 0x7f7e8c074bf0 2024-09-12T01:31:09.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.535+0000 7f7e90a00640 1 -- 172.21.15.1:0/4115981265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e8c1a2340 con 0x7f7e8c074bf0 2024-09-12T01:31:09.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.538+0000 7f7e82a00640 1 -- 172.21.15.1:0/4115981265 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f7e7c042030 con 0x7f7e8c074bf0 2024-09-12T01:31:09.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.539+0000 7f7e9380c740 1 Processor -- start 2024-09-12T01:31:09.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.539+0000 7f7e9380c740 1 -- start start 2024-09-12T01:31:09.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.539+0000 7f7e9380c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a77349d090 0x55a77349d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:09.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.539+0000 7f7e9380c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a77349d9d0 0x55a7734a2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:09.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.539+0000 7f7e9380c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7734a2a60 0x55a7734a4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:09.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.539+0000 7f7e9380c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a773402100 con 0x55a7734a2a60 2024-09-12T01:31:09.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.539+0000 7f7e9380c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a7733aa1b0 con 0x55a77349d9d0 2024-09-12T01:31:09.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.539+0000 7f7e9380c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a7733fe0c0 con 0x55a77349d090 2024-09-12T01:31:09.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.539+0000 7f7e89e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a77349d9d0 0x55a7734a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:09.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.539+0000 7f7e89e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a77349d9d0 0x55a7734a2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:60664/0 (socket says 172.21.15.1:60664) 2024-09-12T01:31:09.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.539+0000 7f7e89e00640 1 -- 172.21.15.1:0/2914110059 learned_addr learned my addr 172.21.15.1:0/2914110059 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:31:09.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.539+0000 7f7e89400640 1 --2- 172.21.15.1:0/2914110059 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7734a2a60 0x55a7734a4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:09.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.540+0000 7f7e8a800640 1 --2- 172.21.15.1:0/2914110059 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a77349d090 0x55a77349d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:09.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.540+0000 7f7e89e00640 1 -- 172.21.15.1:0/2914110059 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a77349d090 msgr2=0x55a77349d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:31:09.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.540+0000 7f7e89e00640 1 --2- 172.21.15.1:0/2914110059 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a77349d090 0x55a77349d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:09.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.540+0000 7f7e89e00640 1 -- 172.21.15.1:0/2914110059 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7734a2a60 msgr2=0x55a7734a4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:31:09.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.540+0000 7f7e89e00640 1 --2- 172.21.15.1:0/2914110059 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7734a2a60 0x55a7734a4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:09.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.540+0000 7f7e89e00640 1 -- 172.21.15.1:0/2914110059 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a773115210 con 0x55a77349d9d0 2024-09-12T01:31:09.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.540+0000 7f7e82a00640 1 --2- 172.21.15.1:0/4115981265 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f7e700a2c10 0x7f7e700a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.540+0000 7f7e89e00640 1 --2- 172.21.15.1:0/2914110059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a77349d9d0 0x55a7734a2490 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c0f3d00 tx=0x7f7e7c0f3df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f741f7a2bacdcde1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:31:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.540+0000 7f7e82a00640 1 -- 172.21.15.1:0/4115981265 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f7e7c0efb90 con 0x7f7e8c074bf0 2024-09-12T01:31:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.540+0000 7f7e8a800640 1 --2- 172.21.15.1:0/4115981265 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f7e700a2c10 0x7f7e700a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.540+0000 7f7e77000640 1 -- 172.21.15.1:0/2914110059 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c0f4040 con 0x55a77349d9d0 2024-09-12T01:31:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.541+0000 7f7e77000640 1 -- 172.21.15.1:0/2914110059 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e7c0bc5a0 con 0x55a77349d9d0 2024-09-12T01:31:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.541+0000 7f7e77000640 1 -- 172.21.15.1:0/2914110059 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c0b8140 con 0x55a77349d9d0 2024-09-12T01:31:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.541+0000 7f7e9380c740 1 -- 172.21.15.1:0/2914110059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a77349d9d0 msgr2=0x55a7734a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:31:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.541+0000 7f7e9380c740 1 --2- 172.21.15.1:0/2914110059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a77349d9d0 0x55a7734a2490 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c0f3d00 tx=0x7f7e7c0f3df0 comp rx=0 tx=0).stop 2024-09-12T01:31:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.541+0000 7f7e8a800640 1 --2- 172.21.15.1:0/4115981265 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f7e700a2c10 0x7f7e700a50d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7e840099a0 tx=0x7f7e84009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:31:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.541+0000 7f7e9380c740 1 -- 172.21.15.1:0/2914110059 shutdown_connections 2024-09-12T01:31:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.541+0000 7f7e9380c740 1 --2- 172.21.15.1:0/2914110059 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7734a2a60 0x55a7734a4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.541+0000 7f7e9380c740 1 --2- 172.21.15.1:0/2914110059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a77349d9d0 0x55a7734a2490 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.541+0000 7f7e9380c740 1 --2- 172.21.15.1:0/2914110059 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a77349d090 0x55a77349d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.541+0000 7f7e9380c740 1 -- 172.21.15.1:0/2914110059 >> 172.21.15.1:0/2914110059 conn(0x55a773498700 msgr2=0x55a773498b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:31:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.541+0000 7f7e9380c740 1 -- 172.21.15.1:0/2914110059 shutdown_connections 2024-09-12T01:31:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.541+0000 7f7e9380c740 1 -- 172.21.15.1:0/2914110059 wait complete. 2024-09-12T01:31:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.542+0000 7f7e9380c740 1 -- start start 2024-09-12T01:31:09.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.542+0000 7f7e9380c740 1 Processor -- start 2024-09-12T01:31:09.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.542+0000 7f7e9380c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a77349d090 0x55a7734a6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:09.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.542+0000 7f7e9380c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a77349d9d0 0x55a7734a7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:09.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.542+0000 7f7e9380c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7734a2a60 0x55a7734ae480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:09.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.542+0000 7f7e9380c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a7734a57f0 con 0x55a7734a2a60 2024-09-12T01:31:09.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.542+0000 7f7e9380c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a77338b090 con 0x55a77349d090 2024-09-12T01:31:09.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.542+0000 7f7e9380c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a7734a5970 con 0x55a77349d9d0 2024-09-12T01:31:09.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.542+0000 7f7e9380c740 10 client.0 _ll_register_callbacks cb 0x55a7733dff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:31:09.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.543+0000 7f7e89e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a77349d090 0x55a7734a6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:09.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.543+0000 7f7e8a800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7734a2a60 0x55a7734ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:09.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.543+0000 7f7e89e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a77349d090 0x55a7734a6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:60674/0 (socket says 172.21.15.1:60674) 2024-09-12T01:31:09.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.543+0000 7f7e89e00640 1 -- 172.21.15.1:0/688850655 learned_addr learned my addr 172.21.15.1:0/688850655 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:31:09.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.543+0000 7f7e89400640 1 --2- 172.21.15.1:0/688850655 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a77349d9d0 0x55a7734a7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:09.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.543+0000 7f7e89e00640 1 -- 172.21.15.1:0/688850655 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a77349d9d0 msgr2=0x55a7734a7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:31:09.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.543+0000 7f7e89e00640 1 --2- 172.21.15.1:0/688850655 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a77349d9d0 0x55a7734a7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:09.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.543+0000 7f7e89e00640 1 -- 172.21.15.1:0/688850655 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7734a2a60 msgr2=0x55a7734ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:31:09.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.543+0000 7f7e89e00640 1 --2- 172.21.15.1:0/688850655 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7734a2a60 0x55a7734ae480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:09.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.543+0000 7f7e89e00640 1 -- 172.21.15.1:0/688850655 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7731e1620 con 0x55a77349d090 2024-09-12T01:31:09.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.543+0000 7f7e89e00640 1 --2- 172.21.15.1:0/688850655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a77349d090 0x55a7734a6f00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c0f8650 tx=0x7f7e7c0ba3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a85a482edd2f080 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:31:09.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.543+0000 7f7e6b400640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-12T01:31:09.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.543+0000 7f7e6b400640 1 -- 172.21.15.1:0/688850655 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c049050 con 0x55a77349d090 2024-09-12T01:31:09.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.544+0000 7f7e9380c740 1 -- 172.21.15.1:0/688850655 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e7c0bc5a0 con 0x55a77349d090 2024-09-12T01:31:09.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.544+0000 7f7e6b400640 1 -- 172.21.15.1:0/688850655 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f7e7c0b9f40 con 0x55a77349d090 2024-09-12T01:31:09.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.544+0000 7f7e6b400640 1 -- 172.21.15.1:0/688850655 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7e7c0bb800 con 0x55a77349d090 2024-09-12T01:31:09.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.544+0000 7f7e9380c740 10 client.25325 fetch_fsmap learned FSMap version 31 2024-09-12T01:31:09.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.544+0000 7f7e9380c740 1 -- 172.21.15.1:0/688850655 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a7734a57f0 con 0x55a77349d090 2024-09-12T01:31:09.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.544+0000 7f7e6b400640 1 -- 172.21.15.1:0/688850655 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e7c0bc5a0 con 0x55a77349d090 2024-09-12T01:31:09.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.544+0000 7f7e6b400640 1 -- 172.21.15.1:0/688850655 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f7e7c0fa040 con 0x55a77349d090 2024-09-12T01:31:09.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.544+0000 7f7e9380c740 10 client.25325 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T01:31:09.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.544+0000 7f7e9380c740 10 client.25325 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:31:09.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.544+0000 7f7e9380c740 20 client.25325 populate_metadata read hostname 'smithi001' 2024-09-12T01:31:09.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.544+0000 7f7e9380c740 1 --2- 172.21.15.1:0/688850655 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55a7734b2780 0x55a7734b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:09.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.544+0000 7f7e9380c740 4 client.25325 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:31:09.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.544+0000 7f7e9380c740 1 -- 172.21.15.1:0/688850655 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a77340f830 con 0x55a7734b2780 2024-09-12T01:31:09.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.545+0000 7f7e89400640 1 --2- 172.21.15.1:0/688850655 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55a7734b2780 0x55a7734b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:09.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.545+0000 7f7e89400640 1 --2- 172.21.15.1:0/688850655 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55a7734b2780 0x55a7734b4ba0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c71d6dc39dbb28a server_cookie=847ebb0aa65ac3fc in_seq=0 out_seq=0 2024-09-12T01:31:09.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.545+0000 7f7e6b400640 10 client.25325 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:31:09.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.546+0000 7f7e6b400640 1 -- 172.21.15.1:0/688850655 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a77340f830 con 0x55a7734b2780 2024-09-12T01:31:09.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.546+0000 7f7e6b400640 10 client.25325 handle_command_reply: tid=0 2024-09-12T01:31:09.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.546+0000 7f7e9380c740 1 client.25325 shutdown 2024-09-12T01:31:09.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.546+0000 7f7e9380c740 1 -- 172.21.15.1:0/688850655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a77349d090 msgr2=0x55a7734a6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:31:09.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.546+0000 7f7e9380c740 1 --2- 172.21.15.1:0/688850655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a77349d090 0x55a7734a6f00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c0f8650 tx=0x7f7e7c0ba3b0 comp rx=0 tx=0).stop 2024-09-12T01:31:09.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.546+0000 7f7e9380c740 1 -- 172.21.15.1:0/688850655 shutdown_connections 2024-09-12T01:31:09.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.546+0000 7f7e9380c740 1 --2- 172.21.15.1:0/688850655 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55a7734b2780 0x55a7734b4ba0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:09.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.546+0000 7f7e9380c740 1 --2- 172.21.15.1:0/688850655 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7734a2a60 0x55a7734ae480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:09.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.546+0000 7f7e9380c740 1 --2- 172.21.15.1:0/688850655 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a77349d9d0 0x55a7734a7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:09.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.546+0000 7f7e9380c740 1 --2- 172.21.15.1:0/688850655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a77349d090 0x55a7734a6f00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:09.552 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.546+0000 7f7e6b400640 0 client.25325 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:31:09.552 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.546+0000 7f7e9380c740 1 -- 172.21.15.1:0/688850655 >> 172.21.15.1:0/688850655 conn(0x55a773498700 msgr2=0x55a773499f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:31:09.552 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.546+0000 7f7e9380c740 1 -- 172.21.15.1:0/688850655 shutdown_connections 2024-09-12T01:31:09.552 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.546+0000 7f7e9380c740 1 -- 172.21.15.1:0/688850655 wait complete. 2024-09-12T01:31:09.552 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.546+0000 7f7e9380c740 20 client.25325 trim_cache size 0 max 16384 2024-09-12T01:31:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.562+0000 7f7e9380c740 1 Processor -- start 2024-09-12T01:31:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.562+0000 7f7e9380c740 1 -- start start 2024-09-12T01:31:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.562+0000 7f7e9380c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a77349d090 0x55a7734a6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.562+0000 7f7e9380c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a77349d9d0 0x55a7734ae3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.562+0000 7f7e9380c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7734a2a60 0x55a7734a71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.562+0000 7f7e9380c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c1ac990 con 0x55a77349d9d0 2024-09-12T01:31:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.562+0000 7f7e9380c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c1ae530 con 0x55a7734a2a60 2024-09-12T01:31:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.562+0000 7f7e9380c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c1ad060 con 0x55a77349d090 2024-09-12T01:31:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.562+0000 7f7e89400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a77349d9d0 0x55a7734ae3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.562+0000 7f7e8a800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7734a2a60 0x55a7734a71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.562+0000 7f7e89400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a77349d9d0 0x55a7734ae3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36518/0 (socket says 172.21.15.1:36518) 2024-09-12T01:31:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.562+0000 7f7e89400640 1 -- 172.21.15.1:0/2872276712 learned_addr learned my addr 172.21.15.1:0/2872276712 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:31:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.562+0000 7f7e89e00640 1 --2- 172.21.15.1:0/2872276712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a77349d090 0x55a7734a6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.563+0000 7f7e89400640 1 -- 172.21.15.1:0/2872276712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a77349d090 msgr2=0x55a7734a6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:31:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.563+0000 7f7e89400640 1 --2- 172.21.15.1:0/2872276712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a77349d090 0x55a7734a6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.563+0000 7f7e89400640 1 -- 172.21.15.1:0/2872276712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7734a2a60 msgr2=0x55a7734a71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:31:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.563+0000 7f7e89400640 1 --2- 172.21.15.1:0/2872276712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7734a2a60 0x55a7734a71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.563+0000 7f7e89400640 1 -- 172.21.15.1:0/2872276712 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7734b5220 con 0x55a77349d9d0 2024-09-12T01:31:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.563+0000 7f7e89e00640 1 --2- 172.21.15.1:0/2872276712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a77349d090 0x55a7734a6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:31:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.563+0000 7f7e89400640 1 --2- 172.21.15.1:0/2872276712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a77349d9d0 0x55a7734ae3a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7e78005940 tx=0x7f7e78014840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa559fab2d78bcfe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:31:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.563+0000 7f7e76600640 1 -- 172.21.15.1:0/2872276712 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e78004030 con 0x55a77349d9d0 2024-09-12T01:31:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.563+0000 7f7e76600640 1 -- 172.21.15.1:0/2872276712 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e780041d0 con 0x55a77349d9d0 2024-09-12T01:31:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.563+0000 7f7e76600640 1 -- 172.21.15.1:0/2872276712 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e780126b0 con 0x55a77349d9d0 2024-09-12T01:31:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.564+0000 7f7e9380c740 1 -- 172.21.15.1:0/2872276712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a77349d9d0 msgr2=0x55a7734ae3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:31:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.564+0000 7f7e9380c740 1 --2- 172.21.15.1:0/2872276712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a77349d9d0 0x55a7734ae3a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7e78005940 tx=0x7f7e78014840 comp rx=0 tx=0).stop 2024-09-12T01:31:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.564+0000 7f7e9380c740 1 -- 172.21.15.1:0/2872276712 shutdown_connections 2024-09-12T01:31:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.564+0000 7f7e9380c740 1 --2- 172.21.15.1:0/2872276712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7734a2a60 0x55a7734a71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.564+0000 7f7e9380c740 1 --2- 172.21.15.1:0/2872276712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a77349d9d0 0x55a7734ae3a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.564+0000 7f7e9380c740 1 --2- 172.21.15.1:0/2872276712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a77349d090 0x55a7734a6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.564+0000 7f7e9380c740 1 -- 172.21.15.1:0/2872276712 >> 172.21.15.1:0/2872276712 conn(0x55a773498700 msgr2=0x55a77349eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:31:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.564+0000 7f7e9380c740 1 -- 172.21.15.1:0/2872276712 shutdown_connections 2024-09-12T01:31:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.564+0000 7f7e9380c740 1 -- 172.21.15.1:0/2872276712 wait complete. 2024-09-12T01:31:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.564+0000 7f7e9380c740 1 -- start start 2024-09-12T01:31:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.565+0000 7f7e9380c740 1 Processor -- start 2024-09-12T01:31:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.565+0000 7f7e9380c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a77349d090 0x55a7734ab2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.565+0000 7f7e9380c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a77349d9d0 0x55a7734ab810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.565+0000 7f7e9380c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7734a2a60 0x55a77351b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.565+0000 7f7e9380c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c1ade30 con 0x55a77349d090 2024-09-12T01:31:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.565+0000 7f7e9380c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c1ad760 con 0x55a7734a2a60 2024-09-12T01:31:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.565+0000 7f7e9380c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c1aa970 con 0x55a77349d9d0 2024-09-12T01:31:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.565+0000 7f7e9380c740 10 client.0 _ll_register_callbacks cb 0x7f7e780095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:31:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.565+0000 7f7e89e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a77349d090 0x55a7734ab2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.565+0000 7f7e8a800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7734a2a60 0x55a77351b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.565+0000 7f7e89e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a77349d090 0x55a7734ab2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36522/0 (socket says 172.21.15.1:36522) 2024-09-12T01:31:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.565+0000 7f7e89e00640 1 -- 172.21.15.1:0/1023936258 learned_addr learned my addr 172.21.15.1:0/1023936258 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:31:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.565+0000 7f7e8a800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7734a2a60 0x55a77351b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:60688/0 (socket says 172.21.15.1:60688) 2024-09-12T01:31:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.565+0000 7f7e89400640 1 --2- 172.21.15.1:0/1023936258 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a77349d9d0 0x55a7734ab810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.566+0000 7f7e8a800640 1 -- 172.21.15.1:0/1023936258 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a77349d9d0 msgr2=0x55a7734ab810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:31:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.566+0000 7f7e8a800640 1 --2- 172.21.15.1:0/1023936258 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a77349d9d0 0x55a7734ab810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.566+0000 7f7e8a800640 1 -- 172.21.15.1:0/1023936258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a77349d090 msgr2=0x55a7734ab2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:31:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.566+0000 7f7e8a800640 1 --2- 172.21.15.1:0/1023936258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a77349d090 0x55a7734ab2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.566+0000 7f7e8a800640 1 -- 172.21.15.1:0/1023936258 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a773499390 con 0x55a7734a2a60 2024-09-12T01:31:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.566+0000 7f7e89e00640 1 --2- 172.21.15.1:0/1023936258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a77349d090 0x55a7734ab2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:31:09.570 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.566+0000 7f7e8a800640 1 --2- 172.21.15.1:0/1023936258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7734a2a60 0x55a77351b470 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7e84012010 tx=0x7f7e84004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2fdfe163674c164 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:31:09.570 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.566+0000 7f7e6b400640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-12T01:31:09.570 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.566+0000 7f7e6b400640 1 -- 172.21.15.1:0/1023936258 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8401eb80 con 0x55a7734a2a60 2024-09-12T01:31:09.570 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.566+0000 7f7e6b400640 1 -- 172.21.15.1:0/1023936258 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f7e8401ed20 con 0x55a7734a2a60 2024-09-12T01:31:09.570 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.566+0000 7f7e9380c740 1 -- 172.21.15.1:0/1023936258 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e780041d0 con 0x55a7734a2a60 2024-09-12T01:31:09.570 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.566+0000 7f7e6b400640 1 -- 172.21.15.1:0/1023936258 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7e84034050 con 0x55a7734a2a60 2024-09-12T01:31:09.570 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.566+0000 7f7e9380c740 10 client.25331 fetch_fsmap learned FSMap version 31 2024-09-12T01:31:09.570 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.567+0000 7f7e9380c740 1 -- 172.21.15.1:0/1023936258 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a77351cbf0 con 0x55a7734a2a60 2024-09-12T01:31:09.570 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.567+0000 7f7e6b400640 1 -- 172.21.15.1:0/1023936258 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e780041d0 con 0x55a7734a2a60 2024-09-12T01:31:09.570 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.567+0000 7f7e6b400640 1 -- 172.21.15.1:0/1023936258 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f7e840057b0 con 0x55a7734a2a60 2024-09-12T01:31:09.570 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.567+0000 7f7e9380c740 10 client.25331 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T01:31:09.570 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.567+0000 7f7e9380c740 10 client.25331 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:31:09.570 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.567+0000 7f7e9380c740 20 client.25331 populate_metadata read hostname 'smithi001' 2024-09-12T01:31:09.570 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.567+0000 7f7e9380c740 1 --2- 172.21.15.1:0/1023936258 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55a77351d650 0x55a77351fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:09.570 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.567+0000 7f7e9380c740 4 client.25331 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:31:09.571 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.567+0000 7f7e9380c740 1 -- 172.21.15.1:0/1023936258 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a773409b80 con 0x55a77351d650 2024-09-12T01:31:09.571 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.567+0000 7f7e89e00640 1 --2- 172.21.15.1:0/1023936258 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55a77351d650 0x55a77351fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:09.571 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.568+0000 7f7e89e00640 1 --2- 172.21.15.1:0/1023936258 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55a77351d650 0x55a77351fa70 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5feb913f45983a7 server_cookie=10392342ed8fcca9 in_seq=0 out_seq=0 2024-09-12T01:31:09.571 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:09.568+0000 7f7e6b400640 10 client.25331 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:31:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:10.664+0000 7f7e6b400640 1 -- 172.21.15.1:0/1023936258 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a773409b80 con 0x55a77351d650 2024-09-12T01:31:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:10.664+0000 7f7e6b400640 10 client.25331 handle_command_reply: tid=0 2024-09-12T01:31:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:10.664+0000 7f7e9380c740 1 client.25331 shutdown 2024-09-12T01:31:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:10.664+0000 7f7e9380c740 1 -- 172.21.15.1:0/1023936258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7734a2a60 msgr2=0x55a77351b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:31:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:10.664+0000 7f7e9380c740 1 --2- 172.21.15.1:0/1023936258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7734a2a60 0x55a77351b470 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7e84012010 tx=0x7f7e84004a40 comp rx=0 tx=0).stop 2024-09-12T01:31:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:10.665+0000 7f7e9380c740 1 -- 172.21.15.1:0/1023936258 shutdown_connections 2024-09-12T01:31:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:10.665+0000 7f7e9380c740 1 --2- 172.21.15.1:0/1023936258 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55a77351d650 0x55a77351fa70 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:10.665+0000 7f7e9380c740 1 --2- 172.21.15.1:0/1023936258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7734a2a60 0x55a77351b470 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:10.665+0000 7f7e9380c740 1 --2- 172.21.15.1:0/1023936258 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a77349d9d0 0x55a7734ab810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:10.665+0000 7f7e6b400640 0 client.25331 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:31:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:10.665+0000 7f7e9380c740 1 --2- 172.21.15.1:0/1023936258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a77349d090 0x55a7734ab2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:10.665+0000 7f7e9380c740 1 -- 172.21.15.1:0/1023936258 >> 172.21.15.1:0/1023936258 conn(0x55a773498700 msgr2=0x55a7734a6c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:31:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:10.665+0000 7f7e9380c740 1 -- 172.21.15.1:0/1023936258 shutdown_connections 2024-09-12T01:31:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:10.665+0000 7f7e9380c740 1 -- 172.21.15.1:0/1023936258 wait complete. 2024-09-12T01:31:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:10.665+0000 7f7e9380c740 20 client.25331 trim_cache size 0 max 16384 2024-09-12T01:31:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-12T01:31:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-12T01:31:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-12T01:31:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-12T01:31:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:10.666+0000 7f7e90a00640 1 -- 172.21.15.1:0/4115981265 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f7e700a2c10 msgr2=0x7f7e700a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:31:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:10.666+0000 7f7e90a00640 1 --2- 172.21.15.1:0/4115981265 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f7e700a2c10 0x7f7e700a50d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7e840099a0 tx=0x7f7e84009500 comp rx=0 tx=0).stop 2024-09-12T01:31:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:10.666+0000 7f7e90a00640 1 -- 172.21.15.1:0/4115981265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e8c074bf0 msgr2=0x7f7e8c19a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:31:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:10.666+0000 7f7e90a00640 1 --2- 172.21.15.1:0/4115981265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e8c074bf0 0x7f7e8c19a090 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c031440 tx=0x7f7e7c032ce0 comp rx=0 tx=0).stop 2024-09-12T01:31:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:10.666+0000 7f7e90a00640 1 -- 172.21.15.1:0/4115981265 shutdown_connections 2024-09-12T01:31:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:10.666+0000 7f7e90a00640 1 --2- 172.21.15.1:0/4115981265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7e8c1099c0 0x7f7e8c1a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:10.666+0000 7f7e90a00640 1 --2- 172.21.15.1:0/4115981265 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f7e700a2c10 0x7f7e700a50d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:10.666+0000 7f7e90a00640 1 --2- 172.21.15.1:0/4115981265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e8c0761d0 0x7f7e8c19a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:10.666+0000 7f7e90a00640 1 --2- 172.21.15.1:0/4115981265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e8c074bf0 0x7f7e8c19a090 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:10.666+0000 7f7e90a00640 1 -- 172.21.15.1:0/4115981265 >> 172.21.15.1:0/4115981265 conn(0x7f7e8c0fd2b0 msgr2=0x7f7e8c105950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:31:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:10.666+0000 7f7e90a00640 1 -- 172.21.15.1:0/4115981265 shutdown_connections 2024-09-12T01:31:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:10.666+0000 7f7e90a00640 1 -- 172.21.15.1:0/4115981265 wait complete. 2024-09-12T01:31:10.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:31:10 smithi001 ceph-mon[30387]: pgmap v445: 129 pgs: 129 active+clean; 16 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 17 op/s 2024-09-12T01:31:10.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:31:10 smithi053 ceph-mon[35158]: pgmap v445: 129 pgs: 129 active+clean; 16 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 17 op/s 2024-09-12T01:31:10.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:31:10 smithi104 ceph-mon[35219]: pgmap v445: 129 pgs: 129 active+clean; 16 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 17 op/s 2024-09-12T01:31:12.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:31:12 smithi001 ceph-mon[30387]: pgmap v446: 129 pgs: 129 active+clean; 18 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 207 op/s 2024-09-12T01:31:12.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:31:12 smithi053 ceph-mon[35158]: pgmap v446: 129 pgs: 129 active+clean; 18 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 207 op/s 2024-09-12T01:31:12.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:31:12 smithi104 ceph-mon[35219]: pgmap v446: 129 pgs: 129 active+clean; 18 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 207 op/s 2024-09-12T01:31:14.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:31:14 smithi053 ceph-mon[35158]: pgmap v447: 129 pgs: 129 active+clean; 16 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 947 KiB/s wr, 318 op/s 2024-09-12T01:31:14.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:31:14 smithi104 ceph-mon[35219]: pgmap v447: 129 pgs: 129 active+clean; 16 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 947 KiB/s wr, 318 op/s 2024-09-12T01:31:14.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:31:14 smithi001 ceph-mon[30387]: pgmap v447: 129 pgs: 129 active+clean; 16 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 947 KiB/s wr, 318 op/s 2024-09-12T01:31:16.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:31:16 smithi053 ceph-mon[35158]: pgmap v448: 129 pgs: 129 active+clean; 18 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 744 KiB/s wr, 513 op/s 2024-09-12T01:31:16.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:31:16 smithi053 ceph-mon[35158]: mgrmap e20: x(active, since 12m), standbys: z, y 2024-09-12T01:31:16.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:31:16 smithi104 ceph-mon[35219]: pgmap v448: 129 pgs: 129 active+clean; 18 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 744 KiB/s wr, 513 op/s 2024-09-12T01:31:16.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:31:16 smithi104 ceph-mon[35219]: mgrmap e20: x(active, since 12m), standbys: z, y 2024-09-12T01:31:16.843 INFO:tasks.workunit.client.0.smithi001.stderr:Updating files: 11% (494/4351) Updating files: 12% (523/4351) Updating files: 13% (566/4351) Updating files: 14% (610/4351) Updating files: 15% (653/4351) Updating files: 16% (697/4351) Updating files: 17% (740/4351) Updating files: 18% (784/4351) Updating files: 19% (827/4351) Updating files: 19% (846/4351) Updating files: 20% (871/4351) Updating files: 21% (914/4351) Updating files: 22% (958/4351) Updating files: 23% (1001/4351) Updating files: 24% (1045/4351) Updating files: 25% (1088/4351) Updating files: 26% (1132/4351) Updating files: 27% (1175/4351) Updating files: 28% (1219/4351) Updating files: 29% (1262/4351) Updating files: 29% (1303/4351) Updating files: 30% (1306/4351) Updating files: 31% (1349/4351) Updating files: 32% (1393/4351) Updating files: 33% (1436/4351) Updating files: 34% (1480/4351) Updating files: 35% (1523/4351) Updating files: 36% (1567/4351) Updating files: 37% (1610/4351) Updating files: 38% (1654/4351) Updating files: 39% (1697/4351) Updating files: 40% (1741/4351) Updating files: 41% (1784/4351) Updating files: 42% (1828/4351) Updating files: 42% (1869/4351) Updating files: 43% (1871/4351) Updating files: 44% (1915/4351) Updating files: 45% (1958/4351) Updating files: 46% (2002/4351) Updating files: 47% (2045/4351) Updating files: 48% (2089/4351) Updating files: 49% (2132/4351) Updating files: 50% (2176/4351) Updating files: 51% (2220/4351) Updating files: 52% (2263/4351) Updating files: 53% (2307/4351) Updating files: 54% (2350/4351) Updating files: 55% (2394/4351) Updating files: 55% (2426/4351) Updating files: 56% (2437/4351) Updating files: 57% (2481/4351) Updating files: 58% (2524/4351) Updating files: 58% (2552/4351) Updating files: 59% (2568/4351) Updating files: 60% (2611/4351) Updating files: 61% (2655/4351) Updating files: 62% (2698/4351) Updating files: 63% (2742/4351) Updating files: 64% (2785/4351) Updating files: 64% (2809/4351) Updating files: 65% (2829/4351) Updating files: 66% (2872/4351) Updating files: 67% (2916/4351) Updating files: 68% (2959/4351) Updating files: 69% (3003/4351) Updating files: 69% (3040/4351) Updating files: 70% (3046/4351) Updating files: 71% (3090/4351) Updating files: 72% (3133/4351) Updating files: 73% (3177/4351) Updating files: 74% (3220/4351) Updating files: 75% (3264/4351) Updating files: 76% (3307/4351) Updating files: 77% (3351/4351) Updating files: 78% (3394/4351) Updating files: 79% (3438/4351) Updating files: 80% (3481/4351) Updating files: 81% (3525/4351) Updating files: 81% (3547/4351) Updating files: 82% (3568/4351) Updating files: 83% (3612/4351) Updating files: 84% (3655/4351) Updating files: 85% (3699/4351) Updating files: 86% (3742/4351) Updating files: 87% (3786/4351) Updating files: 88% (3829/4351) Updating files: 89% (3873/4351) Updating files: 90% (3916/4351) Updating files: 91% (3960/4351) Updating files: 92% (4003/4351) Updating files: 93% (4047/4351) Updating files: 94% (4090/4351) Updating files: 94% (4122/4351) Updating files: 95% (4134/4351) Updating files: 96% (4177/4351) Updating files: 97% (4221/4351) Updating files: 98% (4264/4351) Updating files: 99% (4308/4351) Updating files: 100% (4351/4351) Updating files: 100% (4351/4351), done. 2024-09-12T01:31:16.853 INFO:tasks.workunit.client.0.smithi001.stderr:+ cd xfstests-dev 2024-09-12T01:31:16.854 INFO:tasks.workunit.client.0.smithi001.stderr:+ git checkout 0e5c12dfd008efc2848c98108c9237487e91ef35 2024-09-12T01:31:16.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:31:16 smithi001 ceph-mon[30387]: pgmap v448: 129 pgs: 129 active+clean; 18 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 744 KiB/s wr, 513 op/s 2024-09-12T01:31:16.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:31:16 smithi001 ceph-mon[30387]: mgrmap e20: x(active, since 12m), standbys: z, y 2024-09-12T01:31:18.697 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:31:18 smithi001 ceph-mon[30387]: pgmap v449: 129 pgs: 129 active+clean; 18 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 758 KiB/s wr, 577 op/s 2024-09-12T01:31:18.697 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:31:18 smithi001 ceph-mon[30387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T01:31:18.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:31:18 smithi053 ceph-mon[35158]: pgmap v449: 129 pgs: 129 active+clean; 18 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 758 KiB/s wr, 577 op/s 2024-09-12T01:31:18.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:31:18 smithi053 ceph-mon[35158]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T01:31:18.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:31:18 smithi104 ceph-mon[35219]: pgmap v449: 129 pgs: 129 active+clean; 18 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 758 KiB/s wr, 577 op/s 2024-09-12T01:31:18.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:31:18 smithi104 ceph-mon[35219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T01:31:20.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:31:20 smithi053 ceph-mon[35158]: pgmap v450: 129 pgs: 129 active+clean; 18 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 763 KiB/s wr, 585 op/s 2024-09-12T01:31:20.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:31:20 smithi104 ceph-mon[35219]: pgmap v450: 129 pgs: 129 active+clean; 18 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 763 KiB/s wr, 585 op/s 2024-09-12T01:31:20.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:31:20 smithi001 ceph-mon[30387]: pgmap v450: 129 pgs: 129 active+clean; 18 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 763 KiB/s wr, 585 op/s 2024-09-12T01:31:22.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:31:22 smithi053 ceph-mon[35158]: pgmap v451: 129 pgs: 129 active+clean; 28 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 717 op/s 2024-09-12T01:31:22.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:31:22 smithi104 ceph-mon[35219]: pgmap v451: 129 pgs: 129 active+clean; 28 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 717 op/s 2024-09-12T01:31:22.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:31:22 smithi001 ceph-mon[30387]: pgmap v451: 129 pgs: 129 active+clean; 28 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 717 op/s 2024-09-12T01:31:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.786+0000 7f24eea00640 1 -- 172.21.15.1:0/1887983158 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24e8105900 msgr2=0x7f24e810c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:31:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.786+0000 7f24eea00640 1 --2- 172.21.15.1:0/1887983158 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24e8105900 0x7f24e810c190 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f24e0009be0 tx=0x7f24e002a6c0 comp rx=0 tx=0).stop 2024-09-12T01:31:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.787+0000 7f24eea00640 1 -- 172.21.15.1:0/1887983158 shutdown_connections 2024-09-12T01:31:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.787+0000 7f24eea00640 1 --2- 172.21.15.1:0/1887983158 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24e8105900 0x7f24e810c190 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.787+0000 7f24eea00640 1 --2- 172.21.15.1:0/1887983158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24e8104f40 0x7f24e81053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.787+0000 7f24eea00640 1 --2- 172.21.15.1:0/1887983158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f24e8103d50 0x7f24e8104150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.787+0000 7f24eea00640 1 -- 172.21.15.1:0/1887983158 >> 172.21.15.1:0/1887983158 conn(0x7f24e80ff4b0 msgr2=0x7f24e81018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:31:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.787+0000 7f24eea00640 1 -- 172.21.15.1:0/1887983158 shutdown_connections 2024-09-12T01:31:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.787+0000 7f24eea00640 1 -- 172.21.15.1:0/1887983158 wait complete. 2024-09-12T01:31:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.788+0000 7f24eea00640 1 Processor -- start 2024-09-12T01:31:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.788+0000 7f24eea00640 1 -- start start 2024-09-12T01:31:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.788+0000 7f24eea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24e8103d50 0x7f24e819a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.788+0000 7f24eea00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24e8104f40 0x7f24e819a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.788+0000 7f24eea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f24e8105900 0x7f24e81a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.788+0000 7f24eea00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f24e810f9d0 con 0x7f24e8104f40 2024-09-12T01:31:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.788+0000 7f24eea00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f24e810f850 con 0x7f24e8103d50 2024-09-12T01:31:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.788+0000 7f24eea00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f24e810fb50 con 0x7f24e8105900 2024-09-12T01:31:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.788+0000 7f24e7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24e8103d50 0x7f24e819a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.788+0000 7f24e7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24e8103d50 0x7f24e819a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:58834/0 (socket says 172.21.15.1:58834) 2024-09-12T01:31:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.788+0000 7f24e7e00640 1 -- 172.21.15.1:0/2335877622 learned_addr learned my addr 172.21.15.1:0/2335877622 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:31:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.788+0000 7f24eca00640 1 --2- 172.21.15.1:0/2335877622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f24e8105900 0x7f24e81a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.788+0000 7f24e7400640 1 --2- 172.21.15.1:0/2335877622 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24e8104f40 0x7f24e819a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.788+0000 7f24e7e00640 1 -- 172.21.15.1:0/2335877622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f24e8105900 msgr2=0x7f24e81a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:31:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.788+0000 7f24e7e00640 1 --2- 172.21.15.1:0/2335877622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f24e8105900 0x7f24e81a15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.788+0000 7f24e7e00640 1 -- 172.21.15.1:0/2335877622 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24e8104f40 msgr2=0x7f24e819a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:31:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.788+0000 7f24e7e00640 1 --2- 172.21.15.1:0/2335877622 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24e8104f40 0x7f24e819a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.788+0000 7f24e7e00640 1 -- 172.21.15.1:0/2335877622 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24e81a1c40 con 0x7f24e8103d50 2024-09-12T01:31:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.788+0000 7f24e7e00640 1 --2- 172.21.15.1:0/2335877622 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24e8103d50 0x7f24e819a020 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f24d800e9c0 tx=0x7f24d800edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:31:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.788+0000 7f24e4c00640 1 -- 172.21.15.1:0/2335877622 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d800ccc0 con 0x7f24e8103d50 2024-09-12T01:31:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.788+0000 7f24eea00640 1 -- 172.21.15.1:0/2335877622 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24e81a1f30 con 0x7f24e8103d50 2024-09-12T01:31:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.788+0000 7f24e4c00640 1 -- 172.21.15.1:0/2335877622 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24d80035f0 con 0x7f24e8103d50 2024-09-12T01:31:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.788+0000 7f24e4c00640 1 -- 172.21.15.1:0/2335877622 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d800ce60 con 0x7f24e8103d50 2024-09-12T01:31:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.788+0000 7f24eea00640 1 -- 172.21.15.1:0/2335877622 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24e81a2340 con 0x7f24e8103d50 2024-09-12T01:31:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24f1854740 1 Processor -- start 2024-09-12T01:31:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24e4c00640 1 -- 172.21.15.1:0/2335877622 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f24d80037a0 con 0x7f24e8103d50 2024-09-12T01:31:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24f1854740 1 -- start start 2024-09-12T01:31:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24f1854740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557582b35090 0x557582b35490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24f1854740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557582b359d0 0x557582b3a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24f1854740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557582b3aa60 0x557582b3cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24f1854740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x557582a9a100 con 0x557582b35090 2024-09-12T01:31:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24f1854740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x557582a421b0 con 0x557582b359d0 2024-09-12T01:31:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24f1854740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557582a960c0 con 0x557582b3aa60 2024-09-12T01:31:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24e7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557582b359d0 0x557582b3a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24eca00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557582b35090 0x557582b35490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24e7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557582b359d0 0x557582b3a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:58836/0 (socket says 172.21.15.1:58836) 2024-09-12T01:31:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24eca00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557582b35090 0x557582b35490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47710/0 (socket says 172.21.15.1:47710) 2024-09-12T01:31:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24e7e00640 1 -- 172.21.15.1:0/1411726712 learned_addr learned my addr 172.21.15.1:0/1411726712 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:31:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24e7400640 1 --2- 172.21.15.1:0/1411726712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557582b3aa60 0x557582b3cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24e7e00640 1 -- 172.21.15.1:0/1411726712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557582b3aa60 msgr2=0x557582b3cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:31:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24e7e00640 1 --2- 172.21.15.1:0/1411726712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557582b3aa60 0x557582b3cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24e7e00640 1 -- 172.21.15.1:0/1411726712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557582b35090 msgr2=0x557582b35490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:31:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24e7e00640 1 --2- 172.21.15.1:0/1411726712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557582b35090 0x557582b35490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24e7e00640 1 -- 172.21.15.1:0/1411726712 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575827ad210 con 0x557582b359d0 2024-09-12T01:31:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24eca00640 1 --2- 172.21.15.1:0/1411726712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557582b35090 0x557582b35490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:31:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24e7e00640 1 --2- 172.21.15.1:0/1411726712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557582b359d0 0x557582b3a490 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f24d80d5a80 tx=0x7f24d80d5b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e16b40268ad7f7fd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:31:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24e4c00640 1 --2- 172.21.15.1:0/2335877622 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f24cc0a2b80 0x7f24cc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24dd200640 1 -- 172.21.15.1:0/1411726712 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d80de070 con 0x557582b359d0 2024-09-12T01:31:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24e4c00640 1 -- 172.21.15.1:0/2335877622 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f24d80cf730 con 0x7f24e8103d50 2024-09-12T01:31:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24dd200640 1 -- 172.21.15.1:0/1411726712 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24d809c300 con 0x557582b359d0 2024-09-12T01:31:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24dd200640 1 -- 172.21.15.1:0/1411726712 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d809b560 con 0x557582b359d0 2024-09-12T01:31:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24eca00640 1 --2- 172.21.15.1:0/2335877622 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f24cc0a2b80 0x7f24cc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:23.798 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24f1854740 1 -- 172.21.15.1:0/1411726712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557582b359d0 msgr2=0x557582b3a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:31:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24f1854740 1 --2- 172.21.15.1:0/1411726712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557582b359d0 0x557582b3a490 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f24d80d5a80 tx=0x7f24d80d5b70 comp rx=0 tx=0).stop 2024-09-12T01:31:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24eca00640 1 --2- 172.21.15.1:0/2335877622 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f24cc0a2b80 0x7f24cc0a5040 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f24e002abe0 tx=0x7f24e0005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:31:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24f1854740 1 -- 172.21.15.1:0/1411726712 shutdown_connections 2024-09-12T01:31:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24f1854740 1 --2- 172.21.15.1:0/1411726712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557582b3aa60 0x557582b3cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24f1854740 1 --2- 172.21.15.1:0/1411726712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557582b359d0 0x557582b3a490 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24f1854740 1 --2- 172.21.15.1:0/1411726712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557582b35090 0x557582b35490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24f1854740 1 -- 172.21.15.1:0/1411726712 >> 172.21.15.1:0/1411726712 conn(0x557582b30700 msgr2=0x557582b30b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:31:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24f1854740 1 -- 172.21.15.1:0/1411726712 shutdown_connections 2024-09-12T01:31:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24f1854740 1 -- 172.21.15.1:0/1411726712 wait complete. 2024-09-12T01:31:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24f1854740 1 -- start start 2024-09-12T01:31:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24f1854740 1 Processor -- start 2024-09-12T01:31:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24f1854740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557582b35090 0x557582b3f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24f1854740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557582b359d0 0x557582b3f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24f1854740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557582b3aa60 0x557582b46760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24f1854740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x557582b3da30 con 0x557582b3aa60 2024-09-12T01:31:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24f1854740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x557582a23090 con 0x557582b359d0 2024-09-12T01:31:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24f1854740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557582b3dbb0 con 0x557582b35090 2024-09-12T01:31:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24f1854740 10 client.0 _ll_register_callbacks cb 0x557582a77f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:31:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24e7400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557582b359d0 0x557582b3f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24e7400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557582b359d0 0x557582b3f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:58838/0 (socket says 172.21.15.1:58838) 2024-09-12T01:31:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24e7400640 1 -- 172.21.15.1:0/2326821498 learned_addr learned my addr 172.21.15.1:0/2326821498 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:31:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24eca00640 1 --2- 172.21.15.1:0/2326821498 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557582b3aa60 0x557582b46760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24e7e00640 1 --2- 172.21.15.1:0/2326821498 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557582b35090 0x557582b3f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24e7400640 1 -- 172.21.15.1:0/2326821498 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557582b35090 msgr2=0x557582b3f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:31:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24e7400640 1 --2- 172.21.15.1:0/2326821498 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557582b35090 0x557582b3f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.793+0000 7f24e7400640 1 -- 172.21.15.1:0/2326821498 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557582b3aa60 msgr2=0x557582b46760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:31:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.798+0000 7f24e7400640 1 --2- 172.21.15.1:0/2326821498 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557582b3aa60 0x557582b46760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.798+0000 7f24e7400640 1 -- 172.21.15.1:0/2326821498 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557582879620 con 0x557582b359d0 2024-09-12T01:31:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.798+0000 7f24eca00640 1 --2- 172.21.15.1:0/2326821498 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557582b3aa60 0x557582b46760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:31:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.798+0000 7f24e7400640 1 --2- 172.21.15.1:0/2326821498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557582b359d0 0x557582b3f720 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f24d4012720 tx=0x7f24d4012be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f06a141fc70169d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:31:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.798+0000 7f24d1600640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-12T01:31:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.798+0000 7f24d1600640 1 -- 172.21.15.1:0/2326821498 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d400f920 con 0x557582b359d0 2024-09-12T01:31:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.798+0000 7f24f1854740 1 -- 172.21.15.1:0/2326821498 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24d809c300 con 0x557582b359d0 2024-09-12T01:31:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.798+0000 7f24d1600640 1 -- 172.21.15.1:0/2326821498 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f24d400fac0 con 0x557582b359d0 2024-09-12T01:31:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.798+0000 7f24d1600640 1 -- 172.21.15.1:0/2326821498 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f24d400fcc0 con 0x557582b359d0 2024-09-12T01:31:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.798+0000 7f24f1854740 10 client.25343 fetch_fsmap learned FSMap version 32 2024-09-12T01:31:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.798+0000 7f24f1854740 1 -- 172.21.15.1:0/2326821498 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557582b49670 con 0x557582b359d0 2024-09-12T01:31:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.798+0000 7f24d1600640 1 -- 172.21.15.1:0/2326821498 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24d809c300 con 0x557582b359d0 2024-09-12T01:31:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.798+0000 7f24d1600640 1 -- 172.21.15.1:0/2326821498 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f24d401c420 con 0x557582b359d0 2024-09-12T01:31:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.798+0000 7f24f1854740 10 client.25343 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T01:31:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.798+0000 7f24f1854740 10 client.25343 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:31:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.798+0000 7f24f1854740 20 client.25343 populate_metadata read hostname 'smithi001' 2024-09-12T01:31:23.805 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.798+0000 7f24f1854740 1 --2- 172.21.15.1:0/2326821498 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x557582b4ad60 0x557582b4d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:23.805 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.798+0000 7f24f1854740 4 client.25343 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:31:23.805 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.798+0000 7f24f1854740 1 -- 172.21.15.1:0/2326821498 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557582aa7830 con 0x557582b4ad60 2024-09-12T01:31:23.805 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.800+0000 7f24e7e00640 1 --2- 172.21.15.1:0/2326821498 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x557582b4ad60 0x557582b4d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:23.805 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.800+0000 7f24d1600640 10 client.25343 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:31:23.805 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.800+0000 7f24e7e00640 1 --2- 172.21.15.1:0/2326821498 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x557582b4ad60 0x557582b4d1c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72d322b83bce30dc server_cookie=a76ba92afcb2a05a in_seq=0 out_seq=0 2024-09-12T01:31:23.805 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.801+0000 7f24d1600640 1 -- 172.21.15.1:0/2326821498 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557582aa7830 con 0x557582b4ad60 2024-09-12T01:31:23.805 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.801+0000 7f24d1600640 10 client.25343 handle_command_reply: tid=0 2024-09-12T01:31:23.805 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.801+0000 7f24f1854740 1 client.25343 shutdown 2024-09-12T01:31:23.805 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.801+0000 7f24f1854740 1 -- 172.21.15.1:0/2326821498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557582b359d0 msgr2=0x557582b3f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:31:23.805 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.802+0000 7f24f1854740 1 --2- 172.21.15.1:0/2326821498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557582b359d0 0x557582b3f720 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f24d4012720 tx=0x7f24d4012be0 comp rx=0 tx=0).stop 2024-09-12T01:31:23.805 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.802+0000 7f24f1854740 1 -- 172.21.15.1:0/2326821498 shutdown_connections 2024-09-12T01:31:23.805 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.802+0000 7f24f1854740 1 --2- 172.21.15.1:0/2326821498 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x557582b4ad60 0x557582b4d1c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:23.805 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.802+0000 7f24d1600640 0 client.25343 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:31:23.805 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.802+0000 7f24f1854740 1 --2- 172.21.15.1:0/2326821498 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557582b3aa60 0x557582b46760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.802+0000 7f24f1854740 1 --2- 172.21.15.1:0/2326821498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557582b359d0 0x557582b3f720 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.802+0000 7f24f1854740 1 --2- 172.21.15.1:0/2326821498 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557582b35090 0x557582b3f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.802+0000 7f24f1854740 1 -- 172.21.15.1:0/2326821498 >> 172.21.15.1:0/2326821498 conn(0x557582b30700 msgr2=0x557582b31f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:31:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.802+0000 7f24f1854740 1 -- 172.21.15.1:0/2326821498 shutdown_connections 2024-09-12T01:31:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.802+0000 7f24f1854740 1 -- 172.21.15.1:0/2326821498 wait complete. 2024-09-12T01:31:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.802+0000 7f24f1854740 20 client.25343 trim_cache size 0 max 16384 2024-09-12T01:31:23.817 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.817+0000 7f24f1854740 1 Processor -- start 2024-09-12T01:31:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.817+0000 7f24f1854740 1 -- start start 2024-09-12T01:31:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.817+0000 7f24f1854740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557582b35090 0x557582b35470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.817+0000 7f24f1854740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557582b359d0 0x557582b46720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.817+0000 7f24f1854740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557582b3aa60 0x557582b2f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.817+0000 7f24f1854740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f24e81ac9c0 con 0x557582b3aa60 2024-09-12T01:31:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.817+0000 7f24f1854740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f24e81ae590 con 0x557582b359d0 2024-09-12T01:31:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.817+0000 7f24f1854740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f24e81ad090 con 0x557582b35090 2024-09-12T01:31:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.818+0000 7f24e7e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557582b35090 0x557582b35470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.818+0000 7f24e7400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557582b359d0 0x557582b46720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.818+0000 7f24eca00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557582b3aa60 0x557582b2f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.818+0000 7f24e7400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557582b359d0 0x557582b46720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:58842/0 (socket says 172.21.15.1:58842) 2024-09-12T01:31:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.818+0000 7f24e7e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557582b35090 0x557582b35470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:52776/0 (socket says 172.21.15.1:52776) 2024-09-12T01:31:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.818+0000 7f24e7400640 1 -- 172.21.15.1:0/2379724030 learned_addr learned my addr 172.21.15.1:0/2379724030 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:31:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.818+0000 7f24e7e00640 1 -- 172.21.15.1:0/2379724030 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557582b359d0 msgr2=0x557582b46720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:31:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.818+0000 7f24e7e00640 1 --2- 172.21.15.1:0/2379724030 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557582b359d0 0x557582b46720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.818+0000 7f24e7e00640 1 -- 172.21.15.1:0/2379724030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557582b3aa60 msgr2=0x557582b2f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:31:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.818+0000 7f24e7e00640 1 --2- 172.21.15.1:0/2379724030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557582b3aa60 0x557582b2f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.818+0000 7f24e7e00640 1 -- 172.21.15.1:0/2379724030 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557582b30520 con 0x557582b35090 2024-09-12T01:31:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.818+0000 7f24e7e00640 1 --2- 172.21.15.1:0/2379724030 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557582b35090 0x557582b35470 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f24d80cfab0 tx=0x7f24d8003f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4aef5a90a48972bb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:31:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.819+0000 7f24d3e00640 1 -- 172.21.15.1:0/2379724030 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d80b2070 con 0x557582b35090 2024-09-12T01:31:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.819+0000 7f24d3e00640 1 -- 172.21.15.1:0/2379724030 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24d809c300 con 0x557582b35090 2024-09-12T01:31:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.819+0000 7f24d3e00640 1 -- 172.21.15.1:0/2379724030 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d80de070 con 0x557582b35090 2024-09-12T01:31:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.819+0000 7f24f1854740 1 -- 172.21.15.1:0/2379724030 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557582b35090 msgr2=0x557582b35470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:31:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.819+0000 7f24f1854740 1 --2- 172.21.15.1:0/2379724030 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557582b35090 0x557582b35470 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f24d80cfab0 tx=0x7f24d8003f10 comp rx=0 tx=0).stop 2024-09-12T01:31:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.819+0000 7f24f1854740 1 -- 172.21.15.1:0/2379724030 shutdown_connections 2024-09-12T01:31:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.819+0000 7f24f1854740 1 --2- 172.21.15.1:0/2379724030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557582b3aa60 0x557582b2f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.819+0000 7f24f1854740 1 --2- 172.21.15.1:0/2379724030 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557582b359d0 0x557582b46720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.819+0000 7f24f1854740 1 --2- 172.21.15.1:0/2379724030 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557582b35090 0x557582b35470 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.819+0000 7f24f1854740 1 -- 172.21.15.1:0/2379724030 >> 172.21.15.1:0/2379724030 conn(0x557582b30700 msgr2=0x557582b2f060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:31:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.819+0000 7f24f1854740 1 -- 172.21.15.1:0/2379724030 shutdown_connections 2024-09-12T01:31:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.819+0000 7f24f1854740 1 -- 172.21.15.1:0/2379724030 wait complete. 2024-09-12T01:31:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.820+0000 7f24f1854740 1 -- start start 2024-09-12T01:31:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.820+0000 7f24f1854740 1 Processor -- start 2024-09-12T01:31:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.820+0000 7f24f1854740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557582b35090 0x557582b43a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.820+0000 7f24f1854740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557582b359d0 0x557582b43fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.820+0000 7f24f1854740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557582b3aa60 0x557582b44fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.820+0000 7f24f1854740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f24e81ade90 con 0x557582b35090 2024-09-12T01:31:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.820+0000 7f24f1854740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f24e81ad7c0 con 0x557582b3aa60 2024-09-12T01:31:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.820+0000 7f24f1854740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f24e81aa9a0 con 0x557582b359d0 2024-09-12T01:31:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.820+0000 7f24f1854740 10 client.0 _ll_register_callbacks cb 0x7f24d4009890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:31:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.821+0000 7f24e7e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557582b35090 0x557582b43a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.821+0000 7f24e7400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557582b359d0 0x557582b43fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.821+0000 7f24e7e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557582b35090 0x557582b43a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47736/0 (socket says 172.21.15.1:47736) 2024-09-12T01:31:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.821+0000 7f24e7e00640 1 -- 172.21.15.1:0/1862640764 learned_addr learned my addr 172.21.15.1:0/1862640764 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:31:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.821+0000 7f24e7e00640 1 -- 172.21.15.1:0/1862640764 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557582b359d0 msgr2=0x557582b43fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:31:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.821+0000 7f24e7e00640 1 --2- 172.21.15.1:0/1862640764 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557582b359d0 0x557582b43fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.821+0000 7f24e7e00640 1 -- 172.21.15.1:0/1862640764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557582b3aa60 msgr2=0x557582b44fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T01:31:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.821+0000 7f24e7e00640 1 --2- 172.21.15.1:0/1862640764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557582b3aa60 0x557582b44fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.821+0000 7f24e7e00640 1 -- 172.21.15.1:0/1862640764 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557582b455c0 con 0x557582b35090 2024-09-12T01:31:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.821+0000 7f24e7400640 1 --2- 172.21.15.1:0/1862640764 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557582b359d0 0x557582b43fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:31:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.821+0000 7f24e7e00640 1 --2- 172.21.15.1:0/1862640764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557582b35090 0x557582b43a60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f24d809c4b0 tx=0x7f24d809bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53f6e28e9b4dc56a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:31:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.821+0000 7f24d1600640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-12T01:31:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.821+0000 7f24d1600640 1 -- 172.21.15.1:0/1862640764 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d80b9020 con 0x557582b35090 2024-09-12T01:31:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.821+0000 7f24d1600640 1 -- 172.21.15.1:0/1862640764 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f24d80b3070 con 0x557582b35090 2024-09-12T01:31:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.821+0000 7f24f1854740 1 -- 172.21.15.1:0/1862640764 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24d809c300 con 0x557582b35090 2024-09-12T01:31:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.822+0000 7f24d1600640 1 -- 172.21.15.1:0/1862640764 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f24d80b2030 con 0x557582b35090 2024-09-12T01:31:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.822+0000 7f24f1854740 10 client.15324 fetch_fsmap learned FSMap version 32 2024-09-12T01:31:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.822+0000 7f24f1854740 1 -- 172.21.15.1:0/1862640764 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557582b468d0 con 0x557582b35090 2024-09-12T01:31:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.822+0000 7f24d1600640 1 -- 172.21.15.1:0/1862640764 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24d809c300 con 0x557582b35090 2024-09-12T01:31:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.822+0000 7f24d1600640 1 -- 172.21.15.1:0/1862640764 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f24d8098270 con 0x557582b35090 2024-09-12T01:31:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.822+0000 7f24f1854740 10 client.15324 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T01:31:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.822+0000 7f24f1854740 10 client.15324 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:31:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.822+0000 7f24f1854740 20 client.15324 populate_metadata read hostname 'smithi001' 2024-09-12T01:31:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.822+0000 7f24f1854740 1 --2- 172.21.15.1:0/1862640764 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x557582bb5c90 0x557582bb80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.822+0000 7f24f1854740 4 client.15324 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:31:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.822+0000 7f24f1854740 1 -- 172.21.15.1:0/1862640764 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557582aa1b80 con 0x557582bb5c90 2024-09-12T01:31:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.822+0000 7f24e7400640 1 --2- 172.21.15.1:0/1862640764 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x557582bb5c90 0x557582bb80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.823+0000 7f24e7400640 1 --2- 172.21.15.1:0/1862640764 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x557582bb5c90 0x557582bb80b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ae24dd3d33ebafb server_cookie=71cdbbf92caa7df4 in_seq=0 out_seq=0 2024-09-12T01:31:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:23.823+0000 7f24d1600640 10 client.15324 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:31:24.649 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:31:24 smithi001 ceph-mon[30387]: pgmap v452: 129 pgs: 129 active+clean; 29 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 1.5 MiB/s wr, 559 op/s 2024-09-12T01:31:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:24.678+0000 7f24d1600640 1 -- 172.21.15.1:0/1862640764 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557582aa1b80 con 0x557582bb5c90 2024-09-12T01:31:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:24.678+0000 7f24d1600640 10 client.15324 handle_command_reply: tid=0 2024-09-12T01:31:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:24.678+0000 7f24f1854740 1 client.15324 shutdown 2024-09-12T01:31:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:24.678+0000 7f24f1854740 1 -- 172.21.15.1:0/1862640764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557582b35090 msgr2=0x557582b43a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:31:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:24.678+0000 7f24f1854740 1 --2- 172.21.15.1:0/1862640764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557582b35090 0x557582b43a60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f24d809c4b0 tx=0x7f24d809bc00 comp rx=0 tx=0).stop 2024-09-12T01:31:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:24.678+0000 7f24f1854740 1 -- 172.21.15.1:0/1862640764 shutdown_connections 2024-09-12T01:31:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:24.678+0000 7f24f1854740 1 --2- 172.21.15.1:0/1862640764 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x557582bb5c90 0x557582bb80b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:24.678+0000 7f24f1854740 1 --2- 172.21.15.1:0/1862640764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557582b3aa60 0x557582b44fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:24.678+0000 7f24f1854740 1 --2- 172.21.15.1:0/1862640764 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557582b359d0 0x557582b43fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:24.678+0000 7f24f1854740 1 --2- 172.21.15.1:0/1862640764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557582b35090 0x557582b43a60 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:24.678+0000 7f24f1854740 1 -- 172.21.15.1:0/1862640764 >> 172.21.15.1:0/1862640764 conn(0x557582b30700 msgr2=0x557582b2f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:31:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:24.678+0000 7f24d1600640 0 client.15324 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:31:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:24.678+0000 7f24f1854740 1 -- 172.21.15.1:0/1862640764 shutdown_connections 2024-09-12T01:31:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:24.678+0000 7f24f1854740 1 -- 172.21.15.1:0/1862640764 wait complete. 2024-09-12T01:31:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:24.678+0000 7f24f1854740 20 client.15324 trim_cache size 0 max 16384 2024-09-12T01:31:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-12T01:31:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-12T01:31:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-12T01:31:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-12T01:31:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:24.678+0000 7f24eea00640 1 -- 172.21.15.1:0/2335877622 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f24cc0a2b80 msgr2=0x7f24cc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:31:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:24.678+0000 7f24eea00640 1 --2- 172.21.15.1:0/2335877622 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f24cc0a2b80 0x7f24cc0a5040 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f24e002abe0 tx=0x7f24e0005fd0 comp rx=0 tx=0).stop 2024-09-12T01:31:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:24.678+0000 7f24eea00640 1 -- 172.21.15.1:0/2335877622 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24e8103d50 msgr2=0x7f24e819a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:31:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:24.678+0000 7f24eea00640 1 --2- 172.21.15.1:0/2335877622 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24e8103d50 0x7f24e819a020 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f24d800e9c0 tx=0x7f24d800edc0 comp rx=0 tx=0).stop 2024-09-12T01:31:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:24.678+0000 7f24eea00640 1 -- 172.21.15.1:0/2335877622 shutdown_connections 2024-09-12T01:31:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:24.678+0000 7f24eea00640 1 --2- 172.21.15.1:0/2335877622 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f24cc0a2b80 0x7f24cc0a5040 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:24.678+0000 7f24eea00640 1 --2- 172.21.15.1:0/2335877622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f24e8105900 0x7f24e81a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:24.678+0000 7f24eea00640 1 --2- 172.21.15.1:0/2335877622 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24e8104f40 0x7f24e819a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:24.678+0000 7f24eea00640 1 --2- 172.21.15.1:0/2335877622 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24e8103d50 0x7f24e819a020 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:24.678+0000 7f24eea00640 1 -- 172.21.15.1:0/2335877622 >> 172.21.15.1:0/2335877622 conn(0x7f24e80ff4b0 msgr2=0x7f24e8108260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:31:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:24.678+0000 7f24eea00640 1 -- 172.21.15.1:0/2335877622 shutdown_connections 2024-09-12T01:31:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:24.678+0000 7f24eea00640 1 -- 172.21.15.1:0/2335877622 wait complete. 2024-09-12T01:31:24.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:31:24 smithi053 ceph-mon[35158]: pgmap v452: 129 pgs: 129 active+clean; 29 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 1.5 MiB/s wr, 559 op/s 2024-09-12T01:31:24.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:31:24 smithi104 ceph-mon[35219]: pgmap v452: 129 pgs: 129 active+clean; 29 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 1.5 MiB/s wr, 559 op/s 2024-09-12T01:31:26.582 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:31:26 smithi001 ceph-mon[30387]: pgmap v453: 129 pgs: 129 active+clean; 41 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 580 op/s 2024-09-12T01:31:26.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:31:26 smithi053 ceph-mon[35158]: pgmap v453: 129 pgs: 129 active+clean; 41 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 580 op/s 2024-09-12T01:31:26.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:31:26 smithi104 ceph-mon[35219]: pgmap v453: 129 pgs: 129 active+clean; 41 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 580 op/s 2024-09-12T01:31:28.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:31:28 smithi053 ceph-mon[35158]: pgmap v454: 129 pgs: 129 active+clean; 43 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 619 op/s 2024-09-12T01:31:28.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:31:28 smithi104 ceph-mon[35219]: pgmap v454: 129 pgs: 129 active+clean; 43 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 619 op/s 2024-09-12T01:31:28.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:31:28 smithi001 ceph-mon[30387]: pgmap v454: 129 pgs: 129 active+clean; 43 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 619 op/s 2024-09-12T01:31:29.935 INFO:tasks.workunit.client.0.smithi001.stderr:Note: switching to '0e5c12dfd008efc2848c98108c9237487e91ef35'. 2024-09-12T01:31:29.935 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-09-12T01:31:29.936 INFO:tasks.workunit.client.0.smithi001.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-12T01:31:29.936 INFO:tasks.workunit.client.0.smithi001.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-12T01:31:29.936 INFO:tasks.workunit.client.0.smithi001.stderr:state without impacting any branches by switching back to a branch. 2024-09-12T01:31:29.936 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-09-12T01:31:29.936 INFO:tasks.workunit.client.0.smithi001.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-12T01:31:29.936 INFO:tasks.workunit.client.0.smithi001.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-12T01:31:29.936 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-09-12T01:31:29.936 INFO:tasks.workunit.client.0.smithi001.stderr: git switch -c 2024-09-12T01:31:29.936 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-09-12T01:31:29.936 INFO:tasks.workunit.client.0.smithi001.stderr:Or undo this operation with: 2024-09-12T01:31:29.936 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-09-12T01:31:29.937 INFO:tasks.workunit.client.0.smithi001.stderr: git switch - 2024-09-12T01:31:29.937 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-09-12T01:31:29.937 INFO:tasks.workunit.client.0.smithi001.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-12T01:31:29.937 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-09-12T01:31:29.937 INFO:tasks.workunit.client.0.smithi001.stderr:HEAD is now at 0e5c12df xfs/522: use reflink instead of crc as test feature 2024-09-12T01:31:29.938 INFO:tasks.workunit.client.0.smithi001.stderr:+ make -j4 2024-09-12T01:31:29.953 INFO:tasks.workunit.client.0.smithi001.stdout:libtoolize -cfi 2024-09-12T01:31:30.477 INFO:tasks.workunit.client.0.smithi001.stdout:libtoolize: putting auxiliary files in '.'. 2024-09-12T01:31:30.477 INFO:tasks.workunit.client.0.smithi001.stdout:libtoolize: copying file './config.guess' 2024-09-12T01:31:30.489 INFO:tasks.workunit.client.0.smithi001.stdout:libtoolize: copying file './config.sub' 2024-09-12T01:31:30.506 INFO:tasks.workunit.client.0.smithi001.stdout:libtoolize: copying file './install-sh' 2024-09-12T01:31:30.530 INFO:tasks.workunit.client.0.smithi001.stdout:libtoolize: copying file './ltmain.sh' 2024-09-12T01:31:30.646 INFO:tasks.workunit.client.0.smithi001.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-09-12T01:31:30.646 INFO:tasks.workunit.client.0.smithi001.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-09-12T01:31:30.667 INFO:tasks.workunit.client.0.smithi001.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-09-12T01:31:30.691 INFO:tasks.workunit.client.0.smithi001.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-09-12T01:31:30.714 INFO:tasks.workunit.client.0.smithi001.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-09-12T01:31:30.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:31:30 smithi053 ceph-mon[35158]: pgmap v455: 129 pgs: 129 active+clean; 47 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 569 op/s 2024-09-12T01:31:30.740 INFO:tasks.workunit.client.0.smithi001.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-09-12T01:31:30.769 INFO:tasks.workunit.client.0.smithi001.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-09-12T01:31:30.774 INFO:tasks.workunit.client.0.smithi001.stdout:cp include/install-sh . 2024-09-12T01:31:30.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:31:30 smithi104 ceph-mon[35219]: pgmap v455: 129 pgs: 129 active+clean; 47 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 569 op/s 2024-09-12T01:31:30.830 INFO:tasks.workunit.client.0.smithi001.stdout:aclocal -I m4 2024-09-12T01:31:30.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:31:30 smithi001 ceph-mon[30387]: pgmap v455: 129 pgs: 129 active+clean; 47 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 569 op/s 2024-09-12T01:31:32.183 INFO:tasks.workunit.client.0.smithi001.stdout:autoheader 2024-09-12T01:31:32.561 INFO:tasks.workunit.client.0.smithi001.stdout:autoconf 2024-09-12T01:31:32.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:31:32 smithi053 ceph-mon[35158]: pgmap v456: 129 pgs: 129 active+clean; 49 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 611 op/s 2024-09-12T01:31:32.794 INFO:tasks.workunit.client.0.smithi001.stdout:./configure \ 2024-09-12T01:31:32.794 INFO:tasks.workunit.client.0.smithi001.stdout: --libexecdir=/usr/lib \ 2024-09-12T01:31:32.794 INFO:tasks.workunit.client.0.smithi001.stdout: --exec_prefix=/var/lib 2024-09-12T01:31:32.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:31:32 smithi104 ceph-mon[35219]: pgmap v456: 129 pgs: 129 active+clean; 49 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 611 op/s 2024-09-12T01:31:32.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:31:32 smithi001 ceph-mon[30387]: pgmap v456: 129 pgs: 129 active+clean; 49 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 611 op/s 2024-09-12T01:31:33.073 INFO:tasks.workunit.client.0.smithi001.stdout:checking build system type... x86_64-pc-linux-gnu 2024-09-12T01:31:33.073 INFO:tasks.workunit.client.0.smithi001.stdout:checking host system type... x86_64-pc-linux-gnu 2024-09-12T01:31:33.075 INFO:tasks.workunit.client.0.smithi001.stdout:checking how to print strings... printf 2024-09-12T01:31:33.076 INFO:tasks.workunit.client.0.smithi001.stdout:checking for gcc... gcc 2024-09-12T01:31:33.206 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether the C compiler works... yes 2024-09-12T01:31:33.206 INFO:tasks.workunit.client.0.smithi001.stdout:checking for C compiler default output file name... a.out 2024-09-12T01:31:33.267 INFO:tasks.workunit.client.0.smithi001.stdout:checking for suffix of executables... 2024-09-12T01:31:33.332 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether we are cross compiling... no 2024-09-12T01:31:33.388 INFO:tasks.workunit.client.0.smithi001.stdout:checking for suffix of object files... o 2024-09-12T01:31:33.429 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether we are using the GNU C compiler... yes 2024-09-12T01:31:33.470 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether gcc accepts -g... yes 2024-09-12T01:31:33.581 INFO:tasks.workunit.client.0.smithi001.stdout:checking for gcc option to accept ISO C89... none needed 2024-09-12T01:31:33.589 INFO:tasks.workunit.client.0.smithi001.stdout:checking for a sed that does not truncate output... /usr/bin/sed 2024-09-12T01:31:33.597 INFO:tasks.workunit.client.0.smithi001.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-09-12T01:31:33.599 INFO:tasks.workunit.client.0.smithi001.stdout:checking for egrep... /usr/bin/grep -E 2024-09-12T01:31:33.602 INFO:tasks.workunit.client.0.smithi001.stdout:checking for fgrep... /usr/bin/grep -F 2024-09-12T01:31:33.610 INFO:tasks.workunit.client.0.smithi001.stdout:checking for ld used by gcc... /usr/bin/ld 2024-09-12T01:31:33.614 INFO:tasks.workunit.client.0.smithi001.stdout:checking if the linker (/usr/bin/ld) is GNU ld... yes 2024-09-12T01:31:33.633 INFO:tasks.workunit.client.0.smithi001.stdout:checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B 2024-09-12T01:31:34.671 INFO:tasks.workunit.client.0.smithi001.stdout:checking the name lister (/usr/bin/nm -B) interface... BSD nm 2024-09-12T01:31:34.672 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether ln -s works... yes 2024-09-12T01:31:34.711 INFO:tasks.workunit.client.0.smithi001.stdout:checking the maximum length of command line arguments... 1572864 2024-09-12T01:31:34.717 INFO:tasks.workunit.client.0.smithi001.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-09-12T01:31:34.718 INFO:tasks.workunit.client.0.smithi001.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-09-12T01:31:34.718 INFO:tasks.workunit.client.0.smithi001.stdout:checking for /usr/bin/ld option to reload object files... -r 2024-09-12T01:31:34.718 INFO:tasks.workunit.client.0.smithi001.stdout:checking for objdump... objdump 2024-09-12T01:31:34.718 INFO:tasks.workunit.client.0.smithi001.stdout:checking how to recognize dependent libraries... pass_all 2024-09-12T01:31:34.719 INFO:tasks.workunit.client.0.smithi001.stdout:checking for dlltool... no 2024-09-12T01:31:34.719 INFO:tasks.workunit.client.0.smithi001.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-09-12T01:31:34.720 INFO:tasks.workunit.client.0.smithi001.stdout:checking for ar... ar 2024-09-12T01:31:34.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:31:34 smithi053 ceph-mon[35158]: pgmap v457: 129 pgs: 129 active+clean; 50 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 494 op/s 2024-09-12T01:31:34.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:31:34 smithi104 ceph-mon[35219]: pgmap v457: 129 pgs: 129 active+clean; 50 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 494 op/s 2024-09-12T01:31:34.846 INFO:tasks.workunit.client.0.smithi001.stdout:checking for archiver @FILE support... @ 2024-09-12T01:31:34.847 INFO:tasks.workunit.client.0.smithi001.stdout:checking for strip... strip 2024-09-12T01:31:34.847 INFO:tasks.workunit.client.0.smithi001.stdout:checking for ranlib... ranlib 2024-09-12T01:31:34.848 INFO:tasks.workunit.client.0.smithi001.stdout:checking for gawk... gawk 2024-09-12T01:31:34.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:31:34 smithi001 ceph-mon[30387]: pgmap v457: 129 pgs: 129 active+clean; 50 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 494 op/s 2024-09-12T01:31:35.009 INFO:tasks.workunit.client.0.smithi001.stdout:checking command to parse /usr/bin/nm -B output from gcc object... ok 2024-09-12T01:31:35.012 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sysroot... no 2024-09-12T01:31:35.027 INFO:tasks.workunit.client.0.smithi001.stdout:checking for a working dd... /usr/bin/dd 2024-09-12T01:31:35.042 INFO:tasks.workunit.client.0.smithi001.stdout:checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 2024-09-12T01:31:35.089 INFO:tasks.workunit.client.0.smithi001.stdout:checking for mt... no 2024-09-12T01:31:35.105 INFO:tasks.workunit.client.0.smithi001.stdout:checking if : is a manifest tool... no 2024-09-12T01:31:35.212 INFO:tasks.workunit.client.0.smithi001.stdout:checking how to run the C preprocessor... gcc -E 2024-09-12T01:31:35.517 INFO:tasks.workunit.client.0.smithi001.stdout:checking for ANSI C header files... yes 2024-09-12T01:31:35.572 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/types.h... yes 2024-09-12T01:31:35.636 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/stat.h... yes 2024-09-12T01:31:35.701 INFO:tasks.workunit.client.0.smithi001.stdout:checking for stdlib.h... yes 2024-09-12T01:31:35.767 INFO:tasks.workunit.client.0.smithi001.stdout:checking for string.h... yes 2024-09-12T01:31:35.833 INFO:tasks.workunit.client.0.smithi001.stdout:checking for memory.h... yes 2024-09-12T01:31:35.898 INFO:tasks.workunit.client.0.smithi001.stdout:checking for strings.h... yes 2024-09-12T01:31:35.965 INFO:tasks.workunit.client.0.smithi001.stdout:checking for inttypes.h... yes 2024-09-12T01:31:36.031 INFO:tasks.workunit.client.0.smithi001.stdout:checking for stdint.h... yes 2024-09-12T01:31:36.102 INFO:tasks.workunit.client.0.smithi001.stdout:checking for unistd.h... yes 2024-09-12T01:31:36.178 INFO:tasks.workunit.client.0.smithi001.stdout:checking for dlfcn.h... yes 2024-09-12T01:31:36.189 INFO:tasks.workunit.client.0.smithi001.stdout:checking for objdir... .libs 2024-09-12T01:31:36.376 INFO:tasks.workunit.client.0.smithi001.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-09-12T01:31:36.376 INFO:tasks.workunit.client.0.smithi001.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-09-12T01:31:36.438 INFO:tasks.workunit.client.0.smithi001.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-09-12T01:31:36.500 INFO:tasks.workunit.client.0.smithi001.stdout:checking if gcc static flag -static works... no 2024-09-12T01:31:36.585 INFO:tasks.workunit.client.0.smithi001.stdout:checking if gcc supports -c -o file.o... yes 2024-09-12T01:31:36.585 INFO:tasks.workunit.client.0.smithi001.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-09-12T01:31:36.600 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-09-12T01:31:36.683 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether -lc should be explicitly linked in... no 2024-09-12T01:31:36.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:31:36 smithi053 ceph-mon[35158]: pgmap v458: 129 pgs: 129 active+clean; 50 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 491 op/s 2024-09-12T01:31:36.788 INFO:tasks.workunit.client.0.smithi001.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-09-12T01:31:36.788 INFO:tasks.workunit.client.0.smithi001.stdout:checking how to hardcode library paths into programs... immediate 2024-09-12T01:31:36.791 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether stripping libraries is possible... yes 2024-09-12T01:31:36.791 INFO:tasks.workunit.client.0.smithi001.stdout:checking if libtool supports shared libraries... yes 2024-09-12T01:31:36.791 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether to build shared libraries... yes 2024-09-12T01:31:36.791 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether to build static libraries... yes 2024-09-12T01:31:36.805 INFO:tasks.workunit.client.0.smithi001.stdout:checking for gcc... (cached) gcc 2024-09-12T01:31:36.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:31:36 smithi104 ceph-mon[35219]: pgmap v458: 129 pgs: 129 active+clean; 50 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 491 op/s 2024-09-12T01:31:36.858 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-09-12T01:31:36.858 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether gcc accepts -g... (cached) yes 2024-09-12T01:31:36.859 INFO:tasks.workunit.client.0.smithi001.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-09-12T01:31:36.859 INFO:tasks.workunit.client.0.smithi001.stdout:checking for gmake... /usr/bin/gmake 2024-09-12T01:31:36.859 INFO:tasks.workunit.client.0.smithi001.stderr:./configure: line 12082: test: $(SHELL): binary operator expected 2024-09-12T01:31:36.860 INFO:tasks.workunit.client.0.smithi001.stdout:checking for tar... /bin/tar 2024-09-12T01:31:36.860 INFO:tasks.workunit.client.0.smithi001.stdout:checking for gzip... /bin/gzip 2024-09-12T01:31:36.910 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether gcc -MM is supported... yes 2024-09-12T01:31:36.911 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sort... /bin/sort 2024-09-12T01:31:36.911 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether ln -s works... yes 2024-09-12T01:31:36.911 INFO:tasks.workunit.client.0.smithi001.stdout:checking for rpm... /bin/rpm 2024-09-12T01:31:36.918 INFO:tasks.workunit.client.0.smithi001.stdout:checking for rpmbuild... no 2024-09-12T01:31:36.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:31:36 smithi001 ceph-mon[30387]: pgmap v458: 129 pgs: 129 active+clean; 50 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 491 op/s 2024-09-12T01:31:36.993 INFO:tasks.workunit.client.0.smithi001.stdout:checking assert.h usability... yes 2024-09-12T01:31:37.023 INFO:tasks.workunit.client.0.smithi001.stdout:checking assert.h presence... yes 2024-09-12T01:31:37.023 INFO:tasks.workunit.client.0.smithi001.stdout:checking for assert.h... yes 2024-09-12T01:31:37.137 INFO:tasks.workunit.client.0.smithi001.stdout:checking bstring.h usability... no 2024-09-12T01:31:37.179 INFO:tasks.workunit.client.0.smithi001.stdout:checking bstring.h presence... no 2024-09-12T01:31:37.179 INFO:tasks.workunit.client.0.smithi001.stdout:checking for bstring.h... no 2024-09-12T01:31:37.306 INFO:tasks.workunit.client.0.smithi001.stdout:checking libgen.h usability... yes 2024-09-12T01:31:37.333 INFO:tasks.workunit.client.0.smithi001.stdout:checking libgen.h presence... yes 2024-09-12T01:31:37.333 INFO:tasks.workunit.client.0.smithi001.stdout:checking for libgen.h... yes 2024-09-12T01:31:37.417 INFO:tasks.workunit.client.0.smithi001.stdout:checking dirent.h usability... yes 2024-09-12T01:31:37.444 INFO:tasks.workunit.client.0.smithi001.stdout:checking dirent.h presence... yes 2024-09-12T01:31:37.444 INFO:tasks.workunit.client.0.smithi001.stdout:checking for dirent.h... yes 2024-09-12T01:31:37.518 INFO:tasks.workunit.client.0.smithi001.stdout:checking errno.h usability... yes 2024-09-12T01:31:37.550 INFO:tasks.workunit.client.0.smithi001.stdout:checking errno.h presence... yes 2024-09-12T01:31:37.550 INFO:tasks.workunit.client.0.smithi001.stdout:checking for errno.h... yes 2024-09-12T01:31:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.804+0000 7fb9dd400640 1 -- 172.21.15.1:0/2789886580 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9d81019b0 msgr2=0x7fb9d8101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:31:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.804+0000 7fb9dd400640 1 --2- 172.21.15.1:0/2789886580 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9d81019b0 0x7fb9d8101db0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8009900 tx=0x7fb9c802a410 comp rx=0 tx=0).stop 2024-09-12T01:31:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.804+0000 7fb9dd400640 1 -- 172.21.15.1:0/2789886580 shutdown_connections 2024-09-12T01:31:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.804+0000 7fb9dd400640 1 --2- 172.21.15.1:0/2789886580 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9d8103560 0x7fb9d8109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.804+0000 7fb9dd400640 1 --2- 172.21.15.1:0/2789886580 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9d8102ba0 0x7fb9d8103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.804+0000 7fb9dd400640 1 --2- 172.21.15.1:0/2789886580 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9d81019b0 0x7fb9d8101db0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.804+0000 7fb9dd400640 1 -- 172.21.15.1:0/2789886580 >> 172.21.15.1:0/2789886580 conn(0x7fb9d80fd130 msgr2=0x7fb9d80ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:31:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.804+0000 7fb9dd400640 1 -- 172.21.15.1:0/2789886580 shutdown_connections 2024-09-12T01:31:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.804+0000 7fb9dd400640 1 -- 172.21.15.1:0/2789886580 wait complete. 2024-09-12T01:31:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.805+0000 7fb9dd400640 1 Processor -- start 2024-09-12T01:31:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.805+0000 7fb9dd400640 1 -- start start 2024-09-12T01:31:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.805+0000 7fb9dd400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9d81019b0 0x7fb9d8197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.805+0000 7fb9dd400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9d8102ba0 0x7fb9d8198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.805+0000 7fb9dd400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9d8103560 0x7fb9d819f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.805+0000 7fb9dd400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d810d680 con 0x7fb9d8102ba0 2024-09-12T01:31:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.805+0000 7fb9dd400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d810d500 con 0x7fb9d81019b0 2024-09-12T01:31:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.805+0000 7fb9dd400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d810d800 con 0x7fb9d8103560 2024-09-12T01:31:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.805+0000 7fb9cfe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9d8102ba0 0x7fb9d8198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.806+0000 7fb9cfe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9d8102ba0 0x7fb9d8198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43560/0 (socket says 172.21.15.1:43560) 2024-09-12T01:31:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.806+0000 7fb9cfe00640 1 -- 172.21.15.1:0/3414552338 learned_addr learned my addr 172.21.15.1:0/3414552338 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:31:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.806+0000 7fb9d7e00640 1 --2- 172.21.15.1:0/3414552338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9d81019b0 0x7fb9d8197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.806+0000 7fb9dca00640 1 --2- 172.21.15.1:0/3414552338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9d8103560 0x7fb9d819f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.806+0000 7fb9cfe00640 1 -- 172.21.15.1:0/3414552338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9d8103560 msgr2=0x7fb9d819f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:31:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.806+0000 7fb9cfe00640 1 --2- 172.21.15.1:0/3414552338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9d8103560 0x7fb9d819f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.806+0000 7fb9cfe00640 1 -- 172.21.15.1:0/3414552338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9d81019b0 msgr2=0x7fb9d8197cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:31:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.806+0000 7fb9cfe00640 1 --2- 172.21.15.1:0/3414552338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9d81019b0 0x7fb9d8197cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.806+0000 7fb9cfe00640 1 -- 172.21.15.1:0/3414552338 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9d819f900 con 0x7fb9d8102ba0 2024-09-12T01:31:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.806+0000 7fb9cfe00640 1 --2- 172.21.15.1:0/3414552338 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9d8102ba0 0x7fb9d8198230 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb9c000e9f0 tx=0x7fb9c000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:31:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.806+0000 7fb9d5600640 1 -- 172.21.15.1:0/3414552338 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c000cd80 con 0x7fb9d8102ba0 2024-09-12T01:31:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.807+0000 7fb9d5600640 1 -- 172.21.15.1:0/3414552338 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9c0005500 con 0x7fb9d8102ba0 2024-09-12T01:31:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.807+0000 7fb9dd400640 1 -- 172.21.15.1:0/3414552338 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9d819fbf0 con 0x7fb9d8102ba0 2024-09-12T01:31:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.807+0000 7fb9d5600640 1 -- 172.21.15.1:0/3414552338 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0010690 con 0x7fb9d8102ba0 2024-09-12T01:31:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.807+0000 7fb9dd400640 1 -- 172.21.15.1:0/3414552338 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9d81a00c0 con 0x7fb9d8102ba0 2024-09-12T01:31:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.810+0000 7fb9d5600640 1 -- 172.21.15.1:0/3414552338 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fb9c00046c0 con 0x7fb9d8102ba0 2024-09-12T01:31:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.810+0000 7fb9e1879740 1 Processor -- start 2024-09-12T01:31:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.810+0000 7fb9e1879740 1 -- start start 2024-09-12T01:31:37.812 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.810+0000 7fb9e1879740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d0df48670 0x559d0df48a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:37.812 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.810+0000 7fb9e1879740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0df48fb0 0x559d0df4da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:37.812 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.810+0000 7fb9e1879740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d0df4e040 0x559d0df50500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.810+0000 7fb9e1879740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559d0dea9100 con 0x559d0df48fb0 2024-09-12T01:31:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.810+0000 7fb9e1879740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x559d0de511b0 con 0x559d0df4e040 2024-09-12T01:31:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.810+0000 7fb9e1879740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559d0dea50c0 con 0x559d0df48670 2024-09-12T01:31:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.810+0000 7fb9d7e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0df48fb0 0x559d0df4da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.810+0000 7fb9cfe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d0df4e040 0x559d0df50500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.810+0000 7fb9d7e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0df48fb0 0x559d0df4da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43576/0 (socket says 172.21.15.1:43576) 2024-09-12T01:31:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.810+0000 7fb9d7e00640 1 -- 172.21.15.1:0/1186443078 learned_addr learned my addr 172.21.15.1:0/1186443078 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:31:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.811+0000 7fb9dca00640 1 --2- 172.21.15.1:0/1186443078 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d0df48670 0x559d0df48a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.811+0000 7fb9d7e00640 1 -- 172.21.15.1:0/1186443078 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d0df48670 msgr2=0x559d0df48a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:31:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.811+0000 7fb9d7e00640 1 --2- 172.21.15.1:0/1186443078 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d0df48670 0x559d0df48a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.811+0000 7fb9d7e00640 1 -- 172.21.15.1:0/1186443078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d0df4e040 msgr2=0x559d0df50500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:31:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.811+0000 7fb9d7e00640 1 --2- 172.21.15.1:0/1186443078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d0df4e040 0x559d0df50500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.811+0000 7fb9d7e00640 1 -- 172.21.15.1:0/1186443078 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d0dbbc210 con 0x559d0df48fb0 2024-09-12T01:31:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.811+0000 7fb9cfe00640 1 --2- 172.21.15.1:0/1186443078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d0df4e040 0x559d0df50500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:31:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.811+0000 7fb9d7e00640 1 --2- 172.21.15.1:0/1186443078 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0df48fb0 0x559d0df4da70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8031440 tx=0x7fb9c8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b54f34cdf0f18f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:31:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.811+0000 7fb9d5600640 1 --2- 172.21.15.1:0/3414552338 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fb9bc0a2b70 0x7fb9bc0a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.811+0000 7fb9cd200640 1 -- 172.21.15.1:0/1186443078 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c8035040 con 0x559d0df48fb0 2024-09-12T01:31:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.811+0000 7fb9cd200640 1 -- 172.21.15.1:0/1186443078 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9c8004430 con 0x559d0df48fb0 2024-09-12T01:31:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.811+0000 7fb9cd200640 1 -- 172.21.15.1:0/1186443078 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c80316b0 con 0x559d0df48fb0 2024-09-12T01:31:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.812+0000 7fb9d5600640 1 -- 172.21.15.1:0/3414552338 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fb9c00cf840 con 0x7fb9d8102ba0 2024-09-12T01:31:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.812+0000 7fb9dca00640 1 --2- 172.21.15.1:0/3414552338 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fb9bc0a2b70 0x7fb9bc0a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.812+0000 7fb9e1879740 1 -- 172.21.15.1:0/1186443078 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0df48fb0 msgr2=0x559d0df4da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:31:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.812+0000 7fb9e1879740 1 --2- 172.21.15.1:0/1186443078 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0df48fb0 0x559d0df4da70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8031440 tx=0x7fb9c8004040 comp rx=0 tx=0).stop 2024-09-12T01:31:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.812+0000 7fb9e1879740 1 -- 172.21.15.1:0/1186443078 shutdown_connections 2024-09-12T01:31:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.812+0000 7fb9e1879740 1 --2- 172.21.15.1:0/1186443078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d0df4e040 0x559d0df50500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.812+0000 7fb9e1879740 1 --2- 172.21.15.1:0/1186443078 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0df48fb0 0x559d0df4da70 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.812+0000 7fb9e1879740 1 --2- 172.21.15.1:0/1186443078 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d0df48670 0x559d0df48a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.812+0000 7fb9e1879740 1 -- 172.21.15.1:0/1186443078 >> 172.21.15.1:0/1186443078 conn(0x559d0df3f8d0 msgr2=0x559d0df3fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:31:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.812+0000 7fb9dca00640 1 --2- 172.21.15.1:0/3414552338 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fb9bc0a2b70 0x7fb9bc0a5030 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0005fd0 tx=0x7fb9d0009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:31:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.812+0000 7fb9e1879740 1 -- 172.21.15.1:0/1186443078 shutdown_connections 2024-09-12T01:31:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.812+0000 7fb9e1879740 1 -- 172.21.15.1:0/1186443078 wait complete. 2024-09-12T01:31:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.813+0000 7fb9e1879740 1 -- start start 2024-09-12T01:31:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.813+0000 7fb9e1879740 1 Processor -- start 2024-09-12T01:31:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.813+0000 7fb9e1879740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d0df48670 0x559d0df527e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.813+0000 7fb9e1879740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d0df48fb0 0x559d0df52d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.813+0000 7fb9e1879740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0df4e040 0x559d0df59d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.813+0000 7fb9e1879740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559d0df50fb0 con 0x559d0df4e040 2024-09-12T01:31:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.813+0000 7fb9e1879740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x559d0de32090 con 0x559d0df48fb0 2024-09-12T01:31:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.813+0000 7fb9e1879740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559d0df51130 con 0x559d0df48670 2024-09-12T01:31:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.813+0000 7fb9e1879740 10 client.0 _ll_register_callbacks cb 0x559d0de86f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:31:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.814+0000 7fb9dca00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0df4e040 0x559d0df59d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.814+0000 7fb9cfe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d0df48fb0 0x559d0df52d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.814+0000 7fb9d7e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d0df48670 0x559d0df527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.814+0000 7fb9cfe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d0df48fb0 0x559d0df52d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:35496/0 (socket says 172.21.15.1:35496) 2024-09-12T01:31:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.814+0000 7fb9cfe00640 1 -- 172.21.15.1:0/3665854016 learned_addr learned my addr 172.21.15.1:0/3665854016 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:31:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.814+0000 7fb9cfe00640 1 -- 172.21.15.1:0/3665854016 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d0df48670 msgr2=0x559d0df527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:31:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.814+0000 7fb9cfe00640 1 --2- 172.21.15.1:0/3665854016 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d0df48670 0x559d0df527e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.814+0000 7fb9cfe00640 1 -- 172.21.15.1:0/3665854016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0df4e040 msgr2=0x559d0df59d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:31:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.814+0000 7fb9cfe00640 1 --2- 172.21.15.1:0/3665854016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0df4e040 0x559d0df59d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.814+0000 7fb9cfe00640 1 -- 172.21.15.1:0/3665854016 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d0dc88620 con 0x559d0df48fb0 2024-09-12T01:31:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.814+0000 7fb9dca00640 1 --2- 172.21.15.1:0/3665854016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0df4e040 0x559d0df59d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:31:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.814+0000 7fb9cfe00640 1 --2- 172.21.15.1:0/3665854016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d0df48fb0 0x559d0df52d20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0098650 tx=0x7fb9c0009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e23c8929dfb39cb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:31:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.814+0000 7fb9c5600640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-12T01:31:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.815+0000 7fb9c5600640 1 -- 172.21.15.1:0/3665854016 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c009b850 con 0x559d0df48fb0 2024-09-12T01:31:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.815+0000 7fb9e1879740 1 -- 172.21.15.1:0/3665854016 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9c8004430 con 0x559d0df48fb0 2024-09-12T01:31:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.815+0000 7fb9c5600640 1 -- 172.21.15.1:0/3665854016 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fb9c009b160 con 0x559d0df48fb0 2024-09-12T01:31:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.815+0000 7fb9c5600640 1 -- 172.21.15.1:0/3665854016 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb9c0099e50 con 0x559d0df48fb0 2024-09-12T01:31:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.815+0000 7fb9e1879740 10 client.25358 fetch_fsmap learned FSMap version 32 2024-09-12T01:31:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.815+0000 7fb9e1879740 1 -- 172.21.15.1:0/3665854016 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559d0df5cc70 con 0x559d0df48fb0 2024-09-12T01:31:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.815+0000 7fb9c5600640 1 -- 172.21.15.1:0/3665854016 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9c8004430 con 0x559d0df48fb0 2024-09-12T01:31:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.815+0000 7fb9c5600640 1 -- 172.21.15.1:0/3665854016 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7fb9c009abd0 con 0x559d0df48fb0 2024-09-12T01:31:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.815+0000 7fb9e1879740 10 client.25358 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T01:31:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.815+0000 7fb9e1879740 10 client.25358 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:31:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.816+0000 7fb9e1879740 20 client.25358 populate_metadata read hostname 'smithi001' 2024-09-12T01:31:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.816+0000 7fb9e1879740 1 --2- 172.21.15.1:0/3665854016 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x559d0df5e360 0x559d0df60750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.816+0000 7fb9e1879740 4 client.25358 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:31:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.816+0000 7fb9e1879740 1 -- 172.21.15.1:0/3665854016 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d0deb6830 con 0x559d0df5e360 2024-09-12T01:31:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.816+0000 7fb9d7e00640 1 --2- 172.21.15.1:0/3665854016 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x559d0df5e360 0x559d0df60750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.816+0000 7fb9d7e00640 1 --2- 172.21.15.1:0/3665854016 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x559d0df5e360 0x559d0df60750 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ab89bf9b12e6154 server_cookie=f8776cf56f4af077 in_seq=0 out_seq=0 2024-09-12T01:31:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.816+0000 7fb9c5600640 10 client.25358 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:31:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.817+0000 7fb9c5600640 1 -- 172.21.15.1:0/3665854016 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559d0deb6830 con 0x559d0df5e360 2024-09-12T01:31:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.817+0000 7fb9c5600640 10 client.25358 handle_command_reply: tid=0 2024-09-12T01:31:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.817+0000 7fb9e1879740 1 client.25358 shutdown 2024-09-12T01:31:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.817+0000 7fb9e1879740 1 -- 172.21.15.1:0/3665854016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d0df48fb0 msgr2=0x559d0df52d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:31:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.817+0000 7fb9e1879740 1 --2- 172.21.15.1:0/3665854016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d0df48fb0 0x559d0df52d20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0098650 tx=0x7fb9c0009310 comp rx=0 tx=0).stop 2024-09-12T01:31:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.817+0000 7fb9e1879740 1 -- 172.21.15.1:0/3665854016 shutdown_connections 2024-09-12T01:31:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.817+0000 7fb9e1879740 1 --2- 172.21.15.1:0/3665854016 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x559d0df5e360 0x559d0df60750 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.817+0000 7fb9e1879740 1 --2- 172.21.15.1:0/3665854016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0df4e040 0x559d0df59d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.817+0000 7fb9e1879740 1 --2- 172.21.15.1:0/3665854016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d0df48fb0 0x559d0df52d20 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:37.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.817+0000 7fb9c5600640 0 client.25358 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:31:37.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.817+0000 7fb9e1879740 1 --2- 172.21.15.1:0/3665854016 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d0df48670 0x559d0df527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:37.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.817+0000 7fb9e1879740 1 -- 172.21.15.1:0/3665854016 >> 172.21.15.1:0/3665854016 conn(0x559d0df3f8d0 msgr2=0x559d0df410e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:31:37.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.817+0000 7fb9e1879740 1 -- 172.21.15.1:0/3665854016 shutdown_connections 2024-09-12T01:31:37.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.817+0000 7fb9e1879740 1 -- 172.21.15.1:0/3665854016 wait complete. 2024-09-12T01:31:37.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.818+0000 7fb9e1879740 20 client.25358 trim_cache size 0 max 16384 2024-09-12T01:31:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.833+0000 7fb9e1879740 1 Processor -- start 2024-09-12T01:31:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.833+0000 7fb9e1879740 1 -- start start 2024-09-12T01:31:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.833+0000 7fb9e1879740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d0df48670 0x559d0df3e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.833+0000 7fb9e1879740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d0df48fb0 0x559d0df59d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.833+0000 7fb9e1879740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0df4e040 0x559d0df3e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.833+0000 7fb9e1879740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d81a83c0 con 0x559d0df4e040 2024-09-12T01:31:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.833+0000 7fb9e1879740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d81a9f90 con 0x559d0df48fb0 2024-09-12T01:31:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.833+0000 7fb9e1879740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d81a8a90 con 0x559d0df48670 2024-09-12T01:31:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.834+0000 7fb9cfe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d0df48fb0 0x559d0df59d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.834+0000 7fb9cfe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d0df48fb0 0x559d0df59d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:35510/0 (socket says 172.21.15.1:35510) 2024-09-12T01:31:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.834+0000 7fb9cfe00640 1 -- 172.21.15.1:0/1171574011 learned_addr learned my addr 172.21.15.1:0/1171574011 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:31:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.834+0000 7fb9dca00640 1 --2- 172.21.15.1:0/1171574011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0df4e040 0x559d0df3e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.834+0000 7fb9d7e00640 1 --2- 172.21.15.1:0/1171574011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d0df48670 0x559d0df3e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.834+0000 7fb9cfe00640 1 -- 172.21.15.1:0/1171574011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d0df48670 msgr2=0x559d0df3e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:31:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.834+0000 7fb9cfe00640 1 --2- 172.21.15.1:0/1171574011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d0df48670 0x559d0df3e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.834+0000 7fb9cfe00640 1 -- 172.21.15.1:0/1171574011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0df4e040 msgr2=0x559d0df3e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:31:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.834+0000 7fb9cfe00640 1 --2- 172.21.15.1:0/1171574011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0df4e040 0x559d0df3e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.834+0000 7fb9cfe00640 1 -- 172.21.15.1:0/1171574011 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d0df3f6f0 con 0x559d0df48fb0 2024-09-12T01:31:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.834+0000 7fb9cfe00640 1 --2- 172.21.15.1:0/1171574011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d0df48fb0 0x559d0df59d20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb9c000e9c0 tx=0x7fb9c0009720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98c08e3cb074bcdd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:31:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.834+0000 7fb9c7e00640 1 -- 172.21.15.1:0/1171574011 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c009b670 con 0x559d0df48fb0 2024-09-12T01:31:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.834+0000 7fb9c7e00640 1 -- 172.21.15.1:0/1171574011 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9c009bd10 con 0x559d0df48fb0 2024-09-12T01:31:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.834+0000 7fb9c7e00640 1 -- 172.21.15.1:0/1171574011 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c00a1310 con 0x559d0df48fb0 2024-09-12T01:31:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.835+0000 7fb9e1879740 1 -- 172.21.15.1:0/1171574011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d0df48fb0 msgr2=0x559d0df59d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:31:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.835+0000 7fb9e1879740 1 --2- 172.21.15.1:0/1171574011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d0df48fb0 0x559d0df59d20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb9c000e9c0 tx=0x7fb9c0009720 comp rx=0 tx=0).stop 2024-09-12T01:31:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.835+0000 7fb9e1879740 1 -- 172.21.15.1:0/1171574011 shutdown_connections 2024-09-12T01:31:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.835+0000 7fb9e1879740 1 --2- 172.21.15.1:0/1171574011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0df4e040 0x559d0df3e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.835+0000 7fb9e1879740 1 --2- 172.21.15.1:0/1171574011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d0df48fb0 0x559d0df59d20 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.835+0000 7fb9e1879740 1 --2- 172.21.15.1:0/1171574011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d0df48670 0x559d0df3e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.835+0000 7fb9e1879740 1 -- 172.21.15.1:0/1171574011 >> 172.21.15.1:0/1171574011 conn(0x559d0df3f8d0 msgr2=0x559d0df4a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:31:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.835+0000 7fb9e1879740 1 -- 172.21.15.1:0/1171574011 shutdown_connections 2024-09-12T01:31:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.835+0000 7fb9e1879740 1 -- 172.21.15.1:0/1171574011 wait complete. 2024-09-12T01:31:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.836+0000 7fb9e1879740 1 -- start start 2024-09-12T01:31:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.836+0000 7fb9e1879740 1 Processor -- start 2024-09-12T01:31:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.836+0000 7fb9e1879740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0df48670 0x559d0df56f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.836+0000 7fb9e1879740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d0df48fb0 0x559d0df57470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.836+0000 7fb9e1879740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d0df4e040 0x559d0df58480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.836+0000 7fb9e1879740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d81a9890 con 0x559d0df48670 2024-09-12T01:31:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.836+0000 7fb9e1879740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d81a91c0 con 0x559d0df48fb0 2024-09-12T01:31:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.836+0000 7fb9e1879740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d81a63a0 con 0x559d0df4e040 2024-09-12T01:31:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.836+0000 7fb9e1879740 10 client.0 _ll_register_callbacks cb 0x7fb9c00d5240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:31:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.837+0000 7fb9d7e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0df48670 0x559d0df56f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.837+0000 7fb9d7e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0df48670 0x559d0df56f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43602/0 (socket says 172.21.15.1:43602) 2024-09-12T01:31:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.837+0000 7fb9d7e00640 1 -- 172.21.15.1:0/550903660 learned_addr learned my addr 172.21.15.1:0/550903660 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:31:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.837+0000 7fb9cfe00640 1 --2- 172.21.15.1:0/550903660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d0df48fb0 0x559d0df57470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.837+0000 7fb9dca00640 1 --2- 172.21.15.1:0/550903660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d0df4e040 0x559d0df58480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.837+0000 7fb9d7e00640 1 -- 172.21.15.1:0/550903660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d0df4e040 msgr2=0x559d0df58480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:31:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.837+0000 7fb9d7e00640 1 --2- 172.21.15.1:0/550903660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d0df4e040 0x559d0df58480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.837+0000 7fb9d7e00640 1 -- 172.21.15.1:0/550903660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d0df48fb0 msgr2=0x559d0df57470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:31:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.837+0000 7fb9d7e00640 1 --2- 172.21.15.1:0/550903660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d0df48fb0 0x559d0df57470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.837+0000 7fb9d7e00640 1 -- 172.21.15.1:0/550903660 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d0df58a90 con 0x559d0df48670 2024-09-12T01:31:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.837+0000 7fb9d7e00640 1 --2- 172.21.15.1:0/550903660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0df48670 0x559d0df56f30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb9c802aa00 tx=0x7fb9c80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80deea305248919b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:31:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.837+0000 7fb9c5600640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-12T01:31:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.838+0000 7fb9c5600640 1 -- 172.21.15.1:0/550903660 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c8035040 con 0x559d0df48670 2024-09-12T01:31:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.838+0000 7fb9c5600640 1 -- 172.21.15.1:0/550903660 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fb9c8005220 con 0x559d0df48670 2024-09-12T01:31:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.838+0000 7fb9e1879740 1 -- 172.21.15.1:0/550903660 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559d0deb0b80 con 0x559d0df48670 2024-09-12T01:31:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.838+0000 7fb9c5600640 1 -- 172.21.15.1:0/550903660 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb9c8005420 con 0x559d0df48670 2024-09-12T01:31:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.838+0000 7fb9e1879740 10 client.15342 fetch_fsmap learned FSMap version 32 2024-09-12T01:31:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.838+0000 7fb9e1879740 1 -- 172.21.15.1:0/550903660 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559d0df59e00 con 0x559d0df48670 2024-09-12T01:31:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.838+0000 7fb9c5600640 1 -- 172.21.15.1:0/550903660 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x559d0deb0b80 con 0x559d0df48670 2024-09-12T01:31:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.838+0000 7fb9c5600640 1 -- 172.21.15.1:0/550903660 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7fb9c8031a80 con 0x559d0df48670 2024-09-12T01:31:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.838+0000 7fb9e1879740 10 client.15342 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T01:31:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.838+0000 7fb9e1879740 10 client.15342 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:31:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.838+0000 7fb9e1879740 20 client.15342 populate_metadata read hostname 'smithi001' 2024-09-12T01:31:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.838+0000 7fb9e1879740 1 --2- 172.21.15.1:0/550903660 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x559d0dfc90b0 0x559d0dfcb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.838+0000 7fb9e1879740 4 client.15342 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:31:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.838+0000 7fb9e1879740 1 -- 172.21.15.1:0/550903660 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559d0de2d7c0 con 0x559d0dfc90b0 2024-09-12T01:31:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.839+0000 7fb9cfe00640 1 --2- 172.21.15.1:0/550903660 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x559d0dfc90b0 0x559d0dfcb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.839+0000 7fb9cfe00640 1 --2- 172.21.15.1:0/550903660 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x559d0dfc90b0 0x559d0dfcb4a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26729c7d4335ddde server_cookie=9d9f20bf7dc3e314 in_seq=0 out_seq=0 2024-09-12T01:31:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.839+0000 7fb9c5600640 10 client.15342 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:31:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.847+0000 7fb9c5600640 1 -- 172.21.15.1:0/550903660 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559d0de2d7c0 con 0x559d0dfc90b0 2024-09-12T01:31:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.847+0000 7fb9c5600640 10 client.15342 handle_command_reply: tid=0 2024-09-12T01:31:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.847+0000 7fb9e1879740 1 client.15342 shutdown 2024-09-12T01:31:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.847+0000 7fb9e1879740 1 -- 172.21.15.1:0/550903660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0df48670 msgr2=0x559d0df56f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:31:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.847+0000 7fb9e1879740 1 --2- 172.21.15.1:0/550903660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0df48670 0x559d0df56f30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb9c802aa00 tx=0x7fb9c80045e0 comp rx=0 tx=0).stop 2024-09-12T01:31:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.847+0000 7fb9e1879740 1 -- 172.21.15.1:0/550903660 shutdown_connections 2024-09-12T01:31:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.847+0000 7fb9e1879740 1 --2- 172.21.15.1:0/550903660 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x559d0dfc90b0 0x559d0dfcb4a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.847+0000 7fb9e1879740 1 --2- 172.21.15.1:0/550903660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d0df4e040 0x559d0df58480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.847+0000 7fb9e1879740 1 --2- 172.21.15.1:0/550903660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d0df48fb0 0x559d0df57470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.847+0000 7fb9e1879740 1 --2- 172.21.15.1:0/550903660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559d0df48670 0x559d0df56f30 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.847+0000 7fb9e1879740 1 -- 172.21.15.1:0/550903660 >> 172.21.15.1:0/550903660 conn(0x559d0df3f8d0 msgr2=0x559d0df40780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:31:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.847+0000 7fb9c5600640 0 client.15342 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:31:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.847+0000 7fb9e1879740 1 -- 172.21.15.1:0/550903660 shutdown_connections 2024-09-12T01:31:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.847+0000 7fb9e1879740 1 -- 172.21.15.1:0/550903660 wait complete. 2024-09-12T01:31:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.847+0000 7fb9e1879740 20 client.15342 trim_cache size 0 max 16384 2024-09-12T01:31:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-12T01:31:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-12T01:31:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-12T01:31:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-12T01:31:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.848+0000 7fb9dd400640 1 -- 172.21.15.1:0/3414552338 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fb9bc0a2b70 msgr2=0x7fb9bc0a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:31:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.848+0000 7fb9dd400640 1 --2- 172.21.15.1:0/3414552338 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fb9bc0a2b70 0x7fb9bc0a5030 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0005fd0 tx=0x7fb9d0009500 comp rx=0 tx=0).stop 2024-09-12T01:31:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.848+0000 7fb9dd400640 1 -- 172.21.15.1:0/3414552338 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9d8102ba0 msgr2=0x7fb9d8198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:31:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.848+0000 7fb9dd400640 1 --2- 172.21.15.1:0/3414552338 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9d8102ba0 0x7fb9d8198230 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb9c000e9f0 tx=0x7fb9c000edf0 comp rx=0 tx=0).stop 2024-09-12T01:31:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.848+0000 7fb9dd400640 1 -- 172.21.15.1:0/3414552338 shutdown_connections 2024-09-12T01:31:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.848+0000 7fb9dd400640 1 --2- 172.21.15.1:0/3414552338 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fb9bc0a2b70 0x7fb9bc0a5030 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.848+0000 7fb9dd400640 1 --2- 172.21.15.1:0/3414552338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9d8103560 0x7fb9d819f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.848+0000 7fb9dd400640 1 --2- 172.21.15.1:0/3414552338 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9d8102ba0 0x7fb9d8198230 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.848+0000 7fb9dd400640 1 --2- 172.21.15.1:0/3414552338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9d81019b0 0x7fb9d8197cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.848+0000 7fb9dd400640 1 -- 172.21.15.1:0/3414552338 >> 172.21.15.1:0/3414552338 conn(0x7fb9d80fd130 msgr2=0x7fb9d8105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:31:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.848+0000 7fb9dd400640 1 -- 172.21.15.1:0/3414552338 shutdown_connections 2024-09-12T01:31:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:37.848+0000 7fb9dd400640 1 -- 172.21.15.1:0/3414552338 wait complete. 2024-09-12T01:31:37.919 INFO:tasks.workunit.client.0.smithi001.stdout:checking malloc.h usability... yes 2024-09-12T01:31:37.951 INFO:tasks.workunit.client.0.smithi001.stdout:checking malloc.h presence... yes 2024-09-12T01:31:37.951 INFO:tasks.workunit.client.0.smithi001.stdout:checking for malloc.h... yes 2024-09-12T01:31:38.033 INFO:tasks.workunit.client.0.smithi001.stdout:checking uuid.h usability... no 2024-09-12T01:31:38.083 INFO:tasks.workunit.client.0.smithi001.stdout:checking uuid.h presence... no 2024-09-12T01:31:38.084 INFO:tasks.workunit.client.0.smithi001.stdout:checking for uuid.h... no 2024-09-12T01:31:38.164 INFO:tasks.workunit.client.0.smithi001.stdout:checking uuid/uuid.h usability... yes 2024-09-12T01:31:38.198 INFO:tasks.workunit.client.0.smithi001.stdout:checking uuid/uuid.h presence... yes 2024-09-12T01:31:38.198 INFO:tasks.workunit.client.0.smithi001.stdout:checking for uuid/uuid.h... yes 2024-09-12T01:31:38.278 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/uuid.h usability... no 2024-09-12T01:31:38.322 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/uuid.h presence... no 2024-09-12T01:31:38.322 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/uuid.h... no 2024-09-12T01:31:38.388 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/file.h usability... yes 2024-09-12T01:31:38.415 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/file.h presence... yes 2024-09-12T01:31:38.415 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/file.h... yes 2024-09-12T01:31:38.490 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/fcntl.h usability... yes 2024-09-12T01:31:38.532 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/fcntl.h presence... yes 2024-09-12T01:31:38.532 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/fcntl.h... yes 2024-09-12T01:31:38.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:31:38 smithi104 ceph-mon[35219]: pgmap v459: 129 pgs: 129 active+clean; 50 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 1.1 MiB/s wr, 377 op/s 2024-09-12T01:31:38.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:31:38 smithi001 ceph-mon[30387]: pgmap v459: 129 pgs: 129 active+clean; 50 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 1.1 MiB/s wr, 377 op/s 2024-09-12T01:31:38.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:31:38 smithi053 ceph-mon[35158]: pgmap v459: 129 pgs: 129 active+clean; 50 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 1.1 MiB/s wr, 377 op/s 2024-09-12T01:31:39.066 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/syssgi.h usability... no 2024-09-12T01:31:39.739 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/syssgi.h presence... no 2024-09-12T01:31:39.739 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/syssgi.h... no 2024-09-12T01:31:39.856 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/param.h usability... yes 2024-09-12T01:31:39.887 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/param.h presence... yes 2024-09-12T01:31:39.887 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/param.h... yes 2024-09-12T01:31:39.908 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/stat.h... (cached) yes 2024-09-12T01:31:39.989 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/statvfs.h usability... yes 2024-09-12T01:31:40.016 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/statvfs.h presence... yes 2024-09-12T01:31:40.016 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/statvfs.h... yes 2024-09-12T01:31:40.089 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/time.h usability... yes 2024-09-12T01:31:40.115 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/time.h presence... yes 2024-09-12T01:31:40.115 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/time.h... yes 2024-09-12T01:31:40.188 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/ioctl.h usability... yes 2024-09-12T01:31:40.217 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/ioctl.h presence... yes 2024-09-12T01:31:40.217 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/ioctl.h... yes 2024-09-12T01:31:40.299 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/wait.h usability... yes 2024-09-12T01:31:40.328 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/wait.h presence... yes 2024-09-12T01:31:40.328 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/wait.h... yes 2024-09-12T01:31:40.349 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/types.h... (cached) yes 2024-09-12T01:31:40.361 INFO:tasks.workunit.client.0.smithi001.stdout:checking for strings.h... (cached) yes 2024-09-12T01:31:40.442 INFO:tasks.workunit.client.0.smithi001.stdout:checking err.h usability... yes 2024-09-12T01:31:40.470 INFO:tasks.workunit.client.0.smithi001.stdout:checking err.h presence... yes 2024-09-12T01:31:40.470 INFO:tasks.workunit.client.0.smithi001.stdout:checking for err.h... yes 2024-09-12T01:31:40.542 INFO:tasks.workunit.client.0.smithi001.stdout:checking linux/falloc.h usability... yes 2024-09-12T01:31:40.567 INFO:tasks.workunit.client.0.smithi001.stdout:checking linux/falloc.h presence... yes 2024-09-12T01:31:40.567 INFO:tasks.workunit.client.0.smithi001.stdout:checking for linux/falloc.h... yes 2024-09-12T01:31:40.643 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-09-12T01:31:40.683 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-09-12T01:31:40.683 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/fs/xfs_fsops.h... no 2024-09-12T01:31:40.747 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/fs/xfs_itable.h usability... no 2024-09-12T01:31:40.788 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/fs/xfs_itable.h presence... no 2024-09-12T01:31:40.788 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/fs/xfs_itable.h... no 2024-09-12T01:31:40.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:31:40 smithi104 ceph-mon[35219]: pgmap v460: 129 pgs: 129 active+clean; 50 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 854 KiB/s wr, 140 op/s 2024-09-12T01:31:40.852 INFO:tasks.workunit.client.0.smithi001.stdout:checking xfs/platform_defs.h usability... no 2024-09-12T01:31:40.893 INFO:tasks.workunit.client.0.smithi001.stdout:checking xfs/platform_defs.h presence... no 2024-09-12T01:31:40.893 INFO:tasks.workunit.client.0.smithi001.stdout:checking for xfs/platform_defs.h... no 2024-09-12T01:31:40.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:31:40 smithi001 ceph-mon[30387]: pgmap v460: 129 pgs: 129 active+clean; 50 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 854 KiB/s wr, 140 op/s 2024-09-12T01:31:40.958 INFO:tasks.workunit.client.0.smithi001.stdout:checking btrfs/ioctl.h usability... no 2024-09-12T01:31:40.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:31:40 smithi053 ceph-mon[35158]: pgmap v460: 129 pgs: 129 active+clean; 50 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 854 KiB/s wr, 140 op/s 2024-09-12T01:31:41.002 INFO:tasks.workunit.client.0.smithi001.stdout:checking btrfs/ioctl.h presence... no 2024-09-12T01:31:41.002 INFO:tasks.workunit.client.0.smithi001.stdout:checking for btrfs/ioctl.h... no 2024-09-12T01:31:41.071 INFO:tasks.workunit.client.0.smithi001.stdout:checking cifs/ioctl.h usability... no 2024-09-12T01:31:41.116 INFO:tasks.workunit.client.0.smithi001.stdout:checking cifs/ioctl.h presence... no 2024-09-12T01:31:41.116 INFO:tasks.workunit.client.0.smithi001.stdout:checking for cifs/ioctl.h... no 2024-09-12T01:31:41.187 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/mman.h usability... yes 2024-09-12T01:31:41.214 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/mman.h presence... yes 2024-09-12T01:31:41.215 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/mman.h... yes 2024-09-12T01:31:41.294 INFO:tasks.workunit.client.0.smithi001.stdout:checking linux/ext4.h usability... no 2024-09-12T01:31:41.335 INFO:tasks.workunit.client.0.smithi001.stdout:checking linux/ext4.h presence... no 2024-09-12T01:31:41.335 INFO:tasks.workunit.client.0.smithi001.stdout:checking for linux/ext4.h... no 2024-09-12T01:31:41.377 INFO:tasks.workunit.client.0.smithi001.stdout:checking for xfs/xfs_log_format.h... no 2024-09-12T01:31:41.466 INFO:tasks.workunit.client.0.smithi001.stdout:checking for uuid_compare... no 2024-09-12T01:31:41.531 INFO:tasks.workunit.client.0.smithi001.stdout:checking for uuid_compare in -luuid... yes 2024-09-12T01:31:41.628 INFO:tasks.workunit.client.0.smithi001.stdout:checking for xfs/xfs.h... yes 2024-09-12T01:31:41.683 INFO:tasks.workunit.client.0.smithi001.stdout:checking for xfs/libxfs.h... no 2024-09-12T01:31:41.730 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether xlog_assign_lsn is declared... no 2024-09-12T01:31:41.794 INFO:tasks.workunit.client.0.smithi001.stdout:checking for xfs/xqm.h... yes 2024-09-12T01:31:41.884 INFO:tasks.workunit.client.0.smithi001.stdout:checking xfsctl from xfs/xfs.h... ok 2024-09-12T01:31:41.941 INFO:tasks.workunit.client.0.smithi001.stdout:checking xfs/handle.h usability... yes 2024-09-12T01:31:41.966 INFO:tasks.workunit.client.0.smithi001.stdout:checking xfs/handle.h presence... yes 2024-09-12T01:31:41.966 INFO:tasks.workunit.client.0.smithi001.stdout:checking for xfs/handle.h... yes 2024-09-12T01:31:42.039 INFO:tasks.workunit.client.0.smithi001.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-09-12T01:31:42.098 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/acl.h usability... yes 2024-09-12T01:31:42.127 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/acl.h presence... yes 2024-09-12T01:31:42.127 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/acl.h... yes 2024-09-12T01:31:42.184 INFO:tasks.workunit.client.0.smithi001.stdout:checking acl/libacl.h usability... yes 2024-09-12T01:31:42.213 INFO:tasks.workunit.client.0.smithi001.stdout:checking acl/libacl.h presence... yes 2024-09-12T01:31:42.213 INFO:tasks.workunit.client.0.smithi001.stdout:checking for acl/libacl.h... yes 2024-09-12T01:31:42.281 INFO:tasks.workunit.client.0.smithi001.stdout:checking for acl_init in -lacl... yes 2024-09-12T01:31:42.368 INFO:tasks.workunit.client.0.smithi001.stdout:checking gdbm-ndbm.h usability... no 2024-09-12T01:31:42.409 INFO:tasks.workunit.client.0.smithi001.stdout:checking gdbm-ndbm.h presence... no 2024-09-12T01:31:42.410 INFO:tasks.workunit.client.0.smithi001.stdout:checking for gdbm-ndbm.h... no 2024-09-12T01:31:42.467 INFO:tasks.workunit.client.0.smithi001.stdout:checking gdbm/ndbm.h usability... no 2024-09-12T01:31:42.508 INFO:tasks.workunit.client.0.smithi001.stdout:checking gdbm/ndbm.h presence... no 2024-09-12T01:31:42.509 INFO:tasks.workunit.client.0.smithi001.stdout:checking for gdbm/ndbm.h... no 2024-09-12T01:31:42.565 INFO:tasks.workunit.client.0.smithi001.stdout:checking ndbm.h usability... no 2024-09-12T01:31:42.606 INFO:tasks.workunit.client.0.smithi001.stdout:checking ndbm.h presence... no 2024-09-12T01:31:42.606 INFO:tasks.workunit.client.0.smithi001.stdout:checking for ndbm.h... no 2024-09-12T01:31:42.660 INFO:tasks.workunit.client.0.smithi001.stdout:checking libaio.h usability... yes 2024-09-12T01:31:42.690 INFO:tasks.workunit.client.0.smithi001.stdout:checking libaio.h presence... yes 2024-09-12T01:31:42.690 INFO:tasks.workunit.client.0.smithi001.stdout:checking for libaio.h... yes 2024-09-12T01:31:42.754 INFO:tasks.workunit.client.0.smithi001.stdout:checking liburing.h usability... no 2024-09-12T01:31:42.795 INFO:tasks.workunit.client.0.smithi001.stdout:checking liburing.h presence... no 2024-09-12T01:31:42.796 INFO:tasks.workunit.client.0.smithi001.stdout:checking for liburing.h... no 2024-09-12T01:31:42.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:31:42 smithi104 ceph-mon[35219]: pgmap v461: 129 pgs: 129 active+clean; 53 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 815 KiB/s wr, 226 op/s 2024-09-12T01:31:42.853 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/capability.h usability... no 2024-09-12T01:31:42.897 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/capability.h presence... no 2024-09-12T01:31:42.897 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/capability.h... no 2024-09-12T01:31:42.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:31:42 smithi001 ceph-mon[30387]: pgmap v461: 129 pgs: 129 active+clean; 53 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 815 KiB/s wr, 226 op/s 2024-09-12T01:31:42.949 INFO:tasks.workunit.client.0.smithi001.stdout:checking linux/fiemap.h usability... yes 2024-09-12T01:31:42.974 INFO:tasks.workunit.client.0.smithi001.stdout:checking linux/fiemap.h presence... yes 2024-09-12T01:31:42.974 INFO:tasks.workunit.client.0.smithi001.stdout:checking for linux/fiemap.h... yes 2024-09-12T01:31:42.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:31:42 smithi053 ceph-mon[35158]: pgmap v461: 129 pgs: 129 active+clean; 53 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 815 KiB/s wr, 226 op/s 2024-09-12T01:31:43.041 INFO:tasks.workunit.client.0.smithi001.stdout:checking for fallocate... yes 2024-09-12T01:31:43.128 INFO:tasks.workunit.client.0.smithi001.stdout:checking for open_by_handle_at... yes 2024-09-12T01:31:43.187 INFO:tasks.workunit.client.0.smithi001.stdout:checking linux/fs.h usability... yes 2024-09-12T01:31:43.214 INFO:tasks.workunit.client.0.smithi001.stdout:checking linux/fs.h presence... yes 2024-09-12T01:31:43.214 INFO:tasks.workunit.client.0.smithi001.stdout:checking for linux/fs.h... yes 2024-09-12T01:31:43.288 INFO:tasks.workunit.client.0.smithi001.stdout:checking btrfsutil.h usability... no 2024-09-12T01:31:43.330 INFO:tasks.workunit.client.0.smithi001.stdout:checking btrfsutil.h presence... no 2024-09-12T01:31:43.330 INFO:tasks.workunit.client.0.smithi001.stdout:checking for btrfsutil.h... no 2024-09-12T01:31:43.398 INFO:tasks.workunit.client.0.smithi001.stdout:checking for copy_file_range... yes 2024-09-12T01:31:43.470 INFO:tasks.workunit.client.0.smithi001.stdout:checking for SEEK_DATA... yes 2024-09-12T01:31:43.580 INFO:tasks.workunit.client.0.smithi001.stdout:checking for BMV_OF_SHARED... yes 2024-09-12T01:31:43.654 INFO:tasks.workunit.client.0.smithi001.stdout:checking for nftw... yes 2024-09-12T01:31:43.880 INFO:tasks.workunit.client.0.smithi001.stdout:checking for RLIMIT_NOFILE... yes 2024-09-12T01:31:44.649 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:31:44 smithi001 ceph-mon[30387]: pgmap v462: 129 pgs: 129 active+clean; 56 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 745 KiB/s wr, 203 op/s 2024-09-12T01:31:44.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:31:44 smithi104 ceph-mon[35219]: pgmap v462: 129 pgs: 129 active+clean; 56 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 745 KiB/s wr, 203 op/s 2024-09-12T01:31:44.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:31:44 smithi053 ceph-mon[35158]: pgmap v462: 129 pgs: 129 active+clean; 56 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 745 KiB/s wr, 203 op/s 2024-09-12T01:31:45.838 INFO:tasks.workunit.client.0.smithi001.stdout:checking for XFS_IOC_EXCHANGE_RANGE... no 2024-09-12T01:31:46.004 INFO:tasks.workunit.client.0.smithi001.stdout:checking for FICLONE... yes 2024-09-12T01:31:46.078 INFO:tasks.workunit.client.0.smithi001.stdout:checking for renameat2... yes 2024-09-12T01:31:46.148 INFO:tasks.workunit.client.0.smithi001.stdout:checking for reallocarray... yes 2024-09-12T01:31:46.257 INFO:tasks.workunit.client.0.smithi001.stdout:checking for struct mount_attr... yes 2024-09-12T01:31:46.356 INFO:tasks.workunit.client.0.smithi001.stdout:checking for struct btrfs_qgroup_limit... yes 2024-09-12T01:31:46.447 INFO:tasks.workunit.client.0.smithi001.stdout:checking for struct btrfs_qgroup_inherit... yes 2024-09-12T01:31:46.577 INFO:tasks.workunit.client.0.smithi001.stdout:checking for struct btrfs_ioctl_vol_args... yes 2024-09-12T01:31:46.684 INFO:tasks.workunit.client.0.smithi001.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2024-09-12T01:31:46.779 INFO:tasks.workunit.client.0.smithi001.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2024-09-12T01:31:46.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:31:46 smithi104 ceph-mon[35219]: pgmap v463: 129 pgs: 129 active+clean; 61 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 1.1 MiB/s wr, 220 op/s 2024-09-12T01:31:46.876 INFO:tasks.workunit.client.0.smithi001.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2024-09-12T01:31:46.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:31:46 smithi001 ceph-mon[30387]: pgmap v463: 129 pgs: 129 active+clean; 61 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 1.1 MiB/s wr, 220 op/s 2024-09-12T01:31:46.973 INFO:tasks.workunit.client.0.smithi001.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2024-09-12T01:31:46.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:31:46 smithi053 ceph-mon[35158]: pgmap v463: 129 pgs: 129 active+clean; 61 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 1.1 MiB/s wr, 220 op/s 2024-09-12T01:31:47.043 INFO:tasks.workunit.client.0.smithi001.stdout:checking linux/btrfs.h usability... yes 2024-09-12T01:31:47.072 INFO:tasks.workunit.client.0.smithi001.stdout:checking linux/btrfs.h presence... yes 2024-09-12T01:31:47.072 INFO:tasks.workunit.client.0.smithi001.stdout:checking for linux/btrfs.h... yes 2024-09-12T01:31:47.153 INFO:tasks.workunit.client.0.smithi001.stdout:checking linux/btrfs_tree.h usability... yes 2024-09-12T01:31:47.183 INFO:tasks.workunit.client.0.smithi001.stdout:checking linux/btrfs_tree.h presence... yes 2024-09-12T01:31:47.183 INFO:tasks.workunit.client.0.smithi001.stdout:checking for linux/btrfs_tree.h... yes 2024-09-12T01:31:47.249 INFO:tasks.workunit.client.0.smithi001.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... yes 2024-09-12T01:31:47.308 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether BTRFS_IOC_SNAP_DESTROY_V2 is declared... yes 2024-09-12T01:31:47.340 INFO:tasks.workunit.client.0.smithi001.stdout:configure: creating ./config.status 2024-09-12T01:31:48.197 INFO:tasks.workunit.client.0.smithi001.stdout:config.status: creating include/builddefs 2024-09-12T01:31:48.232 INFO:tasks.workunit.client.0.smithi001.stdout:config.status: creating include/config.h 2024-09-12T01:31:48.254 INFO:tasks.workunit.client.0.smithi001.stdout:config.status: executing libtool commands 2024-09-12T01:31:48.366 INFO:tasks.workunit.client.0.smithi001.stdout:Building include 2024-09-12T01:31:48.366 INFO:tasks.workunit.client.0.smithi001.stdout:Building lib 2024-09-12T01:31:48.366 INFO:tasks.workunit.client.0.smithi001.stdout:Building tests 2024-09-12T01:31:48.375 INFO:tasks.workunit.client.0.smithi001.stdout: [LTDEP] 2024-09-12T01:31:48.380 INFO:tasks.workunit.client.0.smithi001.stdout:Building btrfs 2024-09-12T01:31:48.381 INFO:tasks.workunit.client.0.smithi001.stdout:Building ceph 2024-09-12T01:31:48.382 INFO:tasks.workunit.client.0.smithi001.stdout:Building cifs 2024-09-12T01:31:48.395 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/cifs/group.list 2024-09-12T01:31:48.395 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ceph/group.list 2024-09-12T01:31:48.414 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/btrfs/group.list 2024-09-12T01:31:48.439 INFO:tasks.workunit.client.0.smithi001.stdout:Building ext4 2024-09-12T01:31:48.445 INFO:tasks.workunit.client.0.smithi001.stdout:Building f2fs 2024-09-12T01:31:48.461 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/f2fs/group.list 2024-09-12T01:31:48.462 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ext4/group.list 2024-09-12T01:31:48.518 INFO:tasks.workunit.client.0.smithi001.stdout:Building generic 2024-09-12T01:31:48.596 INFO:tasks.workunit.client.0.smithi001.stdout:Building nfs 2024-09-12T01:31:48.608 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/nfs/group.list 2024-09-12T01:31:48.639 INFO:tasks.workunit.client.0.smithi001.stdout:Building ocfs2 2024-09-12T01:31:48.651 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ocfs2/group.list 2024-09-12T01:31:48.665 INFO:tasks.workunit.client.0.smithi001.stdout:Building overlay 2024-09-12T01:31:48.689 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/overlay/group.list 2024-09-12T01:31:48.692 INFO:tasks.workunit.client.0.smithi001.stdout:Building perf 2024-09-12T01:31:48.705 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/perf/group.list 2024-09-12T01:31:48.721 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/generic/group.list 2024-09-12T01:31:48.739 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] dataascii.lo 2024-09-12T01:31:48.743 INFO:tasks.workunit.client.0.smithi001.stdout:Building selftest 2024-09-12T01:31:48.755 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/selftest/group.list 2024-09-12T01:31:48.798 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] databin.lo 2024-09-12T01:31:48.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:31:48 smithi104 ceph-mon[35219]: pgmap v464: 129 pgs: 129 active+clean; 63 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 1.3 MiB/s wr, 230 op/s 2024-09-12T01:31:48.831 INFO:tasks.workunit.client.0.smithi001.stdout:Building shared 2024-09-12T01:31:48.846 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/shared/group.list 2024-09-12T01:31:48.905 INFO:tasks.workunit.client.0.smithi001.stdout:Building tmpfs 2024-09-12T01:31:48.918 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/tmpfs/group.list 2024-09-12T01:31:48.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:31:48 smithi001 ceph-mon[30387]: pgmap v464: 129 pgs: 129 active+clean; 63 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 1.3 MiB/s wr, 230 op/s 2024-09-12T01:31:48.961 INFO:tasks.workunit.client.0.smithi001.stdout:Building udf 2024-09-12T01:31:48.974 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/udf/group.list 2024-09-12T01:31:48.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:31:48 smithi053 ceph-mon[35158]: pgmap v464: 129 pgs: 129 active+clean; 63 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 1.3 MiB/s wr, 230 op/s 2024-09-12T01:31:48.990 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] datapid.lo 2024-09-12T01:31:49.012 INFO:tasks.workunit.client.0.smithi001.stdout:Building xfs 2024-09-12T01:31:49.075 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] file_lock.lo 2024-09-12T01:31:49.092 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] forker.lo 2024-09-12T01:31:49.110 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/xfs/group.list 2024-09-12T01:31:49.208 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] pattern.lo 2024-09-12T01:31:49.343 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] open_flags.lo 2024-09-12T01:31:49.353 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] random_range.lo 2024-09-12T01:31:49.455 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] string_to_tokens.lo 2024-09-12T01:31:49.470 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] str_to_bytes.lo 2024-09-12T01:31:49.621 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] tlibio.lo 2024-09-12T01:31:49.665 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] write_log.lo 2024-09-12T01:31:49.734 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] random.lo 2024-09-12T01:31:50.016 INFO:tasks.workunit.client.0.smithi001.stdout: [LD] 2024-09-12T01:31:50.208 INFO:tasks.workunit.client.0.smithi001.stdout:Building ltp 2024-09-12T01:31:50.208 INFO:tasks.workunit.client.0.smithi001.stdout:Building src 2024-09-12T01:31:50.208 INFO:tasks.workunit.client.0.smithi001.stdout:Building m4 2024-09-12T01:31:50.209 INFO:tasks.workunit.client.0.smithi001.stdout:Building common 2024-09-12T01:31:50.217 INFO:tasks.workunit.client.0.smithi001.stdout: [DEP] 2024-09-12T01:31:50.230 INFO:tasks.workunit.client.0.smithi001.stdout: [DEP] 2024-09-12T01:31:50.545 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] doio 2024-09-12T01:31:50.545 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] fsstress 2024-09-12T01:31:50.546 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] fsx 2024-09-12T01:31:50.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:31:50 smithi104 ceph-mon[35219]: pgmap v465: 129 pgs: 129 active+clean; 64 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 1.3 MiB/s wr, 215 op/s 2024-09-12T01:31:50.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:31:50 smithi001 ceph-mon[30387]: pgmap v465: 129 pgs: 129 active+clean; 64 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 1.3 MiB/s wr, 215 op/s 2024-09-12T01:31:50.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:31:50 smithi053 ceph-mon[35158]: pgmap v465: 129 pgs: 129 active+clean; 64 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 1.3 MiB/s wr, 215 op/s 2024-09-12T01:31:51.000 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.000+0000 7f4c84600640 1 -- 172.21.15.1:0/2246558729 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c7c103d60 msgr2=0x7f4c7c104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:31:51.000 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.000+0000 7f4c84600640 1 --2- 172.21.15.1:0/2246558729 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c7c103d60 0x7f4c7c104160 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4c70009900 tx=0x7f4c7002a410 comp rx=0 tx=0).stop 2024-09-12T01:31:51.000 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.000+0000 7f4c84600640 1 -- 172.21.15.1:0/2246558729 shutdown_connections 2024-09-12T01:31:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.000+0000 7f4c84600640 1 --2- 172.21.15.1:0/2246558729 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c7c105910 0x7f4c7c10c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.000+0000 7f4c84600640 1 --2- 172.21.15.1:0/2246558729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c7c104f50 0x7f4c7c1053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.000+0000 7f4c84600640 1 --2- 172.21.15.1:0/2246558729 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c7c103d60 0x7f4c7c104160 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.000+0000 7f4c84600640 1 -- 172.21.15.1:0/2246558729 >> 172.21.15.1:0/2246558729 conn(0x7f4c7c0ff4e0 msgr2=0x7f4c7c101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:31:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.001+0000 7f4c84600640 1 -- 172.21.15.1:0/2246558729 shutdown_connections 2024-09-12T01:31:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.001+0000 7f4c84600640 1 -- 172.21.15.1:0/2246558729 wait complete. 2024-09-12T01:31:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.002+0000 7f4c84600640 1 Processor -- start 2024-09-12T01:31:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.002+0000 7f4c84600640 1 -- start start 2024-09-12T01:31:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.002+0000 7f4c84600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c7c103d60 0x7f4c7c19a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.002+0000 7f4c84600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c7c104f50 0x7f4c7c19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.002+0000 7f4c84600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c7c105910 0x7f4c7c1a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.002+0000 7f4c84600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c10f9c0 con 0x7f4c7c104f50 2024-09-12T01:31:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.002+0000 7f4c84600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c10f840 con 0x7f4c7c105910 2024-09-12T01:31:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.002+0000 7f4c84600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c10fb40 con 0x7f4c7c103d60 2024-09-12T01:31:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.002+0000 7f4c81200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c7c104f50 0x7f4c7c19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.003+0000 7f4c81200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c7c104f50 0x7f4c7c19a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48624/0 (socket says 172.21.15.1:48624) 2024-09-12T01:31:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.003+0000 7f4c81c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c7c103d60 0x7f4c7c19a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.003+0000 7f4c81200640 1 -- 172.21.15.1:0/4127526940 learned_addr learned my addr 172.21.15.1:0/4127526940 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:31:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.003+0000 7f4c82600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c7c105910 0x7f4c7c1a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:51.004 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.003+0000 7f4c81200640 1 -- 172.21.15.1:0/4127526940 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c7c103d60 msgr2=0x7f4c7c19a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:31:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.003+0000 7f4c81200640 1 --2- 172.21.15.1:0/4127526940 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c7c103d60 0x7f4c7c19a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.003+0000 7f4c81200640 1 -- 172.21.15.1:0/4127526940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c7c105910 msgr2=0x7f4c7c1a1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:31:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.003+0000 7f4c81200640 1 --2- 172.21.15.1:0/4127526940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c7c105910 0x7f4c7c1a1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.003+0000 7f4c81200640 1 -- 172.21.15.1:0/4127526940 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c7c1a1ca0 con 0x7f4c7c104f50 2024-09-12T01:31:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.003+0000 7f4c82600640 1 --2- 172.21.15.1:0/4127526940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c7c105910 0x7f4c7c1a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:31:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.004+0000 7f4c81200640 1 --2- 172.21.15.1:0/4127526940 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c7c104f50 0x7f4c7c19a5c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4c6800e9f0 tx=0x7f4c6800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:31:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.004+0000 7f4c7a000640 1 -- 172.21.15.1:0/4127526940 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6800cd80 con 0x7f4c7c104f50 2024-09-12T01:31:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.004+0000 7f4c7a000640 1 -- 172.21.15.1:0/4127526940 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c68010910 con 0x7f4c7c104f50 2024-09-12T01:31:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.004+0000 7f4c84600640 1 -- 172.21.15.1:0/4127526940 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c7c1a1f90 con 0x7f4c7c104f50 2024-09-12T01:31:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.004+0000 7f4c7a000640 1 -- 172.21.15.1:0/4127526940 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c68018990 con 0x7f4c7c104f50 2024-09-12T01:31:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.004+0000 7f4c84600640 1 -- 172.21.15.1:0/4127526940 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c7c1a24a0 con 0x7f4c7c104f50 2024-09-12T01:31:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.008+0000 7f4c7a000640 1 -- 172.21.15.1:0/4127526940 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f4c68010ac0 con 0x7f4c7c104f50 2024-09-12T01:31:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.009+0000 7f4c87616740 1 Processor -- start 2024-09-12T01:31:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.009+0000 7f4c87616740 1 -- start start 2024-09-12T01:31:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.009+0000 7f4c87616740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d8abb0090 0x564d8abb0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.009+0000 7f4c87616740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d8abb09d0 0x564d8abb5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.009+0000 7f4c87616740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d8abb5a60 0x564d8abb7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.009+0000 7f4c87616740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564d8ab15100 con 0x564d8abb5a60 2024-09-12T01:31:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.009+0000 7f4c87616740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x564d8aabd1b0 con 0x564d8abb09d0 2024-09-12T01:31:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.009+0000 7f4c87616740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564d8ab110c0 con 0x564d8abb0090 2024-09-12T01:31:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.010+0000 7f4c7a000640 1 --2- 172.21.15.1:0/4127526940 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f4c640a2c00 0x7f4c640a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.010+0000 7f4c81c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d8abb09d0 0x564d8abb5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.010+0000 7f4c81c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d8abb09d0 0x564d8abb5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:46260/0 (socket says 172.21.15.1:46260) 2024-09-12T01:31:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.010+0000 7f4c81c00640 1 -- 172.21.15.1:0/926031491 learned_addr learned my addr 172.21.15.1:0/926031491 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:31:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.010+0000 7f4c7a000640 1 -- 172.21.15.1:0/4127526940 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f4c680cf670 con 0x7f4c7c104f50 2024-09-12T01:31:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.010+0000 7f4c82600640 1 --2- 172.21.15.1:0/926031491 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d8abb0090 0x564d8abb0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.010+0000 7f4c81c00640 1 -- 172.21.15.1:0/926031491 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d8abb0090 msgr2=0x564d8abb0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:31:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.010+0000 7f4c81c00640 1 --2- 172.21.15.1:0/926031491 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d8abb0090 0x564d8abb0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.010+0000 7f4c81c00640 1 -- 172.21.15.1:0/926031491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d8abb5a60 msgr2=0x564d8abb7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:31:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.010+0000 7f4c81c00640 1 --2- 172.21.15.1:0/926031491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d8abb5a60 0x564d8abb7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.010+0000 7f4c81c00640 1 -- 172.21.15.1:0/926031491 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d8a828210 con 0x564d8abb09d0 2024-09-12T01:31:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.011+0000 7f4c81c00640 1 --2- 172.21.15.1:0/926031491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d8abb09d0 0x564d8abb5490 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4c70031610 tx=0x7f4c70005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfd06e04ba8f8074 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:31:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.011+0000 7f4c82600640 1 --2- 172.21.15.1:0/4127526940 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f4c640a2c00 0x7f4c640a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.011+0000 7f4c6ea00640 1 -- 172.21.15.1:0/926031491 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c70040070 con 0x564d8abb09d0 2024-09-12T01:31:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.011+0000 7f4c6ea00640 1 -- 172.21.15.1:0/926031491 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c70004030 con 0x564d8abb09d0 2024-09-12T01:31:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.011+0000 7f4c6ea00640 1 -- 172.21.15.1:0/926031491 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c70035040 con 0x564d8abb09d0 2024-09-12T01:31:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.012+0000 7f4c87616740 1 -- 172.21.15.1:0/926031491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d8abb09d0 msgr2=0x564d8abb5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:31:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.012+0000 7f4c87616740 1 --2- 172.21.15.1:0/926031491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d8abb09d0 0x564d8abb5490 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4c70031610 tx=0x7f4c70005de0 comp rx=0 tx=0).stop 2024-09-12T01:31:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.012+0000 7f4c82600640 1 --2- 172.21.15.1:0/4127526940 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f4c640a2c00 0x7f4c640a50c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4c740099e0 tx=0x7f4c74013040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:31:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.012+0000 7f4c87616740 1 -- 172.21.15.1:0/926031491 shutdown_connections 2024-09-12T01:31:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.013+0000 7f4c87616740 1 --2- 172.21.15.1:0/926031491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d8abb5a60 0x564d8abb7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.013+0000 7f4c87616740 1 --2- 172.21.15.1:0/926031491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d8abb09d0 0x564d8abb5490 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.013+0000 7f4c87616740 1 --2- 172.21.15.1:0/926031491 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d8abb0090 0x564d8abb0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.013+0000 7f4c87616740 1 -- 172.21.15.1:0/926031491 >> 172.21.15.1:0/926031491 conn(0x564d8abab700 msgr2=0x564d8ababb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:31:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.013+0000 7f4c87616740 1 -- 172.21.15.1:0/926031491 shutdown_connections 2024-09-12T01:31:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.013+0000 7f4c87616740 1 -- 172.21.15.1:0/926031491 wait complete. 2024-09-12T01:31:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.013+0000 7f4c87616740 1 -- start start 2024-09-12T01:31:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.014+0000 7f4c87616740 1 Processor -- start 2024-09-12T01:31:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.014+0000 7f4c87616740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d8abb0090 0x564d8abba300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.014+0000 7f4c87616740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d8abb09d0 0x564d8abba840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.014+0000 7f4c87616740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d8abb5a60 0x564d8abc1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:51.024 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.014+0000 7f4c87616740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564d8abb89d0 con 0x564d8abb5a60 2024-09-12T01:31:51.024 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.014+0000 7f4c87616740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x564d8aa9e090 con 0x564d8abb0090 2024-09-12T01:31:51.024 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.014+0000 7f4c87616740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564d8abb8b50 con 0x564d8abb09d0 2024-09-12T01:31:51.024 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.015+0000 7f4c87616740 10 client.0 _ll_register_callbacks cb 0x564d8aaf2f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:31:51.024 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.015+0000 7f4c81c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d8abb0090 0x564d8abba300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:51.024 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.015+0000 7f4c81c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d8abb0090 0x564d8abba300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:46276/0 (socket says 172.21.15.1:46276) 2024-09-12T01:31:51.024 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.015+0000 7f4c81c00640 1 -- 172.21.15.1:0/3309975620 learned_addr learned my addr 172.21.15.1:0/3309975620 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:31:51.024 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.015+0000 7f4c81c00640 1 -- 172.21.15.1:0/3309975620 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d8abb09d0 msgr2=0x564d8abba840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:31:51.024 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.015+0000 7f4c81c00640 1 --2- 172.21.15.1:0/3309975620 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d8abb09d0 0x564d8abba840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:51.024 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.015+0000 7f4c81c00640 1 -- 172.21.15.1:0/3309975620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d8abb5a60 msgr2=0x564d8abc1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:31:51.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.015+0000 7f4c81c00640 1 --2- 172.21.15.1:0/3309975620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d8abb5a60 0x564d8abc1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:51.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.015+0000 7f4c81c00640 1 -- 172.21.15.1:0/3309975620 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d8a8f4620 con 0x564d8abb0090 2024-09-12T01:31:51.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.016+0000 7f4c81c00640 1 --2- 172.21.15.1:0/3309975620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d8abb0090 0x564d8abba300 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4c70005ab0 tx=0x7f4c70032f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2443c7762aa9b29b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:31:51.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.016+0000 7f4c63400640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-12T01:31:51.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.016+0000 7f4c63400640 1 -- 172.21.15.1:0/3309975620 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7004c020 con 0x564d8abb0090 2024-09-12T01:31:51.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.016+0000 7f4c63400640 1 -- 172.21.15.1:0/3309975620 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f4c700316b0 con 0x564d8abb0090 2024-09-12T01:31:51.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.016+0000 7f4c87616740 1 -- 172.21.15.1:0/3309975620 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c70004030 con 0x564d8abb0090 2024-09-12T01:31:51.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.017+0000 7f4c63400640 1 -- 172.21.15.1:0/3309975620 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4c70057050 con 0x564d8abb0090 2024-09-12T01:31:51.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.017+0000 7f4c87616740 10 client.25376 fetch_fsmap learned FSMap version 32 2024-09-12T01:31:51.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.017+0000 7f4c87616740 1 -- 172.21.15.1:0/3309975620 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564d8abc4670 con 0x564d8abb0090 2024-09-12T01:31:51.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.017+0000 7f4c63400640 1 -- 172.21.15.1:0/3309975620 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c70004030 con 0x564d8abb0090 2024-09-12T01:31:51.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.017+0000 7f4c63400640 1 -- 172.21.15.1:0/3309975620 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f4c70004780 con 0x564d8abb0090 2024-09-12T01:31:51.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.018+0000 7f4c87616740 10 client.25376 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T01:31:51.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.018+0000 7f4c87616740 10 client.25376 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:31:51.026 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.018+0000 7f4c87616740 20 client.25376 populate_metadata read hostname 'smithi001' 2024-09-12T01:31:51.026 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.018+0000 7f4c87616740 1 --2- 172.21.15.1:0/3309975620 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x564d8abc59c0 0x564d8abc7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:51.026 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.018+0000 7f4c87616740 4 client.25376 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:31:51.026 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.018+0000 7f4c87616740 1 -- 172.21.15.1:0/3309975620 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d8ab22830 con 0x564d8abc59c0 2024-09-12T01:31:51.026 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.018+0000 7f4c81200640 1 --2- 172.21.15.1:0/3309975620 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x564d8abc59c0 0x564d8abc7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:51.026 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.019+0000 7f4c81200640 1 --2- 172.21.15.1:0/3309975620 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x564d8abc59c0 0x564d8abc7e40 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8e8aeb624ad012e server_cookie=48645035c20be0d1 in_seq=0 out_seq=0 2024-09-12T01:31:51.026 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.019+0000 7f4c63400640 10 client.25376 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:31:51.026 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.020+0000 7f4c63400640 1 -- 172.21.15.1:0/3309975620 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564d8ab22830 con 0x564d8abc59c0 2024-09-12T01:31:51.026 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.020+0000 7f4c63400640 10 client.25376 handle_command_reply: tid=0 2024-09-12T01:31:51.026 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.020+0000 7f4c87616740 1 client.25376 shutdown 2024-09-12T01:31:51.026 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.020+0000 7f4c87616740 1 -- 172.21.15.1:0/3309975620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d8abb0090 msgr2=0x564d8abba300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:31:51.026 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.020+0000 7f4c87616740 1 --2- 172.21.15.1:0/3309975620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d8abb0090 0x564d8abba300 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4c70005ab0 tx=0x7f4c70032f60 comp rx=0 tx=0).stop 2024-09-12T01:31:51.026 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.020+0000 7f4c87616740 1 -- 172.21.15.1:0/3309975620 shutdown_connections 2024-09-12T01:31:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.020+0000 7f4c87616740 1 --2- 172.21.15.1:0/3309975620 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x564d8abc59c0 0x564d8abc7e40 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.020+0000 7f4c87616740 1 --2- 172.21.15.1:0/3309975620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d8abb5a60 0x564d8abc1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.020+0000 7f4c87616740 1 --2- 172.21.15.1:0/3309975620 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d8abb09d0 0x564d8abba840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.020+0000 7f4c63400640 0 client.25376 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:31:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.020+0000 7f4c87616740 1 --2- 172.21.15.1:0/3309975620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d8abb0090 0x564d8abba300 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.020+0000 7f4c87616740 1 -- 172.21.15.1:0/3309975620 >> 172.21.15.1:0/3309975620 conn(0x564d8abab700 msgr2=0x564d8abacf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:31:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.021+0000 7f4c87616740 1 -- 172.21.15.1:0/3309975620 shutdown_connections 2024-09-12T01:31:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.021+0000 7f4c87616740 1 -- 172.21.15.1:0/3309975620 wait complete. 2024-09-12T01:31:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.021+0000 7f4c87616740 20 client.25376 trim_cache size 0 max 16384 2024-09-12T01:31:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.043+0000 7f4c87616740 1 Processor -- start 2024-09-12T01:31:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.043+0000 7f4c87616740 1 -- start start 2024-09-12T01:31:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.043+0000 7f4c87616740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d8abb0090 0x564d8abba8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.043+0000 7f4c87616740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d8abb09d0 0x564d8abc1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.043+0000 7f4c87616740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d8abb5a60 0x564d8abaa770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.043+0000 7f4c87616740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c1ac960 con 0x564d8abb5a60 2024-09-12T01:31:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.043+0000 7f4c87616740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c1ae530 con 0x564d8abb0090 2024-09-12T01:31:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.043+0000 7f4c87616740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c1ad030 con 0x564d8abb09d0 2024-09-12T01:31:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.043+0000 7f4c81200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d8abb09d0 0x564d8abc1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.043+0000 7f4c82600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d8abb5a60 0x564d8abaa770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.043+0000 7f4c81200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d8abb09d0 0x564d8abc1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:37778/0 (socket says 172.21.15.1:37778) 2024-09-12T01:31:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.043+0000 7f4c81200640 1 -- 172.21.15.1:0/1542150344 learned_addr learned my addr 172.21.15.1:0/1542150344 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:31:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.043+0000 7f4c81c00640 1 --2- 172.21.15.1:0/1542150344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d8abb0090 0x564d8abba8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.044+0000 7f4c82600640 1 -- 172.21.15.1:0/1542150344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d8abb09d0 msgr2=0x564d8abc1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:31:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.044+0000 7f4c82600640 1 --2- 172.21.15.1:0/1542150344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d8abb09d0 0x564d8abc1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.044+0000 7f4c82600640 1 -- 172.21.15.1:0/1542150344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d8abb0090 msgr2=0x564d8abba8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:31:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.044+0000 7f4c82600640 1 --2- 172.21.15.1:0/1542150344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d8abb0090 0x564d8abba8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.044+0000 7f4c82600640 1 -- 172.21.15.1:0/1542150344 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d8abb8cd0 con 0x564d8abb5a60 2024-09-12T01:31:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.044+0000 7f4c81c00640 1 --2- 172.21.15.1:0/1542150344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d8abb0090 0x564d8abba8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:31:51.046 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.045+0000 7f4c82600640 1 --2- 172.21.15.1:0/1542150344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d8abb5a60 0x564d8abaa770 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4c74005fd0 tx=0x7f4c74009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1392c903c54b7ce3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:31:51.046 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.045+0000 7f4c6e000640 1 -- 172.21.15.1:0/1542150344 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7400fcb0 con 0x564d8abb5a60 2024-09-12T01:31:51.046 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.045+0000 7f4c6e000640 1 -- 172.21.15.1:0/1542150344 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c7400fe50 con 0x564d8abb5a60 2024-09-12T01:31:51.046 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.045+0000 7f4c6e000640 1 -- 172.21.15.1:0/1542150344 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c74005650 con 0x564d8abb5a60 2024-09-12T01:31:51.046 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.046+0000 7f4c87616740 1 -- 172.21.15.1:0/1542150344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d8abb5a60 msgr2=0x564d8abaa770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:31:51.046 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.046+0000 7f4c87616740 1 --2- 172.21.15.1:0/1542150344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d8abb5a60 0x564d8abaa770 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4c74005fd0 tx=0x7f4c74009500 comp rx=0 tx=0).stop 2024-09-12T01:31:51.046 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.046+0000 7f4c87616740 1 -- 172.21.15.1:0/1542150344 shutdown_connections 2024-09-12T01:31:51.046 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.046+0000 7f4c87616740 1 --2- 172.21.15.1:0/1542150344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d8abb5a60 0x564d8abaa770 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:51.046 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.046+0000 7f4c87616740 1 --2- 172.21.15.1:0/1542150344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d8abb09d0 0x564d8abc1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:51.047 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.046+0000 7f4c87616740 1 --2- 172.21.15.1:0/1542150344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d8abb0090 0x564d8abba8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:51.047 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.046+0000 7f4c87616740 1 -- 172.21.15.1:0/1542150344 >> 172.21.15.1:0/1542150344 conn(0x564d8abab700 msgr2=0x564d8abaa060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:31:51.047 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.047+0000 7f4c87616740 1 -- 172.21.15.1:0/1542150344 shutdown_connections 2024-09-12T01:31:51.047 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.047+0000 7f4c87616740 1 -- 172.21.15.1:0/1542150344 wait complete. 2024-09-12T01:31:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.047+0000 7f4c87616740 1 -- start start 2024-09-12T01:31:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.047+0000 7f4c87616740 1 Processor -- start 2024-09-12T01:31:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.048+0000 7f4c87616740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d8abb0090 0x564d8abbf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.048+0000 7f4c87616740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d8abb09d0 0x564d8abbf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.048+0000 7f4c87616740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d8abb5a60 0x564d8abc0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.048+0000 7f4c87616740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c1ade30 con 0x564d8abb0090 2024-09-12T01:31:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.048+0000 7f4c87616740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c1ad760 con 0x564d8abb09d0 2024-09-12T01:31:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.048+0000 7f4c87616740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c1aa940 con 0x564d8abb5a60 2024-09-12T01:31:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.048+0000 7f4c87616740 10 client.0 _ll_register_callbacks cb 0x7f4c70009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:31:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.048+0000 7f4c81c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d8abb0090 0x564d8abbf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.049+0000 7f4c81200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d8abb09d0 0x564d8abbf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.049+0000 7f4c81c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d8abb0090 0x564d8abbf2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48662/0 (socket says 172.21.15.1:48662) 2024-09-12T01:31:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.049+0000 7f4c81c00640 1 -- 172.21.15.1:0/37944738 learned_addr learned my addr 172.21.15.1:0/37944738 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:31:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.049+0000 7f4c82600640 1 --2- 172.21.15.1:0/37944738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d8abb5a60 0x564d8abc0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.049+0000 7f4c81200640 1 -- 172.21.15.1:0/37944738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d8abb5a60 msgr2=0x564d8abc0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:31:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.049+0000 7f4c81200640 1 --2- 172.21.15.1:0/37944738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d8abb5a60 0x564d8abc0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.049+0000 7f4c81200640 1 -- 172.21.15.1:0/37944738 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d8abb0090 msgr2=0x564d8abbf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:31:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.049+0000 7f4c81200640 1 --2- 172.21.15.1:0/37944738 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d8abb0090 0x564d8abbf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.049+0000 7f4c81200640 1 -- 172.21.15.1:0/37944738 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d8abc0e20 con 0x564d8abb09d0 2024-09-12T01:31:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.049+0000 7f4c82600640 1 --2- 172.21.15.1:0/37944738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d8abb5a60 0x564d8abc0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:31:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.049+0000 7f4c81c00640 1 --2- 172.21.15.1:0/37944738 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d8abb0090 0x564d8abbf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:31:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.049+0000 7f4c81200640 1 --2- 172.21.15.1:0/37944738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d8abb09d0 0x564d8abbf800 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4c68098030 tx=0x7f4c68018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce055166c6345ef1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:31:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.049+0000 7f4c63400640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-12T01:31:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.050+0000 7f4c63400640 1 -- 172.21.15.1:0/37944738 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c680b8070 con 0x564d8abb09d0 2024-09-12T01:31:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.050+0000 7f4c63400640 1 -- 172.21.15.1:0/37944738 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f4c680cf360 con 0x564d8abb09d0 2024-09-12T01:31:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.050+0000 7f4c87616740 1 -- 172.21.15.1:0/37944738 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c7400fe50 con 0x564d8abb09d0 2024-09-12T01:31:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.050+0000 7f4c63400640 1 -- 172.21.15.1:0/37944738 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4c6809de60 con 0x564d8abb09d0 2024-09-12T01:31:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.050+0000 7f4c87616740 10 client.25382 fetch_fsmap learned FSMap version 32 2024-09-12T01:31:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.050+0000 7f4c87616740 1 -- 172.21.15.1:0/37944738 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564d8ac393c0 con 0x564d8abb09d0 2024-09-12T01:31:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.050+0000 7f4c63400640 1 -- 172.21.15.1:0/37944738 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c7400fe50 con 0x564d8abb09d0 2024-09-12T01:31:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.050+0000 7f4c63400640 1 -- 172.21.15.1:0/37944738 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f4c680cf040 con 0x564d8abb09d0 2024-09-12T01:31:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.050+0000 7f4c87616740 10 client.25382 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T01:31:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.050+0000 7f4c87616740 10 client.25382 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:31:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.050+0000 7f4c87616740 20 client.25382 populate_metadata read hostname 'smithi001' 2024-09-12T01:31:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.050+0000 7f4c87616740 1 --2- 172.21.15.1:0/37944738 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x564d8ac39da0 0x564d8ac3c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:31:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.051+0000 7f4c87616740 4 client.25382 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:31:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.051+0000 7f4c87616740 1 -- 172.21.15.1:0/37944738 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564d8ab1cb80 con 0x564d8ac39da0 2024-09-12T01:31:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.051+0000 7f4c81c00640 1 --2- 172.21.15.1:0/37944738 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x564d8ac39da0 0x564d8ac3c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:31:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.051+0000 7f4c63400640 10 client.25382 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:31:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.051+0000 7f4c81c00640 1 --2- 172.21.15.1:0/37944738 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x564d8ac39da0 0x564d8ac3c1c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffcf6fdd9838c8b5 server_cookie=e14ce65afa1e9ee5 in_seq=0 out_seq=0 2024-09-12T01:31:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.061+0000 7f4c63400640 1 -- 172.21.15.1:0/37944738 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564d8ab1cb80 con 0x564d8ac39da0 2024-09-12T01:31:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.061+0000 7f4c63400640 10 client.25382 handle_command_reply: tid=0 2024-09-12T01:31:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.061+0000 7f4c87616740 1 client.25382 shutdown 2024-09-12T01:31:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.061+0000 7f4c87616740 1 -- 172.21.15.1:0/37944738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d8abb09d0 msgr2=0x564d8abbf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:31:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.061+0000 7f4c87616740 1 --2- 172.21.15.1:0/37944738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d8abb09d0 0x564d8abbf800 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4c68098030 tx=0x7f4c68018420 comp rx=0 tx=0).stop 2024-09-12T01:31:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.061+0000 7f4c87616740 1 -- 172.21.15.1:0/37944738 shutdown_connections 2024-09-12T01:31:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.061+0000 7f4c87616740 1 --2- 172.21.15.1:0/37944738 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x564d8ac39da0 0x564d8ac3c1c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.061+0000 7f4c87616740 1 --2- 172.21.15.1:0/37944738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d8abb5a60 0x564d8abc0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.061+0000 7f4c63400640 0 client.25382 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:31:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.061+0000 7f4c87616740 1 --2- 172.21.15.1:0/37944738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d8abb09d0 0x564d8abbf800 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.061+0000 7f4c87616740 1 --2- 172.21.15.1:0/37944738 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d8abb0090 0x564d8abbf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.061+0000 7f4c87616740 1 -- 172.21.15.1:0/37944738 >> 172.21.15.1:0/37944738 conn(0x564d8abab700 msgr2=0x564d8abaa880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:31:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.061+0000 7f4c87616740 1 -- 172.21.15.1:0/37944738 shutdown_connections 2024-09-12T01:31:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.061+0000 7f4c87616740 1 -- 172.21.15.1:0/37944738 wait complete. 2024-09-12T01:31:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.062+0000 7f4c87616740 20 client.25382 trim_cache size 0 max 16384 2024-09-12T01:31:51.063 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-12T01:31:51.063 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-12T01:31:51.063 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-12T01:31:51.063 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-12T01:31:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.062+0000 7f4c84600640 1 -- 172.21.15.1:0/4127526940 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f4c640a2c00 msgr2=0x7f4c640a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:31:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.062+0000 7f4c84600640 1 --2- 172.21.15.1:0/4127526940 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f4c640a2c00 0x7f4c640a50c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4c740099e0 tx=0x7f4c74013040 comp rx=0 tx=0).stop 2024-09-12T01:31:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.062+0000 7f4c84600640 1 -- 172.21.15.1:0/4127526940 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c7c104f50 msgr2=0x7f4c7c19a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:31:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.062+0000 7f4c84600640 1 --2- 172.21.15.1:0/4127526940 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c7c104f50 0x7f4c7c19a5c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4c6800e9f0 tx=0x7f4c6800eeb0 comp rx=0 tx=0).stop 2024-09-12T01:31:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.063+0000 7f4c84600640 1 -- 172.21.15.1:0/4127526940 shutdown_connections 2024-09-12T01:31:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.063+0000 7f4c84600640 1 --2- 172.21.15.1:0/4127526940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c7c105910 0x7f4c7c1a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.063+0000 7f4c84600640 1 --2- 172.21.15.1:0/4127526940 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4c7c104f50 0x7f4c7c19a5c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.063+0000 7f4c84600640 1 --2- 172.21.15.1:0/4127526940 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f4c640a2c00 0x7f4c640a50c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.063+0000 7f4c84600640 1 --2- 172.21.15.1:0/4127526940 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c7c103d60 0x7f4c7c19a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:31:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.063+0000 7f4c84600640 1 -- 172.21.15.1:0/4127526940 >> 172.21.15.1:0/4127526940 conn(0x7f4c7c0ff4e0 msgr2=0x7f4c7c1083c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:31:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.063+0000 7f4c84600640 1 -- 172.21.15.1:0/4127526940 shutdown_connections 2024-09-12T01:31:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:31:51.063+0000 7f4c84600640 1 -- 172.21.15.1:0/4127526940 wait complete. 2024-09-12T01:31:51.420 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] growfiles 2024-09-12T01:31:51.511 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] iogen 2024-09-12T01:31:52.068 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-stress 2024-09-12T01:31:52.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:31:52 smithi104 ceph-mon[35219]: pgmap v466: 129 pgs: 129 active+clean; 65 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 1.4 MiB/s wr, 228 op/s 2024-09-12T01:31:52.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:31:52 smithi001 ceph-mon[30387]: pgmap v466: 129 pgs: 129 active+clean; 65 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 1.4 MiB/s wr, 228 op/s 2024-09-12T01:31:52.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:31:52 smithi053 ceph-mon[35158]: pgmap v466: 129 pgs: 129 active+clean; 65 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 1.4 MiB/s wr, 228 op/s 2024-09-12T01:31:53.744 INFO:tasks.workunit.client.0.smithi001.stdout: [DEP] 2024-09-12T01:31:53.744 INFO:tasks.workunit.client.0.smithi001.stdout: [DEP] 2024-09-12T01:31:53.744 INFO:tasks.workunit.client.0.smithi001.stdout: [DEP] 2024-09-12T01:31:53.744 INFO:tasks.workunit.client.0.smithi001.stdout: [DEP] 2024-09-12T01:31:54.158 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] dirstress 2024-09-12T01:31:54.158 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] fill 2024-09-12T01:31:54.158 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] fill2 2024-09-12T01:31:54.159 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] getpagesize 2024-09-12T01:31:54.420 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] holes 2024-09-12T01:31:54.437 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] lstat64 2024-09-12T01:31:54.538 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] nametest 2024-09-12T01:31:54.541 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] permname 2024-09-12T01:31:54.756 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] randholes 2024-09-12T01:31:54.770 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] runas 2024-09-12T01:31:54.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:31:54 smithi104 ceph-mon[35219]: pgmap v467: 129 pgs: 129 active+clean; 65 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-09-12T01:31:54.872 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] truncfile 2024-09-12T01:31:54.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:31:54 smithi001 ceph-mon[30387]: pgmap v467: 129 pgs: 129 active+clean; 65 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-09-12T01:31:54.970 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] usemem 2024-09-12T01:31:54.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:31:54 smithi053 ceph-mon[35158]: pgmap v467: 129 pgs: 129 active+clean; 65 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-09-12T01:31:55.091 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] mmapcat 2024-09-12T01:31:55.185 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] append_reader 2024-09-12T01:31:55.204 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] append_writer 2024-09-12T01:31:55.244 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] dirperf 2024-09-12T01:31:55.372 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] metaperf 2024-09-12T01:31:55.508 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] devzero 2024-09-12T01:31:55.514 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] feature 2024-09-12T01:31:55.687 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] alloc 2024-09-12T01:31:55.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:31:55 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:31:55.855 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] fault 2024-09-12T01:31:55.872 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] fstest 2024-09-12T01:31:55.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:31:55 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:31:55.950 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_access_root 2024-09-12T01:31:55.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:31:55 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:31:56.125 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] godown 2024-09-12T01:31:56.189 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] resvtest 2024-09-12T01:31:56.289 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] writemod 2024-09-12T01:31:56.329 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] writev_on_pagefault 2024-09-12T01:31:56.478 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] makeextents 2024-09-12T01:31:56.598 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] itrash 2024-09-12T01:31:56.620 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] rename 2024-09-12T01:31:56.679 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] multi_open_unlink 2024-09-12T01:31:56.809 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] unwritten_sync 2024-09-12T01:31:56.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:31:56 smithi104 ceph-mon[35219]: pgmap v468: 129 pgs: 129 active+clean; 65 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 871 KiB/s wr, 102 op/s 2024-09-12T01:31:56.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:31:56 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:31:56.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:31:56 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:31:56.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:31:56 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:31:56.876 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] genhashnames 2024-09-12T01:31:56.887 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_holes 2024-09-12T01:31:56.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:31:56 smithi001 ceph-mon[30387]: pgmap v468: 129 pgs: 129 active+clean; 65 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 871 KiB/s wr, 102 op/s 2024-09-12T01:31:56.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:31:56 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:31:56.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:31:56 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:31:56.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:31:56 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:31:56.966 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_mmap_writev 2024-09-12T01:31:56.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:31:56 smithi053 ceph-mon[35158]: pgmap v468: 129 pgs: 129 active+clean; 65 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 871 KiB/s wr, 102 op/s 2024-09-12T01:31:56.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:31:56 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:31:56.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:31:56 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:31:56.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:31:56 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:31:57.243 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_truncate_cmtime 2024-09-12T01:31:57.243 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] dirhash_collide 2024-09-12T01:31:57.244 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_rename_overwrite 2024-09-12T01:31:57.244 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] holetest 2024-09-12T01:31:57.514 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_truncate_self 2024-09-12T01:31:57.518 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] af_unix 2024-09-12T01:31:57.571 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_mmap_stale_pmd 2024-09-12T01:31:57.624 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_mmap_cow_race 2024-09-12T01:31:57.782 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_mmap_fallocate 2024-09-12T01:31:57.809 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] fsync-err 2024-09-12T01:31:57.855 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_mmap_write_ro 2024-09-12T01:31:57.933 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_ext4_dax_journal_corruption 2024-09-12T01:31:58.056 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_ext4_dax_inline_corruption 2024-09-12T01:31:58.129 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_ofd_locks 2024-09-12T01:31:58.153 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_mmap_collision 2024-09-12T01:31:58.223 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] mmap-write-concurrent 2024-09-12T01:31:58.351 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_get_file_time 2024-09-12T01:31:58.459 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_create_short_dirs 2024-09-12T01:31:58.503 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_create_long_dirs 2024-09-12T01:31:58.532 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_enospc 2024-09-12T01:31:58.658 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_mmap_writev_overlap 2024-09-12T01:31:58.755 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] checkpoint_journal 2024-09-12T01:31:58.805 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] mmap-rw-fault 2024-09-12T01:31:58.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:31:58 smithi104 ceph-mon[35219]: pgmap v469: 129 pgs: 129 active+clean; 65 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 449 KiB/s wr, 75 op/s 2024-09-12T01:31:58.861 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] allocstale 2024-09-12T01:31:58.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:31:58 smithi001 ceph-mon[30387]: pgmap v469: 129 pgs: 129 active+clean; 65 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 449 KiB/s wr, 75 op/s 2024-09-12T01:31:58.973 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_mmap_cow_memory_failure 2024-09-12T01:31:58.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:31:58 smithi053 ceph-mon[35158]: pgmap v469: 129 pgs: 129 active+clean; 65 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 449 KiB/s wr, 75 op/s 2024-09-12T01:31:59.063 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] fake-dump-rootino 2024-09-12T01:31:59.130 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] dio-buf-fault 2024-09-12T01:31:59.183 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] rewinddir-test 2024-09-12T01:31:59.285 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] readdir-while-renames 2024-09-12T01:31:59.351 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] xfsctl 2024-09-12T01:31:59.423 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] bstat 2024-09-12T01:31:59.497 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_mtab 2024-09-12T01:31:59.590 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] getdevicesize 2024-09-12T01:31:59.697 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] preallo_rw_pattern_reader 2024-09-12T01:31:59.779 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] preallo_rw_pattern_writer 2024-09-12T01:31:59.840 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] ftrunc 2024-09-12T01:31:59.882 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] trunc 2024-09-12T01:31:59.997 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] fs_perms 2024-09-12T01:32:00.076 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] testx 2024-09-12T01:32:00.125 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] looptest 2024-09-12T01:32:00.184 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] locktest 2024-09-12T01:32:00.306 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] unwritten_mmap 2024-09-12T01:32:00.336 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] bulkstat_unlink_test 2024-09-12T01:32:00.459 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] deduperace 2024-09-12T01:32:00.613 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] bulkstat_unlink_test_modified 2024-09-12T01:32:00.661 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_dir_offset 2024-09-12T01:32:00.676 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_futimens 2024-09-12T01:32:00.815 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_immutable 2024-09-12T01:32:00.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:32:00 smithi104 ceph-mon[35219]: pgmap v470: 129 pgs: 129 active+clean; 65 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 256 KiB/s wr, 36 op/s 2024-09-12T01:32:00.919 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] stale_handle 2024-09-12T01:32:00.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:32:00 smithi001 ceph-mon[30387]: pgmap v470: 129 pgs: 129 active+clean; 65 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 256 KiB/s wr, 36 op/s 2024-09-12T01:32:00.939 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] pwrite_mmap_blocked 2024-09-12T01:32:00.978 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_dir_offset2 2024-09-12T01:32:00.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:32:00 smithi053 ceph-mon[35158]: pgmap v470: 129 pgs: 129 active+clean; 65 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 256 KiB/s wr, 36 op/s 2024-09-12T01:32:01.205 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] seek_sanity_test 2024-09-12T01:32:01.232 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] seek_copy_test 2024-09-12T01:32:01.324 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_readdir_1 2024-09-12T01:32:01.539 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_readdir_2 2024-09-12T01:32:01.596 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] fsync-tester 2024-09-12T01:32:01.779 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] nsexec 2024-09-12T01:32:01.824 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] cloner 2024-09-12T01:32:01.871 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] renameat2 2024-09-12T01:32:02.009 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_getcwd 2024-09-12T01:32:02.096 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] e4compact 2024-09-12T01:32:02.132 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] test-nextquota 2024-09-12T01:32:02.185 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] punch-alternating 2024-09-12T01:32:02.327 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] attr-list-by-handle-cursor-test 2024-09-12T01:32:02.434 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] listxattr 2024-09-12T01:32:02.464 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] dio-interleaved 2024-09-12T01:32:02.483 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_dir_type 2024-09-12T01:32:02.644 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] dio-invalidate-cache 2024-09-12T01:32:02.748 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] stat_test 2024-09-12T01:32:02.748 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_encrypted_d_revalidate 2024-09-12T01:32:02.785 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] attr_replace_test 2024-09-12T01:32:02.791 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:32:02 smithi053 ceph-mon[35158]: pgmap v471: 129 pgs: 129 active+clean; 65 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 141 KiB/s wr, 32 op/s 2024-09-12T01:32:02.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:32:02 smithi104 ceph-mon[35219]: pgmap v471: 129 pgs: 129 active+clean; 65 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 141 KiB/s wr, 32 op/s 2024-09-12T01:32:02.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:32:02 smithi001 ceph-mon[30387]: pgmap v471: 129 pgs: 129 active+clean; 65 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 141 KiB/s wr, 32 op/s 2024-09-12T01:32:02.999 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] swapon 2024-09-12T01:32:03.037 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] mkswap 2024-09-12T01:32:03.086 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_attr_corruption 2024-09-12T01:32:03.238 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_open_tmpfiles 2024-09-12T01:32:03.308 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] fscrypt-crypt-util 2024-09-12T01:32:03.329 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] bulkstat_null_ocount 2024-09-12T01:32:03.358 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] splice-test 2024-09-12T01:32:03.583 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] chprojid_fail 2024-09-12T01:32:03.617 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] detached_mounts_propagation 2024-09-12T01:32:03.679 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] ext4_resize 2024-09-12T01:32:03.903 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_readdir_3 2024-09-12T01:32:03.920 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] splice2pipe 2024-09-12T01:32:03.954 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] uuid_ioctl 2024-09-12T01:32:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.273+0000 7f5f47e00640 1 -- 172.21.15.1:0/1423256942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f48100210 msgr2=0x7f5f48102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.273+0000 7f5f47e00640 1 --2- 172.21.15.1:0/1423256942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f48100210 0x7f5f48102620 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5f38009900 tx=0x7f5f3802a420 comp rx=0 tx=0).stop 2024-09-12T01:32:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.275+0000 7f5f47e00640 1 -- 172.21.15.1:0/1423256942 shutdown_connections 2024-09-12T01:32:04.275 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.275+0000 7f5f47e00640 1 --2- 172.21.15.1:0/1423256942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f48105510 0x7f5f481059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:04.275 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.275+0000 7f5f47e00640 1 --2- 172.21.15.1:0/1423256942 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f48102b60 0x7f5f48104fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:04.275 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.275+0000 7f5f47e00640 1 --2- 172.21.15.1:0/1423256942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f48100210 0x7f5f48102620 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:04.275 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.275+0000 7f5f47e00640 1 -- 172.21.15.1:0/1423256942 >> 172.21.15.1:0/1423256942 conn(0x7f5f480fc040 msgr2=0x7f5f480fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:04.279 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.277+0000 7f5f47e00640 1 -- 172.21.15.1:0/1423256942 shutdown_connections 2024-09-12T01:32:04.279 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.278+0000 7f5f47e00640 1 -- 172.21.15.1:0/1423256942 wait complete. 2024-09-12T01:32:04.279 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.279+0000 7f5f47e00640 1 Processor -- start 2024-09-12T01:32:04.279 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.279+0000 7f5f47e00640 1 -- start start 2024-09-12T01:32:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.280+0000 7f5f47e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f48102b60 0x7f5f4819e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.280+0000 7f5f47e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f48105510 0x7f5f4819eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.280+0000 7f5f47e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f48106830 0x7f5f48108cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.281+0000 7f5f47e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5f48113d60 con 0x7f5f48105510 2024-09-12T01:32:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.281+0000 7f5f47e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5f48113be0 con 0x7f5f48102b60 2024-09-12T01:32:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.281+0000 7f5f47e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5f48113ee0 con 0x7f5f48106830 2024-09-12T01:32:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.281+0000 7f5f47400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f48106830 0x7f5f48108cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.281+0000 7f5f47400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f48106830 0x7f5f48108cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:37746/0 (socket says 172.21.15.1:37746) 2024-09-12T01:32:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.281+0000 7f5f47400640 1 -- 172.21.15.1:0/110590018 learned_addr learned my addr 172.21.15.1:0/110590018 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:32:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.281+0000 7f5f46a00640 1 --2- 172.21.15.1:0/110590018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f48102b60 0x7f5f4819e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.281+0000 7f5f46000640 1 --2- 172.21.15.1:0/110590018 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f48105510 0x7f5f4819eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.282+0000 7f5f47400640 1 -- 172.21.15.1:0/110590018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f48102b60 msgr2=0x7f5f4819e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.282+0000 7f5f47400640 1 --2- 172.21.15.1:0/110590018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f48102b60 0x7f5f4819e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.282+0000 7f5f47400640 1 -- 172.21.15.1:0/110590018 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f48105510 msgr2=0x7f5f4819eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.282+0000 7f5f47400640 1 --2- 172.21.15.1:0/110590018 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f48105510 0x7f5f4819eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.282+0000 7f5f47400640 1 -- 172.21.15.1:0/110590018 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f481092b0 con 0x7f5f48106830 2024-09-12T01:32:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.282+0000 7f5f46a00640 1 --2- 172.21.15.1:0/110590018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f48102b60 0x7f5f4819e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:32:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.284+0000 7f5f47400640 1 --2- 172.21.15.1:0/110590018 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f48106830 0x7f5f48108cf0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5f4000b970 tx=0x7f5f4000be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.286+0000 7f5f3f400640 1 -- 172.21.15.1:0/110590018 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f4002a070 con 0x7f5f48106830 2024-09-12T01:32:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.286+0000 7f5f3f400640 1 -- 172.21.15.1:0/110590018 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f4001db50 con 0x7f5f48106830 2024-09-12T01:32:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.286+0000 7f5f3f400640 1 -- 172.21.15.1:0/110590018 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f400076e0 con 0x7f5f48106830 2024-09-12T01:32:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.286+0000 7f5f47e00640 1 -- 172.21.15.1:0/110590018 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f481095a0 con 0x7f5f48106830 2024-09-12T01:32:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.286+0000 7f5f47e00640 1 -- 172.21.15.1:0/110590018 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f48109b20 con 0x7f5f48106830 2024-09-12T01:32:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.298+0000 7f5f3f400640 1 -- 172.21.15.1:0/110590018 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f5f4002d070 con 0x7f5f48106830 2024-09-12T01:32:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.303+0000 7f5f3f400640 1 --2- 172.21.15.1:0/110590018 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f5f2c0a2d20 0x7f5f2c0a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.303+0000 7f5f3f400640 1 -- 172.21.15.1:0/110590018 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f5f400dfa40 con 0x7f5f48106830 2024-09-12T01:32:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.305+0000 7f5f46a00640 1 --2- 172.21.15.1:0/110590018 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f5f2c0a2d20 0x7f5f2c0a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.305+0000 7f5f508c7740 1 Processor -- start 2024-09-12T01:32:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.306+0000 7f5f508c7740 1 -- start start 2024-09-12T01:32:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.306+0000 7f5f508c7740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c782e4260 0x557c782e4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.306+0000 7f5f508c7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557c782e4ba0 0x557c782e9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.306+0000 7f5f508c7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557c782e9c30 0x557c782ec0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.306+0000 7f5f508c7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x557c78249100 con 0x557c782e4ba0 2024-09-12T01:32:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.306+0000 7f5f508c7740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x557c781f11b0 con 0x557c782e4260 2024-09-12T01:32:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.306+0000 7f5f508c7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557c782450c0 con 0x557c782e9c30 2024-09-12T01:32:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.306+0000 7f5f46a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c782e4260 0x557c782e4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.306+0000 7f5f46a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c782e4260 0x557c782e4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:42088/0 (socket says 172.21.15.1:42088) 2024-09-12T01:32:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.306+0000 7f5f46a00640 1 -- 172.21.15.1:0/4059156827 learned_addr learned my addr 172.21.15.1:0/4059156827 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:32:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.306+0000 7f5f47400640 1 --2- 172.21.15.1:0/4059156827 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557c782e9c30 0x557c782ec0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.307+0000 7f5f46000640 1 --2- 172.21.15.1:0/4059156827 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557c782e4ba0 0x557c782e9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.307+0000 7f5f46a00640 1 --2- 172.21.15.1:0/110590018 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f5f2c0a2d20 0x7f5f2c0a51e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5f38009230 tx=0x7f5f38031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.307+0000 7f5f46a00640 1 -- 172.21.15.1:0/4059156827 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557c782e9c30 msgr2=0x557c782ec0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.307+0000 7f5f46a00640 1 --2- 172.21.15.1:0/4059156827 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557c782e9c30 0x557c782ec0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.307+0000 7f5f46a00640 1 -- 172.21.15.1:0/4059156827 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557c782e4ba0 msgr2=0x557c782e9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.307+0000 7f5f46a00640 1 --2- 172.21.15.1:0/4059156827 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557c782e4ba0 0x557c782e9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.307+0000 7f5f46a00640 1 -- 172.21.15.1:0/4059156827 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c77f5c210 con 0x557c782e4260 2024-09-12T01:32:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.307+0000 7f5f46000640 1 --2- 172.21.15.1:0/4059156827 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557c782e4ba0 0x557c782e9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:32:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.308+0000 7f5f46a00640 1 --2- 172.21.15.1:0/4059156827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c782e4260 0x557c782e4660 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5f38031610 tx=0x7f5f3803c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44477ee2d299e19b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.310+0000 7f5f33e00640 1 -- 172.21.15.1:0/4059156827 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f38035050 con 0x557c782e4260 2024-09-12T01:32:04.310 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.310+0000 7f5f33e00640 1 -- 172.21.15.1:0/4059156827 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f3804d070 con 0x557c782e4260 2024-09-12T01:32:04.310 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.310+0000 7f5f33e00640 1 -- 172.21.15.1:0/4059156827 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f38048990 con 0x557c782e4260 2024-09-12T01:32:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.311+0000 7f5f508c7740 1 -- 172.21.15.1:0/4059156827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c782e4260 msgr2=0x557c782e4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.311+0000 7f5f508c7740 1 --2- 172.21.15.1:0/4059156827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c782e4260 0x557c782e4660 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5f38031610 tx=0x7f5f3803c000 comp rx=0 tx=0).stop 2024-09-12T01:32:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.311+0000 7f5f508c7740 1 -- 172.21.15.1:0/4059156827 shutdown_connections 2024-09-12T01:32:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.311+0000 7f5f508c7740 1 --2- 172.21.15.1:0/4059156827 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557c782e9c30 0x557c782ec0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.312+0000 7f5f508c7740 1 --2- 172.21.15.1:0/4059156827 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557c782e4ba0 0x557c782e9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.312+0000 7f5f508c7740 1 --2- 172.21.15.1:0/4059156827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c782e4260 0x557c782e4660 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.312+0000 7f5f508c7740 1 -- 172.21.15.1:0/4059156827 >> 172.21.15.1:0/4059156827 conn(0x557c782df8d0 msgr2=0x557c782dfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.312+0000 7f5f508c7740 1 -- 172.21.15.1:0/4059156827 shutdown_connections 2024-09-12T01:32:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.312+0000 7f5f508c7740 1 -- 172.21.15.1:0/4059156827 wait complete. 2024-09-12T01:32:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.313+0000 7f5f508c7740 1 -- start start 2024-09-12T01:32:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.315+0000 7f5f508c7740 1 Processor -- start 2024-09-12T01:32:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.316+0000 7f5f508c7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557c782e4260 0x557c782ee470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.316+0000 7f5f508c7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557c782e4ba0 0x557c782ee9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.316+0000 7f5f508c7740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c782e9c30 0x557c782f59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.316+0000 7f5f508c7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x557c782ecba0 con 0x557c782e4ba0 2024-09-12T01:32:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.316+0000 7f5f508c7740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x557c781d2090 con 0x557c782e9c30 2024-09-12T01:32:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.316+0000 7f5f508c7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557c782ecd20 con 0x557c782e4260 2024-09-12T01:32:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.316+0000 7f5f508c7740 10 client.0 _ll_register_callbacks cb 0x557c78226f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:32:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.317+0000 7f5f46a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557c782e4260 0x557c782ee470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.317+0000 7f5f46000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557c782e4ba0 0x557c782ee9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.318+0000 7f5f46a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557c782e4260 0x557c782ee470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:37778/0 (socket says 172.21.15.1:37778) 2024-09-12T01:32:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.318+0000 7f5f46a00640 1 -- 172.21.15.1:0/3900402607 learned_addr learned my addr 172.21.15.1:0/3900402607 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:32:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.320+0000 7f5f46000640 1 -- 172.21.15.1:0/3900402607 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557c782e4260 msgr2=0x557c782ee470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.320+0000 7f5f46000640 1 --2- 172.21.15.1:0/3900402607 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557c782e4260 0x557c782ee470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.320+0000 7f5f46000640 1 -- 172.21.15.1:0/3900402607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c782e9c30 msgr2=0x557c782f59f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T01:32:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.321+0000 7f5f46000640 1 --2- 172.21.15.1:0/3900402607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c782e9c30 0x557c782f59f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.321+0000 7f5f46000640 1 -- 172.21.15.1:0/3900402607 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c78028620 con 0x557c782e4ba0 2024-09-12T01:32:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.321+0000 7f5f46a00640 1 --2- 172.21.15.1:0/3900402607 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557c782e4260 0x557c782ee470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:32:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.323+0000 7f5f46000640 1 --2- 172.21.15.1:0/3900402607 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557c782e4ba0 0x557c782ee9b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x557c782e55d0 tx=0x7f5f3400e660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4ca2a671626b397 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.324+0000 7f5f30c00640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-12T01:32:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.325+0000 7f5f30c00640 1 -- 172.21.15.1:0/3900402607 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f34018020 con 0x557c782e4ba0 2024-09-12T01:32:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.325+0000 7f5f508c7740 1 -- 172.21.15.1:0/3900402607 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f3804d070 con 0x557c782e4ba0 2024-09-12T01:32:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.327+0000 7f5f30c00640 1 -- 172.21.15.1:0/3900402607 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f5f3400f040 con 0x557c782e4ba0 2024-09-12T01:32:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.327+0000 7f5f30c00640 1 -- 172.21.15.1:0/3900402607 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5f34023050 con 0x557c782e4ba0 2024-09-12T01:32:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.327+0000 7f5f508c7740 10 client.15360 fetch_fsmap learned FSMap version 32 2024-09-12T01:32:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.328+0000 7f5f508c7740 1 -- 172.21.15.1:0/3900402607 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557c782f8930 con 0x557c782e4ba0 2024-09-12T01:32:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.329+0000 7f5f30c00640 1 -- 172.21.15.1:0/3900402607 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f3804d070 con 0x557c782e4ba0 2024-09-12T01:32:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.329+0000 7f5f30c00640 1 -- 172.21.15.1:0/3900402607 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f5f34014810 con 0x557c782e4ba0 2024-09-12T01:32:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.331+0000 7f5f508c7740 10 client.15360 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T01:32:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.331+0000 7f5f508c7740 10 client.15360 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:32:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.331+0000 7f5f508c7740 20 client.15360 populate_metadata read hostname 'smithi001' 2024-09-12T01:32:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.331+0000 7f5f508c7740 1 --2- 172.21.15.1:0/3900402607 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x557c782f9fd0 0x557c782fc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.331+0000 7f5f508c7740 4 client.15360 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:32:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.331+0000 7f5f508c7740 1 -- 172.21.15.1:0/3900402607 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c78256830 con 0x557c782f9fd0 2024-09-12T01:32:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.332+0000 7f5f46a00640 1 --2- 172.21.15.1:0/3900402607 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x557c782f9fd0 0x557c782fc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.333+0000 7f5f46a00640 1 --2- 172.21.15.1:0/3900402607 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x557c782f9fd0 0x557c782fc440 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c038034e39e53434 server_cookie=b223f98f671d55e6 in_seq=0 out_seq=0 2024-09-12T01:32:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.333+0000 7f5f30c00640 10 client.15360 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:32:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.334+0000 7f5f30c00640 1 -- 172.21.15.1:0/3900402607 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557c78256830 con 0x557c782f9fd0 2024-09-12T01:32:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.334+0000 7f5f30c00640 10 client.15360 handle_command_reply: tid=0 2024-09-12T01:32:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.334+0000 7f5f508c7740 1 client.15360 shutdown 2024-09-12T01:32:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.334+0000 7f5f508c7740 1 -- 172.21.15.1:0/3900402607 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557c782e4ba0 msgr2=0x557c782ee9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.334+0000 7f5f508c7740 1 --2- 172.21.15.1:0/3900402607 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557c782e4ba0 0x557c782ee9b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x557c782e55d0 tx=0x7f5f3400e660 comp rx=0 tx=0).stop 2024-09-12T01:32:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.335+0000 7f5f508c7740 1 -- 172.21.15.1:0/3900402607 shutdown_connections 2024-09-12T01:32:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.335+0000 7f5f508c7740 1 --2- 172.21.15.1:0/3900402607 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x557c782f9fd0 0x557c782fc440 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:04.336 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.335+0000 7f5f508c7740 1 --2- 172.21.15.1:0/3900402607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c782e9c30 0x557c782f59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:04.336 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.335+0000 7f5f508c7740 1 --2- 172.21.15.1:0/3900402607 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557c782e4ba0 0x557c782ee9b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:04.336 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.335+0000 7f5f508c7740 1 --2- 172.21.15.1:0/3900402607 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557c782e4260 0x557c782ee470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:04.336 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.335+0000 7f5f508c7740 1 -- 172.21.15.1:0/3900402607 >> 172.21.15.1:0/3900402607 conn(0x557c782df8d0 msgr2=0x557c782e10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:04.336 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.335+0000 7f5f30c00640 0 client.15360 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:32:04.337 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.336+0000 7f5f508c7740 1 -- 172.21.15.1:0/3900402607 shutdown_connections 2024-09-12T01:32:04.337 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.336+0000 7f5f508c7740 1 -- 172.21.15.1:0/3900402607 wait complete. 2024-09-12T01:32:04.337 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.336+0000 7f5f508c7740 20 client.15360 trim_cache size 0 max 16384 2024-09-12T01:32:04.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.379+0000 7f5f508c7740 1 Processor -- start 2024-09-12T01:32:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.379+0000 7f5f508c7740 1 -- start start 2024-09-12T01:32:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.380+0000 7f5f508c7740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c782e4260 0x557c782da8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.380+0000 7f5f508c7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557c782e4ba0 0x557c782f59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.380+0000 7f5f508c7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557c782e9c30 0x557c782fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.380+0000 7f5f508c7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2800fc00 con 0x557c782e4ba0 2024-09-12T01:32:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.380+0000 7f5f508c7740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5f280117d0 con 0x557c782e4260 2024-09-12T01:32:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.380+0000 7f5f508c7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5f280102d0 con 0x557c782e9c30 2024-09-12T01:32:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.380+0000 7f5f46a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c782e4260 0x557c782da8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.380+0000 7f5f46a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c782e4260 0x557c782da8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:42104/0 (socket says 172.21.15.1:42104) 2024-09-12T01:32:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.380+0000 7f5f46a00640 1 -- 172.21.15.1:0/3189697616 learned_addr learned my addr 172.21.15.1:0/3189697616 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:32:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.380+0000 7f5f46000640 1 --2- 172.21.15.1:0/3189697616 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557c782e4ba0 0x557c782f59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.380+0000 7f5f47400640 1 --2- 172.21.15.1:0/3189697616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557c782e9c30 0x557c782fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.380+0000 7f5f46a00640 1 -- 172.21.15.1:0/3189697616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557c782e9c30 msgr2=0x557c782fbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.380+0000 7f5f46a00640 1 --2- 172.21.15.1:0/3189697616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557c782e9c30 0x557c782fbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.380+0000 7f5f46a00640 1 -- 172.21.15.1:0/3189697616 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557c782e4ba0 msgr2=0x557c782f59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.380+0000 7f5f46a00640 1 --2- 172.21.15.1:0/3189697616 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557c782e4ba0 0x557c782f59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.380+0000 7f5f46a00640 1 -- 172.21.15.1:0/3189697616 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c782e90c0 con 0x557c782e4260 2024-09-12T01:32:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.380+0000 7f5f47400640 1 --2- 172.21.15.1:0/3189697616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557c782e9c30 0x557c782fbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:32:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.381+0000 7f5f46a00640 1 --2- 172.21.15.1:0/3189697616 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c782e4260 0x557c782da8d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5f38031610 tx=0x7f5f38009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6dcccd918aab54 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.381+0000 7f5f33400640 1 -- 172.21.15.1:0/3189697616 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3803f040 con 0x557c782e4260 2024-09-12T01:32:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.381+0000 7f5f33400640 1 -- 172.21.15.1:0/3189697616 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f3804d050 con 0x557c782e4260 2024-09-12T01:32:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.381+0000 7f5f33400640 1 -- 172.21.15.1:0/3189697616 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f38035040 con 0x557c782e4260 2024-09-12T01:32:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.382+0000 7f5f508c7740 1 -- 172.21.15.1:0/3189697616 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c782e4260 msgr2=0x557c782da8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.383+0000 7f5f508c7740 1 --2- 172.21.15.1:0/3189697616 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c782e4260 0x557c782da8d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5f38031610 tx=0x7f5f38009680 comp rx=0 tx=0).stop 2024-09-12T01:32:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.383+0000 7f5f508c7740 1 -- 172.21.15.1:0/3189697616 shutdown_connections 2024-09-12T01:32:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.383+0000 7f5f508c7740 1 --2- 172.21.15.1:0/3189697616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557c782e9c30 0x557c782fbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.383+0000 7f5f508c7740 1 --2- 172.21.15.1:0/3189697616 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557c782e4ba0 0x557c782f59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.383+0000 7f5f508c7740 1 --2- 172.21.15.1:0/3189697616 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c782e4260 0x557c782da8d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.383+0000 7f5f508c7740 1 -- 172.21.15.1:0/3189697616 >> 172.21.15.1:0/3189697616 conn(0x557c782df8d0 msgr2=0x557c782ee470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.383+0000 7f5f508c7740 1 -- 172.21.15.1:0/3189697616 shutdown_connections 2024-09-12T01:32:04.384 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.383+0000 7f5f508c7740 1 -- 172.21.15.1:0/3189697616 wait complete. 2024-09-12T01:32:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.385+0000 7f5f508c7740 1 -- start start 2024-09-12T01:32:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.386+0000 7f5f508c7740 1 Processor -- start 2024-09-12T01:32:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.387+0000 7f5f508c7740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c782e4260 0x557c782f2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.387+0000 7f5f508c7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557c782e4ba0 0x557c782f2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.387+0000 7f5f508c7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557c782e9c30 0x557c78362d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.387+0000 7f5f508c7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5f280110d0 con 0x557c782e4ba0 2024-09-12T01:32:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.387+0000 7f5f508c7740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5f28010a00 con 0x557c782e4260 2024-09-12T01:32:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.387+0000 7f5f508c7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2800dbe0 con 0x557c782e9c30 2024-09-12T01:32:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.387+0000 7f5f508c7740 10 client.0 _ll_register_callbacks cb 0x7f5f34006db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:32:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.387+0000 7f5f46a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c782e4260 0x557c782f2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.387+0000 7f5f46a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c782e4260 0x557c782f2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:42118/0 (socket says 172.21.15.1:42118) 2024-09-12T01:32:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.387+0000 7f5f46a00640 1 -- 172.21.15.1:0/1187928088 learned_addr learned my addr 172.21.15.1:0/1187928088 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:32:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.388+0000 7f5f46000640 1 --2- 172.21.15.1:0/1187928088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557c782e4ba0 0x557c782f2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.388+0000 7f5f47400640 1 --2- 172.21.15.1:0/1187928088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557c782e9c30 0x557c78362d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.388+0000 7f5f47400640 1 -- 172.21.15.1:0/1187928088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c782e4260 msgr2=0x557c782f2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.388+0000 7f5f47400640 1 --2- 172.21.15.1:0/1187928088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c782e4260 0x557c782f2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.388+0000 7f5f47400640 1 -- 172.21.15.1:0/1187928088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557c782e4ba0 msgr2=0x557c782f2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.388+0000 7f5f47400640 1 --2- 172.21.15.1:0/1187928088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557c782e4ba0 0x557c782f2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.388+0000 7f5f47400640 1 -- 172.21.15.1:0/1187928088 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c78363250 con 0x557c782e9c30 2024-09-12T01:32:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.388+0000 7f5f46000640 1 --2- 172.21.15.1:0/1187928088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557c782e4ba0 0x557c782f2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:32:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.389+0000 7f5f47400640 1 --2- 172.21.15.1:0/1187928088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557c782e9c30 0x557c78362d10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5f4000b940 tx=0x7f5f40004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9b3189db36177e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.389+0000 7f5f30c00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-12T01:32:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.389+0000 7f5f30c00640 1 -- 172.21.15.1:0/1187928088 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f4001dd00 con 0x557c782e9c30 2024-09-12T01:32:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.390+0000 7f5f30c00640 1 -- 172.21.15.1:0/1187928088 <== mon.1 v2:172.21.15.104:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f5f400b4690 con 0x557c782e9c30 2024-09-12T01:32:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.390+0000 7f5f508c7740 1 -- 172.21.15.1:0/1187928088 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f3804d050 con 0x557c782e9c30 2024-09-12T01:32:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.390+0000 7f5f30c00640 1 -- 172.21.15.1:0/1187928088 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5f400b4890 con 0x557c782e9c30 2024-09-12T01:32:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.391+0000 7f5f508c7740 10 client.25393 fetch_fsmap learned FSMap version 32 2024-09-12T01:32:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.391+0000 7f5f508c7740 1 -- 172.21.15.1:0/1187928088 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557c78364630 con 0x557c782e9c30 2024-09-12T01:32:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.391+0000 7f5f30c00640 1 -- 172.21.15.1:0/1187928088 <== mon.1 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f3804d050 con 0x557c782e9c30 2024-09-12T01:32:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.391+0000 7f5f30c00640 1 -- 172.21.15.1:0/1187928088 <== mon.1 v2:172.21.15.104:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f5f400a7d90 con 0x557c782e9c30 2024-09-12T01:32:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.391+0000 7f5f508c7740 10 client.25393 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T01:32:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.391+0000 7f5f508c7740 10 client.25393 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:32:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.391+0000 7f5f508c7740 20 client.25393 populate_metadata read hostname 'smithi001' 2024-09-12T01:32:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.391+0000 7f5f508c7740 1 --2- 172.21.15.1:0/1187928088 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x557c78365010 0x557c78367430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.391+0000 7f5f508c7740 4 client.25393 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:32:04.392 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.391+0000 7f5f508c7740 1 -- 172.21.15.1:0/1187928088 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557c78250b80 con 0x557c78365010 2024-09-12T01:32:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.396+0000 7f5f46a00640 1 --2- 172.21.15.1:0/1187928088 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x557c78365010 0x557c78367430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:04.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.455+0000 7f5f30c00640 10 client.25393 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:32:04.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.455+0000 7f5f46a00640 1 --2- 172.21.15.1:0/1187928088 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x557c78365010 0x557c78367430 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9914a150ce857cfb server_cookie=2b38c3330a4c3560 in_seq=0 out_seq=0 2024-09-12T01:32:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.479+0000 7f5f30c00640 1 -- 172.21.15.1:0/1187928088 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557c78250b80 con 0x557c78365010 2024-09-12T01:32:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.479+0000 7f5f30c00640 10 client.25393 handle_command_reply: tid=0 2024-09-12T01:32:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.479+0000 7f5f508c7740 1 client.25393 shutdown 2024-09-12T01:32:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.479+0000 7f5f508c7740 1 -- 172.21.15.1:0/1187928088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557c782e9c30 msgr2=0x557c78362d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.479+0000 7f5f508c7740 1 --2- 172.21.15.1:0/1187928088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557c782e9c30 0x557c78362d10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5f4000b940 tx=0x7f5f40004a40 comp rx=0 tx=0).stop 2024-09-12T01:32:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.479+0000 7f5f508c7740 1 -- 172.21.15.1:0/1187928088 shutdown_connections 2024-09-12T01:32:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.479+0000 7f5f508c7740 1 --2- 172.21.15.1:0/1187928088 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x557c78365010 0x557c78367430 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.479+0000 7f5f508c7740 1 --2- 172.21.15.1:0/1187928088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557c782e9c30 0x557c78362d10 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.479+0000 7f5f508c7740 1 --2- 172.21.15.1:0/1187928088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557c782e4ba0 0x557c782f2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.479+0000 7f5f508c7740 1 --2- 172.21.15.1:0/1187928088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557c782e4260 0x557c782f2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.479+0000 7f5f30c00640 0 client.25393 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:32:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.479+0000 7f5f508c7740 1 -- 172.21.15.1:0/1187928088 >> 172.21.15.1:0/1187928088 conn(0x557c782df8d0 msgr2=0x557c782fbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.479+0000 7f5f508c7740 1 -- 172.21.15.1:0/1187928088 shutdown_connections 2024-09-12T01:32:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.479+0000 7f5f508c7740 1 -- 172.21.15.1:0/1187928088 wait complete. 2024-09-12T01:32:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.480+0000 7f5f508c7740 20 client.25393 trim_cache size 0 max 16384 2024-09-12T01:32:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-12T01:32:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-12T01:32:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-12T01:32:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-12T01:32:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.480+0000 7f5f47e00640 1 -- 172.21.15.1:0/110590018 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f5f2c0a2d20 msgr2=0x7f5f2c0a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.480+0000 7f5f47e00640 1 --2- 172.21.15.1:0/110590018 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f5f2c0a2d20 0x7f5f2c0a51e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5f38009230 tx=0x7f5f38031400 comp rx=0 tx=0).stop 2024-09-12T01:32:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.480+0000 7f5f47e00640 1 -- 172.21.15.1:0/110590018 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f48106830 msgr2=0x7f5f48108cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.480+0000 7f5f47e00640 1 --2- 172.21.15.1:0/110590018 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f48106830 0x7f5f48108cf0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5f4000b970 tx=0x7f5f4000be30 comp rx=0 tx=0).stop 2024-09-12T01:32:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.480+0000 7f5f47e00640 1 -- 172.21.15.1:0/110590018 shutdown_connections 2024-09-12T01:32:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.480+0000 7f5f47e00640 1 --2- 172.21.15.1:0/110590018 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f5f2c0a2d20 0x7f5f2c0a51e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.480+0000 7f5f47e00640 1 --2- 172.21.15.1:0/110590018 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f48106830 0x7f5f48108cf0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.480+0000 7f5f47e00640 1 --2- 172.21.15.1:0/110590018 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f48105510 0x7f5f4819eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.480+0000 7f5f47e00640 1 --2- 172.21.15.1:0/110590018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f48102b60 0x7f5f4819e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.480+0000 7f5f47e00640 1 -- 172.21.15.1:0/110590018 >> 172.21.15.1:0/110590018 conn(0x7f5f480fc040 msgr2=0x7f5f481038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.480+0000 7f5f47e00640 1 -- 172.21.15.1:0/110590018 shutdown_connections 2024-09-12T01:32:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:04.480+0000 7f5f47e00640 1 -- 172.21.15.1:0/110590018 wait complete. 2024-09-12T01:32:04.757 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:32:04 smithi001 ceph-mon[30387]: pgmap v472: 129 pgs: 129 active+clean; 65 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 18 op/s 2024-09-12T01:32:04.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:32:04 smithi104 ceph-mon[35219]: pgmap v472: 129 pgs: 129 active+clean; 65 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 18 op/s 2024-09-12T01:32:04.839 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_snapshot_deleted_subvolume 2024-09-12T01:32:04.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:32:04 smithi053 ceph-mon[35158]: pgmap v472: 129 pgs: 129 active+clean; 65 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 18 op/s 2024-09-12T01:32:06.724 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:32:06 smithi001 ceph-mon[30387]: pgmap v473: 129 pgs: 129 active+clean; 66 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 66 KiB/s wr, 20 op/s 2024-09-12T01:32:06.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:32:06 smithi104 ceph-mon[35219]: pgmap v473: 129 pgs: 129 active+clean; 66 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 66 KiB/s wr, 20 op/s 2024-09-12T01:32:06.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:32:06 smithi053 ceph-mon[35158]: pgmap v473: 129 pgs: 129 active+clean; 66 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 66 KiB/s wr, 20 op/s 2024-09-12T01:32:07.191 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] fiemap-fault 2024-09-12T01:32:07.192 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] fiemap-tester 2024-09-12T01:32:07.193 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_stripealign 2024-09-12T01:32:07.463 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_mmap_dio 2024-09-12T01:32:07.463 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] open_by_handle 2024-09-12T01:32:07.464 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] xfsfind 2024-09-12T01:32:07.613 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_reflink_read_race 2024-09-12T01:32:07.775 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] fssum 2024-09-12T01:32:07.822 INFO:tasks.workunit.client.0.smithi001.stdout:Building log-writes 2024-09-12T01:32:07.837 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] replay-log 2024-09-12T01:32:07.866 INFO:tasks.workunit.client.0.smithi001.stdout:Building perf 2024-09-12T01:32:07.876 INFO:tasks.workunit.client.0.smithi001.stdout:Building aio-dio-regress 2024-09-12T01:32:07.889 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aiocp 2024-09-12T01:32:07.964 INFO:tasks.workunit.client.0.smithi001.stdout:Building vfs 2024-09-12T01:32:07.977 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] vfstest 2024-09-12T01:32:08.254 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-dio-append-write-fallocate-race 2024-09-12T01:32:08.344 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-dio-append-write-read-race 2024-09-12T01:32:08.407 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] mount-idmapped 2024-09-12T01:32:08.573 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-dio-cow-race 2024-09-12T01:32:08.658 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-dio-cycle-write 2024-09-12T01:32:08.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:32:08 smithi104 ceph-mon[35219]: pgmap v474: 129 pgs: 129 active+clean; 70 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 455 KiB/s wr, 49 op/s 2024-09-12T01:32:08.864 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-dio-eof-race 2024-09-12T01:32:08.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:32:08 smithi001 ceph-mon[30387]: pgmap v474: 129 pgs: 129 active+clean; 70 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 455 KiB/s wr, 49 op/s 2024-09-12T01:32:08.972 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-dio-extend-stat 2024-09-12T01:32:08.984 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-dio-fcntl-race 2024-09-12T01:32:08.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:32:08 smithi053 ceph-mon[35158]: pgmap v474: 129 pgs: 129 active+clean; 70 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 455 KiB/s wr, 49 op/s 2024-09-12T01:32:09.193 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-dio-hole-filling-race 2024-09-12T01:32:09.258 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-dio-invalidate-failure 2024-09-12T01:32:09.289 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-dio-invalidate-readahead 2024-09-12T01:32:09.476 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aiodio_sparse2 2024-09-12T01:32:09.558 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-dio-subblock-eof-read 2024-09-12T01:32:09.588 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-dio-write-verify 2024-09-12T01:32:09.842 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2024-09-12T01:32:09.862 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2024-09-12T01:32:09.929 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-last-ref-held-by-io 2024-09-12T01:32:10.524 INFO:tasks.workunit.client.0.smithi001.stderr:+ cd .. 2024-09-12T01:32:10.524 INFO:tasks.workunit.client.0.smithi001.stderr:+ cp xfstests-dev/ltp/fsx . 2024-09-12T01:32:10.530 INFO:tasks.workunit.client.0.smithi001.stderr:+ OPTIONS=-z 2024-09-12T01:32:10.530 INFO:tasks.workunit.client.0.smithi001.stderr:+ ./fsx -z 1MB -N 50000 -p 10000 -l 1048576 2024-09-12T01:32:10.531 INFO:tasks.workunit.client.0.smithi001.stdout:Seed set to 1 2024-09-12T01:32:10.570 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-09-12T01:32:10.571 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-09-12T01:32:10.573 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-09-12T01:32:10.573 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-09-12T01:32:10.573 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support clone range, disabling! 2024-09-12T01:32:10.575 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support dedupe range, disabling! 2024-09-12T01:32:10.577 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support exchange range, disabling! 2024-09-12T01:32:10.577 INFO:tasks.workunit.client.0.smithi001.stdout:skipping zero size read 2024-09-12T01:32:10.577 INFO:tasks.workunit.client.0.smithi001.stdout:skipping zero size read 2024-09-12T01:32:10.577 INFO:tasks.workunit.client.0.smithi001.stdout:truncating to largest ever: 0x81d54 2024-09-12T01:32:10.581 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0xb86d7 2024-09-12T01:32:10.590 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0xe627e 2024-09-12T01:32:10.635 INFO:tasks.workunit.client.0.smithi001.stdout:truncating to largest ever: 0xf3194 2024-09-12T01:32:10.665 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0xf3ceb 2024-09-12T01:32:10.671 INFO:tasks.workunit.client.0.smithi001.stdout:truncating to largest ever: 0xf47e4 2024-09-12T01:32:10.785 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0xfaccc 2024-09-12T01:32:10.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:32:10 smithi104 ceph-mon[35219]: pgmap v475: 129 pgs: 129 active+clean; 70 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 469 KiB/s wr, 48 op/s 2024-09-12T01:32:10.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:32:10 smithi001 ceph-mon[30387]: pgmap v475: 129 pgs: 129 active+clean; 70 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 469 KiB/s wr, 48 op/s 2024-09-12T01:32:10.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:32:10 smithi053 ceph-mon[35158]: pgmap v475: 129 pgs: 129 active+clean; 70 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 469 KiB/s wr, 48 op/s 2024-09-12T01:32:11.328 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0xfe423 2024-09-12T01:32:12.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:32:12 smithi104 ceph-mon[35219]: pgmap v476: 129 pgs: 129 active+clean; 71 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 640 KiB/s wr, 73 op/s 2024-09-12T01:32:12.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:32:12 smithi001 ceph-mon[30387]: pgmap v476: 129 pgs: 129 active+clean; 71 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 640 KiB/s wr, 73 op/s 2024-09-12T01:32:12.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:32:12 smithi053 ceph-mon[35158]: pgmap v476: 129 pgs: 129 active+clean; 71 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 640 KiB/s wr, 73 op/s 2024-09-12T01:32:14.388 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0xfe492 2024-09-12T01:32:14.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:32:14 smithi104 ceph-mon[35219]: pgmap v477: 129 pgs: 129 active+clean; 71 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 634 KiB/s wr, 72 op/s 2024-09-12T01:32:14.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:32:14 smithi001 ceph-mon[30387]: pgmap v477: 129 pgs: 129 active+clean; 71 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 634 KiB/s wr, 72 op/s 2024-09-12T01:32:14.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:32:14 smithi053 ceph-mon[35158]: pgmap v477: 129 pgs: 129 active+clean; 71 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 634 KiB/s wr, 72 op/s 2024-09-12T01:32:16.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:32:16 smithi104 ceph-mon[35219]: pgmap v478: 129 pgs: 129 active+clean; 72 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 1.7 MiB/s wr, 204 op/s 2024-09-12T01:32:16.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:32:16 smithi001 ceph-mon[30387]: pgmap v478: 129 pgs: 129 active+clean; 72 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 1.7 MiB/s wr, 204 op/s 2024-09-12T01:32:16.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:32:16 smithi053 ceph-mon[35158]: pgmap v478: 129 pgs: 129 active+clean; 72 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 1.7 MiB/s wr, 204 op/s 2024-09-12T01:32:17.112 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0xff038 2024-09-12T01:32:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.633+0000 7fa243a00640 1 -- 172.21.15.1:0/518841588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa23c1025e0 msgr2=0x7fa23c1049d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.634+0000 7fa243a00640 1 --2- 172.21.15.1:0/518841588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa23c1025e0 0x7fa23c1049d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa230009900 tx=0x7fa23002a410 comp rx=0 tx=0).stop 2024-09-12T01:32:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.634+0000 7fa243a00640 1 -- 172.21.15.1:0/518841588 shutdown_connections 2024-09-12T01:32:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.634+0000 7fa243a00640 1 --2- 172.21.15.1:0/518841588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa23c107840 0x7fa23c109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.634+0000 7fa243a00640 1 --2- 172.21.15.1:0/518841588 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa23c104f10 0x7fa23c107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.634+0000 7fa243a00640 1 --2- 172.21.15.1:0/518841588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa23c1025e0 0x7fa23c1049d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.634+0000 7fa243a00640 1 -- 172.21.15.1:0/518841588 >> 172.21.15.1:0/518841588 conn(0x7fa23c0fc1c0 msgr2=0x7fa23c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.634+0000 7fa243a00640 1 -- 172.21.15.1:0/518841588 shutdown_connections 2024-09-12T01:32:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.634+0000 7fa243a00640 1 -- 172.21.15.1:0/518841588 wait complete. 2024-09-12T01:32:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.635+0000 7fa243a00640 1 Processor -- start 2024-09-12T01:32:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.635+0000 7fa243a00640 1 -- start start 2024-09-12T01:32:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.636+0000 7fa243a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa23c1025e0 0x7fa23c197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.636+0000 7fa243a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa23c104f10 0x7fa23c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.636+0000 7fa243a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa23c107840 0x7fa23c19f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:17.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.636+0000 7fa243a00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa23c10d870 con 0x7fa23c104f10 2024-09-12T01:32:17.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.636+0000 7fa243a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa23c10d6f0 con 0x7fa23c107840 2024-09-12T01:32:17.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.636+0000 7fa243a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa23c10d9f0 con 0x7fa23c1025e0 2024-09-12T01:32:17.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.636+0000 7fa241000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa23c1025e0 0x7fa23c197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.636+0000 7fa23be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa23c104f10 0x7fa23c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.636+0000 7fa241000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa23c1025e0 0x7fa23c197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:54168/0 (socket says 172.21.15.1:54168) 2024-09-12T01:32:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.636+0000 7fa241000640 1 -- 172.21.15.1:0/1054191306 learned_addr learned my addr 172.21.15.1:0/1054191306 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:32:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.636+0000 7fa241a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa23c107840 0x7fa23c19f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.637+0000 7fa241a00640 1 -- 172.21.15.1:0/1054191306 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa23c1025e0 msgr2=0x7fa23c197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.637+0000 7fa241a00640 1 --2- 172.21.15.1:0/1054191306 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa23c1025e0 0x7fa23c197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.637+0000 7fa241a00640 1 -- 172.21.15.1:0/1054191306 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa23c104f10 msgr2=0x7fa23c198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.637+0000 7fa241a00640 1 --2- 172.21.15.1:0/1054191306 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa23c104f10 0x7fa23c198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.637+0000 7fa241a00640 1 -- 172.21.15.1:0/1054191306 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa23c19faf0 con 0x7fa23c107840 2024-09-12T01:32:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.637+0000 7fa241a00640 1 --2- 172.21.15.1:0/1054191306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa23c107840 0x7fa23c19f480 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa234005fd0 tx=0x7fa23400ca50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.637+0000 7fa239600640 1 -- 172.21.15.1:0/1054191306 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa234004d20 con 0x7fa23c107840 2024-09-12T01:32:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.637+0000 7fa239600640 1 -- 172.21.15.1:0/1054191306 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa234015e20 con 0x7fa23c107840 2024-09-12T01:32:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.637+0000 7fa239600640 1 -- 172.21.15.1:0/1054191306 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa234005560 con 0x7fa23c107840 2024-09-12T01:32:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.637+0000 7fa243a00640 1 -- 172.21.15.1:0/1054191306 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa23c100430 con 0x7fa23c107840 2024-09-12T01:32:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.638+0000 7fa243a00640 1 -- 172.21.15.1:0/1054191306 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa23c100930 con 0x7fa23c107840 2024-09-12T01:32:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.641+0000 7fa2469be740 1 Processor -- start 2024-09-12T01:32:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.643+0000 7fa2469be740 1 -- start start 2024-09-12T01:32:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.643+0000 7fa2469be740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f56890090 0x564f56890490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.643+0000 7fa2469be740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564f568909d0 0x564f56895490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.643+0000 7fa2469be740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f56895a60 0x564f56897f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.643+0000 7fa2469be740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564f567f5100 con 0x564f568909d0 2024-09-12T01:32:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.643+0000 7fa2469be740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x564f5679d1b0 con 0x564f56890090 2024-09-12T01:32:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.643+0000 7fa2469be740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564f567f10c0 con 0x564f56895a60 2024-09-12T01:32:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.643+0000 7fa23be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f56890090 0x564f56890490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.644+0000 7fa23be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f56890090 0x564f56890490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:37226/0 (socket says 172.21.15.1:37226) 2024-09-12T01:32:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.644+0000 7fa23be00640 1 -- 172.21.15.1:0/1345008060 learned_addr learned my addr 172.21.15.1:0/1345008060 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:32:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.644+0000 7fa23be00640 1 --2- 172.21.15.1:0/1345008060 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f56895a60 0x564f56897f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.644+0000 7fa241000640 1 --2- 172.21.15.1:0/1345008060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564f568909d0 0x564f56895490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.644+0000 7fa23be00640 1 -- 172.21.15.1:0/1345008060 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f56895a60 msgr2=0x564f56897f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.644+0000 7fa23be00640 1 --2- 172.21.15.1:0/1345008060 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f56895a60 0x564f56897f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.644+0000 7fa23be00640 1 -- 172.21.15.1:0/1345008060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564f568909d0 msgr2=0x564f56895490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.644+0000 7fa23be00640 1 --2- 172.21.15.1:0/1345008060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564f568909d0 0x564f56895490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.644+0000 7fa23be00640 1 -- 172.21.15.1:0/1345008060 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f56508210 con 0x564f56890090 2024-09-12T01:32:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.644+0000 7fa239600640 1 -- 172.21.15.1:0/1054191306 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fa23400bba0 con 0x7fa23c107840 2024-09-12T01:32:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.644+0000 7fa23be00640 1 --2- 172.21.15.1:0/1345008060 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f56890090 0x564f56890490 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa228010810 tx=0x7fa2280096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e823abf25b504a34 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.644+0000 7fa241000640 1 --2- 172.21.15.1:0/1345008060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564f568909d0 0x564f56895490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:32:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.645+0000 7fa22e000640 1 -- 172.21.15.1:0/1345008060 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa228016050 con 0x564f56890090 2024-09-12T01:32:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.645+0000 7fa22e000640 1 -- 172.21.15.1:0/1345008060 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa22800edd0 con 0x564f56890090 2024-09-12T01:32:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.645+0000 7fa22e000640 1 -- 172.21.15.1:0/1345008060 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa228019040 con 0x564f56890090 2024-09-12T01:32:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.646+0000 7fa2469be740 1 -- 172.21.15.1:0/1345008060 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f56890090 msgr2=0x564f56890490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.646+0000 7fa2469be740 1 --2- 172.21.15.1:0/1345008060 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f56890090 0x564f56890490 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa228010810 tx=0x7fa2280096c0 comp rx=0 tx=0).stop 2024-09-12T01:32:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.646+0000 7fa2469be740 1 -- 172.21.15.1:0/1345008060 shutdown_connections 2024-09-12T01:32:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.646+0000 7fa2469be740 1 --2- 172.21.15.1:0/1345008060 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f56895a60 0x564f56897f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.646+0000 7fa2469be740 1 --2- 172.21.15.1:0/1345008060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564f568909d0 0x564f56895490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.646+0000 7fa2469be740 1 --2- 172.21.15.1:0/1345008060 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f56890090 0x564f56890490 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.646+0000 7fa2469be740 1 -- 172.21.15.1:0/1345008060 >> 172.21.15.1:0/1345008060 conn(0x564f5688b700 msgr2=0x564f5688bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.646+0000 7fa239600640 1 --2- 172.21.15.1:0/1054191306 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fa2240a2cd0 0x7fa2240a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.646+0000 7fa239600640 1 -- 172.21.15.1:0/1054191306 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fa2340d7b20 con 0x7fa23c107840 2024-09-12T01:32:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.647+0000 7fa2469be740 1 -- 172.21.15.1:0/1345008060 shutdown_connections 2024-09-12T01:32:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.647+0000 7fa2469be740 1 -- 172.21.15.1:0/1345008060 wait complete. 2024-09-12T01:32:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.647+0000 7fa241000640 1 --2- 172.21.15.1:0/1054191306 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fa2240a2cd0 0x7fa2240a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.648+0000 7fa2469be740 1 -- start start 2024-09-12T01:32:17.648 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.648+0000 7fa241000640 1 --2- 172.21.15.1:0/1054191306 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fa2240a2cd0 0x7fa2240a5190 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa230004c90 tx=0x7fa230000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:17.648 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.648+0000 7fa2469be740 1 Processor -- start 2024-09-12T01:32:17.648 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.649+0000 7fa2469be740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f56890090 0x564f5689a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:17.648 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.649+0000 7fa2469be740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564f568909d0 0x564f5689a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.649+0000 7fa2469be740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f56895a60 0x564f568a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.649+0000 7fa2469be740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564f56898a30 con 0x564f568909d0 2024-09-12T01:32:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.649+0000 7fa2469be740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x564f5677e090 con 0x564f56890090 2024-09-12T01:32:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.649+0000 7fa2469be740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564f56898bb0 con 0x564f56895a60 2024-09-12T01:32:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.649+0000 7fa2469be740 10 client.0 _ll_register_callbacks cb 0x564f567d2f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:32:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.649+0000 7fa241000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f56890090 0x564f5689a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.649+0000 7fa241000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f56890090 0x564f5689a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:37230/0 (socket says 172.21.15.1:37230) 2024-09-12T01:32:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.649+0000 7fa241000640 1 -- 172.21.15.1:0/2667130298 learned_addr learned my addr 172.21.15.1:0/2667130298 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:32:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.649+0000 7fa23be00640 1 --2- 172.21.15.1:0/2667130298 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564f568909d0 0x564f5689a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.649+0000 7fa241a00640 1 --2- 172.21.15.1:0/2667130298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f56895a60 0x564f568a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.650+0000 7fa241000640 1 -- 172.21.15.1:0/2667130298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f56895a60 msgr2=0x564f568a17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.650+0000 7fa241000640 1 --2- 172.21.15.1:0/2667130298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f56895a60 0x564f568a17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.650+0000 7fa241000640 1 -- 172.21.15.1:0/2667130298 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564f568909d0 msgr2=0x564f5689a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.650+0000 7fa241000640 1 --2- 172.21.15.1:0/2667130298 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564f568909d0 0x564f5689a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.650+0000 7fa241000640 1 -- 172.21.15.1:0/2667130298 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f565d4620 con 0x564f56890090 2024-09-12T01:32:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.650+0000 7fa241a00640 1 --2- 172.21.15.1:0/2667130298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f56895a60 0x564f568a17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:32:17.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.650+0000 7fa241000640 1 --2- 172.21.15.1:0/2667130298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f56890090 0x564f5689a240 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa230009230 tx=0x7fa230009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5d16a0e97d1d190 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:17.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.650+0000 7fa222a00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-12T01:32:17.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.651+0000 7fa222a00640 1 -- 172.21.15.1:0/2667130298 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa230031440 con 0x564f56890090 2024-09-12T01:32:17.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.651+0000 7fa222a00640 1 -- 172.21.15.1:0/2667130298 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fa2300315e0 con 0x564f56890090 2024-09-12T01:32:17.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.651+0000 7fa2469be740 1 -- 172.21.15.1:0/2667130298 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa22800edd0 con 0x564f56890090 2024-09-12T01:32:17.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.652+0000 7fa222a00640 1 -- 172.21.15.1:0/2667130298 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa2300317e0 con 0x564f56890090 2024-09-12T01:32:17.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.652+0000 7fa2469be740 10 client.25406 fetch_fsmap learned FSMap version 32 2024-09-12T01:32:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.652+0000 7fa2469be740 1 -- 172.21.15.1:0/2667130298 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564f568a45e0 con 0x564f56890090 2024-09-12T01:32:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.652+0000 7fa222a00640 1 -- 172.21.15.1:0/2667130298 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa22800edd0 con 0x564f56890090 2024-09-12T01:32:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.652+0000 7fa222a00640 1 -- 172.21.15.1:0/2667130298 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7fa23003ea00 con 0x564f56890090 2024-09-12T01:32:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.653+0000 7fa2469be740 10 client.25406 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T01:32:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.653+0000 7fa2469be740 10 client.25406 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:32:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.653+0000 7fa2469be740 20 client.25406 populate_metadata read hostname 'smithi001' 2024-09-12T01:32:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.653+0000 7fa2469be740 1 --2- 172.21.15.1:0/2667130298 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x564f568a5c60 0x564f568a8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.653+0000 7fa2469be740 4 client.25406 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:32:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.653+0000 7fa2469be740 1 -- 172.21.15.1:0/2667130298 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f56802830 con 0x564f568a5c60 2024-09-12T01:32:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.653+0000 7fa23be00640 1 --2- 172.21.15.1:0/2667130298 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x564f568a5c60 0x564f568a8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.654+0000 7fa23be00640 1 --2- 172.21.15.1:0/2667130298 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x564f568a5c60 0x564f568a8120 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4617ddff798138da server_cookie=8a583dc92522108d in_seq=0 out_seq=0 2024-09-12T01:32:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.654+0000 7fa222a00640 10 client.25406 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:32:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.655+0000 7fa222a00640 1 -- 172.21.15.1:0/2667130298 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564f56802830 con 0x564f568a5c60 2024-09-12T01:32:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.655+0000 7fa222a00640 10 client.25406 handle_command_reply: tid=0 2024-09-12T01:32:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.655+0000 7fa2469be740 1 client.25406 shutdown 2024-09-12T01:32:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.655+0000 7fa2469be740 1 -- 172.21.15.1:0/2667130298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f56890090 msgr2=0x564f5689a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.656+0000 7fa2469be740 1 --2- 172.21.15.1:0/2667130298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f56890090 0x564f5689a240 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa230009230 tx=0x7fa230009260 comp rx=0 tx=0).stop 2024-09-12T01:32:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.656+0000 7fa2469be740 1 -- 172.21.15.1:0/2667130298 shutdown_connections 2024-09-12T01:32:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.656+0000 7fa2469be740 1 --2- 172.21.15.1:0/2667130298 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x564f568a5c60 0x564f568a8120 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.656+0000 7fa2469be740 1 --2- 172.21.15.1:0/2667130298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f56895a60 0x564f568a17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.656+0000 7fa222a00640 0 client.25406 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:32:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.656+0000 7fa2469be740 1 --2- 172.21.15.1:0/2667130298 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564f568909d0 0x564f5689a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.656+0000 7fa2469be740 1 --2- 172.21.15.1:0/2667130298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f56890090 0x564f5689a240 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.656+0000 7fa2469be740 1 -- 172.21.15.1:0/2667130298 >> 172.21.15.1:0/2667130298 conn(0x564f5688b700 msgr2=0x564f5688cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.656+0000 7fa2469be740 1 -- 172.21.15.1:0/2667130298 shutdown_connections 2024-09-12T01:32:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.656+0000 7fa2469be740 1 -- 172.21.15.1:0/2667130298 wait complete. 2024-09-12T01:32:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.657+0000 7fa2469be740 20 client.25406 trim_cache size 0 max 16384 2024-09-12T01:32:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.679+0000 7fa2469be740 1 Processor -- start 2024-09-12T01:32:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.679+0000 7fa2469be740 1 -- start start 2024-09-12T01:32:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.680+0000 7fa2469be740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564f56890090 0x564f5688caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.680+0000 7fa2469be740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f568909d0 0x564f568a1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.680+0000 7fa2469be740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f56895a60 0x564f5688d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.680+0000 7fa2469be740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa23c1a8590 con 0x564f56890090 2024-09-12T01:32:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.680+0000 7fa2469be740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa23c1aa160 con 0x564f568909d0 2024-09-12T01:32:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.680+0000 7fa2469be740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa23c1a8c60 con 0x564f56895a60 2024-09-12T01:32:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.680+0000 7fa23be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f568909d0 0x564f568a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.680+0000 7fa241000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564f56890090 0x564f5688caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.680+0000 7fa23be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f568909d0 0x564f568a1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:37244/0 (socket says 172.21.15.1:37244) 2024-09-12T01:32:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.680+0000 7fa23be00640 1 -- 172.21.15.1:0/2330994567 learned_addr learned my addr 172.21.15.1:0/2330994567 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:32:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.680+0000 7fa241a00640 1 --2- 172.21.15.1:0/2330994567 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f56895a60 0x564f5688d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.680+0000 7fa23be00640 1 -- 172.21.15.1:0/2330994567 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f56895a60 msgr2=0x564f5688d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.680+0000 7fa23be00640 1 --2- 172.21.15.1:0/2330994567 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f56895a60 0x564f5688d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.681+0000 7fa23be00640 1 -- 172.21.15.1:0/2330994567 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564f56890090 msgr2=0x564f5688caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.681+0000 7fa23be00640 1 --2- 172.21.15.1:0/2330994567 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564f56890090 0x564f5688caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.681+0000 7fa23be00640 1 -- 172.21.15.1:0/2330994567 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f56886d60 con 0x564f568909d0 2024-09-12T01:32:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.681+0000 7fa23be00640 1 --2- 172.21.15.1:0/2330994567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f568909d0 0x564f568a1780 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x564f5689b720 tx=0x7fa2280093f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c314b1a27ae4d911 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:17.682 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.681+0000 7fa22d600640 1 -- 172.21.15.1:0/2330994567 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22802c020 con 0x564f568909d0 2024-09-12T01:32:17.682 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.681+0000 7fa22d600640 1 -- 172.21.15.1:0/2330994567 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa228009e80 con 0x564f568909d0 2024-09-12T01:32:17.682 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.681+0000 7fa22d600640 1 -- 172.21.15.1:0/2330994567 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa228016040 con 0x564f568909d0 2024-09-12T01:32:17.683 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.682+0000 7fa2469be740 1 -- 172.21.15.1:0/2330994567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f568909d0 msgr2=0x564f568a1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:17.683 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.682+0000 7fa2469be740 1 --2- 172.21.15.1:0/2330994567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f568909d0 0x564f568a1780 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x564f5689b720 tx=0x7fa2280093f0 comp rx=0 tx=0).stop 2024-09-12T01:32:17.683 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.682+0000 7fa2469be740 1 -- 172.21.15.1:0/2330994567 shutdown_connections 2024-09-12T01:32:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.682+0000 7fa2469be740 1 --2- 172.21.15.1:0/2330994567 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f56895a60 0x564f5688d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.682+0000 7fa2469be740 1 --2- 172.21.15.1:0/2330994567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f568909d0 0x564f568a1780 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.682+0000 7fa2469be740 1 --2- 172.21.15.1:0/2330994567 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564f56890090 0x564f5688caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.682+0000 7fa2469be740 1 -- 172.21.15.1:0/2330994567 >> 172.21.15.1:0/2330994567 conn(0x564f5688b700 msgr2=0x564f5688a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.682+0000 7fa2469be740 1 -- 172.21.15.1:0/2330994567 shutdown_connections 2024-09-12T01:32:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.682+0000 7fa2469be740 1 -- 172.21.15.1:0/2330994567 wait complete. 2024-09-12T01:32:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.683+0000 7fa2469be740 1 -- start start 2024-09-12T01:32:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.683+0000 7fa2469be740 1 Processor -- start 2024-09-12T01:32:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.683+0000 7fa2469be740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f56890090 0x564f5689ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.683+0000 7fa2469be740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f568909d0 0x564f5689f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.683+0000 7fa2469be740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564f56895a60 0x564f568a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.684+0000 7fa2469be740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa23c1a9a60 con 0x564f56895a60 2024-09-12T01:32:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.684+0000 7fa2469be740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa23c1a9390 con 0x564f56890090 2024-09-12T01:32:17.685 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.684+0000 7fa2469be740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa23c1a6570 con 0x564f568909d0 2024-09-12T01:32:17.685 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.684+0000 7fa2469be740 10 client.0 _ll_register_callbacks cb 0x7fa228005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:32:17.685 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.684+0000 7fa241000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f56890090 0x564f5689ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:17.685 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.684+0000 7fa241000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f56890090 0x564f5689ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:37252/0 (socket says 172.21.15.1:37252) 2024-09-12T01:32:17.685 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.684+0000 7fa241000640 1 -- 172.21.15.1:0/1215671180 learned_addr learned my addr 172.21.15.1:0/1215671180 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:32:17.685 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.684+0000 7fa241a00640 1 --2- 172.21.15.1:0/1215671180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564f56895a60 0x564f568a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:17.686 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.684+0000 7fa23be00640 1 --2- 172.21.15.1:0/1215671180 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f568909d0 0x564f5689f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:17.686 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.685+0000 7fa241000640 1 -- 172.21.15.1:0/1215671180 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f568909d0 msgr2=0x564f5689f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:17.686 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.685+0000 7fa241000640 1 --2- 172.21.15.1:0/1215671180 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f568909d0 0x564f5689f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:17.686 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.685+0000 7fa241000640 1 -- 172.21.15.1:0/1215671180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564f56895a60 msgr2=0x564f568a03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:17.686 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.685+0000 7fa241000640 1 --2- 172.21.15.1:0/1215671180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564f56895a60 0x564f568a03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:17.686 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.685+0000 7fa241000640 1 -- 172.21.15.1:0/1215671180 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f568a09d0 con 0x564f56890090 2024-09-12T01:32:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.685+0000 7fa241000640 1 --2- 172.21.15.1:0/1215671180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f56890090 0x564f5689ee70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa230009850 tx=0x7fa23005d420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b837f2386121b7fc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.685+0000 7fa222a00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-12T01:32:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.685+0000 7fa23be00640 1 --2- 172.21.15.1:0/1215671180 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f568909d0 0x564f5689f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:32:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.685+0000 7fa222a00640 1 -- 172.21.15.1:0/1215671180 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa230063070 con 0x564f56890090 2024-09-12T01:32:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.685+0000 7fa222a00640 1 -- 172.21.15.1:0/1215671180 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fa230031b80 con 0x564f56890090 2024-09-12T01:32:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.685+0000 7fa2469be740 1 -- 172.21.15.1:0/1215671180 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564f567fcb80 con 0x564f56890090 2024-09-12T01:32:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.686+0000 7fa222a00640 1 -- 172.21.15.1:0/1215671180 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa230071050 con 0x564f56890090 2024-09-12T01:32:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.686+0000 7fa2469be740 10 client.25412 fetch_fsmap learned FSMap version 32 2024-09-12T01:32:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.686+0000 7fa2469be740 1 -- 172.21.15.1:0/1215671180 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564f56910a30 con 0x564f56890090 2024-09-12T01:32:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.686+0000 7fa222a00640 1 -- 172.21.15.1:0/1215671180 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x564f567fcb80 con 0x564f56890090 2024-09-12T01:32:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.687+0000 7fa222a00640 1 -- 172.21.15.1:0/1215671180 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7fa23003e4a0 con 0x564f56890090 2024-09-12T01:32:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.687+0000 7fa2469be740 10 client.25412 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T01:32:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.687+0000 7fa2469be740 10 client.25412 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:32:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.687+0000 7fa2469be740 20 client.25412 populate_metadata read hostname 'smithi001' 2024-09-12T01:32:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.687+0000 7fa2469be740 1 --2- 172.21.15.1:0/1215671180 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x564f568a5c60 0x564f56913420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.687+0000 7fa2469be740 4 client.25412 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:32:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.687+0000 7fa2469be740 1 -- 172.21.15.1:0/1215671180 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564f567797c0 con 0x564f568a5c60 2024-09-12T01:32:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.689+0000 7fa23be00640 1 --2- 172.21.15.1:0/1215671180 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x564f568a5c60 0x564f56913420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.690+0000 7fa222a00640 10 client.25412 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:32:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.690+0000 7fa23be00640 1 --2- 172.21.15.1:0/1215671180 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x564f568a5c60 0x564f56913420 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e17671fcc89e1d4 server_cookie=84d5ae483df58c9c in_seq=0 out_seq=0 2024-09-12T01:32:17.697 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.697+0000 7fa222a00640 1 -- 172.21.15.1:0/1215671180 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564f567797c0 con 0x564f568a5c60 2024-09-12T01:32:17.697 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.697+0000 7fa222a00640 10 client.25412 handle_command_reply: tid=0 2024-09-12T01:32:17.697 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.697+0000 7fa2469be740 1 client.25412 shutdown 2024-09-12T01:32:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.698+0000 7fa2469be740 1 -- 172.21.15.1:0/1215671180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f56890090 msgr2=0x564f5689ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.698+0000 7fa2469be740 1 --2- 172.21.15.1:0/1215671180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f56890090 0x564f5689ee70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa230009850 tx=0x7fa23005d420 comp rx=0 tx=0).stop 2024-09-12T01:32:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.698+0000 7fa2469be740 1 -- 172.21.15.1:0/1215671180 shutdown_connections 2024-09-12T01:32:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.698+0000 7fa2469be740 1 --2- 172.21.15.1:0/1215671180 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x564f568a5c60 0x564f56913420 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.698+0000 7fa2469be740 1 --2- 172.21.15.1:0/1215671180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564f56895a60 0x564f568a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.698+0000 7fa2469be740 1 --2- 172.21.15.1:0/1215671180 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f568909d0 0x564f5689f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.698+0000 7fa2469be740 1 --2- 172.21.15.1:0/1215671180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564f56890090 0x564f5689ee70 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.698+0000 7fa2469be740 1 -- 172.21.15.1:0/1215671180 >> 172.21.15.1:0/1215671180 conn(0x564f5688b700 msgr2=0x564f5688bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.698+0000 7fa222a00640 0 client.25412 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:32:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.698+0000 7fa2469be740 1 -- 172.21.15.1:0/1215671180 shutdown_connections 2024-09-12T01:32:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.698+0000 7fa2469be740 1 -- 172.21.15.1:0/1215671180 wait complete. 2024-09-12T01:32:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.698+0000 7fa2469be740 20 client.25412 trim_cache size 0 max 16384 2024-09-12T01:32:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-12T01:32:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-12T01:32:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-12T01:32:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-12T01:32:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.699+0000 7fa243a00640 1 -- 172.21.15.1:0/1054191306 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fa2240a2cd0 msgr2=0x7fa2240a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.699+0000 7fa243a00640 1 --2- 172.21.15.1:0/1054191306 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fa2240a2cd0 0x7fa2240a5190 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa230004c90 tx=0x7fa230000bc0 comp rx=0 tx=0).stop 2024-09-12T01:32:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.699+0000 7fa243a00640 1 -- 172.21.15.1:0/1054191306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa23c107840 msgr2=0x7fa23c19f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.699+0000 7fa243a00640 1 --2- 172.21.15.1:0/1054191306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa23c107840 0x7fa23c19f480 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa234005fd0 tx=0x7fa23400ca50 comp rx=0 tx=0).stop 2024-09-12T01:32:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.700+0000 7fa243a00640 1 -- 172.21.15.1:0/1054191306 shutdown_connections 2024-09-12T01:32:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.700+0000 7fa243a00640 1 --2- 172.21.15.1:0/1054191306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa23c107840 0x7fa23c19f480 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.700+0000 7fa243a00640 1 --2- 172.21.15.1:0/1054191306 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa23c104f10 0x7fa23c198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.700+0000 7fa243a00640 1 --2- 172.21.15.1:0/1054191306 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fa2240a2cd0 0x7fa2240a5190 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.700+0000 7fa243a00640 1 --2- 172.21.15.1:0/1054191306 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa23c1025e0 0x7fa23c197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.700+0000 7fa243a00640 1 -- 172.21.15.1:0/1054191306 >> 172.21.15.1:0/1054191306 conn(0x7fa23c0fc1c0 msgr2=0x7fa23c103000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.700+0000 7fa243a00640 1 -- 172.21.15.1:0/1054191306 shutdown_connections 2024-09-12T01:32:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:17.700+0000 7fa243a00640 1 -- 172.21.15.1:0/1054191306 wait complete. 2024-09-12T01:32:18.397 INFO:tasks.workunit.client.0.smithi001.stdout:truncating to largest ever: 0xffcac 2024-09-12T01:32:18.564 INFO:tasks.workunit.client.0.smithi001.stdout:10000 copy from 0x2d237 to 0x30552, (0x331b bytes) at 0x82aed 2024-09-12T01:32:18.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:32:18 smithi104 ceph-mon[35219]: pgmap v479: 129 pgs: 129 active+clean; 73 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 1.8 MiB/s wr, 299 op/s 2024-09-12T01:32:18.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:32:18 smithi001 ceph-mon[30387]: pgmap v479: 129 pgs: 129 active+clean; 73 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 1.8 MiB/s wr, 299 op/s 2024-09-12T01:32:18.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:32:18 smithi053 ceph-mon[35158]: pgmap v479: 129 pgs: 129 active+clean; 73 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 1.8 MiB/s wr, 299 op/s 2024-09-12T01:32:20.824 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:32:20 smithi104 ceph-mon[35219]: pgmap v480: 129 pgs: 129 active+clean; 73 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 1.4 MiB/s wr, 268 op/s 2024-09-12T01:32:20.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:32:20 smithi001 ceph-mon[30387]: pgmap v480: 129 pgs: 129 active+clean; 73 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 1.4 MiB/s wr, 268 op/s 2024-09-12T01:32:20.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:32:20 smithi053 ceph-mon[35158]: pgmap v480: 129 pgs: 129 active+clean; 73 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 1.4 MiB/s wr, 268 op/s 2024-09-12T01:32:22.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:32:22 smithi001 ceph-mon[30387]: pgmap v481: 129 pgs: 129 active+clean; 74 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 413 op/s 2024-09-12T01:32:22.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:32:22 smithi053 ceph-mon[35158]: pgmap v481: 129 pgs: 129 active+clean; 74 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 413 op/s 2024-09-12T01:32:23.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:32:22 smithi104 ceph-mon[35219]: pgmap v481: 129 pgs: 129 active+clean; 74 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 413 op/s 2024-09-12T01:32:24.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:32:24 smithi001 ceph-mon[30387]: pgmap v482: 129 pgs: 129 active+clean; 75 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 465 op/s 2024-09-12T01:32:24.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:32:24 smithi053 ceph-mon[35158]: pgmap v482: 129 pgs: 129 active+clean; 75 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 465 op/s 2024-09-12T01:32:25.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:32:24 smithi104 ceph-mon[35219]: pgmap v482: 129 pgs: 129 active+clean; 75 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 465 op/s 2024-09-12T01:32:26.876 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:32:26 smithi104 ceph-mon[35219]: pgmap v483: 129 pgs: 129 active+clean; 74 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.7 MiB/s wr, 592 op/s 2024-09-12T01:32:26.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:32:26 smithi001 ceph-mon[30387]: pgmap v483: 129 pgs: 129 active+clean; 74 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.7 MiB/s wr, 592 op/s 2024-09-12T01:32:26.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:32:26 smithi053 ceph-mon[35158]: pgmap v483: 129 pgs: 129 active+clean; 74 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.7 MiB/s wr, 592 op/s 2024-09-12T01:32:28.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:32:28 smithi001 ceph-mon[30387]: pgmap v484: 129 pgs: 129 active+clean; 75 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 529 op/s 2024-09-12T01:32:28.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:32:28 smithi053 ceph-mon[35158]: pgmap v484: 129 pgs: 129 active+clean; 75 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 529 op/s 2024-09-12T01:32:29.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:32:28 smithi104 ceph-mon[35219]: pgmap v484: 129 pgs: 129 active+clean; 75 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 529 op/s 2024-09-12T01:32:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.848+0000 7fde7cc00640 1 -- 172.21.15.1:0/2228331352 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde78102ba0 msgr2=0x7fde78103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.848+0000 7fde7cc00640 1 --2- 172.21.15.1:0/2228331352 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde78102ba0 0x7fde78103020 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fde64009900 tx=0x7fde6402a410 comp rx=0 tx=0).stop 2024-09-12T01:32:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.848+0000 7fde7cc00640 1 -- 172.21.15.1:0/2228331352 shutdown_connections 2024-09-12T01:32:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.848+0000 7fde7cc00640 1 --2- 172.21.15.1:0/2228331352 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde78103560 0x7fde78109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.848+0000 7fde7cc00640 1 --2- 172.21.15.1:0/2228331352 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde78102ba0 0x7fde78103020 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.848+0000 7fde7cc00640 1 --2- 172.21.15.1:0/2228331352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde781019b0 0x7fde78101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.848+0000 7fde7cc00640 1 -- 172.21.15.1:0/2228331352 >> 172.21.15.1:0/2228331352 conn(0x7fde780fd130 msgr2=0x7fde780ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.848+0000 7fde7cc00640 1 -- 172.21.15.1:0/2228331352 shutdown_connections 2024-09-12T01:32:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.848+0000 7fde7cc00640 1 -- 172.21.15.1:0/2228331352 wait complete. 2024-09-12T01:32:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.849+0000 7fde7cc00640 1 Processor -- start 2024-09-12T01:32:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.849+0000 7fde7cc00640 1 -- start start 2024-09-12T01:32:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.850+0000 7fde7cc00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde781019b0 0x7fde78197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.850+0000 7fde7cc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde78102ba0 0x7fde781981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.850+0000 7fde7cc00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde78103560 0x7fde7819f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.850+0000 7fde7cc00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fde7810d660 con 0x7fde78103560 2024-09-12T01:32:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.850+0000 7fde7cc00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fde7810d4e0 con 0x7fde78102ba0 2024-09-12T01:32:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.850+0000 7fde7cc00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fde7810d7e0 con 0x7fde781019b0 2024-09-12T01:32:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.850+0000 7fde76a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde78102ba0 0x7fde781981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.850+0000 7fde77400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde781019b0 0x7fde78197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.850+0000 7fde76a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde78102ba0 0x7fde781981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:55512/0 (socket says 172.21.15.1:55512) 2024-09-12T01:32:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.850+0000 7fde76a00640 1 -- 172.21.15.1:0/4071664967 learned_addr learned my addr 172.21.15.1:0/4071664967 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:32:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.850+0000 7fde77e00640 1 --2- 172.21.15.1:0/4071664967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde78103560 0x7fde7819f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.850+0000 7fde76a00640 1 -- 172.21.15.1:0/4071664967 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde781019b0 msgr2=0x7fde78197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.850+0000 7fde76a00640 1 --2- 172.21.15.1:0/4071664967 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde781019b0 0x7fde78197c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.850+0000 7fde76a00640 1 -- 172.21.15.1:0/4071664967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde78103560 msgr2=0x7fde7819f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.850+0000 7fde76a00640 1 --2- 172.21.15.1:0/4071664967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde78103560 0x7fde7819f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.850+0000 7fde76a00640 1 -- 172.21.15.1:0/4071664967 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde7819f8c0 con 0x7fde78102ba0 2024-09-12T01:32:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.851+0000 7fde76a00640 1 --2- 172.21.15.1:0/4071664967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde78102ba0 0x7fde781981c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fde64031610 tx=0x7fde64032c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.851+0000 7fde6fe00640 1 -- 172.21.15.1:0/4071664967 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6403f030 con 0x7fde78102ba0 2024-09-12T01:32:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.851+0000 7fde6fe00640 1 -- 172.21.15.1:0/4071664967 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde64003ad0 con 0x7fde78102ba0 2024-09-12T01:32:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.851+0000 7fde6fe00640 1 -- 172.21.15.1:0/4071664967 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde64035070 con 0x7fde78102ba0 2024-09-12T01:32:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.852+0000 7fde7cc00640 1 -- 172.21.15.1:0/4071664967 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde7819fb50 con 0x7fde78102ba0 2024-09-12T01:32:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.852+0000 7fde7cc00640 1 -- 172.21.15.1:0/4071664967 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde7819fff0 con 0x7fde78102ba0 2024-09-12T01:32:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.856+0000 7fde6fe00640 1 -- 172.21.15.1:0/4071664967 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fde64042030 con 0x7fde78102ba0 2024-09-12T01:32:30.856 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.856+0000 7fde81000740 1 Processor -- start 2024-09-12T01:32:30.856 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.856+0000 7fde81000740 1 -- start start 2024-09-12T01:32:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.857+0000 7fde81000740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b13dfd1260 0x55b13dfd1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.857+0000 7fde81000740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b13dfd1ba0 0x55b13dfd6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.857+0000 7fde81000740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b13dfdb040 0x55b13dfdd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.857+0000 7fde81000740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b13df36100 con 0x55b13dfdb040 2024-09-12T01:32:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.857+0000 7fde81000740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b13dede1b0 con 0x55b13dfd1260 2024-09-12T01:32:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.857+0000 7fde81000740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b13df320c0 con 0x55b13dfd1ba0 2024-09-12T01:32:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.857+0000 7fde77400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b13dfd1ba0 0x55b13dfd6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.857+0000 7fde77400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b13dfd1ba0 0x55b13dfd6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:38654/0 (socket says 172.21.15.1:38654) 2024-09-12T01:32:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.857+0000 7fde77400640 1 -- 172.21.15.1:0/1820735276 learned_addr learned my addr 172.21.15.1:0/1820735276 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:32:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.857+0000 7fde77e00640 1 --2- 172.21.15.1:0/1820735276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b13dfd1260 0x55b13dfd1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.857+0000 7fde77400640 1 -- 172.21.15.1:0/1820735276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b13dfd1260 msgr2=0x55b13dfd1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.857+0000 7fde76a00640 1 --2- 172.21.15.1:0/1820735276 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b13dfdb040 0x55b13dfdd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.857+0000 7fde77400640 1 --2- 172.21.15.1:0/1820735276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b13dfd1260 0x55b13dfd1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.857+0000 7fde77400640 1 -- 172.21.15.1:0/1820735276 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b13dfdb040 msgr2=0x55b13dfdd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.858+0000 7fde6fe00640 1 --2- 172.21.15.1:0/4071664967 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fde5c0a2c50 0x7fde5c0a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.858+0000 7fde77400640 1 --2- 172.21.15.1:0/1820735276 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b13dfdb040 0x55b13dfdd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.858+0000 7fde77400640 1 -- 172.21.15.1:0/1820735276 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b13dc49210 con 0x55b13dfd1ba0 2024-09-12T01:32:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.858+0000 7fde76a00640 1 --2- 172.21.15.1:0/1820735276 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b13dfdb040 0x55b13dfdd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:32:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.858+0000 7fde6fe00640 1 -- 172.21.15.1:0/4071664967 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fde640ef9a0 con 0x7fde78102ba0 2024-09-12T01:32:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.858+0000 7fde77e00640 1 --2- 172.21.15.1:0/4071664967 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fde5c0a2c50 0x7fde5c0a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.858+0000 7fde77400640 1 --2- 172.21.15.1:0/1820735276 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b13dfd1ba0 0x55b13dfd6660 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fde68009c60 tx=0x7fde68011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bd8619b868090ba server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.858+0000 7fde63e00640 1 -- 172.21.15.1:0/1820735276 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde68004220 con 0x55b13dfd1ba0 2024-09-12T01:32:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.858+0000 7fde63e00640 1 -- 172.21.15.1:0/1820735276 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde680043c0 con 0x55b13dfd1ba0 2024-09-12T01:32:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.858+0000 7fde63e00640 1 -- 172.21.15.1:0/1820735276 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde68004e10 con 0x55b13dfd1ba0 2024-09-12T01:32:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.860+0000 7fde81000740 1 -- 172.21.15.1:0/1820735276 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b13dfd1ba0 msgr2=0x55b13dfd6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.860+0000 7fde77e00640 1 --2- 172.21.15.1:0/4071664967 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fde5c0a2c50 0x7fde5c0a5110 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fde70005fd0 tx=0x7fde70009740 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.860+0000 7fde81000740 1 --2- 172.21.15.1:0/1820735276 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b13dfd1ba0 0x55b13dfd6660 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fde68009c60 tx=0x7fde68011810 comp rx=0 tx=0).stop 2024-09-12T01:32:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.861+0000 7fde81000740 1 -- 172.21.15.1:0/1820735276 shutdown_connections 2024-09-12T01:32:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.861+0000 7fde81000740 1 --2- 172.21.15.1:0/1820735276 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b13dfdb040 0x55b13dfdd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.861+0000 7fde81000740 1 --2- 172.21.15.1:0/1820735276 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b13dfd1ba0 0x55b13dfd6660 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.861+0000 7fde81000740 1 --2- 172.21.15.1:0/1820735276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b13dfd1260 0x55b13dfd1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.861+0000 7fde81000740 1 -- 172.21.15.1:0/1820735276 >> 172.21.15.1:0/1820735276 conn(0x55b13dfcc8d0 msgr2=0x55b13dfcccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.861+0000 7fde81000740 1 -- 172.21.15.1:0/1820735276 shutdown_connections 2024-09-12T01:32:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.861+0000 7fde81000740 1 -- 172.21.15.1:0/1820735276 wait complete. 2024-09-12T01:32:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.862+0000 7fde81000740 1 -- start start 2024-09-12T01:32:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.862+0000 7fde81000740 1 Processor -- start 2024-09-12T01:32:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.863+0000 7fde81000740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b13dfd1260 0x55b13dfd42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.863+0000 7fde81000740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b13dfd1ba0 0x55b13dfd4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.863+0000 7fde81000740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b13dfdb040 0x55b13dfd57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.863+0000 7fde76a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b13dfd1ba0 0x55b13dfd4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.863+0000 7fde76a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b13dfd1ba0 0x55b13dfd4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:38662/0 (socket says 172.21.15.1:38662) 2024-09-12T01:32:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.863+0000 7fde81000740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b13dfd6f50 con 0x55b13dfdb040 2024-09-12T01:32:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.863+0000 7fde81000740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b13debf090 con 0x55b13dfd1260 2024-09-12T01:32:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.863+0000 7fde81000740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b13dfd70d0 con 0x55b13dfd1ba0 2024-09-12T01:32:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.863+0000 7fde77400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b13dfd1260 0x55b13dfd42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.863+0000 7fde77400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b13dfd1260 0x55b13dfd42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:55520/0 (socket says 172.21.15.1:55520) 2024-09-12T01:32:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.863+0000 7fde77400640 1 -- 172.21.15.1:0/2883280502 learned_addr learned my addr 172.21.15.1:0/2883280502 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:32:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.864+0000 7fde81000740 10 client.0 _ll_register_callbacks cb 0x55b13df13f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:32:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.864+0000 7fde77e00640 1 --2- 172.21.15.1:0/2883280502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b13dfdb040 0x55b13dfd57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.864+0000 7fde76a00640 1 -- 172.21.15.1:0/2883280502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b13dfd1260 msgr2=0x55b13dfd42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.864+0000 7fde76a00640 1 --2- 172.21.15.1:0/2883280502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b13dfd1260 0x55b13dfd42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.864+0000 7fde76a00640 1 -- 172.21.15.1:0/2883280502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b13dfdb040 msgr2=0x55b13dfd57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.864+0000 7fde76a00640 1 --2- 172.21.15.1:0/2883280502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b13dfdb040 0x55b13dfd57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.864+0000 7fde76a00640 1 -- 172.21.15.1:0/2883280502 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b13dd15620 con 0x55b13dfd1ba0 2024-09-12T01:32:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.864+0000 7fde76a00640 1 --2- 172.21.15.1:0/2883280502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b13dfd1ba0 0x55b13dfd4820 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fde64006010 tx=0x7fde6402a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=269ddb997101b1c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.864+0000 7fde60c00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-12T01:32:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.864+0000 7fde60c00640 1 -- 172.21.15.1:0/2883280502 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde640f1e60 con 0x55b13dfd1ba0 2024-09-12T01:32:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.865+0000 7fde81000740 1 -- 172.21.15.1:0/2883280502 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde680043c0 con 0x55b13dfd1ba0 2024-09-12T01:32:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.865+0000 7fde60c00640 1 -- 172.21.15.1:0/2883280502 <== mon.1 v2:172.21.15.104:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fde6403bd00 con 0x55b13dfd1ba0 2024-09-12T01:32:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.865+0000 7fde60c00640 1 -- 172.21.15.1:0/2883280502 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fde64049070 con 0x55b13dfd1ba0 2024-09-12T01:32:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.865+0000 7fde81000740 10 client.25423 fetch_fsmap learned FSMap version 32 2024-09-12T01:32:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.865+0000 7fde81000740 1 -- 172.21.15.1:0/2883280502 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b13dfd7010 con 0x55b13dfd1ba0 2024-09-12T01:32:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.865+0000 7fde60c00640 1 -- 172.21.15.1:0/2883280502 <== mon.1 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde680043c0 con 0x55b13dfd1ba0 2024-09-12T01:32:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.866+0000 7fde60c00640 1 -- 172.21.15.1:0/2883280502 <== mon.1 v2:172.21.15.104:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7fde640b83f0 con 0x55b13dfd1ba0 2024-09-12T01:32:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.866+0000 7fde81000740 10 client.25423 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T01:32:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.866+0000 7fde81000740 10 client.25423 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:32:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.866+0000 7fde81000740 20 client.25423 populate_metadata read hostname 'smithi001' 2024-09-12T01:32:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.866+0000 7fde81000740 1 --2- 172.21.15.1:0/2883280502 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55b13dfef630 0x55b13dff1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.866+0000 7fde81000740 4 client.25423 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:32:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.866+0000 7fde81000740 1 -- 172.21.15.1:0/2883280502 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b13df43830 con 0x55b13dfef630 2024-09-12T01:32:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.867+0000 7fde77400640 1 --2- 172.21.15.1:0/2883280502 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55b13dfef630 0x55b13dff1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.867+0000 7fde77400640 1 --2- 172.21.15.1:0/2883280502 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55b13dfef630 0x55b13dff1a80 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c186e6b77911ace5 server_cookie=a55317a7cf2243b0 in_seq=0 out_seq=0 2024-09-12T01:32:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.867+0000 7fde60c00640 10 client.25423 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:32:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.868+0000 7fde60c00640 1 -- 172.21.15.1:0/2883280502 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b13df43830 con 0x55b13dfef630 2024-09-12T01:32:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.868+0000 7fde60c00640 10 client.25423 handle_command_reply: tid=0 2024-09-12T01:32:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.868+0000 7fde81000740 1 client.25423 shutdown 2024-09-12T01:32:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.868+0000 7fde81000740 1 -- 172.21.15.1:0/2883280502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b13dfd1ba0 msgr2=0x55b13dfd4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.868+0000 7fde81000740 1 --2- 172.21.15.1:0/2883280502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b13dfd1ba0 0x55b13dfd4820 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fde64006010 tx=0x7fde6402a410 comp rx=0 tx=0).stop 2024-09-12T01:32:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.868+0000 7fde81000740 1 -- 172.21.15.1:0/2883280502 shutdown_connections 2024-09-12T01:32:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.868+0000 7fde81000740 1 --2- 172.21.15.1:0/2883280502 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55b13dfef630 0x55b13dff1a80 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.868+0000 7fde81000740 1 --2- 172.21.15.1:0/2883280502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b13dfdb040 0x55b13dfd57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.868+0000 7fde81000740 1 --2- 172.21.15.1:0/2883280502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b13dfd1ba0 0x55b13dfd4820 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.868+0000 7fde60c00640 0 client.25423 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:32:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.868+0000 7fde81000740 1 --2- 172.21.15.1:0/2883280502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b13dfd1260 0x55b13dfd42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.868+0000 7fde81000740 1 -- 172.21.15.1:0/2883280502 >> 172.21.15.1:0/2883280502 conn(0x55b13dfcc8d0 msgr2=0x55b13dfce0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.869+0000 7fde81000740 1 -- 172.21.15.1:0/2883280502 shutdown_connections 2024-09-12T01:32:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.869+0000 7fde81000740 1 -- 172.21.15.1:0/2883280502 wait complete. 2024-09-12T01:32:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.869+0000 7fde81000740 20 client.25423 trim_cache size 0 max 16384 2024-09-12T01:32:30.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.889+0000 7fde81000740 1 Processor -- start 2024-09-12T01:32:30.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.889+0000 7fde81000740 1 -- start start 2024-09-12T01:32:30.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.889+0000 7fde81000740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b13dfd1260 0x55b13dfcccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:30.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.889+0000 7fde81000740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b13dfd1ba0 0x55b13dfcd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:30.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.889+0000 7fde81000740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b13dfdb040 0x55b13dfcd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:30.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.889+0000 7fde81000740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fde781a8360 con 0x55b13dfdb040 2024-09-12T01:32:30.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.889+0000 7fde81000740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fde781a9f30 con 0x55b13dfd1260 2024-09-12T01:32:30.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.889+0000 7fde81000740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fde781a8a30 con 0x55b13dfd1ba0 2024-09-12T01:32:30.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.890+0000 7fde76a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b13dfd1ba0 0x55b13dfcd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:30.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.890+0000 7fde76a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b13dfd1ba0 0x55b13dfcd1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:38668/0 (socket says 172.21.15.1:38668) 2024-09-12T01:32:30.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.890+0000 7fde76a00640 1 -- 172.21.15.1:0/244582576 learned_addr learned my addr 172.21.15.1:0/244582576 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:32:30.891 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.890+0000 7fde77400640 1 --2- 172.21.15.1:0/244582576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b13dfd1260 0x55b13dfcccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:30.891 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.890+0000 7fde77e00640 1 --2- 172.21.15.1:0/244582576 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b13dfdb040 0x55b13dfcd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:30.891 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.890+0000 7fde76a00640 1 -- 172.21.15.1:0/244582576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b13dfd1260 msgr2=0x55b13dfcccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:30.891 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.890+0000 7fde76a00640 1 --2- 172.21.15.1:0/244582576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b13dfd1260 0x55b13dfcccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:30.891 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.890+0000 7fde76a00640 1 -- 172.21.15.1:0/244582576 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b13dfdb040 msgr2=0x55b13dfcd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:30.891 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.890+0000 7fde76a00640 1 --2- 172.21.15.1:0/244582576 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b13dfdb040 0x55b13dfcd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:30.891 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.890+0000 7fde76a00640 1 -- 172.21.15.1:0/244582576 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b13dfcc6f0 con 0x55b13dfd1ba0 2024-09-12T01:32:30.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.891+0000 7fde76a00640 1 --2- 172.21.15.1:0/244582576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b13dfd1ba0 0x55b13dfcd1f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fde64031440 tx=0x7fde64005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac68bb1fee0a354a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:30.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.891+0000 7fde63400640 1 -- 172.21.15.1:0/244582576 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6403b640 con 0x55b13dfd1ba0 2024-09-12T01:32:30.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.891+0000 7fde63400640 1 -- 172.21.15.1:0/244582576 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde640bc510 con 0x55b13dfd1ba0 2024-09-12T01:32:30.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.891+0000 7fde63400640 1 -- 172.21.15.1:0/244582576 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde640b83f0 con 0x55b13dfd1ba0 2024-09-12T01:32:30.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.892+0000 7fde81000740 1 -- 172.21.15.1:0/244582576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b13dfd1ba0 msgr2=0x55b13dfcd1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:30.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.892+0000 7fde81000740 1 --2- 172.21.15.1:0/244582576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b13dfd1ba0 0x55b13dfcd1f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fde64031440 tx=0x7fde64005b50 comp rx=0 tx=0).stop 2024-09-12T01:32:30.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.892+0000 7fde81000740 1 -- 172.21.15.1:0/244582576 shutdown_connections 2024-09-12T01:32:30.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.892+0000 7fde81000740 1 --2- 172.21.15.1:0/244582576 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b13dfdb040 0x55b13dfcd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:30.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.892+0000 7fde81000740 1 --2- 172.21.15.1:0/244582576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b13dfd1ba0 0x55b13dfcd1f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:30.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.892+0000 7fde81000740 1 --2- 172.21.15.1:0/244582576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b13dfd1260 0x55b13dfcccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:30.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.892+0000 7fde81000740 1 -- 172.21.15.1:0/244582576 >> 172.21.15.1:0/244582576 conn(0x55b13dfcc8d0 msgr2=0x55b13dfd7c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:30.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.892+0000 7fde81000740 1 -- 172.21.15.1:0/244582576 shutdown_connections 2024-09-12T01:32:30.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.892+0000 7fde81000740 1 -- 172.21.15.1:0/244582576 wait complete. 2024-09-12T01:32:30.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.892+0000 7fde81000740 1 -- start start 2024-09-12T01:32:30.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.893+0000 7fde81000740 1 Processor -- start 2024-09-12T01:32:30.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.893+0000 7fde81000740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b13dfd1260 0x55b13dfe96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:30.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.893+0000 7fde81000740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b13dfd1ba0 0x55b13dfe9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:30.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.893+0000 7fde81000740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b13dfdb040 0x55b13dfeac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:30.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.893+0000 7fde81000740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fde781a9830 con 0x55b13dfd1260 2024-09-12T01:32:30.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.893+0000 7fde81000740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fde781a9160 con 0x55b13dfd1ba0 2024-09-12T01:32:30.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.893+0000 7fde81000740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fde781a6340 con 0x55b13dfdb040 2024-09-12T01:32:30.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.893+0000 7fde81000740 10 client.0 _ll_register_callbacks cb 0x7fde64009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:32:30.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.894+0000 7fde77400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b13dfd1260 0x55b13dfe96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:30.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.894+0000 7fde76a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b13dfd1ba0 0x55b13dfe9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:30.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.894+0000 7fde76a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b13dfd1ba0 0x55b13dfe9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:55542/0 (socket says 172.21.15.1:55542) 2024-09-12T01:32:30.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.894+0000 7fde76a00640 1 -- 172.21.15.1:0/1305969142 learned_addr learned my addr 172.21.15.1:0/1305969142 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:32:30.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.894+0000 7fde77e00640 1 --2- 172.21.15.1:0/1305969142 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b13dfdb040 0x55b13dfeac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:30.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.894+0000 7fde77400640 1 -- 172.21.15.1:0/1305969142 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b13dfdb040 msgr2=0x55b13dfeac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:30.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.894+0000 7fde77400640 1 --2- 172.21.15.1:0/1305969142 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b13dfdb040 0x55b13dfeac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:30.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.894+0000 7fde77400640 1 -- 172.21.15.1:0/1305969142 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b13dfd1ba0 msgr2=0x55b13dfe9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:30.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.894+0000 7fde77400640 1 --2- 172.21.15.1:0/1305969142 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b13dfd1ba0 0x55b13dfe9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:30.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.894+0000 7fde77400640 1 -- 172.21.15.1:0/1305969142 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b13dfeb220 con 0x55b13dfd1260 2024-09-12T01:32:30.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.895+0000 7fde60c00640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-12T01:32:30.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.895+0000 7fde77400640 1 --2- 172.21.15.1:0/1305969142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b13dfd1260 0x55b13dfe96c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x55b13dfd38b0 tx=0x7fde68004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a3aa700387df54 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:30.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.895+0000 7fde60c00640 1 -- 172.21.15.1:0/1305969142 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6801d040 con 0x55b13dfd1260 2024-09-12T01:32:30.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.895+0000 7fde60c00640 1 -- 172.21.15.1:0/1305969142 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fde68025640 con 0x55b13dfd1260 2024-09-12T01:32:30.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.895+0000 7fde81000740 1 -- 172.21.15.1:0/1305969142 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde640bc510 con 0x55b13dfd1260 2024-09-12T01:32:30.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.895+0000 7fde60c00640 1 -- 172.21.15.1:0/1305969142 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fde6803b050 con 0x55b13dfd1260 2024-09-12T01:32:30.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.896+0000 7fde81000740 10 client.15390 fetch_fsmap learned FSMap version 32 2024-09-12T01:32:30.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.896+0000 7fde81000740 1 -- 172.21.15.1:0/1305969142 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b13e05a360 con 0x55b13dfd1260 2024-09-12T01:32:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.896+0000 7fde60c00640 1 -- 172.21.15.1:0/1305969142 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde640bc510 con 0x55b13dfd1260 2024-09-12T01:32:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.896+0000 7fde60c00640 1 -- 172.21.15.1:0/1305969142 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7fde68023040 con 0x55b13dfd1260 2024-09-12T01:32:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.896+0000 7fde81000740 10 client.15390 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T01:32:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.896+0000 7fde81000740 10 client.15390 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:32:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.896+0000 7fde81000740 20 client.15390 populate_metadata read hostname 'smithi001' 2024-09-12T01:32:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.896+0000 7fde81000740 1 --2- 172.21.15.1:0/1305969142 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55b13e05ab20 0x55b13e05cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.896+0000 7fde81000740 4 client.15390 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:32:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.896+0000 7fde81000740 1 -- 172.21.15.1:0/1305969142 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b13df3db80 con 0x55b13e05ab20 2024-09-12T01:32:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.897+0000 7fde76a00640 1 --2- 172.21.15.1:0/1305969142 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55b13e05ab20 0x55b13e05cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.897+0000 7fde76a00640 1 --2- 172.21.15.1:0/1305969142 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55b13e05ab20 0x55b13e05cf40 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9c700a1c8dc84b4 server_cookie=6fda5829903c7379 in_seq=0 out_seq=0 2024-09-12T01:32:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.897+0000 7fde60c00640 10 client.15390 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:32:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.906+0000 7fde60c00640 1 -- 172.21.15.1:0/1305969142 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b13df3db80 con 0x55b13e05ab20 2024-09-12T01:32:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.906+0000 7fde60c00640 10 client.15390 handle_command_reply: tid=0 2024-09-12T01:32:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.906+0000 7fde81000740 1 client.15390 shutdown 2024-09-12T01:32:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.906+0000 7fde81000740 1 -- 172.21.15.1:0/1305969142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b13dfd1260 msgr2=0x55b13dfe96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.906+0000 7fde81000740 1 --2- 172.21.15.1:0/1305969142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b13dfd1260 0x55b13dfe96c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x55b13dfd38b0 tx=0x7fde68004570 comp rx=0 tx=0).stop 2024-09-12T01:32:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.906+0000 7fde81000740 1 -- 172.21.15.1:0/1305969142 shutdown_connections 2024-09-12T01:32:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.906+0000 7fde81000740 1 --2- 172.21.15.1:0/1305969142 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55b13e05ab20 0x55b13e05cf40 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.906+0000 7fde81000740 1 --2- 172.21.15.1:0/1305969142 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b13dfdb040 0x55b13dfeac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.906+0000 7fde81000740 1 --2- 172.21.15.1:0/1305969142 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b13dfd1ba0 0x55b13dfe9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.906+0000 7fde81000740 1 --2- 172.21.15.1:0/1305969142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b13dfd1260 0x55b13dfe96c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.906+0000 7fde81000740 1 -- 172.21.15.1:0/1305969142 >> 172.21.15.1:0/1305969142 conn(0x55b13dfcc8d0 msgr2=0x55b13dfcb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.906+0000 7fde60c00640 0 client.15390 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:32:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.907+0000 7fde81000740 1 -- 172.21.15.1:0/1305969142 shutdown_connections 2024-09-12T01:32:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.907+0000 7fde81000740 1 -- 172.21.15.1:0/1305969142 wait complete. 2024-09-12T01:32:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.907+0000 7fde81000740 20 client.15390 trim_cache size 0 max 16384 2024-09-12T01:32:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-12T01:32:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-12T01:32:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-12T01:32:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-12T01:32:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.908+0000 7fde7cc00640 1 -- 172.21.15.1:0/4071664967 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fde5c0a2c50 msgr2=0x7fde5c0a5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.908+0000 7fde7cc00640 1 --2- 172.21.15.1:0/4071664967 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fde5c0a2c50 0x7fde5c0a5110 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fde70005fd0 tx=0x7fde70009740 comp rx=0 tx=0).stop 2024-09-12T01:32:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.908+0000 7fde7cc00640 1 -- 172.21.15.1:0/4071664967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde78102ba0 msgr2=0x7fde781981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.908+0000 7fde7cc00640 1 --2- 172.21.15.1:0/4071664967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde78102ba0 0x7fde781981c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fde64031610 tx=0x7fde64032c80 comp rx=0 tx=0).stop 2024-09-12T01:32:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.908+0000 7fde7cc00640 1 -- 172.21.15.1:0/4071664967 shutdown_connections 2024-09-12T01:32:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.908+0000 7fde7cc00640 1 --2- 172.21.15.1:0/4071664967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde78103560 0x7fde7819f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.908+0000 7fde7cc00640 1 --2- 172.21.15.1:0/4071664967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde78102ba0 0x7fde781981c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.908+0000 7fde7cc00640 1 --2- 172.21.15.1:0/4071664967 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fde5c0a2c50 0x7fde5c0a5110 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.908+0000 7fde7cc00640 1 --2- 172.21.15.1:0/4071664967 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde781019b0 0x7fde78197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.908+0000 7fde7cc00640 1 -- 172.21.15.1:0/4071664967 >> 172.21.15.1:0/4071664967 conn(0x7fde780fd130 msgr2=0x7fde78105ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.908+0000 7fde7cc00640 1 -- 172.21.15.1:0/4071664967 shutdown_connections 2024-09-12T01:32:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:30.908+0000 7fde7cc00640 1 -- 172.21.15.1:0/4071664967 wait complete. 2024-09-12T01:32:30.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:32:30 smithi001 ceph-mon[30387]: pgmap v485: 129 pgs: 129 active+clean; 75 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 429 op/s 2024-09-12T01:32:30.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:32:30 smithi053 ceph-mon[35158]: pgmap v485: 129 pgs: 129 active+clean; 75 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 429 op/s 2024-09-12T01:32:31.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:32:30 smithi104 ceph-mon[35219]: pgmap v485: 129 pgs: 129 active+clean; 75 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 429 op/s 2024-09-12T01:32:32.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:32:32 smithi001 ceph-mon[30387]: pgmap v486: 129 pgs: 129 active+clean; 75 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.5 MiB/s wr, 541 op/s 2024-09-12T01:32:32.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:32:32 smithi053 ceph-mon[35158]: pgmap v486: 129 pgs: 129 active+clean; 75 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.5 MiB/s wr, 541 op/s 2024-09-12T01:32:33.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:32:32 smithi104 ceph-mon[35219]: pgmap v486: 129 pgs: 129 active+clean; 75 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.5 MiB/s wr, 541 op/s 2024-09-12T01:32:34.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:32:34 smithi001 ceph-mon[30387]: pgmap v487: 129 pgs: 129 active+clean; 76 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 450 op/s 2024-09-12T01:32:34.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:32:34 smithi053 ceph-mon[35158]: pgmap v487: 129 pgs: 129 active+clean; 76 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 450 op/s 2024-09-12T01:32:35.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:32:34 smithi104 ceph-mon[35219]: pgmap v487: 129 pgs: 129 active+clean; 76 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 450 op/s 2024-09-12T01:32:36.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:32:36 smithi001 ceph-mon[30387]: pgmap v488: 129 pgs: 129 active+clean; 76 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 477 op/s 2024-09-12T01:32:36.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:32:36 smithi053 ceph-mon[35158]: pgmap v488: 129 pgs: 129 active+clean; 76 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 477 op/s 2024-09-12T01:32:37.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:32:36 smithi104 ceph-mon[35219]: pgmap v488: 129 pgs: 129 active+clean; 76 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 477 op/s 2024-09-12T01:32:38.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:32:38 smithi001 ceph-mon[30387]: pgmap v489: 129 pgs: 129 active+clean; 77 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 406 op/s 2024-09-12T01:32:38.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:32:38 smithi053 ceph-mon[35158]: pgmap v489: 129 pgs: 129 active+clean; 77 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 406 op/s 2024-09-12T01:32:39.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:32:38 smithi104 ceph-mon[35219]: pgmap v489: 129 pgs: 129 active+clean; 77 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 406 op/s 2024-09-12T01:32:40.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:32:40 smithi001 ceph-mon[30387]: pgmap v490: 129 pgs: 129 active+clean; 77 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 337 op/s 2024-09-12T01:32:40.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:32:40 smithi053 ceph-mon[35158]: pgmap v490: 129 pgs: 129 active+clean; 77 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 337 op/s 2024-09-12T01:32:41.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:32:40 smithi104 ceph-mon[35219]: pgmap v490: 129 pgs: 129 active+clean; 77 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 337 op/s 2024-09-12T01:32:42.245 INFO:tasks.workunit.client.0.smithi001.stdout:30000 trunc from 0x7bb2f to 0xcdf83 2024-09-12T01:32:42.820 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0xffcdd 2024-09-12T01:32:42.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:32:42 smithi001 ceph-mon[30387]: pgmap v491: 129 pgs: 129 active+clean; 77 MiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 435 op/s 2024-09-12T01:32:42.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:32:42 smithi053 ceph-mon[35158]: pgmap v491: 129 pgs: 129 active+clean; 77 MiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 435 op/s 2024-09-12T01:32:43.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:32:42 smithi104 ceph-mon[35219]: pgmap v491: 129 pgs: 129 active+clean; 77 MiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 435 op/s 2024-09-12T01:32:44.052 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.052+0000 7f8a2a200640 1 -- 172.21.15.1:0/3697687099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a24101b30 msgr2=0x7f8a24101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:44.052 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.052+0000 7f8a2a200640 1 --2- 172.21.15.1:0/3697687099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a24101b30 0x7f8a24101f30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8a14009860 tx=0x7f8a1402a340 comp rx=0 tx=0).stop 2024-09-12T01:32:44.052 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.052+0000 7f8a2a200640 1 -- 172.21.15.1:0/3697687099 shutdown_connections 2024-09-12T01:32:44.053 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.052+0000 7f8a2a200640 1 --2- 172.21.15.1:0/3697687099 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a241036e0 0x7f8a24109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:44.053 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.052+0000 7f8a2a200640 1 --2- 172.21.15.1:0/3697687099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a24102d20 0x7f8a241031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:44.053 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.052+0000 7f8a2a200640 1 --2- 172.21.15.1:0/3697687099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a24101b30 0x7f8a24101f30 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:44.053 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.052+0000 7f8a2a200640 1 -- 172.21.15.1:0/3697687099 >> 172.21.15.1:0/3697687099 conn(0x7f8a240fd2b0 msgr2=0x7f8a240ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:44.053 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.052+0000 7f8a2a200640 1 -- 172.21.15.1:0/3697687099 shutdown_connections 2024-09-12T01:32:44.053 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.052+0000 7f8a2a200640 1 -- 172.21.15.1:0/3697687099 wait complete. 2024-09-12T01:32:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.053+0000 7f8a2a200640 1 Processor -- start 2024-09-12T01:32:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.053+0000 7f8a2a200640 1 -- start start 2024-09-12T01:32:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.054+0000 7f8a2a200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a24101b30 0x7f8a24075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.054+0000 7f8a2a200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a24102d20 0x7f8a240745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.054+0000 7f8a2a200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a241036e0 0x7f8a24074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.054+0000 7f8a2a200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2410d790 con 0x7f8a24102d20 2024-09-12T01:32:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.054+0000 7f8a2a200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2410d610 con 0x7f8a241036e0 2024-09-12T01:32:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.054+0000 7f8a2a200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2410d910 con 0x7f8a24101b30 2024-09-12T01:32:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.054+0000 7f8a23e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a241036e0 0x7f8a24074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.054+0000 7f8a22a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a24102d20 0x7f8a240745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.054+0000 7f8a23e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a241036e0 0x7f8a24074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:38192/0 (socket says 172.21.15.1:38192) 2024-09-12T01:32:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.054+0000 7f8a23e00640 1 -- 172.21.15.1:0/3804203186 learned_addr learned my addr 172.21.15.1:0/3804203186 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:32:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.054+0000 7f8a23400640 1 --2- 172.21.15.1:0/3804203186 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a24101b30 0x7f8a24075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.055+0000 7f8a22a00640 1 -- 172.21.15.1:0/3804203186 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a24101b30 msgr2=0x7f8a24075fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.055+0000 7f8a22a00640 1 --2- 172.21.15.1:0/3804203186 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a24101b30 0x7f8a24075fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.055+0000 7f8a22a00640 1 -- 172.21.15.1:0/3804203186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a241036e0 msgr2=0x7f8a24074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.055+0000 7f8a22a00640 1 --2- 172.21.15.1:0/3804203186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a241036e0 0x7f8a24074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.055+0000 7f8a22a00640 1 -- 172.21.15.1:0/3804203186 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a240753b0 con 0x7f8a24102d20 2024-09-12T01:32:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.055+0000 7f8a22a00640 1 --2- 172.21.15.1:0/3804203186 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a24102d20 0x7f8a240745e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8a1000e9f0 tx=0x7f8a1000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.056+0000 7f8a1be00640 1 -- 172.21.15.1:0/3804203186 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1000ccc0 con 0x7f8a24102d20 2024-09-12T01:32:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.056+0000 7f8a1be00640 1 -- 172.21.15.1:0/3804203186 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a10010910 con 0x7f8a24102d20 2024-09-12T01:32:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.056+0000 7f8a2a200640 1 -- 172.21.15.1:0/3804203186 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a241a1fc0 con 0x7f8a24102d20 2024-09-12T01:32:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.056+0000 7f8a1be00640 1 -- 172.21.15.1:0/3804203186 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a10018960 con 0x7f8a24102d20 2024-09-12T01:32:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.056+0000 7f8a2a200640 1 -- 172.21.15.1:0/3804203186 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a241a2420 con 0x7f8a24102d20 2024-09-12T01:32:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.060+0000 7f8a2cfee740 1 Processor -- start 2024-09-12T01:32:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.060+0000 7f8a2cfee740 1 -- start start 2024-09-12T01:32:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.060+0000 7f8a2cfee740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d1bdc090 0x55f8d1bdc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.060+0000 7f8a2cfee740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8d1bdc9d0 0x55f8d1be1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.060+0000 7f8a2cfee740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f8d1be1a60 0x55f8d1be3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.060+0000 7f8a2cfee740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f8d1b41100 con 0x55f8d1bdc090 2024-09-12T01:32:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.060+0000 7f8a2cfee740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55f8d1ae91b0 con 0x55f8d1be1a60 2024-09-12T01:32:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.060+0000 7f8a2cfee740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f8d1b3d0c0 con 0x55f8d1bdc9d0 2024-09-12T01:32:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.061+0000 7f8a23400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8d1bdc9d0 0x55f8d1be1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.061+0000 7f8a23400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8d1bdc9d0 0x55f8d1be1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:44180/0 (socket says 172.21.15.1:44180) 2024-09-12T01:32:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.061+0000 7f8a23400640 1 -- 172.21.15.1:0/2239650153 learned_addr learned my addr 172.21.15.1:0/2239650153 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:32:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.061+0000 7f8a22a00640 1 --2- 172.21.15.1:0/2239650153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f8d1be1a60 0x55f8d1be3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.061+0000 7f8a1be00640 1 -- 172.21.15.1:0/3804203186 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f8a10010ac0 con 0x7f8a24102d20 2024-09-12T01:32:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.061+0000 7f8a23400640 1 -- 172.21.15.1:0/2239650153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f8d1be1a60 msgr2=0x55f8d1be3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.061+0000 7f8a23400640 1 --2- 172.21.15.1:0/2239650153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f8d1be1a60 0x55f8d1be3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.061+0000 7f8a23400640 1 -- 172.21.15.1:0/2239650153 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d1bdc090 msgr2=0x55f8d1bdc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.061+0000 7f8a23400640 1 --2- 172.21.15.1:0/2239650153 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d1bdc090 0x55f8d1bdc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.061+0000 7f8a23400640 1 -- 172.21.15.1:0/2239650153 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8d1854210 con 0x55f8d1bdc9d0 2024-09-12T01:32:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.061+0000 7f8a23400640 1 --2- 172.21.15.1:0/2239650153 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8d1bdc9d0 0x55f8d1be1490 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8a14009830 tx=0x7f8a14009520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3af6ca73d4f0a150 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.062+0000 7f8a0fe00640 1 -- 172.21.15.1:0/2239650153 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a14037040 con 0x55f8d1bdc9d0 2024-09-12T01:32:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.062+0000 7f8a0fe00640 1 -- 172.21.15.1:0/2239650153 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a14030c50 con 0x55f8d1bdc9d0 2024-09-12T01:32:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.062+0000 7f8a0fe00640 1 -- 172.21.15.1:0/2239650153 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a14004330 con 0x55f8d1bdc9d0 2024-09-12T01:32:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.062+0000 7f8a2cfee740 1 -- 172.21.15.1:0/2239650153 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8d1bdc9d0 msgr2=0x55f8d1be1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.062+0000 7f8a2cfee740 1 --2- 172.21.15.1:0/2239650153 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8d1bdc9d0 0x55f8d1be1490 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8a14009830 tx=0x7f8a14009520 comp rx=0 tx=0).stop 2024-09-12T01:32:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.063+0000 7f8a2cfee740 1 -- 172.21.15.1:0/2239650153 shutdown_connections 2024-09-12T01:32:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.063+0000 7f8a2cfee740 1 --2- 172.21.15.1:0/2239650153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f8d1be1a60 0x55f8d1be3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.063+0000 7f8a2cfee740 1 --2- 172.21.15.1:0/2239650153 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8d1bdc9d0 0x55f8d1be1490 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.063+0000 7f8a2cfee740 1 --2- 172.21.15.1:0/2239650153 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d1bdc090 0x55f8d1bdc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.063+0000 7f8a2cfee740 1 -- 172.21.15.1:0/2239650153 >> 172.21.15.1:0/2239650153 conn(0x55f8d1bd7700 msgr2=0x55f8d1bd7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.063+0000 7f8a1be00640 1 --2- 172.21.15.1:0/3804203186 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f8a080a2c00 0x7f8a080a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.063+0000 7f8a2cfee740 1 -- 172.21.15.1:0/2239650153 shutdown_connections 2024-09-12T01:32:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.063+0000 7f8a1be00640 1 -- 172.21.15.1:0/3804203186 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f8a100cf6f0 con 0x7f8a24102d20 2024-09-12T01:32:44.067 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.063+0000 7f8a2cfee740 1 -- 172.21.15.1:0/2239650153 wait complete. 2024-09-12T01:32:44.067 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.063+0000 7f8a23400640 1 --2- 172.21.15.1:0/3804203186 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f8a080a2c00 0x7f8a080a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:44.067 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.064+0000 7f8a2cfee740 1 -- start start 2024-09-12T01:32:44.068 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.064+0000 7f8a2cfee740 1 Processor -- start 2024-09-12T01:32:44.068 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.065+0000 7f8a23400640 1 --2- 172.21.15.1:0/3804203186 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f8a080a2c00 0x7f8a080a50c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8a14035040 tx=0x7f8a1402a340 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:44.068 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.065+0000 7f8a2cfee740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8d1bdc090 0x55f8d1be6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:44.068 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.065+0000 7f8a2cfee740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f8d1bdc9d0 0x55f8d1be6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:44.068 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.065+0000 7f8a2cfee740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d1be1a60 0x55f8d1bed7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:44.068 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.065+0000 7f8a2cfee740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f8d1be49d0 con 0x55f8d1be1a60 2024-09-12T01:32:44.068 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.065+0000 7f8a2cfee740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55f8d1aca090 con 0x55f8d1bdc9d0 2024-09-12T01:32:44.068 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.065+0000 7f8a2cfee740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f8d1be4b50 con 0x55f8d1bdc090 2024-09-12T01:32:44.068 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.065+0000 7f8a2cfee740 10 client.0 _ll_register_callbacks cb 0x55f8d1b1ef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:32:44.068 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.065+0000 7f8a23400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8d1bdc090 0x55f8d1be6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:44.068 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.065+0000 7f8a23400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8d1bdc090 0x55f8d1be6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:44196/0 (socket says 172.21.15.1:44196) 2024-09-12T01:32:44.069 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.065+0000 7f8a23400640 1 -- 172.21.15.1:0/1993054944 learned_addr learned my addr 172.21.15.1:0/1993054944 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:32:44.069 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.065+0000 7f8a23e00640 1 --2- 172.21.15.1:0/1993054944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d1be1a60 0x55f8d1bed7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:44.069 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.066+0000 7f8a23400640 1 -- 172.21.15.1:0/1993054944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f8d1bdc9d0 msgr2=0x55f8d1be6780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T01:32:44.069 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.066+0000 7f8a23400640 1 --2- 172.21.15.1:0/1993054944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f8d1bdc9d0 0x55f8d1be6780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:44.069 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.066+0000 7f8a23400640 1 -- 172.21.15.1:0/1993054944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d1be1a60 msgr2=0x55f8d1bed7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:44.069 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.066+0000 7f8a23400640 1 --2- 172.21.15.1:0/1993054944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d1be1a60 0x55f8d1bed7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:44.069 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.066+0000 7f8a23400640 1 -- 172.21.15.1:0/1993054944 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8d1920620 con 0x55f8d1bdc090 2024-09-12T01:32:44.069 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.066+0000 7f8a23e00640 1 --2- 172.21.15.1:0/1993054944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d1be1a60 0x55f8d1bed7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:32:44.069 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.066+0000 7f8a23400640 1 --2- 172.21.15.1:0/1993054944 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8d1bdc090 0x55f8d1be6240 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8a14000c00 tx=0x7f8a14009520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50dcf208efb9eb25 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:44.069 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.066+0000 7f8a0cc00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-12T01:32:44.069 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.066+0000 7f8a0cc00640 1 -- 172.21.15.1:0/1993054944 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1404f070 con 0x55f8d1bdc090 2024-09-12T01:32:44.070 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.066+0000 7f8a2cfee740 1 -- 172.21.15.1:0/1993054944 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a14030c50 con 0x55f8d1bdc090 2024-09-12T01:32:44.070 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.066+0000 7f8a0cc00640 1 -- 172.21.15.1:0/1993054944 <== mon.1 v2:172.21.15.104:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f8a14005260 con 0x55f8d1bdc090 2024-09-12T01:32:44.070 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.067+0000 7f8a0cc00640 1 -- 172.21.15.1:0/1993054944 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8a14075050 con 0x55f8d1bdc090 2024-09-12T01:32:44.070 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.067+0000 7f8a2cfee740 10 client.25441 fetch_fsmap learned FSMap version 32 2024-09-12T01:32:44.070 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.067+0000 7f8a2cfee740 1 -- 172.21.15.1:0/1993054944 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f8d1be4a20 con 0x55f8d1bdc090 2024-09-12T01:32:44.070 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.067+0000 7f8a0cc00640 1 -- 172.21.15.1:0/1993054944 <== mon.1 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a14030c50 con 0x55f8d1bdc090 2024-09-12T01:32:44.070 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.067+0000 7f8a0cc00640 1 -- 172.21.15.1:0/1993054944 <== mon.1 v2:172.21.15.104:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f8a14003710 con 0x55f8d1bdc090 2024-09-12T01:32:44.070 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.067+0000 7f8a2cfee740 10 client.25441 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T01:32:44.070 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.067+0000 7f8a2cfee740 10 client.25441 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:32:44.070 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.067+0000 7f8a2cfee740 20 client.25441 populate_metadata read hostname 'smithi001' 2024-09-12T01:32:44.070 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.067+0000 7f8a2cfee740 1 --2- 172.21.15.1:0/1993054944 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55f8d1bf18b0 0x55f8d1bf3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.067+0000 7f8a2cfee740 4 client.25441 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:32:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.067+0000 7f8a2cfee740 1 -- 172.21.15.1:0/1993054944 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8d1b4e830 con 0x55f8d1bf18b0 2024-09-12T01:32:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.068+0000 7f8a22a00640 1 --2- 172.21.15.1:0/1993054944 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55f8d1bf18b0 0x55f8d1bf3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.068+0000 7f8a22a00640 1 --2- 172.21.15.1:0/1993054944 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55f8d1bf18b0 0x55f8d1bf3cc0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b64e4f37452f14bc server_cookie=1f4a90a1b3911694 in_seq=0 out_seq=0 2024-09-12T01:32:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.068+0000 7f8a0cc00640 10 client.25441 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:32:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.069+0000 7f8a0cc00640 1 -- 172.21.15.1:0/1993054944 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f8d1b4e830 con 0x55f8d1bf18b0 2024-09-12T01:32:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.069+0000 7f8a0cc00640 10 client.25441 handle_command_reply: tid=0 2024-09-12T01:32:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.069+0000 7f8a2cfee740 1 client.25441 shutdown 2024-09-12T01:32:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.069+0000 7f8a2cfee740 1 -- 172.21.15.1:0/1993054944 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8d1bdc090 msgr2=0x55f8d1be6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.069+0000 7f8a2cfee740 1 --2- 172.21.15.1:0/1993054944 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8d1bdc090 0x55f8d1be6240 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8a14000c00 tx=0x7f8a14009520 comp rx=0 tx=0).stop 2024-09-12T01:32:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.069+0000 7f8a2cfee740 1 -- 172.21.15.1:0/1993054944 shutdown_connections 2024-09-12T01:32:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.069+0000 7f8a2cfee740 1 --2- 172.21.15.1:0/1993054944 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55f8d1bf18b0 0x55f8d1bf3cc0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.069+0000 7f8a2cfee740 1 --2- 172.21.15.1:0/1993054944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d1be1a60 0x55f8d1bed7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.069+0000 7f8a2cfee740 1 --2- 172.21.15.1:0/1993054944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f8d1bdc9d0 0x55f8d1be6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.069+0000 7f8a2cfee740 1 --2- 172.21.15.1:0/1993054944 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8d1bdc090 0x55f8d1be6240 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.069+0000 7f8a2cfee740 1 -- 172.21.15.1:0/1993054944 >> 172.21.15.1:0/1993054944 conn(0x55f8d1bd7700 msgr2=0x55f8d1bd8f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.070+0000 7f8a0cc00640 0 client.25441 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:32:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.070+0000 7f8a2cfee740 1 -- 172.21.15.1:0/1993054944 shutdown_connections 2024-09-12T01:32:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.070+0000 7f8a2cfee740 1 -- 172.21.15.1:0/1993054944 wait complete. 2024-09-12T01:32:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.070+0000 7f8a2cfee740 20 client.25441 trim_cache size 0 max 16384 2024-09-12T01:32:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.088+0000 7f8a2cfee740 1 Processor -- start 2024-09-12T01:32:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.088+0000 7f8a2cfee740 1 -- start start 2024-09-12T01:32:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.089+0000 7f8a2cfee740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8d1bdc090 0x55f8d1bd6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.089+0000 7f8a2cfee740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f8d1bdc9d0 0x55f8d1bed780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.089+0000 7f8a2cfee740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d1be1a60 0x55f8d1bd65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.089+0000 7f8a2cfee740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8a24071b10 con 0x55f8d1be1a60 2024-09-12T01:32:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.089+0000 7f8a2cfee740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8a240736e0 con 0x55f8d1bdc9d0 2024-09-12T01:32:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.089+0000 7f8a2cfee740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8a240721e0 con 0x55f8d1bdc090 2024-09-12T01:32:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.089+0000 7f8a23e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d1be1a60 0x55f8d1bd65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.089+0000 7f8a23e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d1be1a60 0x55f8d1bd65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38406/0 (socket says 172.21.15.1:38406) 2024-09-12T01:32:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.089+0000 7f8a23e00640 1 -- 172.21.15.1:0/1818291463 learned_addr learned my addr 172.21.15.1:0/1818291463 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:32:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.089+0000 7f8a23400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8d1bdc090 0x55f8d1bd6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.089+0000 7f8a22a00640 1 --2- 172.21.15.1:0/1818291463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f8d1bdc9d0 0x55f8d1bed780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.089+0000 7f8a23400640 1 -- 172.21.15.1:0/1818291463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f8d1bdc9d0 msgr2=0x55f8d1bed780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.089+0000 7f8a23400640 1 --2- 172.21.15.1:0/1818291463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f8d1bdc9d0 0x55f8d1bed780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.089+0000 7f8a23400640 1 -- 172.21.15.1:0/1818291463 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d1be1a60 msgr2=0x55f8d1bd65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.089+0000 7f8a23400640 1 --2- 172.21.15.1:0/1818291463 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d1be1a60 0x55f8d1bd65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.089+0000 7f8a23400640 1 -- 172.21.15.1:0/1818291463 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8d1bd7520 con 0x55f8d1bdc090 2024-09-12T01:32:44.092 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.090+0000 7f8a23400640 1 --2- 172.21.15.1:0/1818291463 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8d1bdc090 0x55f8d1bd6060 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8a14003cf0 tx=0x7f8a14009590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c76ae21956a2b17 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:44.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.090+0000 7f8a0f400640 1 -- 172.21.15.1:0/1818291463 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1404f040 con 0x55f8d1bdc090 2024-09-12T01:32:44.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.090+0000 7f8a0f400640 1 -- 172.21.15.1:0/1818291463 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a1406c070 con 0x55f8d1bdc090 2024-09-12T01:32:44.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.090+0000 7f8a0f400640 1 -- 172.21.15.1:0/1818291463 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a14037040 con 0x55f8d1bdc090 2024-09-12T01:32:44.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.091+0000 7f8a2cfee740 1 -- 172.21.15.1:0/1818291463 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8d1bdc090 msgr2=0x55f8d1bd6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:44.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.091+0000 7f8a2cfee740 1 --2- 172.21.15.1:0/1818291463 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8d1bdc090 0x55f8d1bd6060 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8a14003cf0 tx=0x7f8a14009590 comp rx=0 tx=0).stop 2024-09-12T01:32:44.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.091+0000 7f8a2cfee740 1 -- 172.21.15.1:0/1818291463 shutdown_connections 2024-09-12T01:32:44.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.091+0000 7f8a2cfee740 1 --2- 172.21.15.1:0/1818291463 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d1be1a60 0x55f8d1bd65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:44.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.091+0000 7f8a2cfee740 1 --2- 172.21.15.1:0/1818291463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f8d1bdc9d0 0x55f8d1bed780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:44.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.091+0000 7f8a2cfee740 1 --2- 172.21.15.1:0/1818291463 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8d1bdc090 0x55f8d1bd6060 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:44.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.091+0000 7f8a2cfee740 1 -- 172.21.15.1:0/1818291463 >> 172.21.15.1:0/1818291463 conn(0x55f8d1bd7700 msgr2=0x55f8d1be5ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:44.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.091+0000 7f8a2cfee740 1 -- 172.21.15.1:0/1818291463 shutdown_connections 2024-09-12T01:32:44.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.091+0000 7f8a2cfee740 1 -- 172.21.15.1:0/1818291463 wait complete. 2024-09-12T01:32:44.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.091+0000 7f8a2cfee740 1 -- start start 2024-09-12T01:32:44.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.092+0000 7f8a2cfee740 1 Processor -- start 2024-09-12T01:32:44.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.092+0000 7f8a2cfee740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8d1bdc090 0x55f8d1beaea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:44.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.092+0000 7f8a2cfee740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d1bdc9d0 0x55f8d1beb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:44.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.092+0000 7f8a2cfee740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f8d1be1a60 0x55f8d1bec380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:44.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.092+0000 7f8a2cfee740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8a24072fe0 con 0x55f8d1bdc9d0 2024-09-12T01:32:44.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.092+0000 7f8a2cfee740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8a24072910 con 0x55f8d1be1a60 2024-09-12T01:32:44.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.092+0000 7f8a2cfee740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2406faf0 con 0x55f8d1bdc090 2024-09-12T01:32:44.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.092+0000 7f8a2cfee740 10 client.0 _ll_register_callbacks cb 0x7f8a100d5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:32:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.092+0000 7f8a22a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d1bdc9d0 0x55f8d1beb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.092+0000 7f8a23e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f8d1be1a60 0x55f8d1bec380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.092+0000 7f8a23400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8d1bdc090 0x55f8d1beaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.092+0000 7f8a22a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d1bdc9d0 0x55f8d1beb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38422/0 (socket says 172.21.15.1:38422) 2024-09-12T01:32:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.092+0000 7f8a22a00640 1 -- 172.21.15.1:0/404886821 learned_addr learned my addr 172.21.15.1:0/404886821 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:32:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.093+0000 7f8a23400640 1 -- 172.21.15.1:0/404886821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f8d1be1a60 msgr2=0x55f8d1bec380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.093+0000 7f8a23400640 1 --2- 172.21.15.1:0/404886821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f8d1be1a60 0x55f8d1bec380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.093+0000 7f8a23400640 1 -- 172.21.15.1:0/404886821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d1bdc9d0 msgr2=0x55f8d1beb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.093+0000 7f8a23400640 1 --2- 172.21.15.1:0/404886821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d1bdc9d0 0x55f8d1beb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.093+0000 7f8a23400640 1 -- 172.21.15.1:0/404886821 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8d1bec990 con 0x55f8d1bdc090 2024-09-12T01:32:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.093+0000 7f8a23400640 1 --2- 172.21.15.1:0/404886821 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8d1bdc090 0x55f8d1beaea0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8a14003b80 tx=0x7f8a14030570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43ebc9844a48186c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.093+0000 7f8a0cc00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-12T01:32:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.093+0000 7f8a0cc00640 1 -- 172.21.15.1:0/404886821 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1404f040 con 0x55f8d1bdc090 2024-09-12T01:32:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.093+0000 7f8a0cc00640 1 -- 172.21.15.1:0/404886821 <== mon.1 v2:172.21.15.104:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f8a14003800 con 0x55f8d1bdc090 2024-09-12T01:32:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.093+0000 7f8a2cfee740 1 -- 172.21.15.1:0/404886821 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a1406c070 con 0x55f8d1bdc090 2024-09-12T01:32:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.094+0000 7f8a0cc00640 1 -- 172.21.15.1:0/404886821 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8a14084050 con 0x55f8d1bdc090 2024-09-12T01:32:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.094+0000 7f8a2cfee740 10 client.25447 fetch_fsmap learned FSMap version 32 2024-09-12T01:32:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.094+0000 7f8a2cfee740 1 -- 172.21.15.1:0/404886821 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f8d1c5c5a0 con 0x55f8d1bdc090 2024-09-12T01:32:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.094+0000 7f8a0cc00640 1 -- 172.21.15.1:0/404886821 <== mon.1 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a1406c070 con 0x55f8d1bdc090 2024-09-12T01:32:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.094+0000 7f8a0cc00640 1 -- 172.21.15.1:0/404886821 <== mon.1 v2:172.21.15.104:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f8a140674d0 con 0x55f8d1bdc090 2024-09-12T01:32:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.094+0000 7f8a2cfee740 10 client.25447 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T01:32:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.094+0000 7f8a2cfee740 10 client.25447 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:32:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.094+0000 7f8a2cfee740 20 client.25447 populate_metadata read hostname 'smithi001' 2024-09-12T01:32:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.094+0000 7f8a2cfee740 1 --2- 172.21.15.1:0/404886821 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55f8d1c5ce60 0x55f8d1c5f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.094+0000 7f8a2cfee740 4 client.25447 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:32:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.094+0000 7f8a2cfee740 1 -- 172.21.15.1:0/404886821 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f8d1b48b80 con 0x55f8d1c5ce60 2024-09-12T01:32:44.099 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.095+0000 7f8a22a00640 1 --2- 172.21.15.1:0/404886821 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55f8d1c5ce60 0x55f8d1c5f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:44.099 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.095+0000 7f8a22a00640 1 --2- 172.21.15.1:0/404886821 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55f8d1c5ce60 0x55f8d1c5f280 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b611886f19e50508 server_cookie=cb301b0dd9c35cf5 in_seq=0 out_seq=0 2024-09-12T01:32:44.099 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.095+0000 7f8a0cc00640 10 client.25447 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:32:44.102 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.102+0000 7f8a0cc00640 1 -- 172.21.15.1:0/404886821 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f8d1b48b80 con 0x55f8d1c5ce60 2024-09-12T01:32:44.102 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.102+0000 7f8a0cc00640 10 client.25447 handle_command_reply: tid=0 2024-09-12T01:32:44.102 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.102+0000 7f8a2cfee740 1 client.25447 shutdown 2024-09-12T01:32:44.102 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.102+0000 7f8a2cfee740 1 -- 172.21.15.1:0/404886821 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8d1bdc090 msgr2=0x55f8d1beaea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:44.102 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.102+0000 7f8a2cfee740 1 --2- 172.21.15.1:0/404886821 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8d1bdc090 0x55f8d1beaea0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8a14003b80 tx=0x7f8a14030570 comp rx=0 tx=0).stop 2024-09-12T01:32:44.102 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.102+0000 7f8a2cfee740 1 -- 172.21.15.1:0/404886821 shutdown_connections 2024-09-12T01:32:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.102+0000 7f8a2cfee740 1 --2- 172.21.15.1:0/404886821 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55f8d1c5ce60 0x55f8d1c5f280 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.102+0000 7f8a2cfee740 1 --2- 172.21.15.1:0/404886821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f8d1be1a60 0x55f8d1bec380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.102+0000 7f8a2cfee740 1 --2- 172.21.15.1:0/404886821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d1bdc9d0 0x55f8d1beb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.102+0000 7f8a2cfee740 1 --2- 172.21.15.1:0/404886821 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8d1bdc090 0x55f8d1beaea0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.102+0000 7f8a0cc00640 0 client.25447 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:32:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.102+0000 7f8a2cfee740 1 -- 172.21.15.1:0/404886821 >> 172.21.15.1:0/404886821 conn(0x55f8d1bd7700 msgr2=0x55f8d1bd6880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.102+0000 7f8a2cfee740 1 -- 172.21.15.1:0/404886821 shutdown_connections 2024-09-12T01:32:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.103+0000 7f8a2cfee740 1 -- 172.21.15.1:0/404886821 wait complete. 2024-09-12T01:32:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.103+0000 7f8a2cfee740 20 client.25447 trim_cache size 0 max 16384 2024-09-12T01:32:44.104 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-12T01:32:44.104 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-12T01:32:44.104 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-12T01:32:44.104 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-12T01:32:44.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.103+0000 7f8a2a200640 1 -- 172.21.15.1:0/3804203186 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f8a080a2c00 msgr2=0x7f8a080a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:44.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.103+0000 7f8a2a200640 1 --2- 172.21.15.1:0/3804203186 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f8a080a2c00 0x7f8a080a50c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8a14035040 tx=0x7f8a1402a340 comp rx=0 tx=0).stop 2024-09-12T01:32:44.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.103+0000 7f8a2a200640 1 -- 172.21.15.1:0/3804203186 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a24102d20 msgr2=0x7f8a240745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:44.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.103+0000 7f8a2a200640 1 --2- 172.21.15.1:0/3804203186 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a24102d20 0x7f8a240745e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8a1000e9f0 tx=0x7f8a1000eeb0 comp rx=0 tx=0).stop 2024-09-12T01:32:44.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.104+0000 7f8a2a200640 1 -- 172.21.15.1:0/3804203186 shutdown_connections 2024-09-12T01:32:44.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.104+0000 7f8a2a200640 1 --2- 172.21.15.1:0/3804203186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a241036e0 0x7f8a24074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:44.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.104+0000 7f8a2a200640 1 --2- 172.21.15.1:0/3804203186 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a24102d20 0x7f8a240745e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:44.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.104+0000 7f8a2a200640 1 --2- 172.21.15.1:0/3804203186 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f8a080a2c00 0x7f8a080a50c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:44.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.104+0000 7f8a2a200640 1 --2- 172.21.15.1:0/3804203186 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a24101b30 0x7f8a24075fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:44.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.104+0000 7f8a2a200640 1 -- 172.21.15.1:0/3804203186 >> 172.21.15.1:0/3804203186 conn(0x7f8a240fd2b0 msgr2=0x7f8a24106190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:44.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.105+0000 7f8a2a200640 1 -- 172.21.15.1:0/3804203186 shutdown_connections 2024-09-12T01:32:44.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:44.105+0000 7f8a2a200640 1 -- 172.21.15.1:0/3804203186 wait complete. 2024-09-12T01:32:44.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:32:44 smithi001 ceph-mon[30387]: pgmap v492: 129 pgs: 129 active+clean; 78 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 351 op/s 2024-09-12T01:32:44.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:32:44 smithi053 ceph-mon[35158]: pgmap v492: 129 pgs: 129 active+clean; 78 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 351 op/s 2024-09-12T01:32:45.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:32:44 smithi104 ceph-mon[35219]: pgmap v492: 129 pgs: 129 active+clean; 78 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 351 op/s 2024-09-12T01:32:45.599 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0xffd97 2024-09-12T01:32:46.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:32:46 smithi001 ceph-mon[30387]: pgmap v493: 129 pgs: 129 active+clean; 78 MiB data, 933 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 430 op/s 2024-09-12T01:32:46.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:32:46 smithi053 ceph-mon[35158]: pgmap v493: 129 pgs: 129 active+clean; 78 MiB data, 933 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 430 op/s 2024-09-12T01:32:47.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:32:46 smithi104 ceph-mon[35219]: pgmap v493: 129 pgs: 129 active+clean; 78 MiB data, 933 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 430 op/s 2024-09-12T01:32:48.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:32:48 smithi001 ceph-mon[30387]: pgmap v494: 129 pgs: 129 active+clean; 79 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 331 op/s 2024-09-12T01:32:48.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:32:48 smithi053 ceph-mon[35158]: pgmap v494: 129 pgs: 129 active+clean; 79 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 331 op/s 2024-09-12T01:32:49.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:32:48 smithi104 ceph-mon[35219]: pgmap v494: 129 pgs: 129 active+clean; 79 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 331 op/s 2024-09-12T01:32:50.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:32:50 smithi001 ceph-mon[30387]: pgmap v495: 129 pgs: 129 active+clean; 80 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 325 op/s 2024-09-12T01:32:50.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:32:50 smithi053 ceph-mon[35158]: pgmap v495: 129 pgs: 129 active+clean; 80 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 325 op/s 2024-09-12T01:32:51.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:32:50 smithi104 ceph-mon[35219]: pgmap v495: 129 pgs: 129 active+clean; 80 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 325 op/s 2024-09-12T01:32:52.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:32:52 smithi001 ceph-mon[30387]: pgmap v496: 129 pgs: 129 active+clean; 81 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 414 op/s 2024-09-12T01:32:52.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:32:52 smithi053 ceph-mon[35158]: pgmap v496: 129 pgs: 129 active+clean; 81 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 414 op/s 2024-09-12T01:32:53.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:32:52 smithi104 ceph-mon[35219]: pgmap v496: 129 pgs: 129 active+clean; 81 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 414 op/s 2024-09-12T01:32:54.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:32:54 smithi001 ceph-mon[30387]: pgmap v497: 129 pgs: 129 active+clean; 81 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 317 op/s 2024-09-12T01:32:54.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:32:54 smithi053 ceph-mon[35158]: pgmap v497: 129 pgs: 129 active+clean; 81 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 317 op/s 2024-09-12T01:32:55.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:32:54 smithi104 ceph-mon[35219]: pgmap v497: 129 pgs: 129 active+clean; 81 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 317 op/s 2024-09-12T01:32:56.876 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:32:56 smithi104 ceph-mon[35219]: pgmap v498: 129 pgs: 129 active+clean; 81 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 417 op/s 2024-09-12T01:32:56.876 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:32:56 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:32:56.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:32:56 smithi001 ceph-mon[30387]: pgmap v498: 129 pgs: 129 active+clean; 81 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 417 op/s 2024-09-12T01:32:56.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:32:56 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:32:56.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:32:56 smithi053 ceph-mon[35158]: pgmap v498: 129 pgs: 129 active+clean; 81 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 417 op/s 2024-09-12T01:32:56.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:32:56 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:32:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.246+0000 7f6a97a00640 1 -- 172.21.15.1:0/3558108224 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a90104f10 msgr2=0x7f6a90107300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.246+0000 7f6a97a00640 1 --2- 172.21.15.1:0/3558108224 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a90104f10 0x7f6a90107300 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c009860 tx=0x7f6a7c02a340 comp rx=0 tx=0).stop 2024-09-12T01:32:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.246+0000 7f6a97a00640 1 -- 172.21.15.1:0/3558108224 shutdown_connections 2024-09-12T01:32:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.246+0000 7f6a97a00640 1 --2- 172.21.15.1:0/3558108224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a90107840 0x7f6a90109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.246+0000 7f6a97a00640 1 --2- 172.21.15.1:0/3558108224 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a90104f10 0x7f6a90107300 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.246+0000 7f6a97a00640 1 --2- 172.21.15.1:0/3558108224 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a901025e0 0x7f6a901049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.246+0000 7f6a97a00640 1 -- 172.21.15.1:0/3558108224 >> 172.21.15.1:0/3558108224 conn(0x7f6a900fc1c0 msgr2=0x7f6a900fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.247+0000 7f6a97a00640 1 -- 172.21.15.1:0/3558108224 shutdown_connections 2024-09-12T01:32:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.247+0000 7f6a97a00640 1 -- 172.21.15.1:0/3558108224 wait complete. 2024-09-12T01:32:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.248+0000 7f6a97a00640 1 Processor -- start 2024-09-12T01:32:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.248+0000 7f6a97a00640 1 -- start start 2024-09-12T01:32:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.248+0000 7f6a97a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a901025e0 0x7f6a9019a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.248+0000 7f6a97a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a90104f10 0x7f6a9019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.248+0000 7f6a97a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a90107840 0x7f6a901a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.248+0000 7f6a97a00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9010d7f0 con 0x7f6a901025e0 2024-09-12T01:32:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.248+0000 7f6a97a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9010d670 con 0x7f6a90107840 2024-09-12T01:32:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.248+0000 7f6a97a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9010d970 con 0x7f6a90104f10 2024-09-12T01:32:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.249+0000 7f6a95000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a901025e0 0x7f6a9019a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.249+0000 7f6a8fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a90104f10 0x7f6a9019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.249+0000 7f6a8fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a90104f10 0x7f6a9019a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:51888/0 (socket says 172.21.15.1:51888) 2024-09-12T01:32:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.249+0000 7f6a8fe00640 1 -- 172.21.15.1:0/1002199533 learned_addr learned my addr 172.21.15.1:0/1002199533 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:32:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.249+0000 7f6a95a00640 1 --2- 172.21.15.1:0/1002199533 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a90107840 0x7f6a901a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.249+0000 7f6a8fe00640 1 -- 172.21.15.1:0/1002199533 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a90107840 msgr2=0x7f6a901a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.249+0000 7f6a8fe00640 1 --2- 172.21.15.1:0/1002199533 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a90107840 0x7f6a901a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.249+0000 7f6a8fe00640 1 -- 172.21.15.1:0/1002199533 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a901025e0 msgr2=0x7f6a9019a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.249+0000 7f6a8fe00640 1 --2- 172.21.15.1:0/1002199533 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a901025e0 0x7f6a9019a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.249+0000 7f6a8fe00640 1 -- 172.21.15.1:0/1002199533 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a901a1cf0 con 0x7f6a90104f10 2024-09-12T01:32:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.249+0000 7f6a95000640 1 --2- 172.21.15.1:0/1002199533 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a901025e0 0x7f6a9019a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:32:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.250+0000 7f6a8fe00640 1 --2- 172.21.15.1:0/1002199533 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a90104f10 0x7f6a9019a610 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c035040 tx=0x7f6a7c033d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.250+0000 7f6a8d600640 1 -- 172.21.15.1:0/1002199533 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c004050 con 0x7f6a90104f10 2024-09-12T01:32:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.250+0000 7f6a8d600640 1 -- 172.21.15.1:0/1002199533 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a7c0041f0 con 0x7f6a90104f10 2024-09-12T01:32:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.250+0000 7f6a97a00640 1 -- 172.21.15.1:0/1002199533 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a90192ed0 con 0x7f6a90104f10 2024-09-12T01:32:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.250+0000 7f6a8d600640 1 -- 172.21.15.1:0/1002199533 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c005530 con 0x7f6a90104f10 2024-09-12T01:32:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.250+0000 7f6a97a00640 1 -- 172.21.15.1:0/1002199533 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a901933c0 con 0x7f6a90104f10 2024-09-12T01:32:57.253 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.254+0000 7f6a9aa44740 1 Processor -- start 2024-09-12T01:32:57.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.254+0000 7f6a9aa44740 1 -- start start 2024-09-12T01:32:57.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.254+0000 7f6a8d600640 1 -- 172.21.15.1:0/1002199533 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f6a7c03e980 con 0x7f6a90104f10 2024-09-12T01:32:57.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.254+0000 7f6a9aa44740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56270658f0b0 0x56270658f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:57.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.254+0000 7f6a9aa44740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56270658f9f0 0x5627065944b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:57.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.254+0000 7f6a9aa44740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562706594a80 0x562706596f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:57.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.254+0000 7f6a9aa44740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5627064f4100 con 0x56270658f0b0 2024-09-12T01:32:57.255 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.254+0000 7f6a9aa44740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56270649c1b0 con 0x56270658f9f0 2024-09-12T01:32:57.255 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.254+0000 7f6a9aa44740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5627064f00c0 con 0x562706594a80 2024-09-12T01:32:57.255 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.255+0000 7f6a95a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56270658f0b0 0x56270658f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:57.255 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.255+0000 7f6a95a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56270658f0b0 0x56270658f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60920/0 (socket says 172.21.15.1:60920) 2024-09-12T01:32:57.255 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.255+0000 7f6a95a00640 1 -- 172.21.15.1:0/678568213 learned_addr learned my addr 172.21.15.1:0/678568213 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:32:57.255 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.255+0000 7f6a8fe00640 1 --2- 172.21.15.1:0/678568213 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562706594a80 0x562706596f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:57.255 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.255+0000 7f6a95000640 1 --2- 172.21.15.1:0/678568213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56270658f9f0 0x5627065944b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:57.256 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.255+0000 7f6a95a00640 1 -- 172.21.15.1:0/678568213 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562706594a80 msgr2=0x562706596f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:57.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.255+0000 7f6a95a00640 1 --2- 172.21.15.1:0/678568213 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562706594a80 0x562706596f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:57.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.255+0000 7f6a95a00640 1 -- 172.21.15.1:0/678568213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56270658f9f0 msgr2=0x5627065944b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:57.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.255+0000 7f6a95a00640 1 --2- 172.21.15.1:0/678568213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56270658f9f0 0x5627065944b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:57.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.255+0000 7f6a95a00640 1 -- 172.21.15.1:0/678568213 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562706207210 con 0x56270658f0b0 2024-09-12T01:32:57.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.256+0000 7f6a95a00640 1 --2- 172.21.15.1:0/678568213 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56270658f0b0 0x56270658f4b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6a8800e4d0 tx=0x7f6a880095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f875b4758d621b75 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:57.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.256+0000 7f6a82000640 1 -- 172.21.15.1:0/678568213 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a88003970 con 0x56270658f0b0 2024-09-12T01:32:57.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.256+0000 7f6a82000640 1 -- 172.21.15.1:0/678568213 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a8800d770 con 0x56270658f0b0 2024-09-12T01:32:57.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.256+0000 7f6a82000640 1 -- 172.21.15.1:0/678568213 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a88004440 con 0x56270658f0b0 2024-09-12T01:32:57.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.256+0000 7f6a8d600640 1 --2- 172.21.15.1:0/1002199533 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f6a780a2b80 0x7f6a780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:57.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.257+0000 7f6a9aa44740 1 -- 172.21.15.1:0/678568213 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56270658f0b0 msgr2=0x56270658f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:57.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.257+0000 7f6a9aa44740 1 --2- 172.21.15.1:0/678568213 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56270658f0b0 0x56270658f4b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6a8800e4d0 tx=0x7f6a880095c0 comp rx=0 tx=0).stop 2024-09-12T01:32:57.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.257+0000 7f6a8d600640 1 -- 172.21.15.1:0/1002199533 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f6a7c0f7680 con 0x7f6a90104f10 2024-09-12T01:32:57.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.257+0000 7f6a9aa44740 1 -- 172.21.15.1:0/678568213 shutdown_connections 2024-09-12T01:32:57.258 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.257+0000 7f6a9aa44740 1 --2- 172.21.15.1:0/678568213 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562706594a80 0x562706596f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:57.258 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.257+0000 7f6a9aa44740 1 --2- 172.21.15.1:0/678568213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56270658f9f0 0x5627065944b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:57.258 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.257+0000 7f6a9aa44740 1 --2- 172.21.15.1:0/678568213 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56270658f0b0 0x56270658f4b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:57.258 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.257+0000 7f6a9aa44740 1 -- 172.21.15.1:0/678568213 >> 172.21.15.1:0/678568213 conn(0x56270658a720 msgr2=0x56270658ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:57.258 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.257+0000 7f6a95000640 1 --2- 172.21.15.1:0/1002199533 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f6a780a2b80 0x7f6a780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:57.258 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.257+0000 7f6a9aa44740 1 -- 172.21.15.1:0/678568213 shutdown_connections 2024-09-12T01:32:57.259 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.257+0000 7f6a9aa44740 1 -- 172.21.15.1:0/678568213 wait complete. 2024-09-12T01:32:57.259 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.258+0000 7f6a95000640 1 --2- 172.21.15.1:0/1002199533 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f6a780a2b80 0x7f6a780a5040 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x56270658b5a0 tx=0x7f6a84005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:57.259 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.258+0000 7f6a9aa44740 1 -- start start 2024-09-12T01:32:57.259 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.259+0000 7f6a9aa44740 1 Processor -- start 2024-09-12T01:32:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.259+0000 7f6a9aa44740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56270658f0b0 0x5627065aa220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.259+0000 7f6a9aa44740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56270658f9f0 0x5627065aa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.259+0000 7f6a9aa44740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562706594a80 0x5627065b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.259+0000 7f6a9aa44740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5627065a8a60 con 0x56270658f9f0 2024-09-12T01:32:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.259+0000 7f6a9aa44740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56270647d090 con 0x56270658f0b0 2024-09-12T01:32:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.259+0000 7f6a9aa44740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5627065a8be0 con 0x562706594a80 2024-09-12T01:32:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.259+0000 7f6a9aa44740 10 client.0 _ll_register_callbacks cb 0x5627064d1f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:32:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.260+0000 7f6a8fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56270658f9f0 0x5627065aa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.260+0000 7f6a95000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56270658f0b0 0x5627065aa220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.260+0000 7f6a8fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56270658f9f0 0x5627065aa760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60922/0 (socket says 172.21.15.1:60922) 2024-09-12T01:32:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.260+0000 7f6a8fe00640 1 -- 172.21.15.1:0/3038098881 learned_addr learned my addr 172.21.15.1:0/3038098881 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:32:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.260+0000 7f6a95a00640 1 --2- 172.21.15.1:0/3038098881 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562706594a80 0x5627065b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:57.261 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.260+0000 7f6a95000640 1 -- 172.21.15.1:0/3038098881 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562706594a80 msgr2=0x5627065b17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:57.261 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.260+0000 7f6a95000640 1 --2- 172.21.15.1:0/3038098881 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562706594a80 0x5627065b17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:57.261 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.260+0000 7f6a95000640 1 -- 172.21.15.1:0/3038098881 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56270658f9f0 msgr2=0x5627065aa760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.260+0000 7f6a95000640 1 --2- 172.21.15.1:0/3038098881 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56270658f9f0 0x5627065aa760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.260+0000 7f6a95000640 1 -- 172.21.15.1:0/3038098881 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627062d3620 con 0x56270658f0b0 2024-09-12T01:32:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.261+0000 7f6a95000640 1 --2- 172.21.15.1:0/3038098881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56270658f0b0 0x5627065aa220 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6a84004650 tx=0x7f6a84004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c7dfe9d75b7f9f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.261+0000 7f6a76a00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-12T01:32:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.261+0000 7f6a76a00640 1 -- 172.21.15.1:0/3038098881 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8400f870 con 0x56270658f0b0 2024-09-12T01:32:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.261+0000 7f6a76a00640 1 -- 172.21.15.1:0/3038098881 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f6a8400fa10 con 0x56270658f0b0 2024-09-12T01:32:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.261+0000 7f6a9aa44740 1 -- 172.21.15.1:0/3038098881 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a8800d770 con 0x56270658f0b0 2024-09-12T01:32:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.262+0000 7f6a76a00640 1 -- 172.21.15.1:0/3038098881 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6a8400fc10 con 0x56270658f0b0 2024-09-12T01:32:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.262+0000 7f6a9aa44740 10 client.25457 fetch_fsmap learned FSMap version 32 2024-09-12T01:32:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.262+0000 7f6a9aa44740 1 -- 172.21.15.1:0/3038098881 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5627065b4660 con 0x56270658f0b0 2024-09-12T01:32:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.262+0000 7f6a76a00640 1 -- 172.21.15.1:0/3038098881 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a8800d770 con 0x56270658f0b0 2024-09-12T01:32:57.263 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.262+0000 7f6a76a00640 1 -- 172.21.15.1:0/3038098881 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f6a84027070 con 0x56270658f0b0 2024-09-12T01:32:57.263 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.262+0000 7f6a9aa44740 10 client.25457 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T01:32:57.263 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.262+0000 7f6a9aa44740 10 client.25457 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:32:57.263 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.262+0000 7f6a9aa44740 20 client.25457 populate_metadata read hostname 'smithi001' 2024-09-12T01:32:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.262+0000 7f6a9aa44740 1 --2- 172.21.15.1:0/3038098881 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5627065b5d50 0x5627065b8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.262+0000 7f6a9aa44740 4 client.25457 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:32:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.262+0000 7f6a9aa44740 1 -- 172.21.15.1:0/3038098881 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562706501830 con 0x5627065b5d50 2024-09-12T01:32:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.263+0000 7f6a8fe00640 1 --2- 172.21.15.1:0/3038098881 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5627065b5d50 0x5627065b8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.263+0000 7f6a8fe00640 1 --2- 172.21.15.1:0/3038098881 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5627065b5d50 0x5627065b8180 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f0a5a10a515d1f1 server_cookie=a2145c8d45419b33 in_seq=0 out_seq=0 2024-09-12T01:32:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.263+0000 7f6a76a00640 10 client.25457 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:32:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.264+0000 7f6a76a00640 1 -- 172.21.15.1:0/3038098881 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562706501830 con 0x5627065b5d50 2024-09-12T01:32:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.264+0000 7f6a76a00640 10 client.25457 handle_command_reply: tid=0 2024-09-12T01:32:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.264+0000 7f6a9aa44740 1 client.25457 shutdown 2024-09-12T01:32:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.264+0000 7f6a9aa44740 1 -- 172.21.15.1:0/3038098881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56270658f0b0 msgr2=0x5627065aa220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.264+0000 7f6a9aa44740 1 --2- 172.21.15.1:0/3038098881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56270658f0b0 0x5627065aa220 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6a84004650 tx=0x7f6a84004730 comp rx=0 tx=0).stop 2024-09-12T01:32:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.264+0000 7f6a9aa44740 1 -- 172.21.15.1:0/3038098881 shutdown_connections 2024-09-12T01:32:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.264+0000 7f6a9aa44740 1 --2- 172.21.15.1:0/3038098881 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5627065b5d50 0x5627065b8180 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.264+0000 7f6a9aa44740 1 --2- 172.21.15.1:0/3038098881 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562706594a80 0x5627065b17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.264+0000 7f6a9aa44740 1 --2- 172.21.15.1:0/3038098881 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56270658f9f0 0x5627065aa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.264+0000 7f6a9aa44740 1 --2- 172.21.15.1:0/3038098881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56270658f0b0 0x5627065aa220 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.264+0000 7f6a9aa44740 1 -- 172.21.15.1:0/3038098881 >> 172.21.15.1:0/3038098881 conn(0x56270658a720 msgr2=0x56270658bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.264+0000 7f6a76a00640 0 client.25457 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:32:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.265+0000 7f6a9aa44740 1 -- 172.21.15.1:0/3038098881 shutdown_connections 2024-09-12T01:32:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.265+0000 7f6a9aa44740 1 -- 172.21.15.1:0/3038098881 wait complete. 2024-09-12T01:32:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.265+0000 7f6a9aa44740 20 client.25457 trim_cache size 0 max 16384 2024-09-12T01:32:57.286 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.284+0000 7f6a9aa44740 1 Processor -- start 2024-09-12T01:32:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.284+0000 7f6a9aa44740 1 -- start start 2024-09-12T01:32:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.284+0000 7f6a9aa44740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56270658f0b0 0x5627065aa6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.284+0000 7f6a9aa44740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56270658f9f0 0x5627065b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.284+0000 7f6a9aa44740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562706594a80 0x56270658b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:57.288 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.284+0000 7f6a9aa44740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6a901a85a0 con 0x562706594a80 2024-09-12T01:32:57.288 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.284+0000 7f6a9aa44740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6a901aa170 con 0x56270658f0b0 2024-09-12T01:32:57.288 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.284+0000 7f6a9aa44740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6a901a8c70 con 0x56270658f9f0 2024-09-12T01:32:57.288 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.284+0000 7f6a95a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562706594a80 0x56270658b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:57.288 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.284+0000 7f6a8fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56270658f9f0 0x5627065b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:57.288 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.284+0000 7f6a8fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56270658f9f0 0x5627065b16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:51922/0 (socket says 172.21.15.1:51922) 2024-09-12T01:32:57.288 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.284+0000 7f6a95a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562706594a80 0x56270658b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60930/0 (socket says 172.21.15.1:60930) 2024-09-12T01:32:57.289 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.284+0000 7f6a8fe00640 1 -- 172.21.15.1:0/1752247071 learned_addr learned my addr 172.21.15.1:0/1752247071 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:32:57.289 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.284+0000 7f6a95000640 1 --2- 172.21.15.1:0/1752247071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56270658f0b0 0x5627065aa6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:57.289 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.285+0000 7f6a8fe00640 1 -- 172.21.15.1:0/1752247071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56270658f0b0 msgr2=0x5627065aa6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:57.289 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.285+0000 7f6a8fe00640 1 --2- 172.21.15.1:0/1752247071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56270658f0b0 0x5627065aa6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:57.289 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.285+0000 7f6a8fe00640 1 -- 172.21.15.1:0/1752247071 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562706594a80 msgr2=0x56270658b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:57.289 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.285+0000 7f6a8fe00640 1 --2- 172.21.15.1:0/1752247071 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562706594a80 0x56270658b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.285+0000 7f6a8fe00640 1 -- 172.21.15.1:0/1752247071 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56270658fed0 con 0x56270658f9f0 2024-09-12T01:32:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.285+0000 7f6a95a00640 1 --2- 172.21.15.1:0/1752247071 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562706594a80 0x56270658b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:32:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.285+0000 7f6a95000640 1 --2- 172.21.15.1:0/1752247071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56270658f0b0 0x5627065aa6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:32:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.285+0000 7f6a8fe00640 1 --2- 172.21.15.1:0/1752247071 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56270658f9f0 0x5627065b16c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c0cc230 tx=0x7f6a7c02c040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c04939590d9e0d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.286+0000 7f6a81600640 1 -- 172.21.15.1:0/1752247071 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c04c040 con 0x56270658f9f0 2024-09-12T01:32:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.286+0000 7f6a81600640 1 -- 172.21.15.1:0/1752247071 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a7c0c3ba0 con 0x56270658f9f0 2024-09-12T01:32:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.286+0000 7f6a81600640 1 -- 172.21.15.1:0/1752247071 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c0f4ea0 con 0x56270658f9f0 2024-09-12T01:32:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.286+0000 7f6a9aa44740 1 -- 172.21.15.1:0/1752247071 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56270658f9f0 msgr2=0x5627065b16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.286+0000 7f6a9aa44740 1 --2- 172.21.15.1:0/1752247071 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56270658f9f0 0x5627065b16c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c0cc230 tx=0x7f6a7c02c040 comp rx=0 tx=0).stop 2024-09-12T01:32:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.286+0000 7f6a9aa44740 1 -- 172.21.15.1:0/1752247071 shutdown_connections 2024-09-12T01:32:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.286+0000 7f6a9aa44740 1 --2- 172.21.15.1:0/1752247071 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562706594a80 0x56270658b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.286+0000 7f6a9aa44740 1 --2- 172.21.15.1:0/1752247071 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56270658f9f0 0x5627065b16c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.286+0000 7f6a9aa44740 1 --2- 172.21.15.1:0/1752247071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56270658f0b0 0x5627065aa6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.286+0000 7f6a9aa44740 1 -- 172.21.15.1:0/1752247071 >> 172.21.15.1:0/1752247071 conn(0x56270658a720 msgr2=0x5627065a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.287+0000 7f6a9aa44740 1 -- 172.21.15.1:0/1752247071 shutdown_connections 2024-09-12T01:32:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.287+0000 7f6a9aa44740 1 -- 172.21.15.1:0/1752247071 wait complete. 2024-09-12T01:32:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.287+0000 7f6a9aa44740 1 -- start start 2024-09-12T01:32:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.288+0000 7f6a9aa44740 1 Processor -- start 2024-09-12T01:32:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.288+0000 7f6a9aa44740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56270658f0b0 0x5627065ae3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.288+0000 7f6a9aa44740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56270658f9f0 0x5627065ae900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.288+0000 7f6a9aa44740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562706594a80 0x56270661ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.288+0000 7f6a9aa44740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6a901a9a70 con 0x56270658f9f0 2024-09-12T01:32:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.288+0000 7f6a9aa44740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6a901a93a0 con 0x56270658f0b0 2024-09-12T01:32:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.288+0000 7f6a9aa44740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6a901a6580 con 0x562706594a80 2024-09-12T01:32:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.288+0000 7f6a9aa44740 10 client.0 _ll_register_callbacks cb 0x7f6a84004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:32:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.289+0000 7f6a95000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56270658f0b0 0x5627065ae3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.289+0000 7f6a8fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56270658f9f0 0x5627065ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.289+0000 7f6a95000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56270658f0b0 0x5627065ae3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:41110/0 (socket says 172.21.15.1:41110) 2024-09-12T01:32:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.289+0000 7f6a95000640 1 -- 172.21.15.1:0/2345492471 learned_addr learned my addr 172.21.15.1:0/2345492471 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:32:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.289+0000 7f6a95000640 1 -- 172.21.15.1:0/2345492471 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562706594a80 msgr2=0x56270661ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.289+0000 7f6a95000640 1 --2- 172.21.15.1:0/2345492471 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562706594a80 0x56270661ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.289+0000 7f6a95000640 1 -- 172.21.15.1:0/2345492471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56270658f9f0 msgr2=0x5627065ae900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:32:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.289+0000 7f6a95000640 1 --2- 172.21.15.1:0/2345492471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56270658f9f0 0x5627065ae900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:57.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.289+0000 7f6a95000640 1 -- 172.21.15.1:0/2345492471 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627065b19f0 con 0x56270658f0b0 2024-09-12T01:32:57.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.289+0000 7f6a95000640 1 --2- 172.21.15.1:0/2345492471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56270658f0b0 0x5627065ae3c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6a8401d4f0 tx=0x7f6a8400fb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d2a301b44d0bc7d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:32:57.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.289+0000 7f6a76a00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-12T01:32:57.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.290+0000 7f6a76a00640 1 -- 172.21.15.1:0/2345492471 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8403b060 con 0x56270658f0b0 2024-09-12T01:32:57.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.290+0000 7f6a76a00640 1 -- 172.21.15.1:0/2345492471 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f6a84012240 con 0x56270658f0b0 2024-09-12T01:32:57.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.290+0000 7f6a9aa44740 1 -- 172.21.15.1:0/2345492471 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a7c0c3ba0 con 0x56270658f0b0 2024-09-12T01:32:57.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.290+0000 7f6a76a00640 1 -- 172.21.15.1:0/2345492471 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6a8401da30 con 0x56270658f0b0 2024-09-12T01:32:57.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.290+0000 7f6a9aa44740 10 client.25463 fetch_fsmap learned FSMap version 32 2024-09-12T01:32:57.296 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.290+0000 7f6a9aa44740 1 -- 172.21.15.1:0/2345492471 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56270661ffb0 con 0x56270658f0b0 2024-09-12T01:32:57.296 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.290+0000 7f6a76a00640 1 -- 172.21.15.1:0/2345492471 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a7c0c3ba0 con 0x56270658f0b0 2024-09-12T01:32:57.296 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.291+0000 7f6a76a00640 1 -- 172.21.15.1:0/2345492471 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f6a84027040 con 0x56270658f0b0 2024-09-12T01:32:57.296 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.291+0000 7f6a9aa44740 10 client.25463 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T01:32:57.296 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.291+0000 7f6a9aa44740 10 client.25463 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:32:57.296 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.291+0000 7f6a9aa44740 20 client.25463 populate_metadata read hostname 'smithi001' 2024-09-12T01:32:57.296 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.291+0000 7f6a9aa44740 1 --2- 172.21.15.1:0/2345492471 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5627066209c0 0x562706622de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:32:57.296 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.291+0000 7f6a9aa44740 4 client.25463 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:32:57.296 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.291+0000 7f6a9aa44740 1 -- 172.21.15.1:0/2345492471 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5627064fbb80 con 0x5627066209c0 2024-09-12T01:32:57.296 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.291+0000 7f6a8fe00640 1 --2- 172.21.15.1:0/2345492471 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5627066209c0 0x562706622de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:32:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.292+0000 7f6a8fe00640 1 --2- 172.21.15.1:0/2345492471 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5627066209c0 0x562706622de0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b607245245cb174 server_cookie=ae9d6df388c82330 in_seq=0 out_seq=0 2024-09-12T01:32:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.292+0000 7f6a76a00640 10 client.25463 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:32:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.300+0000 7f6a76a00640 1 -- 172.21.15.1:0/2345492471 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5627064fbb80 con 0x5627066209c0 2024-09-12T01:32:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.300+0000 7f6a76a00640 10 client.25463 handle_command_reply: tid=0 2024-09-12T01:32:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.300+0000 7f6a9aa44740 1 client.25463 shutdown 2024-09-12T01:32:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.300+0000 7f6a9aa44740 1 -- 172.21.15.1:0/2345492471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56270658f0b0 msgr2=0x5627065ae3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.300+0000 7f6a9aa44740 1 --2- 172.21.15.1:0/2345492471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56270658f0b0 0x5627065ae3c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6a8401d4f0 tx=0x7f6a8400fb80 comp rx=0 tx=0).stop 2024-09-12T01:32:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.300+0000 7f6a9aa44740 1 -- 172.21.15.1:0/2345492471 shutdown_connections 2024-09-12T01:32:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.300+0000 7f6a9aa44740 1 --2- 172.21.15.1:0/2345492471 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5627066209c0 0x562706622de0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.300+0000 7f6a9aa44740 1 --2- 172.21.15.1:0/2345492471 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562706594a80 0x56270661ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.300+0000 7f6a9aa44740 1 --2- 172.21.15.1:0/2345492471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56270658f9f0 0x5627065ae900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.300+0000 7f6a9aa44740 1 --2- 172.21.15.1:0/2345492471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56270658f0b0 0x5627065ae3c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.300+0000 7f6a9aa44740 1 -- 172.21.15.1:0/2345492471 >> 172.21.15.1:0/2345492471 conn(0x56270658a720 msgr2=0x562706589060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.300+0000 7f6a76a00640 0 client.25463 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:32:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.300+0000 7f6a9aa44740 1 -- 172.21.15.1:0/2345492471 shutdown_connections 2024-09-12T01:32:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.300+0000 7f6a9aa44740 1 -- 172.21.15.1:0/2345492471 wait complete. 2024-09-12T01:32:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.301+0000 7f6a9aa44740 20 client.25463 trim_cache size 0 max 16384 2024-09-12T01:32:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-12T01:32:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-12T01:32:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-12T01:32:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-12T01:32:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.301+0000 7f6a97a00640 1 -- 172.21.15.1:0/1002199533 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f6a780a2b80 msgr2=0x7f6a780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.301+0000 7f6a97a00640 1 --2- 172.21.15.1:0/1002199533 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f6a780a2b80 0x7f6a780a5040 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x56270658b5a0 tx=0x7f6a84005f90 comp rx=0 tx=0).stop 2024-09-12T01:32:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.301+0000 7f6a97a00640 1 -- 172.21.15.1:0/1002199533 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a90104f10 msgr2=0x7f6a9019a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:32:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.301+0000 7f6a97a00640 1 --2- 172.21.15.1:0/1002199533 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a90104f10 0x7f6a9019a610 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c035040 tx=0x7f6a7c033d90 comp rx=0 tx=0).stop 2024-09-12T01:32:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.302+0000 7f6a97a00640 1 -- 172.21.15.1:0/1002199533 shutdown_connections 2024-09-12T01:32:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.302+0000 7f6a97a00640 1 --2- 172.21.15.1:0/1002199533 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a90107840 0x7f6a901a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:57.304 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.302+0000 7f6a97a00640 1 --2- 172.21.15.1:0/1002199533 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f6a780a2b80 0x7f6a780a5040 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:57.304 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.302+0000 7f6a97a00640 1 --2- 172.21.15.1:0/1002199533 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a90104f10 0x7f6a9019a610 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:57.304 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.302+0000 7f6a97a00640 1 --2- 172.21.15.1:0/1002199533 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a901025e0 0x7f6a9019a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:32:57.304 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.302+0000 7f6a97a00640 1 -- 172.21.15.1:0/1002199533 >> 172.21.15.1:0/1002199533 conn(0x7f6a900fc1c0 msgr2=0x7f6a90102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:32:57.304 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.302+0000 7f6a97a00640 1 -- 172.21.15.1:0/1002199533 shutdown_connections 2024-09-12T01:32:57.304 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:32:57.302+0000 7f6a97a00640 1 -- 172.21.15.1:0/1002199533 wait complete. 2024-09-12T01:32:57.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:32:57 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:32:57.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:32:57 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:32:57.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:32:57 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:32:57.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:32:57 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:32:57.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:32:57 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:32:57.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:32:57 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:32:58.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:32:57 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:32:58.076 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:32:57 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:32:58.076 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:32:57 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:32:58.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:32:58 smithi001 ceph-mon[30387]: pgmap v499: 129 pgs: 129 active+clean; 81 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 282 op/s 2024-09-12T01:32:58.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:32:58 smithi053 ceph-mon[35158]: pgmap v499: 129 pgs: 129 active+clean; 81 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 282 op/s 2024-09-12T01:32:59.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:32:58 smithi104 ceph-mon[35219]: pgmap v499: 129 pgs: 129 active+clean; 81 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 282 op/s 2024-09-12T01:32:59.991 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0xffdb8 2024-09-12T01:33:00.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:33:00 smithi001 ceph-mon[30387]: pgmap v500: 129 pgs: 129 active+clean; 81 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 316 op/s 2024-09-12T01:33:00.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:33:00 smithi053 ceph-mon[35158]: pgmap v500: 129 pgs: 129 active+clean; 81 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 316 op/s 2024-09-12T01:33:01.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:33:00 smithi104 ceph-mon[35219]: pgmap v500: 129 pgs: 129 active+clean; 81 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 316 op/s 2024-09-12T01:33:02.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:33:02 smithi001 ceph-mon[30387]: pgmap v501: 129 pgs: 129 active+clean; 82 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 344 op/s 2024-09-12T01:33:02.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:33:02 smithi053 ceph-mon[35158]: pgmap v501: 129 pgs: 129 active+clean; 82 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 344 op/s 2024-09-12T01:33:03.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:33:02 smithi104 ceph-mon[35219]: pgmap v501: 129 pgs: 129 active+clean; 82 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 344 op/s 2024-09-12T01:33:04.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:33:04 smithi001 ceph-mon[30387]: pgmap v502: 129 pgs: 129 active+clean; 82 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 256 op/s 2024-09-12T01:33:04.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:33:04 smithi053 ceph-mon[35158]: pgmap v502: 129 pgs: 129 active+clean; 82 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 256 op/s 2024-09-12T01:33:05.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:33:04 smithi104 ceph-mon[35219]: pgmap v502: 129 pgs: 129 active+clean; 82 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 256 op/s 2024-09-12T01:33:06.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:33:06 smithi001 ceph-mon[30387]: pgmap v503: 129 pgs: 129 active+clean; 82 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 372 op/s 2024-09-12T01:33:06.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:33:06 smithi053 ceph-mon[35158]: pgmap v503: 129 pgs: 129 active+clean; 82 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 372 op/s 2024-09-12T01:33:07.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:33:06 smithi104 ceph-mon[35219]: pgmap v503: 129 pgs: 129 active+clean; 82 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 372 op/s 2024-09-12T01:33:08.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:33:08 smithi001 ceph-mon[30387]: pgmap v504: 129 pgs: 129 active+clean; 82 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 1.3 MiB/s wr, 246 op/s 2024-09-12T01:33:08.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:33:08 smithi053 ceph-mon[35158]: pgmap v504: 129 pgs: 129 active+clean; 82 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 1.3 MiB/s wr, 246 op/s 2024-09-12T01:33:09.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:33:08 smithi104 ceph-mon[35219]: pgmap v504: 129 pgs: 129 active+clean; 82 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 1.3 MiB/s wr, 246 op/s 2024-09-12T01:33:10.440 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.440+0000 7f7dece00640 1 -- 172.21.15.1:0/4188986680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7de80ff760 msgr2=0x7f7de80ffb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:10.440 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.440+0000 7f7dece00640 1 --2- 172.21.15.1:0/4188986680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7de80ff760 0x7f7de80ffb40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8009900 tx=0x7f7dd802a3f0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.440 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.440+0000 7f7dece00640 1 -- 172.21.15.1:0/4188986680 shutdown_connections 2024-09-12T01:33:10.440 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.440+0000 7f7dece00640 1 --2- 172.21.15.1:0/4188986680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7de8107870 0x7f7de8109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.440 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.440+0000 7f7dece00640 1 --2- 172.21.15.1:0/4188986680 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7de8100110 0x7f7de8107330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.441 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.440+0000 7f7dece00640 1 --2- 172.21.15.1:0/4188986680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7de80ff760 0x7f7de80ffb40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.441 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.440+0000 7f7dece00640 1 -- 172.21.15.1:0/4188986680 >> 172.21.15.1:0/4188986680 conn(0x7f7de80fb4d0 msgr2=0x7f7de80fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:10.441 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.440+0000 7f7dece00640 1 -- 172.21.15.1:0/4188986680 shutdown_connections 2024-09-12T01:33:10.441 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.440+0000 7f7dece00640 1 -- 172.21.15.1:0/4188986680 wait complete. 2024-09-12T01:33:10.442 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.441+0000 7f7dece00640 1 Processor -- start 2024-09-12T01:33:10.442 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.442+0000 7f7dece00640 1 -- start start 2024-09-12T01:33:10.442 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.442+0000 7f7dece00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7de80ff760 0x7f7de8197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:10.442 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.442+0000 7f7dece00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7de8100110 0x7f7de8198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:10.442 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.442+0000 7f7dece00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7de8107870 0x7f7de819f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:10.442 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.442+0000 7f7dece00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7de810d7d0 con 0x7f7de8100110 2024-09-12T01:33:10.442 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.442+0000 7f7dece00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7de810d650 con 0x7f7de8107870 2024-09-12T01:33:10.443 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.442+0000 7f7dece00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7de810d950 con 0x7f7de80ff760 2024-09-12T01:33:10.443 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.442+0000 7f7de5e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7de80ff760 0x7f7de8197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:10.443 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.442+0000 7f7de5e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7de80ff760 0x7f7de8197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:34346/0 (socket says 172.21.15.1:34346) 2024-09-12T01:33:10.443 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.442+0000 7f7de6800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7de8107870 0x7f7de819f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:10.443 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.442+0000 7f7de5e00640 1 -- 172.21.15.1:0/4272807981 learned_addr learned my addr 172.21.15.1:0/4272807981 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:33:10.443 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.442+0000 7f7de5400640 1 --2- 172.21.15.1:0/4272807981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7de8100110 0x7f7de8198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:10.443 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.443+0000 7f7de5e00640 1 -- 172.21.15.1:0/4272807981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7de8107870 msgr2=0x7f7de819f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:10.443 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.443+0000 7f7de5e00640 1 --2- 172.21.15.1:0/4272807981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7de8107870 0x7f7de819f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.443 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.443+0000 7f7de5e00640 1 -- 172.21.15.1:0/4272807981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7de8100110 msgr2=0x7f7de8198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:10.443 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.443+0000 7f7de5e00640 1 --2- 172.21.15.1:0/4272807981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7de8100110 0x7f7de8198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.443 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.443+0000 7f7de5e00640 1 -- 172.21.15.1:0/4272807981 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7de819fa50 con 0x7f7de80ff760 2024-09-12T01:33:10.444 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.443+0000 7f7de5e00640 1 --2- 172.21.15.1:0/4272807981 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7de80ff760 0x7f7de8197e30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8031440 tx=0x7f7dd8032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:10.444 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.443+0000 7f7ddea00640 1 -- 172.21.15.1:0/4272807981 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd803f030 con 0x7f7de80ff760 2024-09-12T01:33:10.445 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.443+0000 7f7ddea00640 1 -- 172.21.15.1:0/4272807981 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dd8003ce0 con 0x7f7de80ff760 2024-09-12T01:33:10.445 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.443+0000 7f7dece00640 1 -- 172.21.15.1:0/4272807981 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7de819fce0 con 0x7f7de80ff760 2024-09-12T01:33:10.445 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.444+0000 7f7dece00640 1 -- 172.21.15.1:0/4272807981 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7de81a01d0 con 0x7f7de80ff760 2024-09-12T01:33:10.445 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.444+0000 7f7ddea00640 1 -- 172.21.15.1:0/4272807981 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd8035070 con 0x7f7de80ff760 2024-09-12T01:33:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.447+0000 7f7defbc7740 1 Processor -- start 2024-09-12T01:33:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.447+0000 7f7defbc7740 1 -- start start 2024-09-12T01:33:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.448+0000 7f7defbc7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf290f4a0 0x55dcf290f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.448+0000 7f7defbc7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcf290fde0 0x55dcf29148a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.448+0000 7f7defbc7740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dcf2914e70 0x55dcf2917330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.448+0000 7f7defbc7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55dcf2870100 con 0x55dcf290fde0 2024-09-12T01:33:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.448+0000 7f7defbc7740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55dcf28181b0 con 0x55dcf2914e70 2024-09-12T01:33:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.448+0000 7f7defbc7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55dcf286c0c0 con 0x55dcf290f4a0 2024-09-12T01:33:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.448+0000 7f7de6800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf290f4a0 0x55dcf290f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.448+0000 7f7de6800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf290f4a0 0x55dcf290f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:34356/0 (socket says 172.21.15.1:34356) 2024-09-12T01:33:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.448+0000 7f7de5400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dcf2914e70 0x55dcf2917330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.448+0000 7f7de6800640 1 -- 172.21.15.1:0/3172483348 learned_addr learned my addr 172.21.15.1:0/3172483348 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:33:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.448+0000 7f7de6800640 1 -- 172.21.15.1:0/3172483348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dcf2914e70 msgr2=0x55dcf2917330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.448+0000 7f7de6800640 1 --2- 172.21.15.1:0/3172483348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dcf2914e70 0x55dcf2917330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.448+0000 7f7de6800640 1 -- 172.21.15.1:0/3172483348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcf290fde0 msgr2=0x55dcf29148a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T01:33:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.448+0000 7f7de6800640 1 --2- 172.21.15.1:0/3172483348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcf290fde0 0x55dcf29148a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.448+0000 7f7de6800640 1 -- 172.21.15.1:0/3172483348 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcf2583210 con 0x55dcf290f4a0 2024-09-12T01:33:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.448+0000 7f7de5400640 1 --2- 172.21.15.1:0/3172483348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dcf2914e70 0x55dcf2917330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:33:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.449+0000 7f7de6800640 1 --2- 172.21.15.1:0/3172483348 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf290f4a0 0x55dcf290f8a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7de0011440 tx=0x7f7de0011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98974b320738ba84 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.449+0000 7f7ddea00640 1 -- 172.21.15.1:0/4272807981 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f7dd8042030 con 0x7f7de80ff760 2024-09-12T01:33:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.449+0000 7f7dd3400640 1 -- 172.21.15.1:0/3172483348 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de0003c30 con 0x55dcf290f4a0 2024-09-12T01:33:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.449+0000 7f7dd3400640 1 -- 172.21.15.1:0/3172483348 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7de0003dd0 con 0x55dcf290f4a0 2024-09-12T01:33:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.449+0000 7f7dd3400640 1 -- 172.21.15.1:0/3172483348 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de000fb70 con 0x55dcf290f4a0 2024-09-12T01:33:10.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.450+0000 7f7defbc7740 1 -- 172.21.15.1:0/3172483348 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf290f4a0 msgr2=0x55dcf290f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:10.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.450+0000 7f7defbc7740 1 --2- 172.21.15.1:0/3172483348 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf290f4a0 0x55dcf290f8a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7de0011440 tx=0x7f7de0011900 comp rx=0 tx=0).stop 2024-09-12T01:33:10.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.450+0000 7f7defbc7740 1 -- 172.21.15.1:0/3172483348 shutdown_connections 2024-09-12T01:33:10.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.450+0000 7f7defbc7740 1 --2- 172.21.15.1:0/3172483348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dcf2914e70 0x55dcf2917330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.450+0000 7f7defbc7740 1 --2- 172.21.15.1:0/3172483348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcf290fde0 0x55dcf29148a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.450+0000 7f7defbc7740 1 --2- 172.21.15.1:0/3172483348 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf290f4a0 0x55dcf290f8a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.450+0000 7f7defbc7740 1 -- 172.21.15.1:0/3172483348 >> 172.21.15.1:0/3172483348 conn(0x55dcf2906700 msgr2=0x55dcf2906b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:10.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.450+0000 7f7defbc7740 1 -- 172.21.15.1:0/3172483348 shutdown_connections 2024-09-12T01:33:10.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.450+0000 7f7defbc7740 1 -- 172.21.15.1:0/3172483348 wait complete. 2024-09-12T01:33:10.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.451+0000 7f7defbc7740 1 -- start start 2024-09-12T01:33:10.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.451+0000 7f7defbc7740 1 Processor -- start 2024-09-12T01:33:10.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.451+0000 7f7defbc7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf290f4a0 0x55dcf29196c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:10.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.451+0000 7f7defbc7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcf290fde0 0x55dcf2919c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:10.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.451+0000 7f7defbc7740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dcf2914e70 0x55dcf2920c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:10.453 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.451+0000 7f7defbc7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55dcf2917e40 con 0x55dcf290fde0 2024-09-12T01:33:10.453 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.451+0000 7f7defbc7740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55dcf27f9090 con 0x55dcf2914e70 2024-09-12T01:33:10.453 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.451+0000 7f7defbc7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55dcf2917fc0 con 0x55dcf290f4a0 2024-09-12T01:33:10.453 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.452+0000 7f7defbc7740 10 client.0 _ll_register_callbacks cb 0x55dcf284df30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:33:10.453 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.452+0000 7f7de6800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf290f4a0 0x55dcf29196c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:10.453 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.452+0000 7f7de6800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf290f4a0 0x55dcf29196c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:34366/0 (socket says 172.21.15.1:34366) 2024-09-12T01:33:10.453 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.452+0000 7f7de6800640 1 -- 172.21.15.1:0/647458665 learned_addr learned my addr 172.21.15.1:0/647458665 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:33:10.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.452+0000 7f7de5400640 1 --2- 172.21.15.1:0/647458665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dcf2914e70 0x55dcf2920c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:10.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.452+0000 7f7ddea00640 1 --2- 172.21.15.1:0/4272807981 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f7dcc0a2a90 0x7f7dcc0a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:10.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.452+0000 7f7de5e00640 1 --2- 172.21.15.1:0/647458665 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcf290fde0 0x55dcf2919c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:10.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.452+0000 7f7de5400640 1 -- 172.21.15.1:0/647458665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf290f4a0 msgr2=0x55dcf29196c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:10.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.452+0000 7f7de5400640 1 --2- 172.21.15.1:0/647458665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf290f4a0 0x55dcf29196c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.452+0000 7f7de5400640 1 -- 172.21.15.1:0/647458665 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcf290fde0 msgr2=0x55dcf2919c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:10.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.452+0000 7f7de5400640 1 --2- 172.21.15.1:0/647458665 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcf290fde0 0x55dcf2919c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.452+0000 7f7de5400640 1 -- 172.21.15.1:0/647458665 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcf264f620 con 0x55dcf2914e70 2024-09-12T01:33:10.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.452+0000 7f7de6800640 1 --2- 172.21.15.1:0/4272807981 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f7dcc0a2a90 0x7f7dcc0a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:10.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.452+0000 7f7ddea00640 1 -- 172.21.15.1:0/4272807981 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f7dd80efbb0 con 0x7f7de80ff760 2024-09-12T01:33:10.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.453+0000 7f7de5400640 1 --2- 172.21.15.1:0/647458665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dcf2914e70 0x55dcf2920c40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7de8199340 tx=0x7f7dd4012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45e77c60a990a3b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:10.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.453+0000 7f7dc7e00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-12T01:33:10.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.453+0000 7f7dc7e00640 1 -- 172.21.15.1:0/647458665 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd400f920 con 0x55dcf2914e70 2024-09-12T01:33:10.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.453+0000 7f7defbc7740 1 -- 172.21.15.1:0/647458665 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7de0003dd0 con 0x55dcf2914e70 2024-09-12T01:33:10.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.453+0000 7f7dc7e00640 1 -- 172.21.15.1:0/647458665 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f7dd400fac0 con 0x55dcf2914e70 2024-09-12T01:33:10.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.453+0000 7f7de6800640 1 --2- 172.21.15.1:0/4272807981 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f7dcc0a2a90 0x7f7dcc0a4f50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7de00049d0 tx=0x7f7de0009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:10.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.453+0000 7f7dc7e00640 1 -- 172.21.15.1:0/647458665 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7dd4025050 con 0x55dcf2914e70 2024-09-12T01:33:10.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.454+0000 7f7defbc7740 10 client.25475 fetch_fsmap learned FSMap version 32 2024-09-12T01:33:10.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.454+0000 7f7defbc7740 1 -- 172.21.15.1:0/647458665 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55dcf2923b50 con 0x55dcf2914e70 2024-09-12T01:33:10.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.454+0000 7f7dc7e00640 1 -- 172.21.15.1:0/647458665 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7de0003dd0 con 0x55dcf2914e70 2024-09-12T01:33:10.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.454+0000 7f7dc7e00640 1 -- 172.21.15.1:0/647458665 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f7dd4010430 con 0x55dcf2914e70 2024-09-12T01:33:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.454+0000 7f7defbc7740 10 client.25475 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T01:33:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.454+0000 7f7defbc7740 10 client.25475 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:33:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.454+0000 7f7defbc7740 20 client.25475 populate_metadata read hostname 'smithi001' 2024-09-12T01:33:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.454+0000 7f7defbc7740 1 --2- 172.21.15.1:0/647458665 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55dcf2925240 0x55dcf2927660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.454+0000 7f7defbc7740 4 client.25475 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:33:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.454+0000 7f7defbc7740 1 -- 172.21.15.1:0/647458665 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcf287d830 con 0x55dcf2925240 2024-09-12T01:33:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.454+0000 7f7de5e00640 1 --2- 172.21.15.1:0/647458665 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55dcf2925240 0x55dcf2927660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.455+0000 7f7dc7e00640 10 client.25475 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:33:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.455+0000 7f7de5e00640 1 --2- 172.21.15.1:0/647458665 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55dcf2925240 0x55dcf2927660 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2b5aa6afebab805 server_cookie=3931a4236f585aba in_seq=0 out_seq=0 2024-09-12T01:33:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.456+0000 7f7dc7e00640 1 -- 172.21.15.1:0/647458665 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dcf287d830 con 0x55dcf2925240 2024-09-12T01:33:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.456+0000 7f7dc7e00640 10 client.25475 handle_command_reply: tid=0 2024-09-12T01:33:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.456+0000 7f7defbc7740 1 client.25475 shutdown 2024-09-12T01:33:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.456+0000 7f7defbc7740 1 -- 172.21.15.1:0/647458665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dcf2914e70 msgr2=0x55dcf2920c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.456+0000 7f7defbc7740 1 --2- 172.21.15.1:0/647458665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dcf2914e70 0x55dcf2920c40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7de8199340 tx=0x7f7dd4012b60 comp rx=0 tx=0).stop 2024-09-12T01:33:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.456+0000 7f7defbc7740 1 -- 172.21.15.1:0/647458665 shutdown_connections 2024-09-12T01:33:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.456+0000 7f7defbc7740 1 --2- 172.21.15.1:0/647458665 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55dcf2925240 0x55dcf2927660 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.456+0000 7f7defbc7740 1 --2- 172.21.15.1:0/647458665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dcf2914e70 0x55dcf2920c40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.456+0000 7f7defbc7740 1 --2- 172.21.15.1:0/647458665 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcf290fde0 0x55dcf2919c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.456+0000 7f7defbc7740 1 --2- 172.21.15.1:0/647458665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf290f4a0 0x55dcf29196c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.456+0000 7f7dc7e00640 0 client.25475 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:33:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.456+0000 7f7defbc7740 1 -- 172.21.15.1:0/647458665 >> 172.21.15.1:0/647458665 conn(0x55dcf2906700 msgr2=0x55dcf2907f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.456+0000 7f7defbc7740 1 -- 172.21.15.1:0/647458665 shutdown_connections 2024-09-12T01:33:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.456+0000 7f7defbc7740 1 -- 172.21.15.1:0/647458665 wait complete. 2024-09-12T01:33:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.457+0000 7f7defbc7740 20 client.25475 trim_cache size 0 max 16384 2024-09-12T01:33:10.475 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.475+0000 7f7defbc7740 1 Processor -- start 2024-09-12T01:33:10.475 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.475+0000 7f7defbc7740 1 -- start start 2024-09-12T01:33:10.475 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.475+0000 7f7defbc7740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dcf290f4a0 0x55dcf290f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.476+0000 7f7defbc7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcf290fde0 0x55dcf2920b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.476+0000 7f7defbc7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf2914e70 0x55dcf2907af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.476+0000 7f7defbc7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc800fbd0 con 0x55dcf290fde0 2024-09-12T01:33:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.476+0000 7f7defbc7740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc80117a0 con 0x55dcf290f4a0 2024-09-12T01:33:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.476+0000 7f7defbc7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc80102a0 con 0x55dcf2914e70 2024-09-12T01:33:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.476+0000 7f7de5e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dcf290f4a0 0x55dcf290f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.476+0000 7f7de5400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcf290fde0 0x55dcf2920b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.476+0000 7f7de6800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf2914e70 0x55dcf2907af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.476+0000 7f7de5e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dcf290f4a0 0x55dcf290f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:51378/0 (socket says 172.21.15.1:51378) 2024-09-12T01:33:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.476+0000 7f7de5400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcf290fde0 0x55dcf2920b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46392/0 (socket says 172.21.15.1:46392) 2024-09-12T01:33:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.476+0000 7f7de5e00640 1 -- 172.21.15.1:0/1377952542 learned_addr learned my addr 172.21.15.1:0/1377952542 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:33:10.477 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.476+0000 7f7de5e00640 1 -- 172.21.15.1:0/1377952542 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf2914e70 msgr2=0x55dcf2907af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:10.477 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.476+0000 7f7de5e00640 1 --2- 172.21.15.1:0/1377952542 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf2914e70 0x55dcf2907af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.477 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.476+0000 7f7de5e00640 1 -- 172.21.15.1:0/1377952542 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcf290fde0 msgr2=0x55dcf2920b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:10.477 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.476+0000 7f7de5e00640 1 --2- 172.21.15.1:0/1377952542 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcf290fde0 0x55dcf2920b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.478 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.476+0000 7f7de5e00640 1 -- 172.21.15.1:0/1377952542 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcf29102c0 con 0x55dcf290f4a0 2024-09-12T01:33:10.478 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.476+0000 7f7de5400640 1 --2- 172.21.15.1:0/1377952542 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcf290fde0 0x55dcf2920b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:33:10.478 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.477+0000 7f7de5e00640 1 --2- 172.21.15.1:0/1377952542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dcf290f4a0 0x55dcf290f880 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7dd80b89f0 tx=0x7f7dd808bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d74a20992a1f6632 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:10.478 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.477+0000 7f7dd2a00640 1 -- 172.21.15.1:0/1377952542 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd8049040 con 0x55dcf290f4a0 2024-09-12T01:33:10.478 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.477+0000 7f7dd2a00640 1 -- 172.21.15.1:0/1377952542 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dd80bc720 con 0x55dcf290f4a0 2024-09-12T01:33:10.478 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.477+0000 7f7dd2a00640 1 -- 172.21.15.1:0/1377952542 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd80ed3d0 con 0x55dcf290f4a0 2024-09-12T01:33:10.478 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.478+0000 7f7defbc7740 1 -- 172.21.15.1:0/1377952542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dcf290f4a0 msgr2=0x55dcf290f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:10.478 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.478+0000 7f7defbc7740 1 --2- 172.21.15.1:0/1377952542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dcf290f4a0 0x55dcf290f880 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7dd80b89f0 tx=0x7f7dd808bed0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.479 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.478+0000 7f7defbc7740 1 -- 172.21.15.1:0/1377952542 shutdown_connections 2024-09-12T01:33:10.479 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.478+0000 7f7defbc7740 1 --2- 172.21.15.1:0/1377952542 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf2914e70 0x55dcf2907af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.479 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.478+0000 7f7defbc7740 1 --2- 172.21.15.1:0/1377952542 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcf290fde0 0x55dcf2920b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.479 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.478+0000 7f7defbc7740 1 --2- 172.21.15.1:0/1377952542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dcf290f4a0 0x55dcf290f880 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.479 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.478+0000 7f7defbc7740 1 -- 172.21.15.1:0/1377952542 >> 172.21.15.1:0/1377952542 conn(0x55dcf2906700 msgr2=0x55dcf2905880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:10.479 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.478+0000 7f7defbc7740 1 -- 172.21.15.1:0/1377952542 shutdown_connections 2024-09-12T01:33:10.479 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.478+0000 7f7defbc7740 1 -- 172.21.15.1:0/1377952542 wait complete. 2024-09-12T01:33:10.479 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.478+0000 7f7defbc7740 1 -- start start 2024-09-12T01:33:10.479 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.479+0000 7f7defbc7740 1 Processor -- start 2024-09-12T01:33:10.481 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.479+0000 7f7defbc7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcf290f4a0 0x55dcf291e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:10.482 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.479+0000 7f7defbc7740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dcf290fde0 0x55dcf291ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:10.482 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.479+0000 7f7defbc7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf2914e70 0x55dcf291fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:10.482 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.479+0000 7f7defbc7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc80110a0 con 0x55dcf290f4a0 2024-09-12T01:33:10.482 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.479+0000 7f7defbc7740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc80109d0 con 0x55dcf290fde0 2024-09-12T01:33:10.482 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.479+0000 7f7defbc7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc800dbb0 con 0x55dcf2914e70 2024-09-12T01:33:10.482 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.479+0000 7f7defbc7740 10 client.0 _ll_register_callbacks cb 0x7f7dd4009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:33:10.482 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.479+0000 7f7de5e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcf290f4a0 0x55dcf291e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:10.482 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.479+0000 7f7de5e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcf290f4a0 0x55dcf291e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46404/0 (socket says 172.21.15.1:46404) 2024-09-12T01:33:10.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.479+0000 7f7de5e00640 1 -- 172.21.15.1:0/2481882318 learned_addr learned my addr 172.21.15.1:0/2481882318 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:33:10.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.479+0000 7f7de5400640 1 --2- 172.21.15.1:0/2481882318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dcf290fde0 0x55dcf291ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:10.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.480+0000 7f7de6800640 1 --2- 172.21.15.1:0/2481882318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf2914e70 0x55dcf291fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:10.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.480+0000 7f7de5400640 1 -- 172.21.15.1:0/2481882318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf2914e70 msgr2=0x55dcf291fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:10.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.480+0000 7f7de5400640 1 --2- 172.21.15.1:0/2481882318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf2914e70 0x55dcf291fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.480+0000 7f7de5400640 1 -- 172.21.15.1:0/2481882318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcf290f4a0 msgr2=0x55dcf291e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:10.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.480+0000 7f7de5400640 1 --2- 172.21.15.1:0/2481882318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcf290f4a0 0x55dcf291e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.480+0000 7f7de5400640 1 -- 172.21.15.1:0/2481882318 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcf2920260 con 0x55dcf290fde0 2024-09-12T01:33:10.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.480+0000 7f7de5e00640 1 --2- 172.21.15.1:0/2481882318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcf290f4a0 0x55dcf291e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:33:10.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.480+0000 7f7de5400640 1 --2- 172.21.15.1:0/2481882318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dcf290fde0 0x55dcf291ec40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4010800 tx=0x7f7dd4009f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23677a1eaf678fee server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:10.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.480+0000 7f7dc7e00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-12T01:33:10.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.480+0000 7f7dc7e00640 1 -- 172.21.15.1:0/2481882318 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd402e060 con 0x55dcf290fde0 2024-09-12T01:33:10.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.481+0000 7f7defbc7740 1 -- 172.21.15.1:0/2481882318 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7dd80bc720 con 0x55dcf290fde0 2024-09-12T01:33:10.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.481+0000 7f7dc7e00640 1 -- 172.21.15.1:0/2481882318 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f7dd40105b0 con 0x55dcf290fde0 2024-09-12T01:33:10.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.481+0000 7f7dc7e00640 1 -- 172.21.15.1:0/2481882318 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7dd4035050 con 0x55dcf290fde0 2024-09-12T01:33:10.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.481+0000 7f7defbc7740 10 client.25481 fetch_fsmap learned FSMap version 32 2024-09-12T01:33:10.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.481+0000 7f7defbc7740 1 -- 172.21.15.1:0/2481882318 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55dcf29903b0 con 0x55dcf290fde0 2024-09-12T01:33:10.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.481+0000 7f7dc7e00640 1 -- 172.21.15.1:0/2481882318 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dd80bc720 con 0x55dcf290fde0 2024-09-12T01:33:10.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.481+0000 7f7dc7e00640 1 -- 172.21.15.1:0/2481882318 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f7dd40035f0 con 0x55dcf290fde0 2024-09-12T01:33:10.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.481+0000 7f7defbc7740 10 client.25481 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T01:33:10.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.481+0000 7f7defbc7740 10 client.25481 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:33:10.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.481+0000 7f7defbc7740 20 client.25481 populate_metadata read hostname 'smithi001' 2024-09-12T01:33:10.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.481+0000 7f7defbc7740 1 --2- 172.21.15.1:0/2481882318 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55dcf2990d90 0x55dcf29931b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:10.485 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.481+0000 7f7defbc7740 4 client.25481 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:33:10.485 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.481+0000 7f7defbc7740 1 -- 172.21.15.1:0/2481882318 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dcf2877b80 con 0x55dcf2990d90 2024-09-12T01:33:10.485 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.482+0000 7f7de5e00640 1 --2- 172.21.15.1:0/2481882318 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55dcf2990d90 0x55dcf29931b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:10.485 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.482+0000 7f7de5e00640 1 --2- 172.21.15.1:0/2481882318 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55dcf2990d90 0x55dcf29931b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a8b3366678cb932 server_cookie=86ab763c2b133706 in_seq=0 out_seq=0 2024-09-12T01:33:10.485 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.482+0000 7f7dc7e00640 10 client.25481 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:33:10.495 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.494+0000 7f7dc7e00640 1 -- 172.21.15.1:0/2481882318 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dcf2877b80 con 0x55dcf2990d90 2024-09-12T01:33:10.495 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.494+0000 7f7dc7e00640 10 client.25481 handle_command_reply: tid=0 2024-09-12T01:33:10.495 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.494+0000 7f7defbc7740 1 client.25481 shutdown 2024-09-12T01:33:10.495 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.494+0000 7f7defbc7740 1 -- 172.21.15.1:0/2481882318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dcf290fde0 msgr2=0x55dcf291ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:10.495 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.494+0000 7f7defbc7740 1 --2- 172.21.15.1:0/2481882318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dcf290fde0 0x55dcf291ec40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4010800 tx=0x7f7dd4009f20 comp rx=0 tx=0).stop 2024-09-12T01:33:10.495 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.494+0000 7f7defbc7740 1 -- 172.21.15.1:0/2481882318 shutdown_connections 2024-09-12T01:33:10.495 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.494+0000 7f7defbc7740 1 --2- 172.21.15.1:0/2481882318 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55dcf2990d90 0x55dcf29931b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.496 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.494+0000 7f7defbc7740 1 --2- 172.21.15.1:0/2481882318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dcf2914e70 0x55dcf291fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.496 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.494+0000 7f7defbc7740 1 --2- 172.21.15.1:0/2481882318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dcf290fde0 0x55dcf291ec40 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.496 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.494+0000 7f7dc7e00640 0 client.25481 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:33:10.496 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.494+0000 7f7defbc7740 1 --2- 172.21.15.1:0/2481882318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dcf290f4a0 0x55dcf291e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.496 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.494+0000 7f7defbc7740 1 -- 172.21.15.1:0/2481882318 >> 172.21.15.1:0/2481882318 conn(0x55dcf2906700 msgr2=0x55dcf2905880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:10.496 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.495+0000 7f7defbc7740 1 -- 172.21.15.1:0/2481882318 shutdown_connections 2024-09-12T01:33:10.496 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.495+0000 7f7defbc7740 1 -- 172.21.15.1:0/2481882318 wait complete. 2024-09-12T01:33:10.496 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.495+0000 7f7defbc7740 20 client.25481 trim_cache size 0 max 16384 2024-09-12T01:33:10.496 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-12T01:33:10.497 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-12T01:33:10.497 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-12T01:33:10.497 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-12T01:33:10.497 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.496+0000 7f7dece00640 1 -- 172.21.15.1:0/4272807981 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f7dcc0a2a90 msgr2=0x7f7dcc0a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:10.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.496+0000 7f7dece00640 1 --2- 172.21.15.1:0/4272807981 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f7dcc0a2a90 0x7f7dcc0a4f50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7de00049d0 tx=0x7f7de0009500 comp rx=0 tx=0).stop 2024-09-12T01:33:10.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.496+0000 7f7dece00640 1 -- 172.21.15.1:0/4272807981 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7de80ff760 msgr2=0x7f7de8197e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:10.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.496+0000 7f7dece00640 1 --2- 172.21.15.1:0/4272807981 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7de80ff760 0x7f7de8197e30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8031440 tx=0x7f7dd8032ce0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.496+0000 7f7dece00640 1 -- 172.21.15.1:0/4272807981 shutdown_connections 2024-09-12T01:33:10.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.496+0000 7f7dece00640 1 --2- 172.21.15.1:0/4272807981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7de8107870 0x7f7de819f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.496+0000 7f7dece00640 1 --2- 172.21.15.1:0/4272807981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7de8100110 0x7f7de8198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.496+0000 7f7dece00640 1 --2- 172.21.15.1:0/4272807981 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f7dcc0a2a90 0x7f7dcc0a4f50 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.496+0000 7f7dece00640 1 --2- 172.21.15.1:0/4272807981 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7de80ff760 0x7f7de8197e30 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:10.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.496+0000 7f7dece00640 1 -- 172.21.15.1:0/4272807981 >> 172.21.15.1:0/4272807981 conn(0x7f7de80fb4d0 msgr2=0x7f7de8103310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:10.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.496+0000 7f7dece00640 1 -- 172.21.15.1:0/4272807981 shutdown_connections 2024-09-12T01:33:10.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:10.496+0000 7f7dece00640 1 -- 172.21.15.1:0/4272807981 wait complete. 2024-09-12T01:33:10.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:33:10 smithi001 ceph-mon[30387]: pgmap v505: 129 pgs: 129 active+clean; 83 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 1.4 MiB/s wr, 282 op/s 2024-09-12T01:33:10.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:33:10 smithi053 ceph-mon[35158]: pgmap v505: 129 pgs: 129 active+clean; 83 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 1.4 MiB/s wr, 282 op/s 2024-09-12T01:33:11.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:33:10 smithi104 ceph-mon[35219]: pgmap v505: 129 pgs: 129 active+clean; 83 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 1.4 MiB/s wr, 282 op/s 2024-09-12T01:33:12.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:33:12 smithi001 ceph-mon[30387]: pgmap v506: 129 pgs: 129 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 311 op/s 2024-09-12T01:33:12.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:33:12 smithi053 ceph-mon[35158]: pgmap v506: 129 pgs: 129 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 311 op/s 2024-09-12T01:33:13.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:33:12 smithi104 ceph-mon[35219]: pgmap v506: 129 pgs: 129 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 311 op/s 2024-09-12T01:33:13.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:33:13 smithi001 ceph-mon[30387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T01:33:13.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:33:13 smithi053 ceph-mon[35158]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T01:33:14.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:33:13 smithi104 ceph-mon[35219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T01:33:14.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:33:14 smithi001 ceph-mon[30387]: pgmap v507: 129 pgs: 129 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 1.2 MiB/s wr, 233 op/s 2024-09-12T01:33:14.952 INFO:tasks.workunit.client.0.smithi001.stdout:50000 copy from 0x898b8 to 0x92015, (0x875d bytes) at 0xb142c 2024-09-12T01:33:14.953 INFO:tasks.workunit.client.0.smithi001.stdout:All 50000 operations completed A-OK! 2024-09-12T01:33:14.957 INFO:tasks.workunit.client.0.smithi001.stderr:+ ./fsx -z 10MB -N 50000 -p 10000 -l 10485760 2024-09-12T01:33:14.962 INFO:tasks.workunit.client.0.smithi001.stdout:Seed set to 1 2024-09-12T01:33:14.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:33:14 smithi053 ceph-mon[35158]: pgmap v507: 129 pgs: 129 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 1.2 MiB/s wr, 233 op/s 2024-09-12T01:33:15.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:33:14 smithi104 ceph-mon[35219]: pgmap v507: 129 pgs: 129 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 1.2 MiB/s wr, 233 op/s 2024-09-12T01:33:15.116 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-09-12T01:33:15.116 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-09-12T01:33:15.119 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-09-12T01:33:15.119 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-09-12T01:33:15.119 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support clone range, disabling! 2024-09-12T01:33:15.123 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support dedupe range, disabling! 2024-09-12T01:33:15.126 INFO:tasks.workunit.client.0.smithi001.stdout:skipping zero length punch hole 2024-09-12T01:33:15.126 INFO:tasks.workunit.client.0.smithi001.stdout:skipping zero length punch hole 2024-09-12T01:33:15.127 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support exchange range, disabling! 2024-09-12T01:33:15.127 INFO:tasks.workunit.client.0.smithi001.stdout:skipping zero length punch hole 2024-09-12T01:33:15.127 INFO:tasks.workunit.client.0.smithi001.stdout:skipping zero size read 2024-09-12T01:33:15.127 INFO:tasks.workunit.client.0.smithi001.stdout:skipping zero length punch hole 2024-09-12T01:33:15.127 INFO:tasks.workunit.client.0.smithi001.stdout:skipping zero size read 2024-09-12T01:33:15.127 INFO:tasks.workunit.client.0.smithi001.stdout:skipping zero length punch hole 2024-09-12T01:33:15.127 INFO:tasks.workunit.client.0.smithi001.stdout:truncating to largest ever: 0x4310e9 2024-09-12T01:33:15.137 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0x954a45 2024-09-12T01:33:15.210 INFO:tasks.workunit.client.0.smithi001.stdout:truncating to largest ever: 0x9bc582 2024-09-12T01:33:15.253 INFO:tasks.workunit.client.0.smithi001.stdout:truncating to largest ever: 0x9eecbc 2024-09-12T01:33:16.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:33:16 smithi001 ceph-mon[30387]: pgmap v508: 129 pgs: 129 active+clean; 84 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 347 op/s 2024-09-12T01:33:16.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:33:16 smithi053 ceph-mon[35158]: pgmap v508: 129 pgs: 129 active+clean; 84 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 347 op/s 2024-09-12T01:33:17.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:33:16 smithi104 ceph-mon[35219]: pgmap v508: 129 pgs: 129 active+clean; 84 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 347 op/s 2024-09-12T01:33:18.796 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0x9fdd84 2024-09-12T01:33:18.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:33:18 smithi001 ceph-mon[30387]: pgmap v509: 129 pgs: 129 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 237 op/s 2024-09-12T01:33:18.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:33:18 smithi053 ceph-mon[35158]: pgmap v509: 129 pgs: 129 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 237 op/s 2024-09-12T01:33:19.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:33:18 smithi104 ceph-mon[35219]: pgmap v509: 129 pgs: 129 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 237 op/s 2024-09-12T01:33:20.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:33:20 smithi001 ceph-mon[30387]: pgmap v510: 129 pgs: 129 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 284 op/s 2024-09-12T01:33:20.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:33:20 smithi053 ceph-mon[35158]: pgmap v510: 129 pgs: 129 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 284 op/s 2024-09-12T01:33:21.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:33:20 smithi104 ceph-mon[35219]: pgmap v510: 129 pgs: 129 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 284 op/s 2024-09-12T01:33:22.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:33:22 smithi001 ceph-mon[30387]: pgmap v511: 129 pgs: 129 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.9 MiB/s wr, 347 op/s 2024-09-12T01:33:22.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:33:22 smithi053 ceph-mon[35158]: pgmap v511: 129 pgs: 129 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.9 MiB/s wr, 347 op/s 2024-09-12T01:33:23.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:33:22 smithi104 ceph-mon[35219]: pgmap v511: 129 pgs: 129 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.9 MiB/s wr, 347 op/s 2024-09-12T01:33:23.635 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.635+0000 7f6f04400640 1 -- 172.21.15.1:0/415544714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6efc101b60 msgr2=0x7f6efc101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:23.635 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.635+0000 7f6f04400640 1 --2- 172.21.15.1:0/415544714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6efc101b60 0x7f6efc101f60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0009900 tx=0x7f6ef002a410 comp rx=0 tx=0).stop 2024-09-12T01:33:23.635 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.635+0000 7f6f04400640 1 -- 172.21.15.1:0/415544714 shutdown_connections 2024-09-12T01:33:23.635 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.635+0000 7f6f04400640 1 --2- 172.21.15.1:0/415544714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6efc103710 0x7f6efc109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:23.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.635+0000 7f6f04400640 1 --2- 172.21.15.1:0/415544714 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6efc102d50 0x7f6efc1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:23.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.635+0000 7f6f04400640 1 --2- 172.21.15.1:0/415544714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6efc101b60 0x7f6efc101f60 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:23.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.635+0000 7f6f04400640 1 -- 172.21.15.1:0/415544714 >> 172.21.15.1:0/415544714 conn(0x7f6efc0fd2e0 msgr2=0x7f6efc0ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:23.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.635+0000 7f6f04400640 1 -- 172.21.15.1:0/415544714 shutdown_connections 2024-09-12T01:33:23.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.635+0000 7f6f04400640 1 -- 172.21.15.1:0/415544714 wait complete. 2024-09-12T01:33:23.638 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.636+0000 7f6f04400640 1 Processor -- start 2024-09-12T01:33:23.638 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.636+0000 7f6f04400640 1 -- start start 2024-09-12T01:33:23.638 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.637+0000 7f6f04400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6efc101b60 0x7f6efc19a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:23.638 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.637+0000 7f6f04400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6efc102d50 0x7f6efc19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:23.638 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.637+0000 7f6f04400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6efc103710 0x7f6efc1a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:23.638 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.637+0000 7f6f04400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc10d810 con 0x7f6efc103710 2024-09-12T01:33:23.638 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.637+0000 7f6f04400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc10d690 con 0x7f6efc101b60 2024-09-12T01:33:23.638 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.637+0000 7f6f04400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc10d990 con 0x7f6efc102d50 2024-09-12T01:33:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.637+0000 7f6f01a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6efc101b60 0x7f6efc19a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.637+0000 7f6f01a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6efc101b60 0x7f6efc19a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:60112/0 (socket says 172.21.15.1:60112) 2024-09-12T01:33:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.637+0000 7f6f01a00640 1 -- 172.21.15.1:0/3667826986 learned_addr learned my addr 172.21.15.1:0/3667826986 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:33:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.637+0000 7f6f01000640 1 --2- 172.21.15.1:0/3667826986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6efc102d50 0x7f6efc19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.637+0000 7f6f02400640 1 --2- 172.21.15.1:0/3667826986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6efc103710 0x7f6efc1a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.637+0000 7f6f01a00640 1 -- 172.21.15.1:0/3667826986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6efc102d50 msgr2=0x7f6efc19a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.637+0000 7f6f01a00640 1 --2- 172.21.15.1:0/3667826986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6efc102d50 0x7f6efc19a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.637+0000 7f6f01a00640 1 -- 172.21.15.1:0/3667826986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6efc103710 msgr2=0x7f6efc1a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.637+0000 7f6f01a00640 1 --2- 172.21.15.1:0/3667826986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6efc103710 0x7f6efc1a1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.637+0000 7f6f01a00640 1 -- 172.21.15.1:0/3667826986 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6efc1a1ce0 con 0x7f6efc101b60 2024-09-12T01:33:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.638+0000 7f6f01a00640 1 --2- 172.21.15.1:0/3667826986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6efc101b60 0x7f6efc19a0c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0031440 tx=0x7f6ef0032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.638+0000 7f6efa000640 1 -- 172.21.15.1:0/3667826986 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef003f030 con 0x7f6efc101b60 2024-09-12T01:33:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.638+0000 7f6efa000640 1 -- 172.21.15.1:0/3667826986 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ef0003ce0 con 0x7f6efc101b60 2024-09-12T01:33:23.640 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.638+0000 7f6efa000640 1 -- 172.21.15.1:0/3667826986 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef0035070 con 0x7f6efc101b60 2024-09-12T01:33:23.640 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.638+0000 7f6f04400640 1 -- 172.21.15.1:0/3667826986 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6efc1a1f70 con 0x7f6efc101b60 2024-09-12T01:33:23.640 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.639+0000 7f6f04400640 1 -- 172.21.15.1:0/3667826986 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6efc1a2320 con 0x7f6efc101b60 2024-09-12T01:33:23.642 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.642+0000 7f6efa000640 1 -- 172.21.15.1:0/3667826986 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f6ef0042030 con 0x7f6efc101b60 2024-09-12T01:33:23.642 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.643+0000 7f6f0741b740 1 Processor -- start 2024-09-12T01:33:23.642 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.643+0000 7f6f0741b740 1 -- start start 2024-09-12T01:33:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.643+0000 7f6f0741b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562218df8090 0x562218df8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.643+0000 7f6f0741b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562218df89d0 0x562218dfd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.643+0000 7f6f0741b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562218dfda60 0x562218dfff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.643+0000 7f6f0741b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562218d5d100 con 0x562218df89d0 2024-09-12T01:33:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.643+0000 7f6f0741b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x562218d051b0 con 0x562218df8090 2024-09-12T01:33:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.643+0000 7f6f0741b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x562218d590c0 con 0x562218dfda60 2024-09-12T01:33:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.643+0000 7f6efa000640 1 --2- 172.21.15.1:0/3667826986 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f6ee40a2b80 0x7f6ee40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.643+0000 7f6f02400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562218df8090 0x562218df8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.644+0000 7f6f02400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562218df8090 0x562218df8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:60116/0 (socket says 172.21.15.1:60116) 2024-09-12T01:33:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.644+0000 7f6f02400640 1 -- 172.21.15.1:0/2227724072 learned_addr learned my addr 172.21.15.1:0/2227724072 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:33:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.644+0000 7f6f01000640 1 --2- 172.21.15.1:0/2227724072 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562218dfda60 0x562218dfff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.644+0000 7f6efa000640 1 -- 172.21.15.1:0/3667826986 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f6ef00ef270 con 0x7f6efc101b60 2024-09-12T01:33:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.644+0000 7f6f01a00640 1 --2- 172.21.15.1:0/2227724072 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562218df89d0 0x562218dfd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.644+0000 7f6f01000640 1 -- 172.21.15.1:0/2227724072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562218df8090 msgr2=0x562218df8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.644+0000 7f6f01000640 1 --2- 172.21.15.1:0/2227724072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562218df8090 0x562218df8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.644+0000 7f6f01000640 1 -- 172.21.15.1:0/2227724072 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562218df89d0 msgr2=0x562218dfd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.644+0000 7f6f01000640 1 --2- 172.21.15.1:0/2227724072 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562218df89d0 0x562218dfd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.644+0000 7f6f01000640 1 -- 172.21.15.1:0/2227724072 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562218a70210 con 0x562218dfda60 2024-09-12T01:33:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.644+0000 7f6f02400640 1 --2- 172.21.15.1:0/3667826986 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f6ee40a2b80 0x7f6ee40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.644+0000 7f6f01000640 1 --2- 172.21.15.1:0/2227724072 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562218dfda60 0x562218dfff20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8009bf0 tx=0x7f6ee8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c9195400175a5ed server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.645+0000 7f6eeea00640 1 -- 172.21.15.1:0/2227724072 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8004050 con 0x562218dfda60 2024-09-12T01:33:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.645+0000 7f6eeea00640 1 -- 172.21.15.1:0/2227724072 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ee8005500 con 0x562218dfda60 2024-09-12T01:33:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.645+0000 7f6eeea00640 1 -- 172.21.15.1:0/2227724072 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8004e40 con 0x562218dfda60 2024-09-12T01:33:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.645+0000 7f6f0741b740 1 -- 172.21.15.1:0/2227724072 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562218dfda60 msgr2=0x562218dfff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.645+0000 7f6f02400640 1 --2- 172.21.15.1:0/3667826986 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f6ee40a2b80 0x7f6ee40a5040 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4009fd0 tx=0x7f6ef4014040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.645+0000 7f6f0741b740 1 --2- 172.21.15.1:0/2227724072 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562218dfda60 0x562218dfff20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8009bf0 tx=0x7f6ee8011810 comp rx=0 tx=0).stop 2024-09-12T01:33:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.645+0000 7f6f0741b740 1 -- 172.21.15.1:0/2227724072 shutdown_connections 2024-09-12T01:33:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.645+0000 7f6f0741b740 1 --2- 172.21.15.1:0/2227724072 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562218dfda60 0x562218dfff20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.645+0000 7f6f0741b740 1 --2- 172.21.15.1:0/2227724072 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562218df89d0 0x562218dfd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.645+0000 7f6f0741b740 1 --2- 172.21.15.1:0/2227724072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562218df8090 0x562218df8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.645+0000 7f6f0741b740 1 -- 172.21.15.1:0/2227724072 >> 172.21.15.1:0/2227724072 conn(0x562218df3700 msgr2=0x562218df3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.646+0000 7f6f0741b740 1 -- 172.21.15.1:0/2227724072 shutdown_connections 2024-09-12T01:33:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.646+0000 7f6f0741b740 1 -- 172.21.15.1:0/2227724072 wait complete. 2024-09-12T01:33:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.646+0000 7f6f0741b740 1 -- start start 2024-09-12T01:33:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.646+0000 7f6f0741b740 1 Processor -- start 2024-09-12T01:33:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.647+0000 7f6f0741b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562218df8090 0x562218e02180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.647+0000 7f6f0741b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562218df89d0 0x562218e026c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.647+0000 7f6f0741b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562218dfda60 0x562218e09700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.647+0000 7f6f0741b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562218e00a30 con 0x562218df8090 2024-09-12T01:33:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.647+0000 7f6f0741b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x562218ce6090 con 0x562218dfda60 2024-09-12T01:33:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.647+0000 7f6f0741b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x562218e00bb0 con 0x562218df89d0 2024-09-12T01:33:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.647+0000 7f6f0741b740 10 client.0 _ll_register_callbacks cb 0x562218d3af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:33:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.647+0000 7f6f01a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562218df8090 0x562218e02180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.647+0000 7f6f01000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562218df89d0 0x562218e026c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.647+0000 7f6f01a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562218df8090 0x562218e02180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36876/0 (socket says 172.21.15.1:36876) 2024-09-12T01:33:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.647+0000 7f6f01a00640 1 -- 172.21.15.1:0/1351063953 learned_addr learned my addr 172.21.15.1:0/1351063953 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:33:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.647+0000 7f6f01000640 1 -- 172.21.15.1:0/1351063953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562218dfda60 msgr2=0x562218e09700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.647+0000 7f6f02400640 1 --2- 172.21.15.1:0/1351063953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562218dfda60 0x562218e09700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.647+0000 7f6f01000640 1 --2- 172.21.15.1:0/1351063953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562218dfda60 0x562218e09700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.647+0000 7f6f01000640 1 -- 172.21.15.1:0/1351063953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562218df8090 msgr2=0x562218e02180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.647+0000 7f6f01000640 1 --2- 172.21.15.1:0/1351063953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562218df8090 0x562218e02180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.647+0000 7f6f01000640 1 -- 172.21.15.1:0/1351063953 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562218b3c620 con 0x562218df89d0 2024-09-12T01:33:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.647+0000 7f6f01a00640 1 --2- 172.21.15.1:0/1351063953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562218df8090 0x562218e02180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:33:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.647+0000 7f6f02400640 1 --2- 172.21.15.1:0/1351063953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562218dfda60 0x562218e09700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:33:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.648+0000 7f6f01000640 1 --2- 172.21.15.1:0/1351063953 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562218df89d0 0x562218e026c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6ee801b040 tx=0x7f6ee80056f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=339f0bc64f00b321 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.648+0000 7f6ee3400640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-12T01:33:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.648+0000 7f6ee3400640 1 -- 172.21.15.1:0/1351063953 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8029070 con 0x562218df89d0 2024-09-12T01:33:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.648+0000 7f6f0741b740 1 -- 172.21.15.1:0/1351063953 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ee8005500 con 0x562218df89d0 2024-09-12T01:33:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.648+0000 7f6ee3400640 1 -- 172.21.15.1:0/1351063953 <== mon.1 v2:172.21.15.104:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f6ee8004d80 con 0x562218df89d0 2024-09-12T01:33:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.648+0000 7f6ee3400640 1 -- 172.21.15.1:0/1351063953 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6ee8035050 con 0x562218df89d0 2024-09-12T01:33:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.648+0000 7f6f0741b740 10 client.25492 fetch_fsmap learned FSMap version 34 2024-09-12T01:33:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.648+0000 7f6f0741b740 1 -- 172.21.15.1:0/1351063953 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562218e0c540 con 0x562218df89d0 2024-09-12T01:33:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.649+0000 7f6ee3400640 1 -- 172.21.15.1:0/1351063953 <== mon.1 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ee8005500 con 0x562218df89d0 2024-09-12T01:33:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.649+0000 7f6ee3400640 1 -- 172.21.15.1:0/1351063953 <== mon.1 v2:172.21.15.104:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7f6ee8023430 con 0x562218df89d0 2024-09-12T01:33:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.649+0000 7f6f0741b740 10 client.25492 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T01:33:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.649+0000 7f6f0741b740 10 client.25492 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:33:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.649+0000 7f6f0741b740 20 client.25492 populate_metadata read hostname 'smithi001' 2024-09-12T01:33:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.649+0000 7f6f0741b740 1 --2- 172.21.15.1:0/1351063953 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x562218e0d9a0 0x562218e0fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.649+0000 7f6f0741b740 4 client.25492 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:33:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.649+0000 7f6f0741b740 1 -- 172.21.15.1:0/1351063953 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562218d6a830 con 0x562218e0d9a0 2024-09-12T01:33:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.649+0000 7f6f01a00640 1 --2- 172.21.15.1:0/1351063953 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x562218e0d9a0 0x562218e0fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.650+0000 7f6f01a00640 1 --2- 172.21.15.1:0/1351063953 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x562218e0d9a0 0x562218e0fd90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91fb4fc04c76f06e server_cookie=17f03d80daeb247f in_seq=0 out_seq=0 2024-09-12T01:33:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.650+0000 7f6ee3400640 10 client.25492 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:33:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.651+0000 7f6ee3400640 1 -- 172.21.15.1:0/1351063953 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562218d6a830 con 0x562218e0d9a0 2024-09-12T01:33:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.651+0000 7f6ee3400640 10 client.25492 handle_command_reply: tid=0 2024-09-12T01:33:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.651+0000 7f6f0741b740 1 client.25492 shutdown 2024-09-12T01:33:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.651+0000 7f6f0741b740 1 -- 172.21.15.1:0/1351063953 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562218df89d0 msgr2=0x562218e026c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.651+0000 7f6f0741b740 1 --2- 172.21.15.1:0/1351063953 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562218df89d0 0x562218e026c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6ee801b040 tx=0x7f6ee80056f0 comp rx=0 tx=0).stop 2024-09-12T01:33:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.651+0000 7f6f0741b740 1 -- 172.21.15.1:0/1351063953 shutdown_connections 2024-09-12T01:33:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.651+0000 7f6f0741b740 1 --2- 172.21.15.1:0/1351063953 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x562218e0d9a0 0x562218e0fd90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.651+0000 7f6f0741b740 1 --2- 172.21.15.1:0/1351063953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562218dfda60 0x562218e09700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.651+0000 7f6f0741b740 1 --2- 172.21.15.1:0/1351063953 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562218df89d0 0x562218e026c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.651+0000 7f6f0741b740 1 --2- 172.21.15.1:0/1351063953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562218df8090 0x562218e02180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.651+0000 7f6ee3400640 0 client.25492 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:33:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.651+0000 7f6f0741b740 1 -- 172.21.15.1:0/1351063953 >> 172.21.15.1:0/1351063953 conn(0x562218df3700 msgr2=0x562218df4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.651+0000 7f6f0741b740 1 -- 172.21.15.1:0/1351063953 shutdown_connections 2024-09-12T01:33:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.651+0000 7f6f0741b740 1 -- 172.21.15.1:0/1351063953 wait complete. 2024-09-12T01:33:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.652+0000 7f6f0741b740 20 client.25492 trim_cache size 0 max 16384 2024-09-12T01:33:23.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.670+0000 7f6f0741b740 1 Processor -- start 2024-09-12T01:33:23.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.670+0000 7f6f0741b740 1 -- start start 2024-09-12T01:33:23.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.670+0000 7f6f0741b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562218df8090 0x562218df8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:23.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.670+0000 7f6f0741b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562218df89d0 0x562218e096c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:23.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.670+0000 7f6f0741b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562218dfda60 0x562218df43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:23.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.670+0000 7f6f0741b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc1ac8e0 con 0x562218df8090 2024-09-12T01:33:23.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.670+0000 7f6f0741b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc1ae450 con 0x562218dfda60 2024-09-12T01:33:23.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.670+0000 7f6f0741b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc1acfb0 con 0x562218df89d0 2024-09-12T01:33:23.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.671+0000 7f6f01000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562218df89d0 0x562218e096c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:23.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.671+0000 7f6f01a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562218df8090 0x562218df8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:23.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.671+0000 7f6f01000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562218df89d0 0x562218e096c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:50168/0 (socket says 172.21.15.1:50168) 2024-09-12T01:33:23.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.671+0000 7f6f01000640 1 -- 172.21.15.1:0/4249930917 learned_addr learned my addr 172.21.15.1:0/4249930917 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:33:23.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.671+0000 7f6f02400640 1 --2- 172.21.15.1:0/4249930917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562218dfda60 0x562218df43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:23.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.671+0000 7f6f01000640 1 -- 172.21.15.1:0/4249930917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562218dfda60 msgr2=0x562218df43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:23.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.671+0000 7f6f01000640 1 --2- 172.21.15.1:0/4249930917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562218dfda60 0x562218df43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:23.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.671+0000 7f6f01000640 1 -- 172.21.15.1:0/4249930917 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562218df8090 msgr2=0x562218df8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:23.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.671+0000 7f6f01000640 1 --2- 172.21.15.1:0/4249930917 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562218df8090 0x562218df8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:23.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.671+0000 7f6f01000640 1 -- 172.21.15.1:0/4249930917 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562218deed60 con 0x562218df89d0 2024-09-12T01:33:23.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.671+0000 7f6f01a00640 1 --2- 172.21.15.1:0/4249930917 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562218df8090 0x562218df8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:33:23.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.671+0000 7f6f01000640 1 --2- 172.21.15.1:0/4249930917 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562218df89d0 0x562218e096c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8009560 tx=0x7f6ee8004600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0b86de795909197 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:23.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.672+0000 7f6eee000640 1 -- 172.21.15.1:0/4249930917 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8029050 con 0x562218df89d0 2024-09-12T01:33:23.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.672+0000 7f6eee000640 1 -- 172.21.15.1:0/4249930917 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ee8023430 con 0x562218df89d0 2024-09-12T01:33:23.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.672+0000 7f6eee000640 1 -- 172.21.15.1:0/4249930917 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8003ea0 con 0x562218df89d0 2024-09-12T01:33:23.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.672+0000 7f6f0741b740 1 -- 172.21.15.1:0/4249930917 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562218df89d0 msgr2=0x562218e096c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:23.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.672+0000 7f6f0741b740 1 --2- 172.21.15.1:0/4249930917 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562218df89d0 0x562218e096c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8009560 tx=0x7f6ee8004600 comp rx=0 tx=0).stop 2024-09-12T01:33:23.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.673+0000 7f6f0741b740 1 -- 172.21.15.1:0/4249930917 shutdown_connections 2024-09-12T01:33:23.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.673+0000 7f6f0741b740 1 --2- 172.21.15.1:0/4249930917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562218dfda60 0x562218df43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:23.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.673+0000 7f6f0741b740 1 --2- 172.21.15.1:0/4249930917 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562218df89d0 0x562218e096c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:23.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.673+0000 7f6f0741b740 1 --2- 172.21.15.1:0/4249930917 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562218df8090 0x562218df8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:23.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.673+0000 7f6f0741b740 1 -- 172.21.15.1:0/4249930917 >> 172.21.15.1:0/4249930917 conn(0x562218df3700 msgr2=0x562218df9b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:23.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.673+0000 7f6f0741b740 1 -- 172.21.15.1:0/4249930917 shutdown_connections 2024-09-12T01:33:23.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.673+0000 7f6f0741b740 1 -- 172.21.15.1:0/4249930917 wait complete. 2024-09-12T01:33:23.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.673+0000 7f6f0741b740 1 -- start start 2024-09-12T01:33:23.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.674+0000 7f6f0741b740 1 Processor -- start 2024-09-12T01:33:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.674+0000 7f6f0741b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562218df8090 0x562218e06660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.674+0000 7f6f0741b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562218dfda60 0x562218e06ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.674+0000 7f6f0741b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562218e74630 0x562218e76a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.674+0000 7f6f0741b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc1add50 con 0x562218df8090 2024-09-12T01:33:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.674+0000 7f6f0741b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc1ad680 con 0x562218e74630 2024-09-12T01:33:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.674+0000 7f6f0741b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc1aa8c0 con 0x562218dfda60 2024-09-12T01:33:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.674+0000 7f6f0741b740 10 client.0 _ll_register_callbacks cb 0x7f6ef4009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:33:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.675+0000 7f6f01000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562218dfda60 0x562218e06ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.675+0000 7f6f02400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562218e74630 0x562218e76a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.675+0000 7f6f01000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562218dfda60 0x562218e06ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:50184/0 (socket says 172.21.15.1:50184) 2024-09-12T01:33:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.675+0000 7f6f01000640 1 -- 172.21.15.1:0/1100237875 learned_addr learned my addr 172.21.15.1:0/1100237875 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:33:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.675+0000 7f6f01000640 1 -- 172.21.15.1:0/1100237875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562218e74630 msgr2=0x562218e76a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.675+0000 7f6f01000640 1 --2- 172.21.15.1:0/1100237875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562218e74630 0x562218e76a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.675+0000 7f6f01000640 1 -- 172.21.15.1:0/1100237875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562218df8090 msgr2=0x562218e06660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.675+0000 7f6f01000640 1 --2- 172.21.15.1:0/1100237875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562218df8090 0x562218e06660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.675+0000 7f6f01000640 1 -- 172.21.15.1:0/1100237875 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562218e76f60 con 0x562218dfda60 2024-09-12T01:33:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.675+0000 7f6f01000640 1 --2- 172.21.15.1:0/1100237875 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562218dfda60 0x562218e06ba0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8009bf0 tx=0x7f6ee8023650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd3da3fd3ebeef15 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.675+0000 7f6ee3400640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-12T01:33:23.677 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.676+0000 7f6ee3400640 1 -- 172.21.15.1:0/1100237875 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee803d020 con 0x562218dfda60 2024-09-12T01:33:23.677 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.676+0000 7f6ee3400640 1 -- 172.21.15.1:0/1100237875 <== mon.1 v2:172.21.15.104:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f6ee8003c20 con 0x562218dfda60 2024-09-12T01:33:23.678 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.676+0000 7f6f0741b740 1 -- 172.21.15.1:0/1100237875 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ee8023430 con 0x562218dfda60 2024-09-12T01:33:23.678 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.676+0000 7f6ee3400640 1 -- 172.21.15.1:0/1100237875 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6ee8029070 con 0x562218dfda60 2024-09-12T01:33:23.678 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.676+0000 7f6f0741b740 10 client.25498 fetch_fsmap learned FSMap version 34 2024-09-12T01:33:23.678 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.676+0000 7f6f0741b740 1 -- 172.21.15.1:0/1100237875 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562218e78270 con 0x562218dfda60 2024-09-12T01:33:23.678 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.677+0000 7f6ee3400640 1 -- 172.21.15.1:0/1100237875 <== mon.1 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ee8023430 con 0x562218dfda60 2024-09-12T01:33:23.678 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.677+0000 7f6ee3400640 1 -- 172.21.15.1:0/1100237875 <== mon.1 v2:172.21.15.104:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7f6ee802e940 con 0x562218dfda60 2024-09-12T01:33:23.678 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.677+0000 7f6f0741b740 10 client.25498 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T01:33:23.678 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.677+0000 7f6f0741b740 10 client.25498 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:33:23.678 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.677+0000 7f6f0741b740 20 client.25498 populate_metadata read hostname 'smithi001' 2024-09-12T01:33:23.678 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.677+0000 7f6f0741b740 1 --2- 172.21.15.1:0/1100237875 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x562218e78b60 0x562218e7af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:23.678 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.677+0000 7f6f0741b740 4 client.25498 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:33:23.678 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.677+0000 7f6f0741b740 1 -- 172.21.15.1:0/1100237875 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562218d64b80 con 0x562218e78b60 2024-09-12T01:33:23.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.678+0000 7f6f01a00640 1 --2- 172.21.15.1:0/1100237875 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x562218e78b60 0x562218e7af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:23.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.679+0000 7f6f01a00640 1 --2- 172.21.15.1:0/1100237875 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x562218e78b60 0x562218e7af50 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcd203d78cb03842 server_cookie=5c813b365ea966bd in_seq=0 out_seq=0 2024-09-12T01:33:23.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.679+0000 7f6ee3400640 10 client.25498 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:33:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.687+0000 7f6ee3400640 1 -- 172.21.15.1:0/1100237875 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562218d64b80 con 0x562218e78b60 2024-09-12T01:33:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.687+0000 7f6ee3400640 10 client.25498 handle_command_reply: tid=0 2024-09-12T01:33:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.687+0000 7f6f0741b740 1 client.25498 shutdown 2024-09-12T01:33:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.688+0000 7f6f0741b740 1 -- 172.21.15.1:0/1100237875 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562218dfda60 msgr2=0x562218e06ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.688+0000 7f6f0741b740 1 --2- 172.21.15.1:0/1100237875 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562218dfda60 0x562218e06ba0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8009bf0 tx=0x7f6ee8023650 comp rx=0 tx=0).stop 2024-09-12T01:33:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.688+0000 7f6f0741b740 1 -- 172.21.15.1:0/1100237875 shutdown_connections 2024-09-12T01:33:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.688+0000 7f6f0741b740 1 --2- 172.21.15.1:0/1100237875 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x562218e78b60 0x562218e7af50 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.688+0000 7f6f0741b740 1 --2- 172.21.15.1:0/1100237875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562218e74630 0x562218e76a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.688+0000 7f6f0741b740 1 --2- 172.21.15.1:0/1100237875 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562218dfda60 0x562218e06ba0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.688+0000 7f6f0741b740 1 --2- 172.21.15.1:0/1100237875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562218df8090 0x562218e06660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.688+0000 7f6ee3400640 0 client.25498 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:33:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.688+0000 7f6f0741b740 1 -- 172.21.15.1:0/1100237875 >> 172.21.15.1:0/1100237875 conn(0x562218df3700 msgr2=0x562218df2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.688+0000 7f6f0741b740 1 -- 172.21.15.1:0/1100237875 shutdown_connections 2024-09-12T01:33:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.688+0000 7f6f0741b740 1 -- 172.21.15.1:0/1100237875 wait complete. 2024-09-12T01:33:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.688+0000 7f6f0741b740 20 client.25498 trim_cache size 0 max 16384 2024-09-12T01:33:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-12T01:33:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-12T01:33:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-12T01:33:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-12T01:33:23.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.689+0000 7f6f04400640 1 -- 172.21.15.1:0/3667826986 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f6ee40a2b80 msgr2=0x7f6ee40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:23.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.689+0000 7f6f04400640 1 --2- 172.21.15.1:0/3667826986 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f6ee40a2b80 0x7f6ee40a5040 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4009fd0 tx=0x7f6ef4014040 comp rx=0 tx=0).stop 2024-09-12T01:33:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.689+0000 7f6f04400640 1 -- 172.21.15.1:0/3667826986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6efc101b60 msgr2=0x7f6efc19a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.689+0000 7f6f04400640 1 --2- 172.21.15.1:0/3667826986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6efc101b60 0x7f6efc19a0c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0031440 tx=0x7f6ef0032ce0 comp rx=0 tx=0).stop 2024-09-12T01:33:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.689+0000 7f6f04400640 1 -- 172.21.15.1:0/3667826986 shutdown_connections 2024-09-12T01:33:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.689+0000 7f6f04400640 1 --2- 172.21.15.1:0/3667826986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6efc103710 0x7f6efc1a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.689+0000 7f6f04400640 1 --2- 172.21.15.1:0/3667826986 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f6ee40a2b80 0x7f6ee40a5040 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.689+0000 7f6f04400640 1 --2- 172.21.15.1:0/3667826986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6efc102d50 0x7f6efc19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.689+0000 7f6f04400640 1 --2- 172.21.15.1:0/3667826986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6efc101b60 0x7f6efc19a0c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.689+0000 7f6f04400640 1 -- 172.21.15.1:0/3667826986 >> 172.21.15.1:0/3667826986 conn(0x7f6efc0fd2e0 msgr2=0x7f6efc108520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.689+0000 7f6f04400640 1 -- 172.21.15.1:0/3667826986 shutdown_connections 2024-09-12T01:33:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:23.689+0000 7f6f04400640 1 -- 172.21.15.1:0/3667826986 wait complete. 2024-09-12T01:33:23.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:33:23 smithi053 ceph-mon[35158]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T01:33:24.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:33:23 smithi104 ceph-mon[35219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T01:33:24.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:33:23 smithi001 ceph-mon[30387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T01:33:24.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:33:24 smithi053 ceph-mon[35158]: pgmap v512: 129 pgs: 129 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.6 MiB/s wr, 311 op/s 2024-09-12T01:33:25.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:33:24 smithi104 ceph-mon[35219]: pgmap v512: 129 pgs: 129 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.6 MiB/s wr, 311 op/s 2024-09-12T01:33:25.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:33:24 smithi001 ceph-mon[30387]: pgmap v512: 129 pgs: 129 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.6 MiB/s wr, 311 op/s 2024-09-12T01:33:26.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:33:26 smithi053 ceph-mon[35158]: pgmap v513: 129 pgs: 129 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.0 MiB/s wr, 418 op/s 2024-09-12T01:33:27.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:33:26 smithi104 ceph-mon[35219]: pgmap v513: 129 pgs: 129 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.0 MiB/s wr, 418 op/s 2024-09-12T01:33:27.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:33:26 smithi001 ceph-mon[30387]: pgmap v513: 129 pgs: 129 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.0 MiB/s wr, 418 op/s 2024-09-12T01:33:28.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:33:28 smithi053 ceph-mon[35158]: pgmap v514: 129 pgs: 129 active+clean; 89 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.8 MiB/s wr, 379 op/s 2024-09-12T01:33:29.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:33:28 smithi104 ceph-mon[35219]: pgmap v514: 129 pgs: 129 active+clean; 89 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.8 MiB/s wr, 379 op/s 2024-09-12T01:33:29.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:33:28 smithi001 ceph-mon[30387]: pgmap v514: 129 pgs: 129 active+clean; 89 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.8 MiB/s wr, 379 op/s 2024-09-12T01:33:29.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:33:29 smithi053 ceph-mon[35158]: pgmap v515: 129 pgs: 129 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.9 MiB/s wr, 424 op/s 2024-09-12T01:33:30.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:33:29 smithi104 ceph-mon[35219]: pgmap v515: 129 pgs: 129 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.9 MiB/s wr, 424 op/s 2024-09-12T01:33:30.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:33:29 smithi001 ceph-mon[30387]: pgmap v515: 129 pgs: 129 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.9 MiB/s wr, 424 op/s 2024-09-12T01:33:32.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:33:31 smithi001 ceph-mon[30387]: pgmap v516: 129 pgs: 129 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 2.5 MiB/s wr, 472 op/s 2024-09-12T01:33:32.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:33:31 smithi053 ceph-mon[35158]: pgmap v516: 129 pgs: 129 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 2.5 MiB/s wr, 472 op/s 2024-09-12T01:33:32.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:33:31 smithi104 ceph-mon[35219]: pgmap v516: 129 pgs: 129 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 2.5 MiB/s wr, 472 op/s 2024-09-12T01:33:34.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:33:33 smithi001 ceph-mon[30387]: pgmap v517: 129 pgs: 129 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.0 MiB/s wr, 397 op/s 2024-09-12T01:33:34.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:33:33 smithi053 ceph-mon[35158]: pgmap v517: 129 pgs: 129 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.0 MiB/s wr, 397 op/s 2024-09-12T01:33:34.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:33:33 smithi104 ceph-mon[35219]: pgmap v517: 129 pgs: 129 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.0 MiB/s wr, 397 op/s 2024-09-12T01:33:36.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:33:35 smithi001 ceph-mon[30387]: pgmap v518: 129 pgs: 129 active+clean; 90 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.2 MiB/s wr, 459 op/s 2024-09-12T01:33:36.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:33:35 smithi053 ceph-mon[35158]: pgmap v518: 129 pgs: 129 active+clean; 90 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.2 MiB/s wr, 459 op/s 2024-09-12T01:33:36.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:33:35 smithi104 ceph-mon[35219]: pgmap v518: 129 pgs: 129 active+clean; 90 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.2 MiB/s wr, 459 op/s 2024-09-12T01:33:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.833+0000 7fcbaaa00640 1 -- 172.21.15.1:0/399469471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcba41019b0 msgr2=0x7fcba4101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.833+0000 7fcbaaa00640 1 --2- 172.21.15.1:0/399469471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcba41019b0 0x7fcba4101db0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcb98009900 tx=0x7fcb9802a410 comp rx=0 tx=0).stop 2024-09-12T01:33:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.834+0000 7fcbaaa00640 1 -- 172.21.15.1:0/399469471 shutdown_connections 2024-09-12T01:33:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.834+0000 7fcbaaa00640 1 --2- 172.21.15.1:0/399469471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcba4103560 0x7fcba4109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.834+0000 7fcbaaa00640 1 --2- 172.21.15.1:0/399469471 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcba4102ba0 0x7fcba4103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.834+0000 7fcbaaa00640 1 --2- 172.21.15.1:0/399469471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcba41019b0 0x7fcba4101db0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.834+0000 7fcbaaa00640 1 -- 172.21.15.1:0/399469471 >> 172.21.15.1:0/399469471 conn(0x7fcba40fd130 msgr2=0x7fcba40ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.834+0000 7fcbaaa00640 1 -- 172.21.15.1:0/399469471 shutdown_connections 2024-09-12T01:33:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.834+0000 7fcbaaa00640 1 -- 172.21.15.1:0/399469471 wait complete. 2024-09-12T01:33:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.835+0000 7fcbaaa00640 1 Processor -- start 2024-09-12T01:33:36.836 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.835+0000 7fcbaaa00640 1 -- start start 2024-09-12T01:33:36.836 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.835+0000 7fcbaaa00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcba41019b0 0x7fcba4199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:36.836 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.835+0000 7fcbaaa00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcba4102ba0 0x7fcba419a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:36.836 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.835+0000 7fcbaaa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcba4103560 0x7fcba41a1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:36.836 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.835+0000 7fcbaaa00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcba410d660 con 0x7fcba4102ba0 2024-09-12T01:33:36.836 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.835+0000 7fcbaaa00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcba410d4e0 con 0x7fcba4103560 2024-09-12T01:33:36.836 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.835+0000 7fcbaaa00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcba410d7e0 con 0x7fcba41019b0 2024-09-12T01:33:36.836 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.836+0000 7fcbaa000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcba4103560 0x7fcba41a1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:36.836 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.836+0000 7fcbaa000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcba4103560 0x7fcba41a1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:48942/0 (socket says 172.21.15.1:48942) 2024-09-12T01:33:36.836 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.836+0000 7fcba9600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcba41019b0 0x7fcba4199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:36.836 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.836+0000 7fcbaa000640 1 -- 172.21.15.1:0/1785277169 learned_addr learned my addr 172.21.15.1:0/1785277169 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:33:36.836 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.836+0000 7fcba8c00640 1 --2- 172.21.15.1:0/1785277169 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcba4102ba0 0x7fcba419a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:36.836 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.836+0000 7fcbaa000640 1 -- 172.21.15.1:0/1785277169 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcba41019b0 msgr2=0x7fcba4199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:36.837 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.836+0000 7fcbaa000640 1 --2- 172.21.15.1:0/1785277169 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcba41019b0 0x7fcba4199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.837 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.836+0000 7fcbaa000640 1 -- 172.21.15.1:0/1785277169 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcba4102ba0 msgr2=0x7fcba419a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:36.837 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.836+0000 7fcbaa000640 1 --2- 172.21.15.1:0/1785277169 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcba4102ba0 0x7fcba419a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.837 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.836+0000 7fcbaa000640 1 -- 172.21.15.1:0/1785277169 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcba41a1ae0 con 0x7fcba4103560 2024-09-12T01:33:36.838 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.836+0000 7fcbaa000640 1 --2- 172.21.15.1:0/1785277169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcba4103560 0x7fcba41a1470 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c00eb10 tx=0x7fcb9c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:36.838 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.837+0000 7fcba2000640 1 -- 172.21.15.1:0/1785277169 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c004b20 con 0x7fcba4103560 2024-09-12T01:33:36.838 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.837+0000 7fcba2000640 1 -- 172.21.15.1:0/1785277169 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb9c004cc0 con 0x7fcba4103560 2024-09-12T01:33:36.838 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.837+0000 7fcbaaa00640 1 -- 172.21.15.1:0/1785277169 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcba41a1dd0 con 0x7fcba4103560 2024-09-12T01:33:36.838 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.837+0000 7fcba2000640 1 -- 172.21.15.1:0/1785277169 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c010760 con 0x7fcba4103560 2024-09-12T01:33:36.838 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.837+0000 7fcbaaa00640 1 -- 172.21.15.1:0/1785277169 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcba41a2240 con 0x7fcba4103560 2024-09-12T01:33:36.839 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.840+0000 7fcbaede5740 1 Processor -- start 2024-09-12T01:33:36.841 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.841+0000 7fcba2000640 1 -- 172.21.15.1:0/1785277169 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fcb9c005020 con 0x7fcba4103560 2024-09-12T01:33:36.841 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.841+0000 7fcbaede5740 1 -- start start 2024-09-12T01:33:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.841+0000 7fcbaede5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562f909c260 0x5562f909c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.842+0000 7fcbaede5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5562f909cba0 0x5562f90a1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.842+0000 7fcbaede5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562f90a1c30 0x5562f90a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.842+0000 7fcbaede5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5562f9001100 con 0x5562f90a1c30 2024-09-12T01:33:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.842+0000 7fcbaede5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5562f8fa91b0 con 0x5562f909c260 2024-09-12T01:33:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.842+0000 7fcbaede5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5562f8ffd0c0 con 0x5562f909cba0 2024-09-12T01:33:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.842+0000 7fcba9600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5562f909cba0 0x5562f90a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.842+0000 7fcba9600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5562f909cba0 0x5562f90a1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:37112/0 (socket says 172.21.15.1:37112) 2024-09-12T01:33:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.842+0000 7fcba9600640 1 -- 172.21.15.1:0/17649992 learned_addr learned my addr 172.21.15.1:0/17649992 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:33:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.842+0000 7fcba8c00640 1 --2- 172.21.15.1:0/17649992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562f909c260 0x5562f909c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.842+0000 7fcba8c00640 1 --2- 172.21.15.1:0/17649992 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562f90a1c30 0x5562f90a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.842+0000 7fcba9600640 1 -- 172.21.15.1:0/17649992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562f909c260 msgr2=0x5562f909c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.842+0000 7fcba9600640 1 --2- 172.21.15.1:0/17649992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562f909c260 0x5562f909c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.842+0000 7fcba9600640 1 -- 172.21.15.1:0/17649992 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562f90a1c30 msgr2=0x5562f90a40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.842+0000 7fcba9600640 1 --2- 172.21.15.1:0/17649992 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562f90a1c30 0x5562f90a40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.842+0000 7fcba9600640 1 -- 172.21.15.1:0/17649992 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562f8d14210 con 0x5562f909cba0 2024-09-12T01:33:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.842+0000 7fcba8c00640 1 --2- 172.21.15.1:0/17649992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562f909c260 0x5562f909c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:33:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.843+0000 7fcba9600640 1 --2- 172.21.15.1:0/17649992 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5562f909cba0 0x5562f90a1660 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcb98006010 tx=0x7fcb98004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88af60f5c151ff36 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.843+0000 7fcba2000640 1 --2- 172.21.15.1:0/1785277169 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fcb8c0a2b70 0x7fcb8c0a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.843+0000 7fcba2000640 1 -- 172.21.15.1:0/1785277169 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fcb9c0cf9e0 con 0x7fcba4103560 2024-09-12T01:33:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.843+0000 7fcba8c00640 1 --2- 172.21.15.1:0/1785277169 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fcb8c0a2b70 0x7fcb8c0a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.844+0000 7fcba8c00640 1 --2- 172.21.15.1:0/1785277169 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fcb8c0a2b70 0x7fcb8c0a5030 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcb90009730 tx=0x7fcb90005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.845+0000 7fcb96600640 1 -- 172.21.15.1:0/17649992 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb98035040 con 0x5562f909cba0 2024-09-12T01:33:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.845+0000 7fcb96600640 1 -- 172.21.15.1:0/17649992 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb98004430 con 0x5562f909cba0 2024-09-12T01:33:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.845+0000 7fcb96600640 1 -- 172.21.15.1:0/17649992 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb980316b0 con 0x5562f909cba0 2024-09-12T01:33:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.846+0000 7fcbaede5740 1 -- 172.21.15.1:0/17649992 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5562f909cba0 msgr2=0x5562f90a1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.846+0000 7fcbaede5740 1 --2- 172.21.15.1:0/17649992 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5562f909cba0 0x5562f90a1660 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcb98006010 tx=0x7fcb98004040 comp rx=0 tx=0).stop 2024-09-12T01:33:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.846+0000 7fcbaede5740 1 -- 172.21.15.1:0/17649992 shutdown_connections 2024-09-12T01:33:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.846+0000 7fcbaede5740 1 --2- 172.21.15.1:0/17649992 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562f90a1c30 0x5562f90a40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.846+0000 7fcbaede5740 1 --2- 172.21.15.1:0/17649992 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5562f909cba0 0x5562f90a1660 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.846+0000 7fcbaede5740 1 --2- 172.21.15.1:0/17649992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562f909c260 0x5562f909c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.846+0000 7fcbaede5740 1 -- 172.21.15.1:0/17649992 >> 172.21.15.1:0/17649992 conn(0x5562f90978d0 msgr2=0x5562f9097cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.846+0000 7fcbaede5740 1 -- 172.21.15.1:0/17649992 shutdown_connections 2024-09-12T01:33:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.846+0000 7fcbaede5740 1 -- 172.21.15.1:0/17649992 wait complete. 2024-09-12T01:33:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.847+0000 7fcbaede5740 1 -- start start 2024-09-12T01:33:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.847+0000 7fcbaede5740 1 Processor -- start 2024-09-12T01:33:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.848+0000 7fcbaede5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562f909c260 0x5562f90a6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.848+0000 7fcbaede5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5562f909cba0 0x5562f90a69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.848+0000 7fcbaede5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562f90a1c30 0x5562f90ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.848+0000 7fcbaede5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5562f90a4ba0 con 0x5562f90a1c30 2024-09-12T01:33:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.848+0000 7fcbaede5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5562f8f8a090 con 0x5562f909c260 2024-09-12T01:33:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.848+0000 7fcbaede5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5562f90a4d20 con 0x5562f909cba0 2024-09-12T01:33:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.848+0000 7fcbaede5740 10 client.0 _ll_register_callbacks cb 0x5562f8fdef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:33:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.848+0000 7fcba9600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562f909c260 0x5562f90a6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.848+0000 7fcba9600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562f909c260 0x5562f90a6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:48956/0 (socket says 172.21.15.1:48956) 2024-09-12T01:33:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.848+0000 7fcba9600640 1 -- 172.21.15.1:0/2392024377 learned_addr learned my addr 172.21.15.1:0/2392024377 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:33:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.848+0000 7fcba8c00640 1 --2- 172.21.15.1:0/2392024377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5562f909cba0 0x5562f90a69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.848+0000 7fcbaa000640 1 --2- 172.21.15.1:0/2392024377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562f90a1c30 0x5562f90ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.849+0000 7fcbaa000640 1 -- 172.21.15.1:0/2392024377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5562f909cba0 msgr2=0x5562f90a69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.849+0000 7fcbaa000640 1 --2- 172.21.15.1:0/2392024377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5562f909cba0 0x5562f90a69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.849+0000 7fcbaa000640 1 -- 172.21.15.1:0/2392024377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562f909c260 msgr2=0x5562f90a6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.849+0000 7fcbaa000640 1 --2- 172.21.15.1:0/2392024377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562f909c260 0x5562f90a6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.849+0000 7fcbaa000640 1 -- 172.21.15.1:0/2392024377 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562f8de0620 con 0x5562f90a1c30 2024-09-12T01:33:36.850 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.850+0000 7fcbaa000640 1 --2- 172.21.15.1:0/2392024377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562f90a1c30 0x5562f90ad9f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c0cd0f0 tx=0x7fcb9c09b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1b579f67be1cc5a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:36.850 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.851+0000 7fcb8aa00640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-12T01:33:36.850 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.851+0000 7fcb8aa00640 1 -- 172.21.15.1:0/2392024377 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c0a4830 con 0x5562f90a1c30 2024-09-12T01:33:36.850 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.851+0000 7fcb8aa00640 1 -- 172.21.15.1:0/2392024377 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fcb9c005310 con 0x5562f90a1c30 2024-09-12T01:33:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.851+0000 7fcbaede5740 1 -- 172.21.15.1:0/2392024377 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb98004430 con 0x5562f90a1c30 2024-09-12T01:33:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.851+0000 7fcb8aa00640 1 -- 172.21.15.1:0/2392024377 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcb9c0a0200 con 0x5562f90a1c30 2024-09-12T01:33:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.851+0000 7fcbaede5740 10 client.15465 fetch_fsmap learned FSMap version 34 2024-09-12T01:33:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.851+0000 7fcbaede5740 1 -- 172.21.15.1:0/2392024377 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5562f90c18c0 con 0x5562f90a1c30 2024-09-12T01:33:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.852+0000 7fcb8aa00640 1 -- 172.21.15.1:0/2392024377 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb98004430 con 0x5562f90a1c30 2024-09-12T01:33:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.852+0000 7fcb8aa00640 1 -- 172.21.15.1:0/2392024377 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7fcb9c09ad10 con 0x5562f90a1c30 2024-09-12T01:33:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.852+0000 7fcbaede5740 10 client.15465 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T01:33:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.852+0000 7fcbaede5740 10 client.15465 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:33:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.852+0000 7fcbaede5740 20 client.15465 populate_metadata read hostname 'smithi001' 2024-09-12T01:33:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.852+0000 7fcbaede5740 1 --2- 172.21.15.1:0/2392024377 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5562f90c3030 0x5562f90c5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.852+0000 7fcbaede5740 4 client.15465 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:33:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.852+0000 7fcbaede5740 1 -- 172.21.15.1:0/2392024377 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562f900e830 con 0x5562f90c3030 2024-09-12T01:33:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.853+0000 7fcba9600640 1 --2- 172.21.15.1:0/2392024377 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5562f90c3030 0x5562f90c5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.854+0000 7fcb8aa00640 10 client.15465 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:33:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.854+0000 7fcba9600640 1 --2- 172.21.15.1:0/2392024377 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5562f90c3030 0x5562f90c5480 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15b7b9734b041b1 server_cookie=c89e52f6023b3b62 in_seq=0 out_seq=0 2024-09-12T01:33:36.856 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.857+0000 7fcb8aa00640 1 -- 172.21.15.1:0/2392024377 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5562f900e830 con 0x5562f90c3030 2024-09-12T01:33:36.856 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.857+0000 7fcb8aa00640 10 client.15465 handle_command_reply: tid=0 2024-09-12T01:33:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.857+0000 7fcbaede5740 1 client.15465 shutdown 2024-09-12T01:33:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.857+0000 7fcbaede5740 1 -- 172.21.15.1:0/2392024377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562f90a1c30 msgr2=0x5562f90ad9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.857+0000 7fcbaede5740 1 --2- 172.21.15.1:0/2392024377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562f90a1c30 0x5562f90ad9f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c0cd0f0 tx=0x7fcb9c09b870 comp rx=0 tx=0).stop 2024-09-12T01:33:36.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.857+0000 7fcbaede5740 1 -- 172.21.15.1:0/2392024377 shutdown_connections 2024-09-12T01:33:36.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.857+0000 7fcbaede5740 1 --2- 172.21.15.1:0/2392024377 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5562f90c3030 0x5562f90c5480 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.857+0000 7fcbaede5740 1 --2- 172.21.15.1:0/2392024377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562f90a1c30 0x5562f90ad9f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.857+0000 7fcb8aa00640 0 client.15465 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:33:36.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.857+0000 7fcbaede5740 1 --2- 172.21.15.1:0/2392024377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5562f909cba0 0x5562f90a69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.857+0000 7fcbaede5740 1 --2- 172.21.15.1:0/2392024377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562f909c260 0x5562f90a6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.857+0000 7fcbaede5740 1 -- 172.21.15.1:0/2392024377 >> 172.21.15.1:0/2392024377 conn(0x5562f90978d0 msgr2=0x5562f90990e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:36.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.858+0000 7fcbaede5740 1 -- 172.21.15.1:0/2392024377 shutdown_connections 2024-09-12T01:33:36.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.858+0000 7fcbaede5740 1 -- 172.21.15.1:0/2392024377 wait complete. 2024-09-12T01:33:36.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.858+0000 7fcbaede5740 20 client.15465 trim_cache size 0 max 16384 2024-09-12T01:33:36.879 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.878+0000 7fcbaede5740 1 Processor -- start 2024-09-12T01:33:36.879 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.878+0000 7fcbaede5740 1 -- start start 2024-09-12T01:33:36.879 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.879+0000 7fcbaede5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562f909c260 0x5562f9097cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:36.879 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.879+0000 7fcbaede5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5562f909cba0 0x5562f90ad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:36.879 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.879+0000 7fcbaede5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562f90a1c30 0x5562f90981f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:36.879 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.879+0000 7fcbaede5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcba41a3f30 con 0x5562f909c260 2024-09-12T01:33:36.879 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.879+0000 7fcbaede5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcba41a5aa0 con 0x5562f90a1c30 2024-09-12T01:33:36.879 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.879+0000 7fcbaede5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcba41a4600 con 0x5562f909cba0 2024-09-12T01:33:36.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.879+0000 7fcbaa000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562f90a1c30 0x5562f90981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:36.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.879+0000 7fcbaa000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562f90a1c30 0x5562f90981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:48970/0 (socket says 172.21.15.1:48970) 2024-09-12T01:33:36.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.879+0000 7fcba8c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5562f909cba0 0x5562f90ad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:36.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.879+0000 7fcbaa000640 1 -- 172.21.15.1:0/1701282453 learned_addr learned my addr 172.21.15.1:0/1701282453 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:33:36.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.879+0000 7fcbaa000640 1 -- 172.21.15.1:0/1701282453 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5562f909cba0 msgr2=0x5562f90ad9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:36.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.879+0000 7fcbaa000640 1 --2- 172.21.15.1:0/1701282453 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5562f909cba0 0x5562f90ad9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.879+0000 7fcbaa000640 1 -- 172.21.15.1:0/1701282453 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562f909c260 msgr2=0x5562f9097cb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T01:33:36.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.879+0000 7fcbaa000640 1 --2- 172.21.15.1:0/1701282453 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562f909c260 0x5562f9097cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.879+0000 7fcbaa000640 1 -- 172.21.15.1:0/1701282453 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562f90976f0 con 0x5562f90a1c30 2024-09-12T01:33:36.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.879+0000 7fcbaa000640 1 --2- 172.21.15.1:0/1701282453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562f90a1c30 0x5562f90981f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c005880 tx=0x7fcb9c009220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b77651aa9417fa3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:36.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.880+0000 7fcb95c00640 1 -- 172.21.15.1:0/1701282453 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c0a4ac0 con 0x5562f90a1c30 2024-09-12T01:33:36.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.880+0000 7fcb95c00640 1 -- 172.21.15.1:0/1701282453 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb9c09e070 con 0x5562f90a1c30 2024-09-12T01:33:36.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.880+0000 7fcb95c00640 1 -- 172.21.15.1:0/1701282453 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c005570 con 0x5562f90a1c30 2024-09-12T01:33:36.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.880+0000 7fcbaede5740 1 -- 172.21.15.1:0/1701282453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562f90a1c30 msgr2=0x5562f90981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:36.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.880+0000 7fcbaede5740 1 --2- 172.21.15.1:0/1701282453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562f90a1c30 0x5562f90981f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c005880 tx=0x7fcb9c009220 comp rx=0 tx=0).stop 2024-09-12T01:33:36.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.880+0000 7fcbaede5740 1 -- 172.21.15.1:0/1701282453 shutdown_connections 2024-09-12T01:33:36.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.880+0000 7fcbaede5740 1 --2- 172.21.15.1:0/1701282453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562f90a1c30 0x5562f90981f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.881+0000 7fcbaede5740 1 --2- 172.21.15.1:0/1701282453 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5562f909cba0 0x5562f90ad9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.881+0000 7fcbaede5740 1 --2- 172.21.15.1:0/1701282453 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562f909c260 0x5562f9097cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.881+0000 7fcbaede5740 1 -- 172.21.15.1:0/1701282453 >> 172.21.15.1:0/1701282453 conn(0x5562f90978d0 msgr2=0x5562f90a6470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:36.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.881+0000 7fcbaede5740 1 -- 172.21.15.1:0/1701282453 shutdown_connections 2024-09-12T01:33:36.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.881+0000 7fcbaede5740 1 -- 172.21.15.1:0/1701282453 wait complete. 2024-09-12T01:33:36.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.881+0000 7fcbaede5740 1 -- start start 2024-09-12T01:33:36.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.881+0000 7fcbaede5740 1 Processor -- start 2024-09-12T01:33:36.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.882+0000 7fcbaede5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562f909c260 0x5562f90aad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:36.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.882+0000 7fcbaede5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5562f909cba0 0x5562f90ab260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:36.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.882+0000 7fcbaede5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562f90a1c30 0x5562f90ac220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:36.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.882+0000 7fcbaede5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcba41a53a0 con 0x5562f90a1c30 2024-09-12T01:33:36.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.882+0000 7fcbaede5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcba41a4cd0 con 0x5562f909c260 2024-09-12T01:33:36.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.882+0000 7fcbaede5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcba4073e60 con 0x5562f909cba0 2024-09-12T01:33:36.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.882+0000 7fcbaede5740 10 client.0 _ll_register_callbacks cb 0x7fcb9c0057d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:33:36.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.882+0000 7fcba9600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562f909c260 0x5562f90aad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:36.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.882+0000 7fcba9600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562f909c260 0x5562f90aad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:48984/0 (socket says 172.21.15.1:48984) 2024-09-12T01:33:36.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.882+0000 7fcba9600640 1 -- 172.21.15.1:0/1572307927 learned_addr learned my addr 172.21.15.1:0/1572307927 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:33:36.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.882+0000 7fcbaa000640 1 --2- 172.21.15.1:0/1572307927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562f90a1c30 0x5562f90ac220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:36.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.882+0000 7fcba8c00640 1 --2- 172.21.15.1:0/1572307927 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5562f909cba0 0x5562f90ab260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:36.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.882+0000 7fcba9600640 1 -- 172.21.15.1:0/1572307927 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5562f909cba0 msgr2=0x5562f90ab260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:36.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.882+0000 7fcba9600640 1 --2- 172.21.15.1:0/1572307927 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5562f909cba0 0x5562f90ab260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.882+0000 7fcba9600640 1 -- 172.21.15.1:0/1572307927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562f90a1c30 msgr2=0x5562f90ac220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:36.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.882+0000 7fcba9600640 1 --2- 172.21.15.1:0/1572307927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562f90a1c30 0x5562f90ac220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.882+0000 7fcba9600640 1 -- 172.21.15.1:0/1572307927 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562f90ac830 con 0x5562f909c260 2024-09-12T01:33:36.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.882+0000 7fcba8c00640 1 --2- 172.21.15.1:0/1572307927 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5562f909cba0 0x5562f90ab260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:33:36.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.883+0000 7fcba9600640 1 --2- 172.21.15.1:0/1572307927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562f909c260 0x5562f90aad20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcb98009340 tx=0x7fcb980045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=848e589ca8800640 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:36.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.883+0000 7fcb8aa00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-12T01:33:36.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.883+0000 7fcb8aa00640 1 -- 172.21.15.1:0/1572307927 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb98035040 con 0x5562f909c260 2024-09-12T01:33:36.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.883+0000 7fcb8aa00640 1 -- 172.21.15.1:0/1572307927 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fcb98031ac0 con 0x5562f909c260 2024-09-12T01:33:36.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.883+0000 7fcbaede5740 1 -- 172.21.15.1:0/1572307927 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb9c09e070 con 0x5562f909c260 2024-09-12T01:33:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.883+0000 7fcb8aa00640 1 -- 172.21.15.1:0/1572307927 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcb9805d050 con 0x5562f909c260 2024-09-12T01:33:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.883+0000 7fcbaede5740 10 client.25514 fetch_fsmap learned FSMap version 34 2024-09-12T01:33:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.883+0000 7fcbaede5740 1 -- 172.21.15.1:0/1572307927 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5562f90adba0 con 0x5562f909c260 2024-09-12T01:33:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.883+0000 7fcb8aa00640 1 -- 172.21.15.1:0/1572307927 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb9c09e070 con 0x5562f909c260 2024-09-12T01:33:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.884+0000 7fcb8aa00640 1 -- 172.21.15.1:0/1572307927 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7fcb9804f560 con 0x5562f909c260 2024-09-12T01:33:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.884+0000 7fcbaede5740 10 client.25514 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T01:33:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.884+0000 7fcbaede5740 10 client.25514 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:33:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.884+0000 7fcbaede5740 20 client.25514 populate_metadata read hostname 'smithi001' 2024-09-12T01:33:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.884+0000 7fcbaede5740 1 --2- 172.21.15.1:0/1572307927 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5562f912df20 0x5562f9130310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.884+0000 7fcbaede5740 4 client.25514 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:33:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.884+0000 7fcbaede5740 1 -- 172.21.15.1:0/1572307927 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5562f9008b80 con 0x5562f912df20 2024-09-12T01:33:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.884+0000 7fcba8c00640 1 --2- 172.21.15.1:0/1572307927 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5562f912df20 0x5562f9130310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:36.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.885+0000 7fcba8c00640 1 --2- 172.21.15.1:0/1572307927 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5562f912df20 0x5562f9130310 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5859b50371b8c19 server_cookie=b6f89a3e9acd5a70 in_seq=0 out_seq=0 2024-09-12T01:33:36.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.885+0000 7fcb8aa00640 10 client.25514 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:33:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.894+0000 7fcb8aa00640 1 -- 172.21.15.1:0/1572307927 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5562f9008b80 con 0x5562f912df20 2024-09-12T01:33:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.894+0000 7fcb8aa00640 10 client.25514 handle_command_reply: tid=0 2024-09-12T01:33:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.894+0000 7fcbaede5740 1 client.25514 shutdown 2024-09-12T01:33:36.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.894+0000 7fcbaede5740 1 -- 172.21.15.1:0/1572307927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562f909c260 msgr2=0x5562f90aad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:36.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.894+0000 7fcbaede5740 1 --2- 172.21.15.1:0/1572307927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562f909c260 0x5562f90aad20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcb98009340 tx=0x7fcb980045e0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.894+0000 7fcbaede5740 1 -- 172.21.15.1:0/1572307927 shutdown_connections 2024-09-12T01:33:36.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.894+0000 7fcbaede5740 1 --2- 172.21.15.1:0/1572307927 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5562f912df20 0x5562f9130310 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.894+0000 7fcbaede5740 1 --2- 172.21.15.1:0/1572307927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562f90a1c30 0x5562f90ac220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.894+0000 7fcbaede5740 1 --2- 172.21.15.1:0/1572307927 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5562f909cba0 0x5562f90ab260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.894+0000 7fcbaede5740 1 --2- 172.21.15.1:0/1572307927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562f909c260 0x5562f90aad20 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.894+0000 7fcb8aa00640 0 client.25514 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:33:36.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.894+0000 7fcbaede5740 1 -- 172.21.15.1:0/1572307927 >> 172.21.15.1:0/1572307927 conn(0x5562f90978d0 msgr2=0x5562f90961f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:36.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.894+0000 7fcbaede5740 1 -- 172.21.15.1:0/1572307927 shutdown_connections 2024-09-12T01:33:36.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.894+0000 7fcbaede5740 1 -- 172.21.15.1:0/1572307927 wait complete. 2024-09-12T01:33:36.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.895+0000 7fcbaede5740 20 client.25514 trim_cache size 0 max 16384 2024-09-12T01:33:36.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-12T01:33:36.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-12T01:33:36.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-12T01:33:36.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-12T01:33:36.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.896+0000 7fcbaaa00640 1 -- 172.21.15.1:0/1785277169 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fcb8c0a2b70 msgr2=0x7fcb8c0a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:36.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.896+0000 7fcbaaa00640 1 --2- 172.21.15.1:0/1785277169 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fcb8c0a2b70 0x7fcb8c0a5030 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcb90009730 tx=0x7fcb90005f90 comp rx=0 tx=0).stop 2024-09-12T01:33:36.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.896+0000 7fcbaaa00640 1 -- 172.21.15.1:0/1785277169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcba4103560 msgr2=0x7fcba41a1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:36.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.896+0000 7fcbaaa00640 1 --2- 172.21.15.1:0/1785277169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcba4103560 0x7fcba41a1470 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c00eb10 tx=0x7fcb9c00c2d0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.896+0000 7fcbaaa00640 1 -- 172.21.15.1:0/1785277169 shutdown_connections 2024-09-12T01:33:36.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.896+0000 7fcbaaa00640 1 --2- 172.21.15.1:0/1785277169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcba4103560 0x7fcba41a1470 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.896+0000 7fcbaaa00640 1 --2- 172.21.15.1:0/1785277169 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcba4102ba0 0x7fcba419a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.896+0000 7fcbaaa00640 1 --2- 172.21.15.1:0/1785277169 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fcb8c0a2b70 0x7fcb8c0a5030 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.896+0000 7fcbaaa00640 1 --2- 172.21.15.1:0/1785277169 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcba41019b0 0x7fcba4199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:36.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.896+0000 7fcbaaa00640 1 -- 172.21.15.1:0/1785277169 >> 172.21.15.1:0/1785277169 conn(0x7fcba40fd130 msgr2=0x7fcba4074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:36.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.896+0000 7fcbaaa00640 1 -- 172.21.15.1:0/1785277169 shutdown_connections 2024-09-12T01:33:36.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:36.896+0000 7fcbaaa00640 1 -- 172.21.15.1:0/1785277169 wait complete. 2024-09-12T01:33:38.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:33:37 smithi001 ceph-mon[30387]: pgmap v519: 129 pgs: 129 active+clean; 90 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.2 MiB/s wr, 420 op/s 2024-09-12T01:33:38.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:33:37 smithi053 ceph-mon[35158]: pgmap v519: 129 pgs: 129 active+clean; 90 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.2 MiB/s wr, 420 op/s 2024-09-12T01:33:38.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:33:37 smithi104 ceph-mon[35219]: pgmap v519: 129 pgs: 129 active+clean; 90 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.2 MiB/s wr, 420 op/s 2024-09-12T01:33:39.975 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0x9ff773 2024-09-12T01:33:40.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:33:39 smithi001 ceph-mon[30387]: pgmap v520: 129 pgs: 129 active+clean; 91 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.9 MiB/s wr, 394 op/s 2024-09-12T01:33:40.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:33:39 smithi053 ceph-mon[35158]: pgmap v520: 129 pgs: 129 active+clean; 91 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.9 MiB/s wr, 394 op/s 2024-09-12T01:33:40.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:33:39 smithi104 ceph-mon[35219]: pgmap v520: 129 pgs: 129 active+clean; 91 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.9 MiB/s wr, 394 op/s 2024-09-12T01:33:40.625 INFO:tasks.workunit.client.0.smithi001.stdout:20000 punch from 0x34ddcb to 0x35324f, (0x5484 bytes) 2024-09-12T01:33:42.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:33:41 smithi001 ceph-mon[30387]: pgmap v521: 129 pgs: 129 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.3 MiB/s wr, 427 op/s 2024-09-12T01:33:42.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:33:41 smithi053 ceph-mon[35158]: pgmap v521: 129 pgs: 129 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.3 MiB/s wr, 427 op/s 2024-09-12T01:33:42.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:33:41 smithi104 ceph-mon[35219]: pgmap v521: 129 pgs: 129 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.3 MiB/s wr, 427 op/s 2024-09-12T01:33:44.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:33:43 smithi001 ceph-mon[30387]: pgmap v522: 129 pgs: 129 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.8 MiB/s wr, 354 op/s 2024-09-12T01:33:44.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:33:43 smithi053 ceph-mon[35158]: pgmap v522: 129 pgs: 129 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.8 MiB/s wr, 354 op/s 2024-09-12T01:33:44.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:33:43 smithi104 ceph-mon[35219]: pgmap v522: 129 pgs: 129 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.8 MiB/s wr, 354 op/s 2024-09-12T01:33:46.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:33:45 smithi001 ceph-mon[30387]: pgmap v523: 129 pgs: 129 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.2 MiB/s wr, 451 op/s 2024-09-12T01:33:46.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:33:45 smithi053 ceph-mon[35158]: pgmap v523: 129 pgs: 129 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.2 MiB/s wr, 451 op/s 2024-09-12T01:33:46.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:33:45 smithi104 ceph-mon[35219]: pgmap v523: 129 pgs: 129 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.2 MiB/s wr, 451 op/s 2024-09-12T01:33:48.181 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:33:47 smithi001 ceph-mon[30387]: pgmap v524: 129 pgs: 129 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.3 MiB/s wr, 440 op/s 2024-09-12T01:33:48.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:33:47 smithi053 ceph-mon[35158]: pgmap v524: 129 pgs: 129 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.3 MiB/s wr, 440 op/s 2024-09-12T01:33:48.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:33:47 smithi104 ceph-mon[35219]: pgmap v524: 129 pgs: 129 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.3 MiB/s wr, 440 op/s 2024-09-12T01:33:50.039 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.039+0000 7f038c600640 1 -- 172.21.15.1:0/2499189675 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0384100a30 msgr2=0x7f0384100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:50.039 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.039+0000 7f038c600640 1 --2- 172.21.15.1:0/2499189675 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0384100a30 0x7f0384100e30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0378009900 tx=0x7f037802a410 comp rx=0 tx=0).stop 2024-09-12T01:33:50.039 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.039+0000 7f038c600640 1 -- 172.21.15.1:0/2499189675 shutdown_connections 2024-09-12T01:33:50.039 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.039+0000 7f038c600640 1 --2- 172.21.15.1:0/2499189675 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03841025e0 0x7f0384108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.039 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.039+0000 7f038c600640 1 --2- 172.21.15.1:0/2499189675 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0384101c20 0x7f03841020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.039 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.039+0000 7f038c600640 1 --2- 172.21.15.1:0/2499189675 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0384100a30 0x7f0384100e30 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.040 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.039+0000 7f038c600640 1 -- 172.21.15.1:0/2499189675 >> 172.21.15.1:0/2499189675 conn(0x7f03840fc190 msgr2=0x7f03840fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:50.040 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.039+0000 7f038c600640 1 -- 172.21.15.1:0/2499189675 shutdown_connections 2024-09-12T01:33:50.040 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.039+0000 7f038c600640 1 -- 172.21.15.1:0/2499189675 wait complete. 2024-09-12T01:33:50.040 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.040+0000 7f038c600640 1 Processor -- start 2024-09-12T01:33:50.040 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.041+0000 7f038c600640 1 -- start start 2024-09-12T01:33:50.041 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.041+0000 7f038c600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0384100a30 0x7f0384197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:50.041 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.041+0000 7f038c600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0384101c20 0x7f03841983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:50.041 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.041+0000 7f038c600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03841025e0 0x7f03841a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:50.041 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.041+0000 7f038c600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f038410c700 con 0x7f0384100a30 2024-09-12T01:33:50.042 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.041+0000 7f038c600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f038410c580 con 0x7f0384101c20 2024-09-12T01:33:50.042 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.041+0000 7f038c600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f038410c880 con 0x7f03841025e0 2024-09-12T01:33:50.042 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.042+0000 7f0389c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0384100a30 0x7f0384197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:50.042 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.042+0000 7f0389200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0384101c20 0x7f03841983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:50.042 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.042+0000 7f0389c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0384100a30 0x7f0384197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51954/0 (socket says 172.21.15.1:51954) 2024-09-12T01:33:50.042 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.042+0000 7f0389c00640 1 -- 172.21.15.1:0/629678370 learned_addr learned my addr 172.21.15.1:0/629678370 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:33:50.042 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.042+0000 7f0389200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0384101c20 0x7f03841983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:34922/0 (socket says 172.21.15.1:34922) 2024-09-12T01:33:50.042 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.042+0000 7f038a600640 1 --2- 172.21.15.1:0/629678370 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03841025e0 0x7f03841a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:50.043 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.042+0000 7f038a600640 1 -- 172.21.15.1:0/629678370 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0384101c20 msgr2=0x7f03841983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:50.043 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.042+0000 7f038a600640 1 --2- 172.21.15.1:0/629678370 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0384101c20 0x7f03841983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.043 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.042+0000 7f038a600640 1 -- 172.21.15.1:0/629678370 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0384100a30 msgr2=0x7f0384197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:50.043 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.042+0000 7f038a600640 1 --2- 172.21.15.1:0/629678370 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0384100a30 0x7f0384197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.044 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.042+0000 7f038a600640 1 -- 172.21.15.1:0/629678370 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f038419d430 con 0x7f03841025e0 2024-09-12T01:33:50.044 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.042+0000 7f0389c00640 1 --2- 172.21.15.1:0/629678370 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0384100a30 0x7f0384197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:33:50.044 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.042+0000 7f038a600640 1 --2- 172.21.15.1:0/629678370 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03841025e0 0x7f03841a1840 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f037c00e940 tx=0x7f037c00ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:50.044 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.043+0000 7f0382000640 1 -- 172.21.15.1:0/629678370 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f037c004d20 con 0x7f03841025e0 2024-09-12T01:33:50.044 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.043+0000 7f0382000640 1 -- 172.21.15.1:0/629678370 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f037c005020 con 0x7f03841025e0 2024-09-12T01:33:50.044 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.043+0000 7f038c600640 1 -- 172.21.15.1:0/629678370 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f038419d720 con 0x7f03841025e0 2024-09-12T01:33:50.044 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.043+0000 7f0382000640 1 -- 172.21.15.1:0/629678370 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f037c0116b0 con 0x7f03841025e0 2024-09-12T01:33:50.044 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.043+0000 7f038c600640 1 -- 172.21.15.1:0/629678370 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f038419dc20 con 0x7f03841025e0 2024-09-12T01:33:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.046+0000 7f0382000640 1 -- 172.21.15.1:0/629678370 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f037c011850 con 0x7f03841025e0 2024-09-12T01:33:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.047+0000 7f038f3bc740 1 Processor -- start 2024-09-12T01:33:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.047+0000 7f0382000640 1 --2- 172.21.15.1:0/629678370 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f036c0a2bc0 0x7f036c0a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.048+0000 7f0382000640 1 -- 172.21.15.1:0/629678370 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f037c0cf430 con 0x7f03841025e0 2024-09-12T01:33:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.048+0000 7f038f3bc740 1 -- start start 2024-09-12T01:33:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.048+0000 7f0389200640 1 --2- 172.21.15.1:0/629678370 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f036c0a2bc0 0x7f036c0a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.048+0000 7f038f3bc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d1ad1cc090 0x55d1ad1cc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.048+0000 7f038f3bc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1ad1cc9d0 0x55d1ad1d1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.048+0000 7f038f3bc740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d1ad1d1a60 0x55d1ad1d3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.048+0000 7f038f3bc740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d1ad131100 con 0x55d1ad1d1a60 2024-09-12T01:33:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.048+0000 7f038f3bc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d1ad0d91b0 con 0x55d1ad1cc9d0 2024-09-12T01:33:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.048+0000 7f038f3bc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d1ad12d0c0 con 0x55d1ad1cc090 2024-09-12T01:33:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.048+0000 7f038a600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d1ad1d1a60 0x55d1ad1d3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.048+0000 7f0389200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1ad1cc9d0 0x55d1ad1d1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.048+0000 7f0389c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d1ad1cc090 0x55d1ad1cc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.048+0000 7f038a600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d1ad1d1a60 0x55d1ad1d3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51966/0 (socket says 172.21.15.1:51966) 2024-09-12T01:33:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.048+0000 7f038a600640 1 -- 172.21.15.1:0/2271020683 learned_addr learned my addr 172.21.15.1:0/2271020683 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:33:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.049+0000 7f0389c00640 1 -- 172.21.15.1:0/2271020683 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1ad1cc9d0 msgr2=0x55d1ad1d1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.049+0000 7f0389c00640 1 --2- 172.21.15.1:0/2271020683 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1ad1cc9d0 0x55d1ad1d1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.049+0000 7f0389200640 1 --2- 172.21.15.1:0/629678370 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f036c0a2bc0 0x7f036c0a5080 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f03841993a0 tx=0x7f0370005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.049+0000 7f0389c00640 1 -- 172.21.15.1:0/2271020683 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d1ad1d1a60 msgr2=0x55d1ad1d3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.049+0000 7f0389c00640 1 --2- 172.21.15.1:0/2271020683 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d1ad1d1a60 0x55d1ad1d3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.049+0000 7f0389c00640 1 -- 172.21.15.1:0/2271020683 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1ace44210 con 0x55d1ad1cc090 2024-09-12T01:33:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.049+0000 7f0389c00640 1 --2- 172.21.15.1:0/2271020683 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d1ad1cc090 0x55d1ad1cc490 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0378031610 tx=0x7f0378004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6e1852724562aa4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.049+0000 7f0376a00640 1 -- 172.21.15.1:0/2271020683 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0378035040 con 0x55d1ad1cc090 2024-09-12T01:33:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.049+0000 7f0376a00640 1 -- 172.21.15.1:0/2271020683 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0378004be0 con 0x55d1ad1cc090 2024-09-12T01:33:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.049+0000 7f0376a00640 1 -- 172.21.15.1:0/2271020683 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0378031c10 con 0x55d1ad1cc090 2024-09-12T01:33:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.050+0000 7f038f3bc740 1 -- 172.21.15.1:0/2271020683 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d1ad1cc090 msgr2=0x55d1ad1cc490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.050+0000 7f038f3bc740 1 --2- 172.21.15.1:0/2271020683 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d1ad1cc090 0x55d1ad1cc490 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0378031610 tx=0x7f0378004610 comp rx=0 tx=0).stop 2024-09-12T01:33:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.050+0000 7f038f3bc740 1 -- 172.21.15.1:0/2271020683 shutdown_connections 2024-09-12T01:33:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.050+0000 7f038f3bc740 1 --2- 172.21.15.1:0/2271020683 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d1ad1d1a60 0x55d1ad1d3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.052 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.050+0000 7f038f3bc740 1 --2- 172.21.15.1:0/2271020683 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1ad1cc9d0 0x55d1ad1d1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.052 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.050+0000 7f038f3bc740 1 --2- 172.21.15.1:0/2271020683 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d1ad1cc090 0x55d1ad1cc490 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.052 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.050+0000 7f038f3bc740 1 -- 172.21.15.1:0/2271020683 >> 172.21.15.1:0/2271020683 conn(0x55d1ad1c7700 msgr2=0x55d1ad1c7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:50.052 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.051+0000 7f038f3bc740 1 -- 172.21.15.1:0/2271020683 shutdown_connections 2024-09-12T01:33:50.052 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.051+0000 7f038f3bc740 1 -- 172.21.15.1:0/2271020683 wait complete. 2024-09-12T01:33:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.051+0000 7f038f3bc740 1 -- start start 2024-09-12T01:33:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.051+0000 7f038f3bc740 1 Processor -- start 2024-09-12T01:33:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.052+0000 7f038f3bc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d1ad1cc090 0x55d1ad1d6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.052+0000 7f038f3bc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1ad1cc9d0 0x55d1ad1d6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.052+0000 7f038f3bc740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d1ad1d1a60 0x55d1ad1dd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.052+0000 7f038f3bc740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d1ad1d49d0 con 0x55d1ad1d1a60 2024-09-12T01:33:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.052+0000 7f038f3bc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d1ad0ba090 con 0x55d1ad1cc9d0 2024-09-12T01:33:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.052+0000 7f038f3bc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d1ad1d4b50 con 0x55d1ad1cc090 2024-09-12T01:33:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.052+0000 7f038f3bc740 10 client.0 _ll_register_callbacks cb 0x55d1ad10ef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:33:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.052+0000 7f0389200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1ad1cc9d0 0x55d1ad1d6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.052+0000 7f0389200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1ad1cc9d0 0x55d1ad1d6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:34930/0 (socket says 172.21.15.1:34930) 2024-09-12T01:33:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.052+0000 7f0389200640 1 -- 172.21.15.1:0/716714507 learned_addr learned my addr 172.21.15.1:0/716714507 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:33:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.052+0000 7f038a600640 1 --2- 172.21.15.1:0/716714507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d1ad1d1a60 0x55d1ad1dd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.052+0000 7f0389c00640 1 --2- 172.21.15.1:0/716714507 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d1ad1cc090 0x55d1ad1d6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.053+0000 7f0389200640 1 -- 172.21.15.1:0/716714507 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d1ad1cc090 msgr2=0x55d1ad1d6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.053+0000 7f0389200640 1 --2- 172.21.15.1:0/716714507 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d1ad1cc090 0x55d1ad1d6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.053+0000 7f0389200640 1 -- 172.21.15.1:0/716714507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d1ad1d1a60 msgr2=0x55d1ad1dd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.053+0000 7f0389200640 1 --2- 172.21.15.1:0/716714507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d1ad1d1a60 0x55d1ad1dd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.053+0000 7f0389200640 1 -- 172.21.15.1:0/716714507 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1acf10620 con 0x55d1ad1cc9d0 2024-09-12T01:33:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.053+0000 7f0389c00640 1 --2- 172.21.15.1:0/716714507 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d1ad1cc090 0x55d1ad1d6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:33:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.053+0000 7f0389200640 1 --2- 172.21.15.1:0/716714507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1ad1cc9d0 0x55d1ad1d6750 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f037001c990 tx=0x7f0370004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b36bd5ce0b9b173 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.053+0000 7f036b400640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-12T01:33:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.053+0000 7f036b400640 1 -- 172.21.15.1:0/716714507 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f037002a060 con 0x55d1ad1cc9d0 2024-09-12T01:33:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.053+0000 7f038f3bc740 1 -- 172.21.15.1:0/716714507 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0378004be0 con 0x55d1ad1cc9d0 2024-09-12T01:33:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.053+0000 7f036b400640 1 -- 172.21.15.1:0/716714507 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f0370009040 con 0x55d1ad1cc9d0 2024-09-12T01:33:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.054+0000 7f036b400640 1 -- 172.21.15.1:0/716714507 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0370031050 con 0x55d1ad1cc9d0 2024-09-12T01:33:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.054+0000 7f038f3bc740 10 client.25526 fetch_fsmap learned FSMap version 34 2024-09-12T01:33:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.054+0000 7f038f3bc740 1 -- 172.21.15.1:0/716714507 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d1ad1e06a0 con 0x55d1ad1cc9d0 2024-09-12T01:33:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.055+0000 7f036b400640 1 -- 172.21.15.1:0/716714507 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0378004be0 con 0x55d1ad1cc9d0 2024-09-12T01:33:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.055+0000 7f036b400640 1 -- 172.21.15.1:0/716714507 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7f037001a030 con 0x55d1ad1cc9d0 2024-09-12T01:33:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.055+0000 7f038f3bc740 10 client.25526 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T01:33:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.055+0000 7f038f3bc740 10 client.25526 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:33:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.055+0000 7f038f3bc740 20 client.25526 populate_metadata read hostname 'smithi001' 2024-09-12T01:33:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.055+0000 7f038f3bc740 1 --2- 172.21.15.1:0/716714507 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55d1ad1e1d20 0x55d1ad1e4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:50.058 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.055+0000 7f038f3bc740 4 client.25526 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:33:50.058 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.055+0000 7f038f3bc740 1 -- 172.21.15.1:0/716714507 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1ad13e830 con 0x55d1ad1e1d20 2024-09-12T01:33:50.058 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.055+0000 7f0389c00640 1 --2- 172.21.15.1:0/716714507 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55d1ad1e1d20 0x55d1ad1e4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:50.058 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.056+0000 7f0389c00640 1 --2- 172.21.15.1:0/716714507 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55d1ad1e1d20 0x55d1ad1e4170 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a78611a207cabe8 server_cookie=2029f2914340f534 in_seq=0 out_seq=0 2024-09-12T01:33:50.058 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.056+0000 7f036b400640 10 client.25526 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:33:50.059 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.056+0000 7f036b400640 1 -- 172.21.15.1:0/716714507 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d1ad13e830 con 0x55d1ad1e1d20 2024-09-12T01:33:50.059 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.056+0000 7f036b400640 10 client.25526 handle_command_reply: tid=0 2024-09-12T01:33:50.059 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.057+0000 7f038f3bc740 1 client.25526 shutdown 2024-09-12T01:33:50.059 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.057+0000 7f038f3bc740 1 -- 172.21.15.1:0/716714507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1ad1cc9d0 msgr2=0x55d1ad1d6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:50.059 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.057+0000 7f038f3bc740 1 --2- 172.21.15.1:0/716714507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1ad1cc9d0 0x55d1ad1d6750 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f037001c990 tx=0x7f0370004730 comp rx=0 tx=0).stop 2024-09-12T01:33:50.059 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.057+0000 7f038f3bc740 1 -- 172.21.15.1:0/716714507 shutdown_connections 2024-09-12T01:33:50.059 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.057+0000 7f038f3bc740 1 --2- 172.21.15.1:0/716714507 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55d1ad1e1d20 0x55d1ad1e4170 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.060 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.057+0000 7f038f3bc740 1 --2- 172.21.15.1:0/716714507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d1ad1d1a60 0x55d1ad1dd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.060 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.057+0000 7f038f3bc740 1 --2- 172.21.15.1:0/716714507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1ad1cc9d0 0x55d1ad1d6750 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.060 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.057+0000 7f038f3bc740 1 --2- 172.21.15.1:0/716714507 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d1ad1cc090 0x55d1ad1d6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.060 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.057+0000 7f038f3bc740 1 -- 172.21.15.1:0/716714507 >> 172.21.15.1:0/716714507 conn(0x55d1ad1c7700 msgr2=0x55d1ad1c8f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:50.060 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.057+0000 7f036b400640 0 client.25526 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:33:50.060 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.057+0000 7f038f3bc740 1 -- 172.21.15.1:0/716714507 shutdown_connections 2024-09-12T01:33:50.060 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.057+0000 7f038f3bc740 1 -- 172.21.15.1:0/716714507 wait complete. 2024-09-12T01:33:50.060 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.057+0000 7f038f3bc740 20 client.25526 trim_cache size 0 max 16384 2024-09-12T01:33:50.077 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.077+0000 7f038f3bc740 1 Processor -- start 2024-09-12T01:33:50.077 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.077+0000 7f038f3bc740 1 -- start start 2024-09-12T01:33:50.077 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.077+0000 7f038f3bc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d1ad1c7700 0x55d1ad1c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:50.077 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.077+0000 7f038f3bc740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d1ad1cc090 0x55d1ad1dd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:50.077 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.077+0000 7f038f3bc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1ad1cc9d0 0x55d1ad1e8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:50.077 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.077+0000 7f038f3bc740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f03841b0de0 con 0x55d1ad1cc090 2024-09-12T01:33:50.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.077+0000 7f038f3bc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f03841b2950 con 0x55d1ad1cc9d0 2024-09-12T01:33:50.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.077+0000 7f038f3bc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f03841b14b0 con 0x55d1ad1c7700 2024-09-12T01:33:50.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.077+0000 7f0389c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d1ad1c7700 0x55d1ad1c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:50.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.077+0000 7f038a600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1ad1cc9d0 0x55d1ad1e8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:50.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.077+0000 7f038a600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1ad1cc9d0 0x55d1ad1e8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:34936/0 (socket says 172.21.15.1:34936) 2024-09-12T01:33:50.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.077+0000 7f0389200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d1ad1cc090 0x55d1ad1dd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:50.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.077+0000 7f0389c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d1ad1c7700 0x55d1ad1c6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:44362/0 (socket says 172.21.15.1:44362) 2024-09-12T01:33:50.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.077+0000 7f038a600640 1 -- 172.21.15.1:0/3638593532 learned_addr learned my addr 172.21.15.1:0/3638593532 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:33:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.078+0000 7f0389c00640 1 -- 172.21.15.1:0/3638593532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1ad1cc9d0 msgr2=0x55d1ad1e8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.078+0000 7f0389c00640 1 --2- 172.21.15.1:0/3638593532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1ad1cc9d0 0x55d1ad1e8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.078+0000 7f0389c00640 1 -- 172.21.15.1:0/3638593532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d1ad1cc090 msgr2=0x55d1ad1dd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.078+0000 7f0389c00640 1 --2- 172.21.15.1:0/3638593532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d1ad1cc090 0x55d1ad1dd750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.078+0000 7f0389c00640 1 -- 172.21.15.1:0/3638593532 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1ad1c2d60 con 0x55d1ad1c7700 2024-09-12T01:33:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.078+0000 7f038a600640 1 --2- 172.21.15.1:0/3638593532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1ad1cc9d0 0x55d1ad1e8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:33:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.078+0000 7f0389c00640 1 --2- 172.21.15.1:0/3638593532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d1ad1c7700 0x55d1ad1c6770 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f037802aa00 tx=0x7f0378000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9aa6432bc898952 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.078+0000 7f0376000640 1 -- 172.21.15.1:0/3638593532 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0378035040 con 0x55d1ad1c7700 2024-09-12T01:33:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.079+0000 7f0376000640 1 -- 172.21.15.1:0/3638593532 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0378004980 con 0x55d1ad1c7700 2024-09-12T01:33:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.079+0000 7f0376000640 1 -- 172.21.15.1:0/3638593532 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0378031da0 con 0x55d1ad1c7700 2024-09-12T01:33:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.079+0000 7f038f3bc740 1 -- 172.21.15.1:0/3638593532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d1ad1c7700 msgr2=0x55d1ad1c6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.079+0000 7f038f3bc740 1 --2- 172.21.15.1:0/3638593532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d1ad1c7700 0x55d1ad1c6770 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f037802aa00 tx=0x7f0378000bc0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.079+0000 7f038f3bc740 1 -- 172.21.15.1:0/3638593532 shutdown_connections 2024-09-12T01:33:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.079+0000 7f038f3bc740 1 --2- 172.21.15.1:0/3638593532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1ad1cc9d0 0x55d1ad1e8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.079+0000 7f038f3bc740 1 --2- 172.21.15.1:0/3638593532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d1ad1cc090 0x55d1ad1dd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.079+0000 7f038f3bc740 1 --2- 172.21.15.1:0/3638593532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d1ad1c7700 0x55d1ad1c6770 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.079+0000 7f038f3bc740 1 -- 172.21.15.1:0/3638593532 >> 172.21.15.1:0/3638593532 conn(0x55d1ad1d15f0 msgr2=0x55d1ad1c6060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.079+0000 7f038f3bc740 1 -- 172.21.15.1:0/3638593532 shutdown_connections 2024-09-12T01:33:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.079+0000 7f038f3bc740 1 -- 172.21.15.1:0/3638593532 wait complete. 2024-09-12T01:33:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.080+0000 7f038f3bc740 1 -- start start 2024-09-12T01:33:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.080+0000 7f038f3bc740 1 Processor -- start 2024-09-12T01:33:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.080+0000 7f038f3bc740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d1ad1c7700 0x55d1ad1db510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.080+0000 7f038f3bc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1ad1cc090 0x55d1ad1dba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.080+0000 7f038f3bc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d1ad1cc9d0 0x55d1ad1dca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.080+0000 7f038f3bc740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f03841b2250 con 0x55d1ad1c7700 2024-09-12T01:33:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.080+0000 7f038f3bc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f03841b1b80 con 0x55d1ad1cc090 2024-09-12T01:33:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.080+0000 7f038f3bc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f03841aede0 con 0x55d1ad1cc9d0 2024-09-12T01:33:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.081+0000 7f038f3bc740 10 client.0 _ll_register_callbacks cb 0x7f03700099b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:33:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.081+0000 7f0389c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d1ad1c7700 0x55d1ad1db510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.081+0000 7f0389200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1ad1cc090 0x55d1ad1dba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.081+0000 7f038a600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d1ad1cc9d0 0x55d1ad1dca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.081+0000 7f0389200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1ad1cc090 0x55d1ad1dba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:34948/0 (socket says 172.21.15.1:34948) 2024-09-12T01:33:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.081+0000 7f038a600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d1ad1cc9d0 0x55d1ad1dca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:44366/0 (socket says 172.21.15.1:44366) 2024-09-12T01:33:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.081+0000 7f0389c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d1ad1c7700 0x55d1ad1db510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51994/0 (socket says 172.21.15.1:51994) 2024-09-12T01:33:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.081+0000 7f0389200640 1 -- 172.21.15.1:0/1609633734 learned_addr learned my addr 172.21.15.1:0/1609633734 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:33:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.081+0000 7f038a600640 1 -- 172.21.15.1:0/1609633734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1ad1cc090 msgr2=0x55d1ad1dba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.081+0000 7f038a600640 1 --2- 172.21.15.1:0/1609633734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1ad1cc090 0x55d1ad1dba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.081+0000 7f038a600640 1 -- 172.21.15.1:0/1609633734 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d1ad1c7700 msgr2=0x55d1ad1db510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:33:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.081+0000 7f038a600640 1 --2- 172.21.15.1:0/1609633734 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d1ad1c7700 0x55d1ad1db510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.081+0000 7f038a600640 1 -- 172.21.15.1:0/1609633734 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1ad1dd020 con 0x55d1ad1cc9d0 2024-09-12T01:33:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.081+0000 7f0389c00640 1 --2- 172.21.15.1:0/1609633734 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d1ad1c7700 0x55d1ad1db510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:33:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.081+0000 7f0389200640 1 --2- 172.21.15.1:0/1609633734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1ad1cc090 0x55d1ad1dba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:33:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.081+0000 7f038a600640 1 --2- 172.21.15.1:0/1609633734 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d1ad1cc9d0 0x55d1ad1dca10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f037c0ccf10 tx=0x7f037c00e8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f16a83f84a56191a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:33:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.081+0000 7f036b400640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-12T01:33:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.082+0000 7f036b400640 1 -- 172.21.15.1:0/1609633734 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f037c01f040 con 0x55d1ad1cc9d0 2024-09-12T01:33:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.082+0000 7f036b400640 1 -- 172.21.15.1:0/1609633734 <== mon.1 v2:172.21.15.104:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f037c0054d0 con 0x55d1ad1cc9d0 2024-09-12T01:33:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.082+0000 7f038f3bc740 1 -- 172.21.15.1:0/1609633734 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0378004980 con 0x55d1ad1cc9d0 2024-09-12T01:33:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.082+0000 7f036b400640 1 -- 172.21.15.1:0/1609633734 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f037c011b40 con 0x55d1ad1cc9d0 2024-09-12T01:33:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.082+0000 7f038f3bc740 10 client.25531 fetch_fsmap learned FSMap version 34 2024-09-12T01:33:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.082+0000 7f038f3bc740 1 -- 172.21.15.1:0/1609633734 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d1ad24d170 con 0x55d1ad1cc9d0 2024-09-12T01:33:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.082+0000 7f036b400640 1 -- 172.21.15.1:0/1609633734 <== mon.1 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0378004980 con 0x55d1ad1cc9d0 2024-09-12T01:33:50.086 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.082+0000 7f036b400640 1 -- 172.21.15.1:0/1609633734 <== mon.1 v2:172.21.15.104:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7f037c00f040 con 0x55d1ad1cc9d0 2024-09-12T01:33:50.086 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.082+0000 7f038f3bc740 10 client.25531 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T01:33:50.086 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.082+0000 7f038f3bc740 10 client.25531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:33:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.082+0000 7f038f3bc740 20 client.25531 populate_metadata read hostname 'smithi001' 2024-09-12T01:33:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.082+0000 7f038f3bc740 1 --2- 172.21.15.1:0/1609633734 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55d1ad1d1a60 0x55d1ad24fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:33:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.082+0000 7f038f3bc740 4 client.25531 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:33:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.082+0000 7f038f3bc740 1 -- 172.21.15.1:0/1609633734 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d1ad138b80 con 0x55d1ad1d1a60 2024-09-12T01:33:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.083+0000 7f0389c00640 1 --2- 172.21.15.1:0/1609633734 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55d1ad1d1a60 0x55d1ad24fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:33:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.083+0000 7f0389c00640 1 --2- 172.21.15.1:0/1609633734 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55d1ad1d1a60 0x55d1ad24fb90 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb83ebf6b5e56e87 server_cookie=f134530fffa4fc2d in_seq=0 out_seq=0 2024-09-12T01:33:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.083+0000 7f036b400640 10 client.25531 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:33:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.092+0000 7f036b400640 1 -- 172.21.15.1:0/1609633734 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d1ad138b80 con 0x55d1ad1d1a60 2024-09-12T01:33:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.092+0000 7f036b400640 10 client.25531 handle_command_reply: tid=0 2024-09-12T01:33:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.092+0000 7f038f3bc740 1 client.25531 shutdown 2024-09-12T01:33:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.092+0000 7f038f3bc740 1 -- 172.21.15.1:0/1609633734 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d1ad1cc9d0 msgr2=0x55d1ad1dca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.092+0000 7f038f3bc740 1 --2- 172.21.15.1:0/1609633734 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d1ad1cc9d0 0x55d1ad1dca10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f037c0ccf10 tx=0x7f037c00e8c0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.092+0000 7f038f3bc740 1 -- 172.21.15.1:0/1609633734 shutdown_connections 2024-09-12T01:33:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.092+0000 7f038f3bc740 1 --2- 172.21.15.1:0/1609633734 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55d1ad1d1a60 0x55d1ad24fb90 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.092+0000 7f038f3bc740 1 --2- 172.21.15.1:0/1609633734 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d1ad1cc9d0 0x55d1ad1dca10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.092+0000 7f038f3bc740 1 --2- 172.21.15.1:0/1609633734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1ad1cc090 0x55d1ad1dba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.092+0000 7f038f3bc740 1 --2- 172.21.15.1:0/1609633734 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d1ad1c7700 0x55d1ad1db510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.092+0000 7f038f3bc740 1 -- 172.21.15.1:0/1609633734 >> 172.21.15.1:0/1609633734 conn(0x55d1ad1d15f0 msgr2=0x55d1ad1c8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.092+0000 7f036b400640 0 client.25531 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:33:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.092+0000 7f038f3bc740 1 -- 172.21.15.1:0/1609633734 shutdown_connections 2024-09-12T01:33:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.092+0000 7f038f3bc740 1 -- 172.21.15.1:0/1609633734 wait complete. 2024-09-12T01:33:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.092+0000 7f038f3bc740 20 client.25531 trim_cache size 0 max 16384 2024-09-12T01:33:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-12T01:33:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-12T01:33:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-12T01:33:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-12T01:33:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.093+0000 7f038c600640 1 -- 172.21.15.1:0/629678370 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f036c0a2bc0 msgr2=0x7f036c0a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.093+0000 7f038c600640 1 --2- 172.21.15.1:0/629678370 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f036c0a2bc0 0x7f036c0a5080 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f03841993a0 tx=0x7f0370005f90 comp rx=0 tx=0).stop 2024-09-12T01:33:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.093+0000 7f038c600640 1 -- 172.21.15.1:0/629678370 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03841025e0 msgr2=0x7f03841a1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:33:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.093+0000 7f038c600640 1 --2- 172.21.15.1:0/629678370 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03841025e0 0x7f03841a1840 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f037c00e940 tx=0x7f037c00ee00 comp rx=0 tx=0).stop 2024-09-12T01:33:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.093+0000 7f038c600640 1 -- 172.21.15.1:0/629678370 shutdown_connections 2024-09-12T01:33:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.093+0000 7f038c600640 1 --2- 172.21.15.1:0/629678370 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f036c0a2bc0 0x7f036c0a5080 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.093+0000 7f038c600640 1 --2- 172.21.15.1:0/629678370 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03841025e0 0x7f03841a1840 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.093+0000 7f038c600640 1 --2- 172.21.15.1:0/629678370 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0384101c20 0x7f03841983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.093+0000 7f038c600640 1 --2- 172.21.15.1:0/629678370 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0384100a30 0x7f0384197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:33:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.094+0000 7f038c600640 1 -- 172.21.15.1:0/629678370 >> 172.21.15.1:0/629678370 conn(0x7f03840fc190 msgr2=0x7f03840fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:33:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.094+0000 7f038c600640 1 -- 172.21.15.1:0/629678370 shutdown_connections 2024-09-12T01:33:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:33:50.094+0000 7f038c600640 1 -- 172.21.15.1:0/629678370 wait complete. 2024-09-12T01:33:50.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:33:49 smithi001 ceph-mon[30387]: pgmap v525: 129 pgs: 129 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.8 MiB/s wr, 372 op/s 2024-09-12T01:33:50.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:33:49 smithi053 ceph-mon[35158]: pgmap v525: 129 pgs: 129 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.8 MiB/s wr, 372 op/s 2024-09-12T01:33:50.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:33:49 smithi104 ceph-mon[35219]: pgmap v525: 129 pgs: 129 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.8 MiB/s wr, 372 op/s 2024-09-12T01:33:52.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:33:51 smithi001 ceph-mon[30387]: pgmap v526: 129 pgs: 129 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.5 MiB/s wr, 471 op/s 2024-09-12T01:33:52.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:33:51 smithi053 ceph-mon[35158]: pgmap v526: 129 pgs: 129 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.5 MiB/s wr, 471 op/s 2024-09-12T01:33:52.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:33:51 smithi104 ceph-mon[35219]: pgmap v526: 129 pgs: 129 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.5 MiB/s wr, 471 op/s 2024-09-12T01:33:53.393 INFO:tasks.workunit.client.0.smithi001.stdout:30000 trunc from 0x7a5444 to 0x143a5 2024-09-12T01:33:54.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:33:53 smithi001 ceph-mon[30387]: pgmap v527: 129 pgs: 129 active+clean; 91 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.1 MiB/s wr, 412 op/s 2024-09-12T01:33:54.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:33:53 smithi053 ceph-mon[35158]: pgmap v527: 129 pgs: 129 active+clean; 91 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.1 MiB/s wr, 412 op/s 2024-09-12T01:33:54.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:33:53 smithi104 ceph-mon[35219]: pgmap v527: 129 pgs: 129 active+clean; 91 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.1 MiB/s wr, 412 op/s 2024-09-12T01:33:56.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:33:55 smithi001 ceph-mon[30387]: pgmap v528: 129 pgs: 129 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.4 MiB/s wr, 485 op/s 2024-09-12T01:33:56.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:33:55 smithi053 ceph-mon[35158]: pgmap v528: 129 pgs: 129 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.4 MiB/s wr, 485 op/s 2024-09-12T01:33:56.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:33:55 smithi104 ceph-mon[35219]: pgmap v528: 129 pgs: 129 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.4 MiB/s wr, 485 op/s 2024-09-12T01:33:58.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:33:57 smithi001 ceph-mon[30387]: pgmap v529: 129 pgs: 129 active+clean; 95 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 2.3 MiB/s wr, 430 op/s 2024-09-12T01:33:58.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:33:57 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:33:58.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:33:57 smithi053 ceph-mon[35158]: pgmap v529: 129 pgs: 129 active+clean; 95 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 2.3 MiB/s wr, 430 op/s 2024-09-12T01:33:58.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:33:57 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:33:58.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:33:57 smithi104 ceph-mon[35219]: pgmap v529: 129 pgs: 129 active+clean; 95 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 2.3 MiB/s wr, 430 op/s 2024-09-12T01:33:58.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:33:57 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:33:59.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:33:58 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:33:59.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:33:58 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:33:59.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:33:58 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:33:59.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:33:58 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:33:59.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:33:58 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:33:59.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:33:58 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:33:59.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:33:58 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:33:59.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:33:58 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:33:59.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:33:58 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:34:00.297 INFO:tasks.workunit.client.0.smithi001.stdout:truncating to largest ever: 0x9ffb01 2024-09-12T01:34:00.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:34:00 smithi001 ceph-mon[30387]: pgmap v530: 129 pgs: 129 active+clean; 95 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.8 MiB/s wr, 346 op/s 2024-09-12T01:34:00.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:34:00 smithi053 ceph-mon[35158]: pgmap v530: 129 pgs: 129 active+clean; 95 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.8 MiB/s wr, 346 op/s 2024-09-12T01:34:01.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:34:00 smithi104 ceph-mon[35219]: pgmap v530: 129 pgs: 129 active+clean; 95 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.8 MiB/s wr, 346 op/s 2024-09-12T01:34:02.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:34:02 smithi001 ceph-mon[30387]: pgmap v531: 129 pgs: 129 active+clean; 97 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 2.6 MiB/s wr, 485 op/s 2024-09-12T01:34:02.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:34:02 smithi053 ceph-mon[35158]: pgmap v531: 129 pgs: 129 active+clean; 97 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 2.6 MiB/s wr, 485 op/s 2024-09-12T01:34:03.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:34:02 smithi104 ceph-mon[35219]: pgmap v531: 129 pgs: 129 active+clean; 97 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 2.6 MiB/s wr, 485 op/s 2024-09-12T01:34:03.238 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.238+0000 7fb6f6600640 1 -- 172.21.15.1:0/1173504185 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6f01036f0 msgr2=0x7fb6f0109f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:03.238 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.238+0000 7fb6f6600640 1 --2- 172.21.15.1:0/1173504185 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6f01036f0 0x7fb6f0109f80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8009be0 tx=0x7fb6e802a6d0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.238 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.238+0000 7fb6f6600640 1 -- 172.21.15.1:0/1173504185 shutdown_connections 2024-09-12T01:34:03.238 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.238+0000 7fb6f6600640 1 --2- 172.21.15.1:0/1173504185 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6f01036f0 0x7fb6f0109f80 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.238 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.238+0000 7fb6f6600640 1 --2- 172.21.15.1:0/1173504185 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6f0102d30 0x7fb6f01031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.238 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.238+0000 7fb6f6600640 1 --2- 172.21.15.1:0/1173504185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6f0101b40 0x7fb6f0101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.238 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.238+0000 7fb6f6600640 1 -- 172.21.15.1:0/1173504185 >> 172.21.15.1:0/1173504185 conn(0x7fb6f00fd2e0 msgr2=0x7fb6f00ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:03.238 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.238+0000 7fb6f6600640 1 -- 172.21.15.1:0/1173504185 shutdown_connections 2024-09-12T01:34:03.239 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.238+0000 7fb6f6600640 1 -- 172.21.15.1:0/1173504185 wait complete. 2024-09-12T01:34:03.241 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.239+0000 7fb6f6600640 1 Processor -- start 2024-09-12T01:34:03.241 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.239+0000 7fb6f6600640 1 -- start start 2024-09-12T01:34:03.241 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.239+0000 7fb6f6600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6f0101b40 0x7fb6f0075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:03.241 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.239+0000 7fb6f6600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6f0102d30 0x7fb6f00745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:03.241 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.239+0000 7fb6f6600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6f01036f0 0x7fb6f0074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:03.241 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.239+0000 7fb6f6600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f010d7a0 con 0x7fb6f01036f0 2024-09-12T01:34:03.242 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.239+0000 7fb6f6600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f010d620 con 0x7fb6f0101b40 2024-09-12T01:34:03.242 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.239+0000 7fb6f6600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f010d920 con 0x7fb6f0102d30 2024-09-12T01:34:03.242 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.240+0000 7fb6eea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6f0102d30 0x7fb6f00745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:03.242 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.240+0000 7fb6ef400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6f0101b40 0x7fb6f0075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:03.242 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.240+0000 7fb6eea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6f0102d30 0x7fb6f00745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:55250/0 (socket says 172.21.15.1:55250) 2024-09-12T01:34:03.242 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.240+0000 7fb6eea00640 1 -- 172.21.15.1:0/2900807395 learned_addr learned my addr 172.21.15.1:0/2900807395 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:34:03.242 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.240+0000 7fb6efe00640 1 --2- 172.21.15.1:0/2900807395 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6f01036f0 0x7fb6f0074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:03.242 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.240+0000 7fb6eea00640 1 -- 172.21.15.1:0/2900807395 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6f0101b40 msgr2=0x7fb6f0075fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:03.242 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.240+0000 7fb6eea00640 1 --2- 172.21.15.1:0/2900807395 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6f0101b40 0x7fb6f0075fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.242 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.240+0000 7fb6eea00640 1 -- 172.21.15.1:0/2900807395 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6f01036f0 msgr2=0x7fb6f0074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:03.242 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.240+0000 7fb6eea00640 1 --2- 172.21.15.1:0/2900807395 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6f01036f0 0x7fb6f0074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.242 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.240+0000 7fb6eea00640 1 -- 172.21.15.1:0/2900807395 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f00753b0 con 0x7fb6f0102d30 2024-09-12T01:34:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.240+0000 7fb6eea00640 1 --2- 172.21.15.1:0/2900807395 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6f0102d30 0x7fb6f00745e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc00e9c0 tx=0x7fb6dc00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.240+0000 7fb6e7e00640 1 -- 172.21.15.1:0/2900807395 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc00cde0 con 0x7fb6f0102d30 2024-09-12T01:34:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.241+0000 7fb6e7e00640 1 -- 172.21.15.1:0/2900807395 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6dc0035f0 con 0x7fb6f0102d30 2024-09-12T01:34:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.241+0000 7fb6f6600640 1 -- 172.21.15.1:0/2900807395 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6f01a1f20 con 0x7fb6f0102d30 2024-09-12T01:34:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.241+0000 7fb6e7e00640 1 -- 172.21.15.1:0/2900807395 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc010690 con 0x7fb6f0102d30 2024-09-12T01:34:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.241+0000 7fb6f6600640 1 -- 172.21.15.1:0/2900807395 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6f01a2390 con 0x7fb6f0102d30 2024-09-12T01:34:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.244+0000 7fb6e7e00640 1 -- 172.21.15.1:0/2900807395 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fb6dc0037a0 con 0x7fb6f0102d30 2024-09-12T01:34:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.244+0000 7fb6f949c740 1 Processor -- start 2024-09-12T01:34:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.244+0000 7fb6f949c740 1 -- start start 2024-09-12T01:34:03.246 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.244+0000 7fb6f949c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc88e50090 0x55bc88e50490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:03.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.244+0000 7fb6f949c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc88e509d0 0x55bc88e55490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:03.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.244+0000 7fb6f949c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc88e55a60 0x55bc88e57f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:03.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.244+0000 7fb6f949c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55bc88db5100 con 0x55bc88e509d0 2024-09-12T01:34:03.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.244+0000 7fb6f949c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55bc88d5d1b0 con 0x55bc88e50090 2024-09-12T01:34:03.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.244+0000 7fb6f949c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55bc88db10c0 con 0x55bc88e55a60 2024-09-12T01:34:03.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.245+0000 7fb6efe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc88e50090 0x55bc88e50490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:03.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.245+0000 7fb6efe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc88e50090 0x55bc88e50490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:40006/0 (socket says 172.21.15.1:40006) 2024-09-12T01:34:03.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.245+0000 7fb6efe00640 1 -- 172.21.15.1:0/557135822 learned_addr learned my addr 172.21.15.1:0/557135822 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:34:03.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.245+0000 7fb6eea00640 1 --2- 172.21.15.1:0/557135822 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc88e55a60 0x55bc88e57f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:03.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.245+0000 7fb6ef400640 1 --2- 172.21.15.1:0/557135822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc88e509d0 0x55bc88e55490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:03.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.245+0000 7fb6efe00640 1 -- 172.21.15.1:0/557135822 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc88e55a60 msgr2=0x55bc88e57f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:03.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.245+0000 7fb6efe00640 1 --2- 172.21.15.1:0/557135822 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc88e55a60 0x55bc88e57f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.245+0000 7fb6efe00640 1 -- 172.21.15.1:0/557135822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc88e509d0 msgr2=0x55bc88e55490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:03.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.245+0000 7fb6efe00640 1 --2- 172.21.15.1:0/557135822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc88e509d0 0x55bc88e55490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.245+0000 7fb6efe00640 1 -- 172.21.15.1:0/557135822 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc88ac8210 con 0x55bc88e50090 2024-09-12T01:34:03.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.245+0000 7fb6ef400640 1 --2- 172.21.15.1:0/557135822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc88e509d0 0x55bc88e55490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:34:03.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.245+0000 7fb6efe00640 1 --2- 172.21.15.1:0/557135822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc88e50090 0x55bc88e50490 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8009be0 tx=0x7fb6e8005eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=694c741f0f872237 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:03.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.246+0000 7fb6e4c00640 1 -- 172.21.15.1:0/557135822 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8041030 con 0x55bc88e50090 2024-09-12T01:34:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.246+0000 7fb6e4c00640 1 -- 172.21.15.1:0/557135822 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6e8030c70 con 0x55bc88e50090 2024-09-12T01:34:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.246+0000 7fb6e4c00640 1 -- 172.21.15.1:0/557135822 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8036040 con 0x55bc88e50090 2024-09-12T01:34:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.246+0000 7fb6e7e00640 1 --2- 172.21.15.1:0/2900807395 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fb6d40a2c00 0x7fb6d40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.246+0000 7fb6e7e00640 1 -- 172.21.15.1:0/2900807395 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fb6dc0cf620 con 0x7fb6f0102d30 2024-09-12T01:34:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.246+0000 7fb6ef400640 1 --2- 172.21.15.1:0/2900807395 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fb6d40a2c00 0x7fb6d40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.247+0000 7fb6f949c740 1 -- 172.21.15.1:0/557135822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc88e50090 msgr2=0x55bc88e50490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.247+0000 7fb6f949c740 1 --2- 172.21.15.1:0/557135822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc88e50090 0x55bc88e50490 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8009be0 tx=0x7fb6e8005eb0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.247+0000 7fb6f949c740 1 -- 172.21.15.1:0/557135822 shutdown_connections 2024-09-12T01:34:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.247+0000 7fb6f949c740 1 --2- 172.21.15.1:0/557135822 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc88e55a60 0x55bc88e57f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.247+0000 7fb6f949c740 1 --2- 172.21.15.1:0/557135822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc88e509d0 0x55bc88e55490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.247+0000 7fb6f949c740 1 --2- 172.21.15.1:0/557135822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc88e50090 0x55bc88e50490 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.247+0000 7fb6f949c740 1 -- 172.21.15.1:0/557135822 >> 172.21.15.1:0/557135822 conn(0x55bc88e4b700 msgr2=0x55bc88e4bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.247+0000 7fb6f949c740 1 -- 172.21.15.1:0/557135822 shutdown_connections 2024-09-12T01:34:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.247+0000 7fb6ef400640 1 --2- 172.21.15.1:0/2900807395 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fb6d40a2c00 0x7fb6d40a50c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x55bc88e51400 tx=0x7fb6e000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.247+0000 7fb6f949c740 1 -- 172.21.15.1:0/557135822 wait complete. 2024-09-12T01:34:03.253 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.248+0000 7fb6f949c740 1 -- start start 2024-09-12T01:34:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.248+0000 7fb6f949c740 1 Processor -- start 2024-09-12T01:34:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.248+0000 7fb6f949c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc88e50090 0x55bc88e5a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.248+0000 7fb6f949c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc88e509d0 0x55bc88e5a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.248+0000 7fb6f949c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc88e55a60 0x55bc88e61710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.248+0000 7fb6f949c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55bc88e589d0 con 0x55bc88e55a60 2024-09-12T01:34:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.248+0000 7fb6f949c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55bc88d3e090 con 0x55bc88e50090 2024-09-12T01:34:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.248+0000 7fb6f949c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55bc88e58b50 con 0x55bc88e509d0 2024-09-12T01:34:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.248+0000 7fb6f949c740 10 client.0 _ll_register_callbacks cb 0x55bc88d92f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:34:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.249+0000 7fb6ef400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc88e50090 0x55bc88e5a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.249+0000 7fb6ef400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc88e50090 0x55bc88e5a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:40014/0 (socket says 172.21.15.1:40014) 2024-09-12T01:34:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.249+0000 7fb6ef400640 1 -- 172.21.15.1:0/626113318 learned_addr learned my addr 172.21.15.1:0/626113318 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:34:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.249+0000 7fb6eea00640 1 --2- 172.21.15.1:0/626113318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc88e509d0 0x55bc88e5a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:03.255 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.249+0000 7fb6efe00640 1 --2- 172.21.15.1:0/626113318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc88e55a60 0x55bc88e61710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:03.255 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.249+0000 7fb6ef400640 1 -- 172.21.15.1:0/626113318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc88e509d0 msgr2=0x55bc88e5a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:03.255 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.249+0000 7fb6ef400640 1 --2- 172.21.15.1:0/626113318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc88e509d0 0x55bc88e5a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.255 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.249+0000 7fb6ef400640 1 -- 172.21.15.1:0/626113318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc88e55a60 msgr2=0x55bc88e61710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:03.255 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.249+0000 7fb6ef400640 1 --2- 172.21.15.1:0/626113318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc88e55a60 0x55bc88e61710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.255 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.249+0000 7fb6ef400640 1 -- 172.21.15.1:0/626113318 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc88b94620 con 0x55bc88e50090 2024-09-12T01:34:03.255 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.249+0000 7fb6eea00640 1 --2- 172.21.15.1:0/626113318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc88e509d0 0x55bc88e5a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:34:03.255 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.249+0000 7fb6ef400640 1 --2- 172.21.15.1:0/626113318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc88e50090 0x55bc88e5a190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0010790 tx=0x7fb6e00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13ca90866082f055 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:03.255 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.249+0000 7fb6d9600640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-12T01:34:03.255 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.250+0000 7fb6d9600640 1 -- 172.21.15.1:0/626113318 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e001f980 con 0x55bc88e50090 2024-09-12T01:34:03.255 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.250+0000 7fb6f949c740 1 -- 172.21.15.1:0/626113318 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6e8030c70 con 0x55bc88e50090 2024-09-12T01:34:03.255 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.250+0000 7fb6d9600640 1 -- 172.21.15.1:0/626113318 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fb6e001fb20 con 0x55bc88e50090 2024-09-12T01:34:03.255 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.250+0000 7fb6d9600640 1 -- 172.21.15.1:0/626113318 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb6e0030050 con 0x55bc88e50090 2024-09-12T01:34:03.255 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.250+0000 7fb6f949c740 10 client.25544 fetch_fsmap learned FSMap version 34 2024-09-12T01:34:03.256 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.250+0000 7fb6f949c740 1 -- 172.21.15.1:0/626113318 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55bc88e64620 con 0x55bc88e50090 2024-09-12T01:34:03.256 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.250+0000 7fb6d9600640 1 -- 172.21.15.1:0/626113318 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6e8030c70 con 0x55bc88e50090 2024-09-12T01:34:03.256 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.250+0000 7fb6d9600640 1 -- 172.21.15.1:0/626113318 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7fb6e001d380 con 0x55bc88e50090 2024-09-12T01:34:03.256 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.250+0000 7fb6f949c740 10 client.25544 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T01:34:03.256 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.250+0000 7fb6f949c740 10 client.25544 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:34:03.256 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.250+0000 7fb6f949c740 20 client.25544 populate_metadata read hostname 'smithi001' 2024-09-12T01:34:03.256 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.251+0000 7fb6f949c740 1 --2- 172.21.15.1:0/626113318 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55bc88e65ca0 0x55bc88e680f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:03.256 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.251+0000 7fb6f949c740 4 client.25544 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:34:03.256 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.251+0000 7fb6f949c740 1 -- 172.21.15.1:0/626113318 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc88dc2830 con 0x55bc88e65ca0 2024-09-12T01:34:03.256 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.251+0000 7fb6eea00640 1 --2- 172.21.15.1:0/626113318 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55bc88e65ca0 0x55bc88e680f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:03.256 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.251+0000 7fb6eea00640 1 --2- 172.21.15.1:0/626113318 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55bc88e65ca0 0x55bc88e680f0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6527d4ba9f9884d server_cookie=47ccbc0e856b17f0 in_seq=0 out_seq=0 2024-09-12T01:34:03.256 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.251+0000 7fb6d9600640 10 client.25544 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:34:03.256 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.252+0000 7fb6d9600640 1 -- 172.21.15.1:0/626113318 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bc88dc2830 con 0x55bc88e65ca0 2024-09-12T01:34:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.252+0000 7fb6d9600640 10 client.25544 handle_command_reply: tid=0 2024-09-12T01:34:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.252+0000 7fb6f949c740 1 client.25544 shutdown 2024-09-12T01:34:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.253+0000 7fb6f949c740 1 -- 172.21.15.1:0/626113318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc88e50090 msgr2=0x55bc88e5a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.253+0000 7fb6f949c740 1 --2- 172.21.15.1:0/626113318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc88e50090 0x55bc88e5a190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0010790 tx=0x7fb6e00046c0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.253+0000 7fb6f949c740 1 -- 172.21.15.1:0/626113318 shutdown_connections 2024-09-12T01:34:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.253+0000 7fb6f949c740 1 --2- 172.21.15.1:0/626113318 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55bc88e65ca0 0x55bc88e680f0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.253+0000 7fb6f949c740 1 --2- 172.21.15.1:0/626113318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc88e55a60 0x55bc88e61710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.253+0000 7fb6f949c740 1 --2- 172.21.15.1:0/626113318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc88e509d0 0x55bc88e5a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.253+0000 7fb6f949c740 1 --2- 172.21.15.1:0/626113318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc88e50090 0x55bc88e5a190 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.253+0000 7fb6f949c740 1 -- 172.21.15.1:0/626113318 >> 172.21.15.1:0/626113318 conn(0x55bc88e4b700 msgr2=0x55bc88e4cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.253+0000 7fb6d9600640 0 client.25544 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:34:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.253+0000 7fb6f949c740 1 -- 172.21.15.1:0/626113318 shutdown_connections 2024-09-12T01:34:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.253+0000 7fb6f949c740 1 -- 172.21.15.1:0/626113318 wait complete. 2024-09-12T01:34:03.258 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.254+0000 7fb6f949c740 20 client.25544 trim_cache size 0 max 16384 2024-09-12T01:34:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.273+0000 7fb6f949c740 1 Processor -- start 2024-09-12T01:34:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.273+0000 7fb6f949c740 1 -- start start 2024-09-12T01:34:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.273+0000 7fb6f949c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc88e50090 0x55bc88e5a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.273+0000 7fb6f949c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc88e509d0 0x55bc88e616d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.273+0000 7fb6f949c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc88e55a60 0x55bc88e4a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.273+0000 7fb6f949c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f01ac920 con 0x55bc88e50090 2024-09-12T01:34:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.273+0000 7fb6f949c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f01ae490 con 0x55bc88e55a60 2024-09-12T01:34:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.273+0000 7fb6f949c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f01acff0 con 0x55bc88e509d0 2024-09-12T01:34:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.273+0000 7fb6eea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc88e509d0 0x55bc88e616d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.273+0000 7fb6efe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc88e55a60 0x55bc88e4a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.273+0000 7fb6ef400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc88e50090 0x55bc88e5a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.273+0000 7fb6efe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc88e55a60 0x55bc88e4a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:40028/0 (socket says 172.21.15.1:40028) 2024-09-12T01:34:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.273+0000 7fb6efe00640 1 -- 172.21.15.1:0/3890924527 learned_addr learned my addr 172.21.15.1:0/3890924527 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:34:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.273+0000 7fb6eea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc88e509d0 0x55bc88e616d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:55284/0 (socket says 172.21.15.1:55284) 2024-09-12T01:34:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.274+0000 7fb6efe00640 1 -- 172.21.15.1:0/3890924527 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc88e509d0 msgr2=0x55bc88e616d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.274+0000 7fb6efe00640 1 --2- 172.21.15.1:0/3890924527 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc88e509d0 0x55bc88e616d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.274+0000 7fb6efe00640 1 -- 172.21.15.1:0/3890924527 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc88e50090 msgr2=0x55bc88e5a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.274+0000 7fb6efe00640 1 --2- 172.21.15.1:0/3890924527 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc88e50090 0x55bc88e5a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.274+0000 7fb6efe00640 1 -- 172.21.15.1:0/3890924527 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc88e51660 con 0x55bc88e55a60 2024-09-12T01:34:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.274+0000 7fb6eea00640 1 --2- 172.21.15.1:0/3890924527 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc88e509d0 0x55bc88e616d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:34:03.276 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.274+0000 7fb6efe00640 1 --2- 172.21.15.1:0/3890924527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc88e55a60 0x55bc88e4a770 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8030d20 tx=0x7fb6e8000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8764943e9af12bfd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:03.276 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.274+0000 7fb6dbe00640 1 -- 172.21.15.1:0/3890924527 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8041040 con 0x55bc88e55a60 2024-09-12T01:34:03.276 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.274+0000 7fb6dbe00640 1 -- 172.21.15.1:0/3890924527 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6e804a070 con 0x55bc88e55a60 2024-09-12T01:34:03.276 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.274+0000 7fb6dbe00640 1 -- 172.21.15.1:0/3890924527 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8036040 con 0x55bc88e55a60 2024-09-12T01:34:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.275+0000 7fb6f949c740 1 -- 172.21.15.1:0/3890924527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc88e55a60 msgr2=0x55bc88e4a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.275+0000 7fb6f949c740 1 --2- 172.21.15.1:0/3890924527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc88e55a60 0x55bc88e4a770 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8030d20 tx=0x7fb6e8000f40 comp rx=0 tx=0).stop 2024-09-12T01:34:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.275+0000 7fb6f949c740 1 -- 172.21.15.1:0/3890924527 shutdown_connections 2024-09-12T01:34:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.275+0000 7fb6f949c740 1 --2- 172.21.15.1:0/3890924527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc88e55a60 0x55bc88e4a770 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.275+0000 7fb6f949c740 1 --2- 172.21.15.1:0/3890924527 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc88e509d0 0x55bc88e616d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.275+0000 7fb6f949c740 1 --2- 172.21.15.1:0/3890924527 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc88e50090 0x55bc88e5a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.275+0000 7fb6f949c740 1 -- 172.21.15.1:0/3890924527 >> 172.21.15.1:0/3890924527 conn(0x55bc88e4b700 msgr2=0x55bc88e4a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.275+0000 7fb6f949c740 1 -- 172.21.15.1:0/3890924527 shutdown_connections 2024-09-12T01:34:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.275+0000 7fb6f949c740 1 -- 172.21.15.1:0/3890924527 wait complete. 2024-09-12T01:34:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.276+0000 7fb6f949c740 1 -- start start 2024-09-12T01:34:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.276+0000 7fb6f949c740 1 Processor -- start 2024-09-12T01:34:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.276+0000 7fb6f949c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc88e50090 0x55bc88e5e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.276+0000 7fb6f949c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc88e509d0 0x55bc88e5ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.276+0000 7fb6f949c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc88e55a60 0x55bc88e5fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.276+0000 7fb6f949c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f01add90 con 0x55bc88e55a60 2024-09-12T01:34:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.276+0000 7fb6f949c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f01ad6c0 con 0x55bc88e50090 2024-09-12T01:34:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.276+0000 7fb6f949c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f01aa900 con 0x55bc88e509d0 2024-09-12T01:34:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.276+0000 7fb6f949c740 10 client.0 _ll_register_callbacks cb 0x7fb6e0005cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:34:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.277+0000 7fb6eea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc88e509d0 0x55bc88e5ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.277+0000 7fb6efe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc88e55a60 0x55bc88e5fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.277+0000 7fb6eea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc88e509d0 0x55bc88e5ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:55296/0 (socket says 172.21.15.1:55296) 2024-09-12T01:34:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.277+0000 7fb6eea00640 1 -- 172.21.15.1:0/3277730995 learned_addr learned my addr 172.21.15.1:0/3277730995 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:34:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.277+0000 7fb6ef400640 1 --2- 172.21.15.1:0/3277730995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc88e50090 0x55bc88e5e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.277+0000 7fb6eea00640 1 -- 172.21.15.1:0/3277730995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc88e50090 msgr2=0x55bc88e5e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.277+0000 7fb6eea00640 1 --2- 172.21.15.1:0/3277730995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc88e50090 0x55bc88e5e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.277+0000 7fb6eea00640 1 -- 172.21.15.1:0/3277730995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc88e55a60 msgr2=0x55bc88e5fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.277+0000 7fb6eea00640 1 --2- 172.21.15.1:0/3277730995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc88e55a60 0x55bc88e5fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.277+0000 7fb6eea00640 1 -- 172.21.15.1:0/3277730995 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc88e60340 con 0x55bc88e509d0 2024-09-12T01:34:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.277+0000 7fb6eea00640 1 --2- 172.21.15.1:0/3277730995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc88e509d0 0x55bc88e5ed70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc09b790 tx=0x7fb6dc09b4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f812471b8faf710b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.277+0000 7fb6d9600640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-12T01:34:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.278+0000 7fb6d9600640 1 -- 172.21.15.1:0/3277730995 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc0a4700 con 0x55bc88e509d0 2024-09-12T01:34:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.278+0000 7fb6d9600640 1 -- 172.21.15.1:0/3277730995 <== mon.1 v2:172.21.15.104:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fb6dc003b10 con 0x55bc88e509d0 2024-09-12T01:34:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.278+0000 7fb6f949c740 1 -- 172.21.15.1:0/3277730995 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6e804a070 con 0x55bc88e509d0 2024-09-12T01:34:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.278+0000 7fb6d9600640 1 -- 172.21.15.1:0/3277730995 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb6dc0cf2d0 con 0x55bc88e509d0 2024-09-12T01:34:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.278+0000 7fb6f949c740 10 client.25549 fetch_fsmap learned FSMap version 34 2024-09-12T01:34:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.278+0000 7fb6f949c740 1 -- 172.21.15.1:0/3277730995 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55bc88e61720 con 0x55bc88e509d0 2024-09-12T01:34:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.278+0000 7fb6d9600640 1 -- 172.21.15.1:0/3277730995 <== mon.1 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6e804a070 con 0x55bc88e509d0 2024-09-12T01:34:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.278+0000 7fb6d9600640 1 -- 172.21.15.1:0/3277730995 <== mon.1 v2:172.21.15.104:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7fb6dc0ba190 con 0x55bc88e509d0 2024-09-12T01:34:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.278+0000 7fb6f949c740 10 client.25549 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T01:34:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.278+0000 7fb6f949c740 10 client.25549 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:34:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.278+0000 7fb6f949c740 20 client.25549 populate_metadata read hostname 'smithi001' 2024-09-12T01:34:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.278+0000 7fb6f949c740 1 --2- 172.21.15.1:0/3277730995 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55bc88ed0eb0 0x55bc88ed32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.278+0000 7fb6f949c740 4 client.25549 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:34:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.278+0000 7fb6f949c740 1 -- 172.21.15.1:0/3277730995 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bc88dbcb80 con 0x55bc88ed0eb0 2024-09-12T01:34:03.283 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.279+0000 7fb6ef400640 1 --2- 172.21.15.1:0/3277730995 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55bc88ed0eb0 0x55bc88ed32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:03.283 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.279+0000 7fb6ef400640 1 --2- 172.21.15.1:0/3277730995 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55bc88ed0eb0 0x55bc88ed32d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0d33e6115ff63ca server_cookie=f294b2e83695b8 in_seq=0 out_seq=0 2024-09-12T01:34:03.283 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.279+0000 7fb6d9600640 10 client.25549 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:34:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.289+0000 7fb6d9600640 1 -- 172.21.15.1:0/3277730995 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bc88dbcb80 con 0x55bc88ed0eb0 2024-09-12T01:34:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.289+0000 7fb6d9600640 10 client.25549 handle_command_reply: tid=0 2024-09-12T01:34:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.289+0000 7fb6f949c740 1 client.25549 shutdown 2024-09-12T01:34:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.289+0000 7fb6f949c740 1 -- 172.21.15.1:0/3277730995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc88e509d0 msgr2=0x55bc88e5ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.289+0000 7fb6f949c740 1 --2- 172.21.15.1:0/3277730995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc88e509d0 0x55bc88e5ed70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc09b790 tx=0x7fb6dc09b4b0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.289+0000 7fb6f949c740 1 -- 172.21.15.1:0/3277730995 shutdown_connections 2024-09-12T01:34:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.289+0000 7fb6f949c740 1 --2- 172.21.15.1:0/3277730995 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55bc88ed0eb0 0x55bc88ed32d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.289+0000 7fb6f949c740 1 --2- 172.21.15.1:0/3277730995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc88e55a60 0x55bc88e5fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.290+0000 7fb6f949c740 1 --2- 172.21.15.1:0/3277730995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc88e509d0 0x55bc88e5ed70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.290+0000 7fb6f949c740 1 --2- 172.21.15.1:0/3277730995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bc88e50090 0x55bc88e5e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.290+0000 7fb6f949c740 1 -- 172.21.15.1:0/3277730995 >> 172.21.15.1:0/3277730995 conn(0x55bc88e4b700 msgr2=0x55bc88e4a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.290+0000 7fb6d9600640 0 client.25549 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:34:03.291 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.290+0000 7fb6f949c740 1 -- 172.21.15.1:0/3277730995 shutdown_connections 2024-09-12T01:34:03.291 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.290+0000 7fb6f949c740 1 -- 172.21.15.1:0/3277730995 wait complete. 2024-09-12T01:34:03.291 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.290+0000 7fb6f949c740 20 client.25549 trim_cache size 0 max 16384 2024-09-12T01:34:03.291 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-12T01:34:03.291 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-12T01:34:03.291 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-12T01:34:03.292 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-12T01:34:03.292 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.290+0000 7fb6f6600640 1 -- 172.21.15.1:0/2900807395 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fb6d40a2c00 msgr2=0x7fb6d40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:03.292 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.290+0000 7fb6f6600640 1 --2- 172.21.15.1:0/2900807395 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fb6d40a2c00 0x7fb6d40a50c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x55bc88e51400 tx=0x7fb6e000b390 comp rx=0 tx=0).stop 2024-09-12T01:34:03.293 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.290+0000 7fb6f6600640 1 -- 172.21.15.1:0/2900807395 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6f0102d30 msgr2=0x7fb6f00745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:03.293 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.290+0000 7fb6f6600640 1 --2- 172.21.15.1:0/2900807395 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6f0102d30 0x7fb6f00745e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc00e9c0 tx=0x7fb6dc00ee80 comp rx=0 tx=0).stop 2024-09-12T01:34:03.293 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.291+0000 7fb6f6600640 1 -- 172.21.15.1:0/2900807395 shutdown_connections 2024-09-12T01:34:03.293 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.291+0000 7fb6f6600640 1 --2- 172.21.15.1:0/2900807395 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6f01036f0 0x7fb6f0074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.293 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.291+0000 7fb6f6600640 1 --2- 172.21.15.1:0/2900807395 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fb6d40a2c00 0x7fb6d40a50c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.293 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.291+0000 7fb6f6600640 1 --2- 172.21.15.1:0/2900807395 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6f0102d30 0x7fb6f00745e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.293 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.291+0000 7fb6f6600640 1 --2- 172.21.15.1:0/2900807395 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6f0101b40 0x7fb6f0075fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:03.293 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.291+0000 7fb6f6600640 1 -- 172.21.15.1:0/2900807395 >> 172.21.15.1:0/2900807395 conn(0x7fb6f00fd2e0 msgr2=0x7fb6f0106100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:03.293 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.291+0000 7fb6f6600640 1 -- 172.21.15.1:0/2900807395 shutdown_connections 2024-09-12T01:34:03.294 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:03.291+0000 7fb6f6600640 1 -- 172.21.15.1:0/2900807395 wait complete. 2024-09-12T01:34:04.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:34:04 smithi001 ceph-mon[30387]: pgmap v532: 129 pgs: 129 active+clean; 98 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.0 MiB/s wr, 360 op/s 2024-09-12T01:34:04.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:34:04 smithi053 ceph-mon[35158]: pgmap v532: 129 pgs: 129 active+clean; 98 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.0 MiB/s wr, 360 op/s 2024-09-12T01:34:05.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:34:04 smithi104 ceph-mon[35219]: pgmap v532: 129 pgs: 129 active+clean; 98 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.0 MiB/s wr, 360 op/s 2024-09-12T01:34:06.823 INFO:tasks.workunit.client.0.smithi001.stdout:40000 mapwrite 0x1becad thru 0x1c6a92 (0x7de6 bytes) 2024-09-12T01:34:06.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:34:06 smithi001 ceph-mon[30387]: pgmap v533: 129 pgs: 129 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.2 MiB/s wr, 431 op/s 2024-09-12T01:34:06.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:34:06 smithi053 ceph-mon[35158]: pgmap v533: 129 pgs: 129 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.2 MiB/s wr, 431 op/s 2024-09-12T01:34:07.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:34:06 smithi104 ceph-mon[35219]: pgmap v533: 129 pgs: 129 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.2 MiB/s wr, 431 op/s 2024-09-12T01:34:08.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:34:08 smithi001 ceph-mon[30387]: pgmap v534: 129 pgs: 129 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.2 MiB/s wr, 404 op/s 2024-09-12T01:34:08.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:34:08 smithi053 ceph-mon[35158]: pgmap v534: 129 pgs: 129 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.2 MiB/s wr, 404 op/s 2024-09-12T01:34:09.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:34:08 smithi104 ceph-mon[35219]: pgmap v534: 129 pgs: 129 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.2 MiB/s wr, 404 op/s 2024-09-12T01:34:10.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:34:10 smithi001 ceph-mon[30387]: pgmap v535: 129 pgs: 129 active+clean; 94 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.8 MiB/s wr, 337 op/s 2024-09-12T01:34:10.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:34:10 smithi053 ceph-mon[35158]: pgmap v535: 129 pgs: 129 active+clean; 94 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.8 MiB/s wr, 337 op/s 2024-09-12T01:34:11.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:34:10 smithi104 ceph-mon[35219]: pgmap v535: 129 pgs: 129 active+clean; 94 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.8 MiB/s wr, 337 op/s 2024-09-12T01:34:12.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:34:12 smithi001 ceph-mon[30387]: pgmap v536: 129 pgs: 129 active+clean; 94 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 2.4 MiB/s wr, 489 op/s 2024-09-12T01:34:12.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:34:12 smithi053 ceph-mon[35158]: pgmap v536: 129 pgs: 129 active+clean; 94 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 2.4 MiB/s wr, 489 op/s 2024-09-12T01:34:13.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:34:12 smithi104 ceph-mon[35219]: pgmap v536: 129 pgs: 129 active+clean; 94 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 2.4 MiB/s wr, 489 op/s 2024-09-12T01:34:14.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:34:14 smithi001 ceph-mon[30387]: pgmap v537: 129 pgs: 129 active+clean; 94 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.9 MiB/s wr, 375 op/s 2024-09-12T01:34:14.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:34:14 smithi053 ceph-mon[35158]: pgmap v537: 129 pgs: 129 active+clean; 94 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.9 MiB/s wr, 375 op/s 2024-09-12T01:34:15.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:34:14 smithi104 ceph-mon[35219]: pgmap v537: 129 pgs: 129 active+clean; 94 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.9 MiB/s wr, 375 op/s 2024-09-12T01:34:16.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.436+0000 7fe48d400640 1 -- 172.21.15.1:0/721491234 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe488102ba0 msgr2=0x7fe488103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:16.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.436+0000 7fe48d400640 1 --2- 172.21.15.1:0/721491234 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe488102ba0 0x7fe488103020 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe474009900 tx=0x7fe47402a3f0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.436+0000 7fe48d400640 1 -- 172.21.15.1:0/721491234 shutdown_connections 2024-09-12T01:34:16.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.436+0000 7fe48d400640 1 --2- 172.21.15.1:0/721491234 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe488103560 0x7fe488109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.436+0000 7fe48d400640 1 --2- 172.21.15.1:0/721491234 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe488102ba0 0x7fe488103020 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.436+0000 7fe48d400640 1 --2- 172.21.15.1:0/721491234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4881019b0 0x7fe488101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.436+0000 7fe48d400640 1 -- 172.21.15.1:0/721491234 >> 172.21.15.1:0/721491234 conn(0x7fe4880fd130 msgr2=0x7fe4880ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:16.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.436+0000 7fe48d400640 1 -- 172.21.15.1:0/721491234 shutdown_connections 2024-09-12T01:34:16.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.437+0000 7fe48d400640 1 -- 172.21.15.1:0/721491234 wait complete. 2024-09-12T01:34:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.438+0000 7fe48d400640 1 Processor -- start 2024-09-12T01:34:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.438+0000 7fe48d400640 1 -- start start 2024-09-12T01:34:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.438+0000 7fe48d400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4881019b0 0x7fe488075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.438+0000 7fe48d400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe488102ba0 0x7fe4880745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.438+0000 7fe48d400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe488103560 0x7fe488074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.438+0000 7fe48d400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe48810d680 con 0x7fe488103560 2024-09-12T01:34:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.438+0000 7fe48d400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe48810d500 con 0x7fe4881019b0 2024-09-12T01:34:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.438+0000 7fe48d400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe48810d800 con 0x7fe488102ba0 2024-09-12T01:34:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.439+0000 7fe487e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4881019b0 0x7fe488075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.439+0000 7fe487e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4881019b0 0x7fe488075fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:60608/0 (socket says 172.21.15.1:60608) 2024-09-12T01:34:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.439+0000 7fe487e00640 1 -- 172.21.15.1:0/1704737342 learned_addr learned my addr 172.21.15.1:0/1704737342 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:34:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.439+0000 7fe487e00640 1 -- 172.21.15.1:0/1704737342 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe488102ba0 msgr2=0x7fe4880745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.439+0000 7fe487e00640 1 --2- 172.21.15.1:0/1704737342 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe488102ba0 0x7fe4880745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.439+0000 7fe487e00640 1 -- 172.21.15.1:0/1704737342 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe488103560 msgr2=0x7fe488074b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T01:34:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.439+0000 7fe487e00640 1 --2- 172.21.15.1:0/1704737342 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe488103560 0x7fe488074b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.439+0000 7fe487e00640 1 -- 172.21.15.1:0/1704737342 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4880753b0 con 0x7fe4881019b0 2024-09-12T01:34:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.440+0000 7fe487e00640 1 --2- 172.21.15.1:0/1704737342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4881019b0 0x7fe488075fa0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe47800e9c0 tx=0x7fe47800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.440+0000 7fe484c00640 1 -- 172.21.15.1:0/1704737342 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47800cd80 con 0x7fe4881019b0 2024-09-12T01:34:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.440+0000 7fe48d400640 1 -- 172.21.15.1:0/1704737342 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4881a1df0 con 0x7fe4881019b0 2024-09-12T01:34:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.440+0000 7fe48d400640 1 -- 172.21.15.1:0/1704737342 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4881a2250 con 0x7fe4881019b0 2024-09-12T01:34:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.440+0000 7fe484c00640 1 -- 172.21.15.1:0/1704737342 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4780035f0 con 0x7fe4881019b0 2024-09-12T01:34:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.440+0000 7fe484c00640 1 -- 172.21.15.1:0/1704737342 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe478010690 con 0x7fe4881019b0 2024-09-12T01:34:16.444 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.444+0000 7fe4917f2740 1 Processor -- start 2024-09-12T01:34:16.444 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.444+0000 7fe4917f2740 1 -- start start 2024-09-12T01:34:16.444 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.445+0000 7fe4917f2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559624600260 0x559624600660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:16.444 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.445+0000 7fe4917f2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559624600ba0 0x559624605660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:16.444 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.445+0000 7fe4917f2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559624605c30 0x5596246080f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:16.445 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.445+0000 7fe4917f2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559624565100 con 0x559624600ba0 2024-09-12T01:34:16.445 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.445+0000 7fe4917f2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55962450d1b0 con 0x559624605c30 2024-09-12T01:34:16.445 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.445+0000 7fe4917f2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5596245610c0 con 0x559624600260 2024-09-12T01:34:16.445 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.445+0000 7fe484c00640 1 -- 172.21.15.1:0/1704737342 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fe4780037a0 con 0x7fe4881019b0 2024-09-12T01:34:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.445+0000 7fe48ca00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559624600260 0x559624600660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.445+0000 7fe487400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559624605c30 0x5596246080f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.445+0000 7fe48ca00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559624600260 0x559624600660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:55678/0 (socket says 172.21.15.1:55678) 2024-09-12T01:34:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.445+0000 7fe48ca00640 1 -- 172.21.15.1:0/3027988921 learned_addr learned my addr 172.21.15.1:0/3027988921 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:34:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.445+0000 7fe487e00640 1 --2- 172.21.15.1:0/3027988921 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559624600ba0 0x559624605660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.445+0000 7fe487400640 1 -- 172.21.15.1:0/3027988921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559624600260 msgr2=0x559624600660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.445+0000 7fe487400640 1 --2- 172.21.15.1:0/3027988921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559624600260 0x559624600660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.445+0000 7fe487400640 1 -- 172.21.15.1:0/3027988921 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559624600ba0 msgr2=0x559624605660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.445+0000 7fe487400640 1 --2- 172.21.15.1:0/3027988921 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559624600ba0 0x559624605660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.446+0000 7fe487400640 1 -- 172.21.15.1:0/3027988921 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559624278210 con 0x559624605c30 2024-09-12T01:34:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.446+0000 7fe487400640 1 --2- 172.21.15.1:0/3027988921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559624605c30 0x5596246080f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe47402a910 tx=0x7fe474000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d48c566d40ce6eaa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.446+0000 7fe47d200640 1 -- 172.21.15.1:0/3027988921 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47403e070 con 0x559624605c30 2024-09-12T01:34:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.446+0000 7fe47d200640 1 -- 172.21.15.1:0/3027988921 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe474005530 con 0x559624605c30 2024-09-12T01:34:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.446+0000 7fe47d200640 1 -- 172.21.15.1:0/3027988921 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe474035040 con 0x559624605c30 2024-09-12T01:34:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.446+0000 7fe484c00640 1 --2- 172.21.15.1:0/1704737342 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe46c0ab400 0x7fe46c0ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.447+0000 7fe484c00640 1 -- 172.21.15.1:0/1704737342 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fe4780cf7b0 con 0x7fe4881019b0 2024-09-12T01:34:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.447+0000 7fe48ca00640 1 --2- 172.21.15.1:0/1704737342 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe46c0ab400 0x7fe46c0ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.447+0000 7fe4917f2740 1 -- 172.21.15.1:0/3027988921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559624605c30 msgr2=0x5596246080f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.447+0000 7fe4917f2740 1 --2- 172.21.15.1:0/3027988921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559624605c30 0x5596246080f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe47402a910 tx=0x7fe474000bc0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.447+0000 7fe4917f2740 1 -- 172.21.15.1:0/3027988921 shutdown_connections 2024-09-12T01:34:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.447+0000 7fe4917f2740 1 --2- 172.21.15.1:0/3027988921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559624605c30 0x5596246080f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.447+0000 7fe4917f2740 1 --2- 172.21.15.1:0/3027988921 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559624600ba0 0x559624605660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.447+0000 7fe4917f2740 1 --2- 172.21.15.1:0/3027988921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559624600260 0x559624600660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.447+0000 7fe4917f2740 1 -- 172.21.15.1:0/3027988921 >> 172.21.15.1:0/3027988921 conn(0x5596245fb8d0 msgr2=0x5596245fbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.447+0000 7fe4917f2740 1 -- 172.21.15.1:0/3027988921 shutdown_connections 2024-09-12T01:34:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.447+0000 7fe48ca00640 1 --2- 172.21.15.1:0/1704737342 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe46c0ab400 0x7fe46c0ad8c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe4800049d0 tx=0x7fe480009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.448+0000 7fe4917f2740 1 -- 172.21.15.1:0/3027988921 wait complete. 2024-09-12T01:34:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.448+0000 7fe4917f2740 1 -- start start 2024-09-12T01:34:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.448+0000 7fe4917f2740 1 Processor -- start 2024-09-12T01:34:16.450 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.449+0000 7fe4917f2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559624600260 0x55962460a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:16.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.449+0000 7fe4917f2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559624600ba0 0x55962460a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:16.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.449+0000 7fe4917f2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559624605c30 0x559624611910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:16.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.449+0000 7fe4917f2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559624608ba0 con 0x559624605c30 2024-09-12T01:34:16.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.449+0000 7fe4917f2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5596244ee090 con 0x559624600260 2024-09-12T01:34:16.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.449+0000 7fe4917f2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559624608d20 con 0x559624600ba0 2024-09-12T01:34:16.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.449+0000 7fe487400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559624600ba0 0x55962460a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:16.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.449+0000 7fe487400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559624600ba0 0x55962460a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:55686/0 (socket says 172.21.15.1:55686) 2024-09-12T01:34:16.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.449+0000 7fe487400640 1 -- 172.21.15.1:0/4221864287 learned_addr learned my addr 172.21.15.1:0/4221864287 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:34:16.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.449+0000 7fe48ca00640 1 --2- 172.21.15.1:0/4221864287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559624605c30 0x559624611910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:16.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.449+0000 7fe487e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559624600260 0x55962460a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:16.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.449+0000 7fe487400640 1 -- 172.21.15.1:0/4221864287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559624600260 msgr2=0x55962460a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:16.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.450+0000 7fe487400640 1 --2- 172.21.15.1:0/4221864287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559624600260 0x55962460a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.450+0000 7fe487400640 1 -- 172.21.15.1:0/4221864287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559624605c30 msgr2=0x559624611910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:16.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.450+0000 7fe487400640 1 --2- 172.21.15.1:0/4221864287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559624605c30 0x559624611910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.450+0000 7fe487400640 1 -- 172.21.15.1:0/4221864287 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559624344620 con 0x559624600ba0 2024-09-12T01:34:16.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.450+0000 7fe48ca00640 1 --2- 172.21.15.1:0/4221864287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559624605c30 0x559624611910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:34:16.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.450+0000 7fe487400640 1 --2- 172.21.15.1:0/4221864287 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559624600ba0 0x55962460a8d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe474005c40 tx=0x7fe474004970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca7a1f36054907ca server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:16.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.450+0000 7fe471600640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-12T01:34:16.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.450+0000 7fe487e00640 1 --2- 172.21.15.1:0/4221864287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559624600260 0x55962460a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:34:16.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.450+0000 7fe4917f2740 10 client.0 _ll_register_callbacks cb 0x559624542f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:34:16.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.450+0000 7fe471600640 1 -- 172.21.15.1:0/4221864287 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47403e070 con 0x559624600ba0 2024-09-12T01:34:16.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.450+0000 7fe4917f2740 1 -- 172.21.15.1:0/4221864287 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe474005530 con 0x559624600ba0 2024-09-12T01:34:16.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.451+0000 7fe471600640 1 -- 172.21.15.1:0/4221864287 <== mon.1 v2:172.21.15.104:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fe474005250 con 0x559624600ba0 2024-09-12T01:34:16.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.451+0000 7fe471600640 1 -- 172.21.15.1:0/4221864287 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe474055050 con 0x559624600ba0 2024-09-12T01:34:16.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.451+0000 7fe4917f2740 10 client.25558 fetch_fsmap learned FSMap version 35 2024-09-12T01:34:16.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.451+0000 7fe4917f2740 1 -- 172.21.15.1:0/4221864287 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({fsmap=35}) -- 0x559624608c00 con 0x559624600ba0 2024-09-12T01:34:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.451+0000 7fe471600640 1 -- 172.21.15.1:0/4221864287 <== mon.1 v2:172.21.15.104:3300/0 4 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fe4740040c0 con 0x559624600ba0 2024-09-12T01:34:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.451+0000 7fe4917f2740 10 client.25558 fetch_fsmap finished waiting for FSMap version 35 2024-09-12T01:34:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.451+0000 7fe4917f2740 10 client.25558 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:34:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.451+0000 7fe4917f2740 20 client.25558 populate_metadata read hostname 'smithi001' 2024-09-12T01:34:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.451+0000 7fe4917f2740 1 --2- 172.21.15.1:0/4221864287 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x559624626810 0x559624628ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.451+0000 7fe4917f2740 4 client.25558 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:34:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.451+0000 7fe4917f2740 1 -- 172.21.15.1:0/4221864287 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559624572830 con 0x559624626810 2024-09-12T01:34:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.452+0000 7fe487e00640 1 --2- 172.21.15.1:0/4221864287 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x559624626810 0x559624628ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.452+0000 7fe487e00640 1 --2- 172.21.15.1:0/4221864287 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x559624626810 0x559624628ca0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c877553f0333b61 server_cookie=76349b5616b3aa5a in_seq=0 out_seq=0 2024-09-12T01:34:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.452+0000 7fe471600640 10 client.25558 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:34:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.453+0000 7fe471600640 1 -- 172.21.15.1:0/4221864287 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559624572830 con 0x559624626810 2024-09-12T01:34:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.453+0000 7fe471600640 10 client.25558 handle_command_reply: tid=0 2024-09-12T01:34:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.453+0000 7fe4917f2740 1 client.25558 shutdown 2024-09-12T01:34:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.453+0000 7fe4917f2740 1 -- 172.21.15.1:0/4221864287 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559624600ba0 msgr2=0x55962460a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.454+0000 7fe4917f2740 1 --2- 172.21.15.1:0/4221864287 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559624600ba0 0x55962460a8d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe474005c40 tx=0x7fe474004970 comp rx=0 tx=0).stop 2024-09-12T01:34:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.454+0000 7fe4917f2740 1 -- 172.21.15.1:0/4221864287 shutdown_connections 2024-09-12T01:34:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.454+0000 7fe4917f2740 1 --2- 172.21.15.1:0/4221864287 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x559624626810 0x559624628ca0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.454+0000 7fe4917f2740 1 --2- 172.21.15.1:0/4221864287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559624605c30 0x559624611910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.454+0000 7fe4917f2740 1 --2- 172.21.15.1:0/4221864287 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559624600ba0 0x55962460a8d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.454+0000 7fe4917f2740 1 --2- 172.21.15.1:0/4221864287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559624600260 0x55962460a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.454+0000 7fe471600640 0 client.25558 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:34:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.454+0000 7fe4917f2740 1 -- 172.21.15.1:0/4221864287 >> 172.21.15.1:0/4221864287 conn(0x5596245fb8d0 msgr2=0x5596245fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.454+0000 7fe4917f2740 1 -- 172.21.15.1:0/4221864287 shutdown_connections 2024-09-12T01:34:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.454+0000 7fe4917f2740 1 -- 172.21.15.1:0/4221864287 wait complete. 2024-09-12T01:34:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.454+0000 7fe4917f2740 20 client.25558 trim_cache size 0 max 16384 2024-09-12T01:34:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.478+0000 7fe4917f2740 1 Processor -- start 2024-09-12T01:34:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.478+0000 7fe4917f2740 1 -- start start 2024-09-12T01:34:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.479+0000 7fe4917f2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559624600260 0x5596245fbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.479+0000 7fe4917f2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559624600ba0 0x5596246118d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.479+0000 7fe4917f2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559624605c30 0x5596245fc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.479+0000 7fe4917f2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe46800fb60 con 0x559624600260 2024-09-12T01:34:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.479+0000 7fe4917f2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe4680116a0 con 0x559624600ba0 2024-09-12T01:34:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.479+0000 7fe4917f2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe468010230 con 0x559624605c30 2024-09-12T01:34:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.479+0000 7fe487e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559624600260 0x5596245fbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.479+0000 7fe487e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559624600260 0x5596245fbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41528/0 (socket says 172.21.15.1:41528) 2024-09-12T01:34:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.479+0000 7fe487e00640 1 -- 172.21.15.1:0/3908244026 learned_addr learned my addr 172.21.15.1:0/3908244026 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:34:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.479+0000 7fe487400640 1 --2- 172.21.15.1:0/3908244026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559624600ba0 0x5596246118d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.479+0000 7fe48ca00640 1 --2- 172.21.15.1:0/3908244026 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559624605c30 0x5596245fc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.480+0000 7fe487400640 1 -- 172.21.15.1:0/3908244026 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559624605c30 msgr2=0x5596245fc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.480+0000 7fe487400640 1 --2- 172.21.15.1:0/3908244026 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559624605c30 0x5596245fc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.480+0000 7fe487400640 1 -- 172.21.15.1:0/3908244026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559624600260 msgr2=0x5596245fbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.480+0000 7fe487400640 1 --2- 172.21.15.1:0/3908244026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559624600260 0x5596245fbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.480+0000 7fe487400640 1 -- 172.21.15.1:0/3908244026 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559624601830 con 0x559624600ba0 2024-09-12T01:34:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.480+0000 7fe487400640 1 --2- 172.21.15.1:0/3908244026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559624600ba0 0x5596246118d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe47402a910 tx=0x7fe474047000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9b832b491346514 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.480+0000 7fe473e00640 1 -- 172.21.15.1:0/3908244026 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe474035040 con 0x559624600ba0 2024-09-12T01:34:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.480+0000 7fe473e00640 1 -- 172.21.15.1:0/3908244026 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe474005530 con 0x559624600ba0 2024-09-12T01:34:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.480+0000 7fe473e00640 1 -- 172.21.15.1:0/3908244026 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47405a740 con 0x559624600ba0 2024-09-12T01:34:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.481+0000 7fe4917f2740 1 -- 172.21.15.1:0/3908244026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559624600ba0 msgr2=0x5596246118d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.481+0000 7fe4917f2740 1 --2- 172.21.15.1:0/3908244026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559624600ba0 0x5596246118d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe47402a910 tx=0x7fe474047000 comp rx=0 tx=0).stop 2024-09-12T01:34:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.481+0000 7fe4917f2740 1 -- 172.21.15.1:0/3908244026 shutdown_connections 2024-09-12T01:34:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.481+0000 7fe4917f2740 1 --2- 172.21.15.1:0/3908244026 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559624605c30 0x5596245fc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.481+0000 7fe4917f2740 1 --2- 172.21.15.1:0/3908244026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559624600ba0 0x5596246118d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.481+0000 7fe4917f2740 1 --2- 172.21.15.1:0/3908244026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559624600260 0x5596245fbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.481+0000 7fe4917f2740 1 -- 172.21.15.1:0/3908244026 >> 172.21.15.1:0/3908244026 conn(0x5596245fb8d0 msgr2=0x5596246053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.481+0000 7fe4917f2740 1 -- 172.21.15.1:0/3908244026 shutdown_connections 2024-09-12T01:34:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.481+0000 7fe4917f2740 1 -- 172.21.15.1:0/3908244026 wait complete. 2024-09-12T01:34:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.482+0000 7fe4917f2740 1 -- start start 2024-09-12T01:34:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.482+0000 7fe4917f2740 1 Processor -- start 2024-09-12T01:34:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.482+0000 7fe4917f2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559624600260 0x55962460eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.483+0000 7fe4917f2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559624600ba0 0x55962460f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.483+0000 7fe4917f2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559624605c30 0x5596246101f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.483+0000 7fe4917f2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe468010fd0 con 0x559624600260 2024-09-12T01:34:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.483+0000 7fe4917f2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe468010900 con 0x559624600ba0 2024-09-12T01:34:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.483+0000 7fe4917f2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe46800dad0 con 0x559624605c30 2024-09-12T01:34:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.483+0000 7fe487e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559624600260 0x55962460eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.483+0000 7fe487e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559624600260 0x55962460eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41544/0 (socket says 172.21.15.1:41544) 2024-09-12T01:34:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.483+0000 7fe487e00640 1 -- 172.21.15.1:0/2437959581 learned_addr learned my addr 172.21.15.1:0/2437959581 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:34:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.483+0000 7fe487400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559624600ba0 0x55962460f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:16.485 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.483+0000 7fe4917f2740 10 client.0 _ll_register_callbacks cb 0x7fe480009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:34:16.485 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.483+0000 7fe48ca00640 1 --2- 172.21.15.1:0/2437959581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559624605c30 0x5596246101f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:16.485 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.483+0000 7fe487400640 1 -- 172.21.15.1:0/2437959581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559624605c30 msgr2=0x5596246101f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:16.485 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.483+0000 7fe487400640 1 --2- 172.21.15.1:0/2437959581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559624605c30 0x5596246101f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.485 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.483+0000 7fe487400640 1 -- 172.21.15.1:0/2437959581 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559624600260 msgr2=0x55962460eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:16.485 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.483+0000 7fe487400640 1 --2- 172.21.15.1:0/2437959581 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559624600260 0x55962460eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.485 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.483+0000 7fe487400640 1 -- 172.21.15.1:0/2437959581 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559624610800 con 0x559624600ba0 2024-09-12T01:34:16.486 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.484+0000 7fe487400640 1 --2- 172.21.15.1:0/2437959581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559624600ba0 0x55962460f1e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x559624628ed0 tx=0x7fe474004ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8313e1b910433c79 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.484+0000 7fe471600640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-12T01:34:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.484+0000 7fe471600640 1 -- 172.21.15.1:0/2437959581 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe474035070 con 0x559624600ba0 2024-09-12T01:34:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.484+0000 7fe4917f2740 1 -- 172.21.15.1:0/2437959581 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe474005530 con 0x559624600ba0 2024-09-12T01:34:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.484+0000 7fe471600640 1 -- 172.21.15.1:0/2437959581 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fe4740318c0 con 0x559624600ba0 2024-09-12T01:34:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.484+0000 7fe471600640 1 -- 172.21.15.1:0/2437959581 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe474071050 con 0x559624600ba0 2024-09-12T01:34:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.484+0000 7fe4917f2740 10 client.25565 fetch_fsmap learned FSMap version 35 2024-09-12T01:34:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.484+0000 7fe4917f2740 1 -- 172.21.15.1:0/2437959581 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559624611b10 con 0x559624600ba0 2024-09-12T01:34:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.485+0000 7fe471600640 1 -- 172.21.15.1:0/2437959581 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe474005530 con 0x559624600ba0 2024-09-12T01:34:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.485+0000 7fe471600640 1 -- 172.21.15.1:0/2437959581 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fe47405abd0 con 0x559624600ba0 2024-09-12T01:34:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.485+0000 7fe4917f2740 10 client.25565 fetch_fsmap finished waiting for FSMap version 35 2024-09-12T01:34:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.485+0000 7fe4917f2740 10 client.25565 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:34:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.485+0000 7fe4917f2740 20 client.25565 populate_metadata read hostname 'smithi001' 2024-09-12T01:34:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.485+0000 7fe4917f2740 1 --2- 172.21.15.1:0/2437959581 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x559624691c00 0x559624694020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.485+0000 7fe4917f2740 4 client.25565 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:34:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.485+0000 7fe4917f2740 1 -- 172.21.15.1:0/2437959581 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55962456cb80 con 0x559624691c00 2024-09-12T01:34:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.485+0000 7fe487e00640 1 --2- 172.21.15.1:0/2437959581 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x559624691c00 0x559624694020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.486+0000 7fe487e00640 1 --2- 172.21.15.1:0/2437959581 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x559624691c00 0x559624694020 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c2a09b2a71ecbe1 server_cookie=a08bb757e73777a9 in_seq=0 out_seq=0 2024-09-12T01:34:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.486+0000 7fe471600640 10 client.25565 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:34:16.495 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.495+0000 7fe471600640 1 -- 172.21.15.1:0/2437959581 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55962456cb80 con 0x559624691c00 2024-09-12T01:34:16.495 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.495+0000 7fe471600640 10 client.25565 handle_command_reply: tid=0 2024-09-12T01:34:16.495 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.495+0000 7fe4917f2740 1 client.25565 shutdown 2024-09-12T01:34:16.496 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.495+0000 7fe4917f2740 1 -- 172.21.15.1:0/2437959581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559624600ba0 msgr2=0x55962460f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:16.496 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.495+0000 7fe4917f2740 1 --2- 172.21.15.1:0/2437959581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559624600ba0 0x55962460f1e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x559624628ed0 tx=0x7fe474004ba0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.496 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.496+0000 7fe4917f2740 1 -- 172.21.15.1:0/2437959581 shutdown_connections 2024-09-12T01:34:16.496 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.496+0000 7fe4917f2740 1 --2- 172.21.15.1:0/2437959581 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x559624691c00 0x559624694020 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.496 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.496+0000 7fe4917f2740 1 --2- 172.21.15.1:0/2437959581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559624605c30 0x5596246101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.496 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.496+0000 7fe4917f2740 1 --2- 172.21.15.1:0/2437959581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559624600ba0 0x55962460f1e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.496 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.496+0000 7fe4917f2740 1 --2- 172.21.15.1:0/2437959581 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559624600260 0x55962460eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.496 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.496+0000 7fe4917f2740 1 -- 172.21.15.1:0/2437959581 >> 172.21.15.1:0/2437959581 conn(0x5596245fb8d0 msgr2=0x5596245fa1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:16.496 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.496+0000 7fe471600640 0 client.25565 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:34:16.496 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.496+0000 7fe4917f2740 1 -- 172.21.15.1:0/2437959581 shutdown_connections 2024-09-12T01:34:16.496 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.496+0000 7fe4917f2740 1 -- 172.21.15.1:0/2437959581 wait complete. 2024-09-12T01:34:16.496 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.496+0000 7fe4917f2740 20 client.25565 trim_cache size 0 max 16384 2024-09-12T01:34:16.497 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-12T01:34:16.497 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-12T01:34:16.497 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-12T01:34:16.497 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-12T01:34:16.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.497+0000 7fe48d400640 1 -- 172.21.15.1:0/1704737342 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe46c0ab400 msgr2=0x7fe46c0ad8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:16.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.497+0000 7fe48d400640 1 --2- 172.21.15.1:0/1704737342 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe46c0ab400 0x7fe46c0ad8c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe4800049d0 tx=0x7fe480009500 comp rx=0 tx=0).stop 2024-09-12T01:34:16.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.497+0000 7fe48d400640 1 -- 172.21.15.1:0/1704737342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4881019b0 msgr2=0x7fe488075fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:16.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.497+0000 7fe48d400640 1 --2- 172.21.15.1:0/1704737342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4881019b0 0x7fe488075fa0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe47800e9c0 tx=0x7fe47800ee80 comp rx=0 tx=0).stop 2024-09-12T01:34:16.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.497+0000 7fe48d400640 1 -- 172.21.15.1:0/1704737342 shutdown_connections 2024-09-12T01:34:16.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.497+0000 7fe48d400640 1 --2- 172.21.15.1:0/1704737342 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe488103560 0x7fe488074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.497+0000 7fe48d400640 1 --2- 172.21.15.1:0/1704737342 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fe46c0ab400 0x7fe46c0ad8c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.497+0000 7fe48d400640 1 --2- 172.21.15.1:0/1704737342 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe488102ba0 0x7fe4880745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.497+0000 7fe48d400640 1 --2- 172.21.15.1:0/1704737342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4881019b0 0x7fe488075fa0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:16.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.497+0000 7fe48d400640 1 -- 172.21.15.1:0/1704737342 >> 172.21.15.1:0/1704737342 conn(0x7fe4880fd130 msgr2=0x7fe488105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:16.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.497+0000 7fe48d400640 1 -- 172.21.15.1:0/1704737342 shutdown_connections 2024-09-12T01:34:16.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:16.497+0000 7fe48d400640 1 -- 172.21.15.1:0/1704737342 wait complete. 2024-09-12T01:34:16.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:34:16 smithi001 ceph-mon[30387]: pgmap v538: 129 pgs: 129 active+clean; 97 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.0 MiB/s wr, 396 op/s 2024-09-12T01:34:16.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:34:16 smithi001 ceph-mon[30387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T01:34:16.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:34:16 smithi053 ceph-mon[35158]: pgmap v538: 129 pgs: 129 active+clean; 97 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.0 MiB/s wr, 396 op/s 2024-09-12T01:34:16.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:34:16 smithi053 ceph-mon[35158]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T01:34:17.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:34:16 smithi104 ceph-mon[35219]: pgmap v538: 129 pgs: 129 active+clean; 97 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.0 MiB/s wr, 396 op/s 2024-09-12T01:34:17.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:34:16 smithi104 ceph-mon[35219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T01:34:18.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:34:18 smithi001 ceph-mon[30387]: pgmap v539: 129 pgs: 129 active+clean; 98 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.2 MiB/s wr, 417 op/s 2024-09-12T01:34:18.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:34:18 smithi053 ceph-mon[35158]: pgmap v539: 129 pgs: 129 active+clean; 98 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.2 MiB/s wr, 417 op/s 2024-09-12T01:34:19.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:34:18 smithi104 ceph-mon[35219]: pgmap v539: 129 pgs: 129 active+clean; 98 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.2 MiB/s wr, 417 op/s 2024-09-12T01:34:20.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:34:20 smithi001 ceph-mon[30387]: pgmap v540: 129 pgs: 129 active+clean; 98 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.8 MiB/s wr, 345 op/s 2024-09-12T01:34:20.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:34:20 smithi053 ceph-mon[35158]: pgmap v540: 129 pgs: 129 active+clean; 98 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.8 MiB/s wr, 345 op/s 2024-09-12T01:34:20.994 INFO:tasks.workunit.client.0.smithi001.stdout:50000 write 0x5304f4 thru 0x53fa11 (0xf51e bytes) 2024-09-12T01:34:20.994 INFO:tasks.workunit.client.0.smithi001.stdout:All 50000 operations completed A-OK! 2024-09-12T01:34:21.000 INFO:tasks.workunit.client.0.smithi001.stderr:+ ./fsx -z 100MB -N 50000 -p 10000 -l 104857600 2024-09-12T01:34:21.004 INFO:tasks.workunit.client.0.smithi001.stdout:Seed set to 1 2024-09-12T01:34:21.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:34:20 smithi104 ceph-mon[35219]: pgmap v540: 129 pgs: 129 active+clean; 98 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.8 MiB/s wr, 345 op/s 2024-09-12T01:34:22.383 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-09-12T01:34:22.383 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-09-12T01:34:22.385 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-09-12T01:34:22.385 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-09-12T01:34:22.385 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support clone range, disabling! 2024-09-12T01:34:22.389 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support dedupe range, disabling! 2024-09-12T01:34:22.391 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support exchange range, disabling! 2024-09-12T01:34:22.391 INFO:tasks.workunit.client.0.smithi001.stdout:skipping zero size read 2024-09-12T01:34:22.409 INFO:tasks.workunit.client.0.smithi001.stdout:truncating to largest ever: 0x1c37b90 2024-09-12T01:34:22.417 INFO:tasks.workunit.client.0.smithi001.stdout:truncating to largest ever: 0x5bd330d 2024-09-12T01:34:22.442 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0x637bf99 2024-09-12T01:34:22.854 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0x63d9b9c 2024-09-12T01:34:22.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:34:22 smithi001 ceph-mon[30387]: pgmap v541: 129 pgs: 129 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.5 MiB/s wr, 483 op/s 2024-09-12T01:34:22.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:34:22 smithi053 ceph-mon[35158]: pgmap v541: 129 pgs: 129 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.5 MiB/s wr, 483 op/s 2024-09-12T01:34:23.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:34:22 smithi104 ceph-mon[35219]: pgmap v541: 129 pgs: 129 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.5 MiB/s wr, 483 op/s 2024-09-12T01:34:24.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:34:24 smithi001 ceph-mon[30387]: pgmap v542: 129 pgs: 129 active+clean; 94 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.9 MiB/s wr, 351 op/s 2024-09-12T01:34:24.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:34:24 smithi053 ceph-mon[35158]: pgmap v542: 129 pgs: 129 active+clean; 94 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.9 MiB/s wr, 351 op/s 2024-09-12T01:34:25.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:34:24 smithi104 ceph-mon[35219]: pgmap v542: 129 pgs: 129 active+clean; 94 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.9 MiB/s wr, 351 op/s 2024-09-12T01:34:26.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:34:26 smithi001 ceph-mon[30387]: pgmap v543: 129 pgs: 129 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.9 MiB/s wr, 363 op/s 2024-09-12T01:34:26.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:34:26 smithi053 ceph-mon[35158]: pgmap v543: 129 pgs: 129 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.9 MiB/s wr, 363 op/s 2024-09-12T01:34:27.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:34:26 smithi104 ceph-mon[35219]: pgmap v543: 129 pgs: 129 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.9 MiB/s wr, 363 op/s 2024-09-12T01:34:28.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:34:28 smithi053 ceph-mon[35158]: pgmap v544: 129 pgs: 129 active+clean; 110 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.8 MiB/s wr, 397 op/s 2024-09-12T01:34:29.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:34:28 smithi104 ceph-mon[35219]: pgmap v544: 129 pgs: 129 active+clean; 110 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.8 MiB/s wr, 397 op/s 2024-09-12T01:34:29.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:34:28 smithi001 ceph-mon[30387]: pgmap v544: 129 pgs: 129 active+clean; 110 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.8 MiB/s wr, 397 op/s 2024-09-12T01:34:29.640 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.640+0000 7f90bca00640 1 -- 172.21.15.1:0/2001827769 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90b81024c0 msgr2=0x7f90b81048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:29.641 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.640+0000 7f90bca00640 1 --2- 172.21.15.1:0/2001827769 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90b81024c0 0x7f90b81048b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f90a8009900 tx=0x7f90a802a3f0 comp rx=0 tx=0).stop 2024-09-12T01:34:29.641 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.640+0000 7f90bca00640 1 -- 172.21.15.1:0/2001827769 shutdown_connections 2024-09-12T01:34:29.641 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.640+0000 7f90bca00640 1 --2- 172.21.15.1:0/2001827769 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90b8107720 0x7f90b8109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:29.641 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.640+0000 7f90bca00640 1 --2- 172.21.15.1:0/2001827769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90b8104df0 0x7f90b81071e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:29.641 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.640+0000 7f90bca00640 1 --2- 172.21.15.1:0/2001827769 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90b81024c0 0x7f90b81048b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:29.641 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.640+0000 7f90bca00640 1 -- 172.21.15.1:0/2001827769 >> 172.21.15.1:0/2001827769 conn(0x7f90b80fc040 msgr2=0x7f90b80fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:29.641 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.640+0000 7f90bca00640 1 -- 172.21.15.1:0/2001827769 shutdown_connections 2024-09-12T01:34:29.641 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.640+0000 7f90bca00640 1 -- 172.21.15.1:0/2001827769 wait complete. 2024-09-12T01:34:29.641 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.641+0000 7f90bca00640 1 Processor -- start 2024-09-12T01:34:29.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.641+0000 7f90bca00640 1 -- start start 2024-09-12T01:34:29.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.641+0000 7f90bca00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90b81024c0 0x7f90b8101e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:29.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.642+0000 7f90bca00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90b8104df0 0x7f90b8100470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:29.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.642+0000 7f90bca00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90b8107720 0x7f90b81009b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:29.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.642+0000 7f90bca00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f90b810d660 con 0x7f90b81024c0 2024-09-12T01:34:29.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.642+0000 7f90bca00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f90b810d4e0 con 0x7f90b8104df0 2024-09-12T01:34:29.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.642+0000 7f90bca00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f90b810d7e0 con 0x7f90b8107720 2024-09-12T01:34:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.642+0000 7f90b6800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90b81024c0 0x7f90b8101e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.642+0000 7f90b6800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90b81024c0 0x7f90b8101e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39424/0 (socket says 172.21.15.1:39424) 2024-09-12T01:34:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.642+0000 7f90b6800640 1 -- 172.21.15.1:0/3856311639 learned_addr learned my addr 172.21.15.1:0/3856311639 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:34:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.642+0000 7f90b5e00640 1 --2- 172.21.15.1:0/3856311639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90b8104df0 0x7f90b8100470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.642+0000 7f90b7200640 1 --2- 172.21.15.1:0/3856311639 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90b8107720 0x7f90b81009b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.642+0000 7f90b5e00640 1 -- 172.21.15.1:0/3856311639 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90b8107720 msgr2=0x7f90b81009b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.642+0000 7f90b5e00640 1 --2- 172.21.15.1:0/3856311639 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90b8107720 0x7f90b81009b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.642+0000 7f90b5e00640 1 -- 172.21.15.1:0/3856311639 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90b81024c0 msgr2=0x7f90b8101e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.642+0000 7f90b5e00640 1 --2- 172.21.15.1:0/3856311639 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90b81024c0 0x7f90b8101e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.642+0000 7f90b5e00640 1 -- 172.21.15.1:0/3856311639 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b8101240 con 0x7f90b8104df0 2024-09-12T01:34:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.642+0000 7f90b6800640 1 --2- 172.21.15.1:0/3856311639 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90b81024c0 0x7f90b8101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:34:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.643+0000 7f90b5e00640 1 --2- 172.21.15.1:0/3856311639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90b8104df0 0x7f90b8100470 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f90a400e9c0 tx=0x7f90a400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.643+0000 7f90af400640 1 -- 172.21.15.1:0/3856311639 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a400ccc0 con 0x7f90b8104df0 2024-09-12T01:34:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.643+0000 7f90af400640 1 -- 172.21.15.1:0/3856311639 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90a4005020 con 0x7f90b8104df0 2024-09-12T01:34:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.643+0000 7f90bca00640 1 -- 172.21.15.1:0/3856311639 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90b81a1e60 con 0x7f90b8104df0 2024-09-12T01:34:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.643+0000 7f90af400640 1 -- 172.21.15.1:0/3856311639 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a4004a00 con 0x7f90b8104df0 2024-09-12T01:34:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.643+0000 7f90bca00640 1 -- 172.21.15.1:0/3856311639 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90b81a2270 con 0x7f90b8104df0 2024-09-12T01:34:29.650 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.647+0000 7f90c01be740 1 Processor -- start 2024-09-12T01:34:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.648+0000 7f90c01be740 1 -- start start 2024-09-12T01:34:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.648+0000 7f90af400640 1 -- 172.21.15.1:0/3856311639 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f90a40051d0 con 0x7f90b8104df0 2024-09-12T01:34:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.648+0000 7f90c01be740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb2be3e260 0x55eb2be3e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.648+0000 7f90c01be740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb2be3eba0 0x55eb2be43660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.648+0000 7f90c01be740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb2be43c30 0x55eb2be460f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.648+0000 7f90c01be740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55eb2bda3100 con 0x55eb2be3eba0 2024-09-12T01:34:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.648+0000 7f90c01be740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55eb2bd4b1b0 con 0x55eb2be43c30 2024-09-12T01:34:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.648+0000 7f90c01be740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55eb2bd9f0c0 con 0x55eb2be3e260 2024-09-12T01:34:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.648+0000 7f90b7200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb2be3e260 0x55eb2be3e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.648+0000 7f90b5e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb2be43c30 0x55eb2be460f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.648+0000 7f90b7200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb2be3e260 0x55eb2be3e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:52422/0 (socket says 172.21.15.1:52422) 2024-09-12T01:34:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.648+0000 7f90b7200640 1 -- 172.21.15.1:0/3958383630 learned_addr learned my addr 172.21.15.1:0/3958383630 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:34:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.648+0000 7f90b6800640 1 --2- 172.21.15.1:0/3958383630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb2be3eba0 0x55eb2be43660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.649+0000 7f90b7200640 1 -- 172.21.15.1:0/3958383630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb2be43c30 msgr2=0x55eb2be460f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.649+0000 7f90b7200640 1 --2- 172.21.15.1:0/3958383630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb2be43c30 0x55eb2be460f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.649+0000 7f90b7200640 1 -- 172.21.15.1:0/3958383630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb2be3eba0 msgr2=0x55eb2be43660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.649+0000 7f90b7200640 1 --2- 172.21.15.1:0/3958383630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb2be3eba0 0x55eb2be43660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:29.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.649+0000 7f90b7200640 1 -- 172.21.15.1:0/3958383630 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb2bab6210 con 0x55eb2be3e260 2024-09-12T01:34:29.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.649+0000 7f90b7200640 1 --2- 172.21.15.1:0/3958383630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb2be3e260 0x55eb2be3e660 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f90b0009f90 tx=0x7f90b00118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4ee9c09c6d5b435 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:29.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.649+0000 7f90a3400640 1 -- 172.21.15.1:0/3958383630 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b0004d20 con 0x55eb2be3e260 2024-09-12T01:34:29.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.649+0000 7f90a3400640 1 -- 172.21.15.1:0/3958383630 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90b0004050 con 0x55eb2be3e260 2024-09-12T01:34:29.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.649+0000 7f90a3400640 1 -- 172.21.15.1:0/3958383630 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b0005260 con 0x55eb2be3e260 2024-09-12T01:34:29.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.650+0000 7f90af400640 1 --2- 172.21.15.1:0/3856311639 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f909c0a2b70 0x7f909c0a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:29.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.650+0000 7f90af400640 1 -- 172.21.15.1:0/3856311639 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f90a40cf2f0 con 0x7f90b8104df0 2024-09-12T01:34:29.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.650+0000 7f90b6800640 1 --2- 172.21.15.1:0/3856311639 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f909c0a2b70 0x7f909c0a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:29.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.650+0000 7f90c01be740 1 -- 172.21.15.1:0/3958383630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb2be3e260 msgr2=0x55eb2be3e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:29.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.650+0000 7f90c01be740 1 --2- 172.21.15.1:0/3958383630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb2be3e260 0x55eb2be3e660 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f90b0009f90 tx=0x7f90b00118d0 comp rx=0 tx=0).stop 2024-09-12T01:34:29.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.650+0000 7f90c01be740 1 -- 172.21.15.1:0/3958383630 shutdown_connections 2024-09-12T01:34:29.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.650+0000 7f90c01be740 1 --2- 172.21.15.1:0/3958383630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb2be43c30 0x55eb2be460f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:29.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.650+0000 7f90c01be740 1 --2- 172.21.15.1:0/3958383630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb2be3eba0 0x55eb2be43660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:29.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.650+0000 7f90c01be740 1 --2- 172.21.15.1:0/3958383630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb2be3e260 0x55eb2be3e660 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:29.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.650+0000 7f90c01be740 1 -- 172.21.15.1:0/3958383630 >> 172.21.15.1:0/3958383630 conn(0x55eb2be398d0 msgr2=0x55eb2be39cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:29.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.650+0000 7f90c01be740 1 -- 172.21.15.1:0/3958383630 shutdown_connections 2024-09-12T01:34:29.655 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.651+0000 7f90c01be740 1 -- 172.21.15.1:0/3958383630 wait complete. 2024-09-12T01:34:29.655 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.651+0000 7f90b6800640 1 --2- 172.21.15.1:0/3856311639 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f909c0a2b70 0x7f909c0a5030 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f90a8031440 tx=0x7f90a80315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:29.655 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.651+0000 7f90c01be740 1 -- start start 2024-09-12T01:34:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.651+0000 7f90c01be740 1 Processor -- start 2024-09-12T01:34:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.652+0000 7f90c01be740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb2be3e260 0x55eb2be48410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.652+0000 7f90c01be740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb2be3eba0 0x55eb2be48950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.652+0000 7f90c01be740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb2be43c30 0x55eb2be4f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.652+0000 7f90c01be740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55eb2be46c00 con 0x55eb2be43c30 2024-09-12T01:34:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.652+0000 7f90c01be740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55eb2bd2c090 con 0x55eb2be3eba0 2024-09-12T01:34:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.652+0000 7f90c01be740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55eb2be46d80 con 0x55eb2be3e260 2024-09-12T01:34:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.652+0000 7f90c01be740 10 client.0 _ll_register_callbacks cb 0x55eb2bd80f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:34:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.652+0000 7f90b5e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb2be3eba0 0x55eb2be48950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.652+0000 7f90b6800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb2be3e260 0x55eb2be48410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.652+0000 7f90b6800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb2be3e260 0x55eb2be48410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:52426/0 (socket says 172.21.15.1:52426) 2024-09-12T01:34:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.652+0000 7f90b6800640 1 -- 172.21.15.1:0/3305451262 learned_addr learned my addr 172.21.15.1:0/3305451262 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:34:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.652+0000 7f90b5e00640 1 --2- 172.21.15.1:0/3305451262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb2be3eba0 0x55eb2be48950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:56698/0 (socket says 172.21.15.1:56698) 2024-09-12T01:34:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.652+0000 7f90b7200640 1 --2- 172.21.15.1:0/3305451262 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb2be43c30 0x55eb2be4f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.652+0000 7f90b5e00640 1 -- 172.21.15.1:0/3305451262 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb2be3e260 msgr2=0x55eb2be48410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.652+0000 7f90b5e00640 1 --2- 172.21.15.1:0/3305451262 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb2be3e260 0x55eb2be48410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.652+0000 7f90b5e00640 1 -- 172.21.15.1:0/3305451262 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb2be43c30 msgr2=0x55eb2be4f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.652+0000 7f90b5e00640 1 --2- 172.21.15.1:0/3305451262 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb2be43c30 0x55eb2be4f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.653+0000 7f90b5e00640 1 -- 172.21.15.1:0/3305451262 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb2bb82620 con 0x55eb2be3eba0 2024-09-12T01:34:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.653+0000 7f90b7200640 1 --2- 172.21.15.1:0/3305451262 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb2be43c30 0x55eb2be4f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:34:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.653+0000 7f90b5e00640 1 --2- 172.21.15.1:0/3305451262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb2be3eba0 0x55eb2be48950 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f90a40ccab0 tx=0x7f90a4097b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3cf9624b3da21ec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.653+0000 7f9097e00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-12T01:34:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.653+0000 7f9097e00640 1 -- 172.21.15.1:0/3305451262 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a401c070 con 0x55eb2be3eba0 2024-09-12T01:34:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.653+0000 7f9097e00640 1 -- 172.21.15.1:0/3305451262 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f90a40860c0 con 0x55eb2be3eba0 2024-09-12T01:34:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.653+0000 7f90c01be740 1 -- 172.21.15.1:0/3305451262 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90b0004050 con 0x55eb2be3eba0 2024-09-12T01:34:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.653+0000 7f9097e00640 1 -- 172.21.15.1:0/3305451262 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f90a409b300 con 0x55eb2be3eba0 2024-09-12T01:34:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.653+0000 7f90c01be740 10 client.25577 fetch_fsmap learned FSMap version 35 2024-09-12T01:34:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.653+0000 7f90c01be740 1 -- 172.21.15.1:0/3305451262 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55eb2be52850 con 0x55eb2be3eba0 2024-09-12T01:34:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.654+0000 7f9097e00640 1 -- 172.21.15.1:0/3305451262 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90b0004050 con 0x55eb2be3eba0 2024-09-12T01:34:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.654+0000 7f9097e00640 1 -- 172.21.15.1:0/3305451262 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f90a40afce0 con 0x55eb2be3eba0 2024-09-12T01:34:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.654+0000 7f90c01be740 10 client.25577 fetch_fsmap finished waiting for FSMap version 35 2024-09-12T01:34:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.654+0000 7f90c01be740 10 client.25577 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:34:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.654+0000 7f90c01be740 20 client.25577 populate_metadata read hostname 'smithi001' 2024-09-12T01:34:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.654+0000 7f90c01be740 1 --2- 172.21.15.1:0/3305451262 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55eb2be53f20 0x55eb2be56330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.654+0000 7f90c01be740 4 client.25577 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:34:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.654+0000 7f90c01be740 1 -- 172.21.15.1:0/3305451262 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb2bdb0830 con 0x55eb2be53f20 2024-09-12T01:34:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.654+0000 7f90b6800640 1 --2- 172.21.15.1:0/3305451262 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55eb2be53f20 0x55eb2be56330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.655+0000 7f90b6800640 1 --2- 172.21.15.1:0/3305451262 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55eb2be53f20 0x55eb2be56330 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78445785bc8ca258 server_cookie=9157909c5252435e in_seq=0 out_seq=0 2024-09-12T01:34:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.655+0000 7f9097e00640 10 client.25577 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:34:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.656+0000 7f9097e00640 1 -- 172.21.15.1:0/3305451262 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eb2bdb0830 con 0x55eb2be53f20 2024-09-12T01:34:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.656+0000 7f9097e00640 10 client.25577 handle_command_reply: tid=0 2024-09-12T01:34:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.656+0000 7f90c01be740 1 client.25577 shutdown 2024-09-12T01:34:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.657+0000 7f90c01be740 1 -- 172.21.15.1:0/3305451262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb2be3eba0 msgr2=0x55eb2be48950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.657+0000 7f90c01be740 1 --2- 172.21.15.1:0/3305451262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb2be3eba0 0x55eb2be48950 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f90a40ccab0 tx=0x7f90a4097b20 comp rx=0 tx=0).stop 2024-09-12T01:34:29.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.657+0000 7f90c01be740 1 -- 172.21.15.1:0/3305451262 shutdown_connections 2024-09-12T01:34:29.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.657+0000 7f90c01be740 1 --2- 172.21.15.1:0/3305451262 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55eb2be53f20 0x55eb2be56330 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:29.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.657+0000 7f90c01be740 1 --2- 172.21.15.1:0/3305451262 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb2be43c30 0x55eb2be4f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:29.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.657+0000 7f90c01be740 1 --2- 172.21.15.1:0/3305451262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb2be3eba0 0x55eb2be48950 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:29.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.657+0000 7f90c01be740 1 --2- 172.21.15.1:0/3305451262 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb2be3e260 0x55eb2be48410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:29.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.657+0000 7f90c01be740 1 -- 172.21.15.1:0/3305451262 >> 172.21.15.1:0/3305451262 conn(0x55eb2be398d0 msgr2=0x55eb2be3b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:29.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.657+0000 7f9097e00640 0 client.25577 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:34:29.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.657+0000 7f90c01be740 1 -- 172.21.15.1:0/3305451262 shutdown_connections 2024-09-12T01:34:29.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.657+0000 7f90c01be740 1 -- 172.21.15.1:0/3305451262 wait complete. 2024-09-12T01:34:29.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.657+0000 7f90c01be740 20 client.25577 trim_cache size 0 max 16384 2024-09-12T01:34:29.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.680+0000 7f90c01be740 1 Processor -- start 2024-09-12T01:34:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.680+0000 7f90c01be740 1 -- start start 2024-09-12T01:34:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.680+0000 7f90c01be740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb2be3e260 0x55eb2be348d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.680+0000 7f90c01be740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb2be3eba0 0x55eb2be4f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.680+0000 7f90c01be740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb2be43c30 0x55eb2be3aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.680+0000 7f90c01be740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f90b81ac890 con 0x55eb2be3eba0 2024-09-12T01:34:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.680+0000 7f90c01be740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f90b81ae3d0 con 0x55eb2be43c30 2024-09-12T01:34:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.680+0000 7f90c01be740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f90b81acf60 con 0x55eb2be3e260 2024-09-12T01:34:29.682 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.681+0000 7f90b6800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb2be3e260 0x55eb2be348d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:29.682 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.681+0000 7f90b6800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb2be3e260 0x55eb2be348d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:52442/0 (socket says 172.21.15.1:52442) 2024-09-12T01:34:29.682 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.681+0000 7f90b6800640 1 -- 172.21.15.1:0/1575832170 learned_addr learned my addr 172.21.15.1:0/1575832170 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:34:29.682 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.682+0000 7f90b7200640 1 --2- 172.21.15.1:0/1575832170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb2be43c30 0x55eb2be3aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:29.682 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.682+0000 7f90b5e00640 1 --2- 172.21.15.1:0/1575832170 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb2be3eba0 0x55eb2be4f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:29.683 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.682+0000 7f90b6800640 1 -- 172.21.15.1:0/1575832170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb2be43c30 msgr2=0x55eb2be3aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:29.683 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.682+0000 7f90b6800640 1 --2- 172.21.15.1:0/1575832170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb2be43c30 0x55eb2be3aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:29.683 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.683+0000 7f90b6800640 1 -- 172.21.15.1:0/1575832170 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb2be3eba0 msgr2=0x55eb2be4f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:29.683 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.683+0000 7f90b6800640 1 --2- 172.21.15.1:0/1575832170 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb2be3eba0 0x55eb2be4f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:29.683 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.683+0000 7f90b6800640 1 -- 172.21.15.1:0/1575832170 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb2be569b0 con 0x55eb2be3e260 2024-09-12T01:34:29.686 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.687+0000 7f90b6800640 1 --2- 172.21.15.1:0/1575832170 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb2be3e260 0x55eb2be348d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f90a80314e0 tx=0x7f90a803e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db45b18b0c02e496 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:29.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.687+0000 7f90a2a00640 1 -- 172.21.15.1:0/1575832170 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a8042070 con 0x55eb2be3e260 2024-09-12T01:34:29.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.687+0000 7f90a2a00640 1 -- 172.21.15.1:0/1575832170 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90a8004300 con 0x55eb2be3e260 2024-09-12T01:34:29.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.687+0000 7f90a2a00640 1 -- 172.21.15.1:0/1575832170 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a8035040 con 0x55eb2be3e260 2024-09-12T01:34:29.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.688+0000 7f90c01be740 1 -- 172.21.15.1:0/1575832170 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb2be3e260 msgr2=0x55eb2be348d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:29.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.688+0000 7f90c01be740 1 --2- 172.21.15.1:0/1575832170 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb2be3e260 0x55eb2be348d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f90a80314e0 tx=0x7f90a803e000 comp rx=0 tx=0).stop 2024-09-12T01:34:29.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.689+0000 7f90c01be740 1 -- 172.21.15.1:0/1575832170 shutdown_connections 2024-09-12T01:34:29.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.689+0000 7f90c01be740 1 --2- 172.21.15.1:0/1575832170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb2be43c30 0x55eb2be3aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:29.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.689+0000 7f90c01be740 1 --2- 172.21.15.1:0/1575832170 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb2be3eba0 0x55eb2be4f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:29.689 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.689+0000 7f90c01be740 1 --2- 172.21.15.1:0/1575832170 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb2be3e260 0x55eb2be348d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:29.689 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.689+0000 7f90c01be740 1 -- 172.21.15.1:0/1575832170 >> 172.21.15.1:0/1575832170 conn(0x55eb2be398d0 msgr2=0x55eb2be3a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:29.689 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.690+0000 7f90c01be740 1 -- 172.21.15.1:0/1575832170 shutdown_connections 2024-09-12T01:34:29.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.690+0000 7f90c01be740 1 -- 172.21.15.1:0/1575832170 wait complete. 2024-09-12T01:34:29.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.690+0000 7f90c01be740 1 -- start start 2024-09-12T01:34:29.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.691+0000 7f90c01be740 1 Processor -- start 2024-09-12T01:34:29.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.691+0000 7f90c01be740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb2be3e260 0x55eb2be4d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:29.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.691+0000 7f90c01be740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb2be3eba0 0x55eb2be4d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:29.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.691+0000 7f90c01be740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb2be43c30 0x55eb2be4e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:29.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.691+0000 7f90c01be740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f90b81add00 con 0x55eb2be3e260 2024-09-12T01:34:29.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.691+0000 7f90c01be740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f90b81ad630 con 0x55eb2be3eba0 2024-09-12T01:34:29.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.691+0000 7f90c01be740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f90b81aa870 con 0x55eb2be43c30 2024-09-12T01:34:29.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.691+0000 7f90c01be740 10 client.0 _ll_register_callbacks cb 0x7f90a40d2ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:34:29.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.691+0000 7f90b7200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb2be43c30 0x55eb2be4e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:29.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.691+0000 7f90b7200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb2be43c30 0x55eb2be4e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:52448/0 (socket says 172.21.15.1:52448) 2024-09-12T01:34:29.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.691+0000 7f90b7200640 1 -- 172.21.15.1:0/2658202717 learned_addr learned my addr 172.21.15.1:0/2658202717 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:34:29.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.691+0000 7f90b5e00640 1 --2- 172.21.15.1:0/2658202717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb2be3eba0 0x55eb2be4d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:29.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.692+0000 7f90b6800640 1 --2- 172.21.15.1:0/2658202717 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb2be3e260 0x55eb2be4d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:29.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.692+0000 7f90b7200640 1 -- 172.21.15.1:0/2658202717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb2be3eba0 msgr2=0x55eb2be4d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:29.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.692+0000 7f90b7200640 1 --2- 172.21.15.1:0/2658202717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb2be3eba0 0x55eb2be4d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:29.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.692+0000 7f90b7200640 1 -- 172.21.15.1:0/2658202717 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb2be3e260 msgr2=0x55eb2be4d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:29.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.692+0000 7f90b7200640 1 --2- 172.21.15.1:0/2658202717 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb2be3e260 0x55eb2be4d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:29.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.692+0000 7f90b7200640 1 -- 172.21.15.1:0/2658202717 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb2be4eed0 con 0x55eb2be43c30 2024-09-12T01:34:29.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.692+0000 7f90b7200640 1 --2- 172.21.15.1:0/2658202717 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb2be43c30 0x55eb2be4e8c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f90b0006010 tx=0x7f90b0009d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9c92650c2c3a43d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:29.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.692+0000 7f9097e00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-12T01:34:29.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.692+0000 7f9097e00640 1 -- 172.21.15.1:0/2658202717 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b0005550 con 0x55eb2be43c30 2024-09-12T01:34:29.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.693+0000 7f9097e00640 1 -- 172.21.15.1:0/2658202717 <== mon.1 v2:172.21.15.104:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f90b00056f0 con 0x55eb2be43c30 2024-09-12T01:34:29.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.693+0000 7f90c01be740 1 -- 172.21.15.1:0/2658202717 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90a8004300 con 0x55eb2be43c30 2024-09-12T01:34:29.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.693+0000 7f9097e00640 1 -- 172.21.15.1:0/2658202717 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f90b0034050 con 0x55eb2be43c30 2024-09-12T01:34:29.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.693+0000 7f90c01be740 10 client.25582 fetch_fsmap learned FSMap version 35 2024-09-12T01:34:29.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.693+0000 7f90c01be740 1 -- 172.21.15.1:0/2658202717 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55eb2bebef10 con 0x55eb2be43c30 2024-09-12T01:34:29.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.693+0000 7f9097e00640 1 -- 172.21.15.1:0/2658202717 <== mon.1 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90a8004300 con 0x55eb2be43c30 2024-09-12T01:34:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.693+0000 7f9097e00640 1 -- 172.21.15.1:0/2658202717 <== mon.1 v2:172.21.15.104:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f90b0019550 con 0x55eb2be43c30 2024-09-12T01:34:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.693+0000 7f90c01be740 10 client.25582 fetch_fsmap finished waiting for FSMap version 35 2024-09-12T01:34:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.693+0000 7f90c01be740 10 client.25582 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:34:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.693+0000 7f90c01be740 20 client.25582 populate_metadata read hostname 'smithi001' 2024-09-12T01:34:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.693+0000 7f90c01be740 1 --2- 172.21.15.1:0/2658202717 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55eb2bebf680 0x55eb2bec1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.693+0000 7f90c01be740 4 client.25582 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:34:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.693+0000 7f90c01be740 1 -- 172.21.15.1:0/2658202717 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eb2bdaab80 con 0x55eb2bebf680 2024-09-12T01:34:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.694+0000 7f90b6800640 1 --2- 172.21.15.1:0/2658202717 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55eb2bebf680 0x55eb2bec1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.694+0000 7f90b6800640 1 --2- 172.21.15.1:0/2658202717 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55eb2bebf680 0x55eb2bec1aa0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=810a556116afa0f2 server_cookie=989753890b7fa8aa in_seq=0 out_seq=0 2024-09-12T01:34:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.694+0000 7f9097e00640 10 client.25582 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:34:29.705 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.704+0000 7f9097e00640 1 -- 172.21.15.1:0/2658202717 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eb2bdaab80 con 0x55eb2bebf680 2024-09-12T01:34:29.705 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.704+0000 7f9097e00640 10 client.25582 handle_command_reply: tid=0 2024-09-12T01:34:29.705 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.704+0000 7f90c01be740 1 client.25582 shutdown 2024-09-12T01:34:29.705 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.705+0000 7f90c01be740 1 -- 172.21.15.1:0/2658202717 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb2be43c30 msgr2=0x55eb2be4e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:29.705 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.705+0000 7f90c01be740 1 --2- 172.21.15.1:0/2658202717 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb2be43c30 0x55eb2be4e8c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f90b0006010 tx=0x7f90b0009d60 comp rx=0 tx=0).stop 2024-09-12T01:34:29.705 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.705+0000 7f90c01be740 1 -- 172.21.15.1:0/2658202717 shutdown_connections 2024-09-12T01:34:29.705 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.705+0000 7f90c01be740 1 --2- 172.21.15.1:0/2658202717 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55eb2bebf680 0x55eb2bec1aa0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:29.705 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.705+0000 7f90c01be740 1 --2- 172.21.15.1:0/2658202717 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb2be43c30 0x55eb2be4e8c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:29.705 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.705+0000 7f90c01be740 1 --2- 172.21.15.1:0/2658202717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb2be3eba0 0x55eb2be4d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:29.705 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.705+0000 7f9097e00640 0 client.25582 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:34:29.706 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.705+0000 7f90c01be740 1 --2- 172.21.15.1:0/2658202717 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb2be3e260 0x55eb2be4d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:29.706 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.705+0000 7f90c01be740 1 -- 172.21.15.1:0/2658202717 >> 172.21.15.1:0/2658202717 conn(0x55eb2be398d0 msgr2=0x55eb2be3b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:29.706 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.705+0000 7f90c01be740 1 -- 172.21.15.1:0/2658202717 shutdown_connections 2024-09-12T01:34:29.706 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.705+0000 7f90c01be740 1 -- 172.21.15.1:0/2658202717 wait complete. 2024-09-12T01:34:29.706 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.705+0000 7f90c01be740 20 client.25582 trim_cache size 0 max 16384 2024-09-12T01:34:29.706 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-12T01:34:29.706 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-12T01:34:29.706 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-12T01:34:29.706 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-12T01:34:29.707 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.706+0000 7f90bca00640 1 -- 172.21.15.1:0/3856311639 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f909c0a2b70 msgr2=0x7f909c0a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:29.707 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.706+0000 7f90bca00640 1 --2- 172.21.15.1:0/3856311639 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f909c0a2b70 0x7f909c0a5030 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f90a8031440 tx=0x7f90a80315b0 comp rx=0 tx=0).stop 2024-09-12T01:34:29.707 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.706+0000 7f90bca00640 1 -- 172.21.15.1:0/3856311639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90b8104df0 msgr2=0x7f90b8100470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:29.707 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.706+0000 7f90bca00640 1 --2- 172.21.15.1:0/3856311639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90b8104df0 0x7f90b8100470 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f90a400e9c0 tx=0x7f90a400ee80 comp rx=0 tx=0).stop 2024-09-12T01:34:29.707 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.706+0000 7f90bca00640 1 -- 172.21.15.1:0/3856311639 shutdown_connections 2024-09-12T01:34:29.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.706+0000 7f90bca00640 1 --2- 172.21.15.1:0/3856311639 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f909c0a2b70 0x7f909c0a5030 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:29.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.706+0000 7f90bca00640 1 --2- 172.21.15.1:0/3856311639 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90b8107720 0x7f90b81009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:29.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.706+0000 7f90bca00640 1 --2- 172.21.15.1:0/3856311639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90b8104df0 0x7f90b8100470 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:29.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.706+0000 7f90bca00640 1 --2- 172.21.15.1:0/3856311639 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90b81024c0 0x7f90b8101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:29.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.706+0000 7f90bca00640 1 -- 172.21.15.1:0/3856311639 >> 172.21.15.1:0/3856311639 conn(0x7f90b80fc040 msgr2=0x7f90b8102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:29.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.706+0000 7f90bca00640 1 -- 172.21.15.1:0/3856311639 shutdown_connections 2024-09-12T01:34:29.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:29.706+0000 7f90bca00640 1 -- 172.21.15.1:0/3856311639 wait complete. 2024-09-12T01:34:30.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:34:30 smithi053 ceph-mon[35158]: pgmap v545: 129 pgs: 129 active+clean; 110 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.2 MiB/s wr, 279 op/s 2024-09-12T01:34:31.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:34:30 smithi104 ceph-mon[35219]: pgmap v545: 129 pgs: 129 active+clean; 110 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.2 MiB/s wr, 279 op/s 2024-09-12T01:34:31.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:34:30 smithi001 ceph-mon[30387]: pgmap v545: 129 pgs: 129 active+clean; 110 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.2 MiB/s wr, 279 op/s 2024-09-12T01:34:32.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:34:32 smithi053 ceph-mon[35158]: pgmap v546: 129 pgs: 129 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.6 MiB/s wr, 438 op/s 2024-09-12T01:34:33.074 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:34:32 smithi104 ceph-mon[35219]: pgmap v546: 129 pgs: 129 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.6 MiB/s wr, 438 op/s 2024-09-12T01:34:33.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:34:32 smithi001 ceph-mon[30387]: pgmap v546: 129 pgs: 129 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.6 MiB/s wr, 438 op/s 2024-09-12T01:34:34.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:34:34 smithi053 ceph-mon[35158]: pgmap v547: 129 pgs: 129 active+clean; 114 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 996 KiB/s wr, 334 op/s 2024-09-12T01:34:35.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:34:34 smithi104 ceph-mon[35219]: pgmap v547: 129 pgs: 129 active+clean; 114 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 996 KiB/s wr, 334 op/s 2024-09-12T01:34:35.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:34:34 smithi001 ceph-mon[30387]: pgmap v547: 129 pgs: 129 active+clean; 114 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 996 KiB/s wr, 334 op/s 2024-09-12T01:34:36.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:34:36 smithi053 ceph-mon[35158]: pgmap v548: 129 pgs: 129 active+clean; 114 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.0 MiB/s wr, 390 op/s 2024-09-12T01:34:37.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:34:36 smithi104 ceph-mon[35219]: pgmap v548: 129 pgs: 129 active+clean; 114 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.0 MiB/s wr, 390 op/s 2024-09-12T01:34:37.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:34:36 smithi001 ceph-mon[30387]: pgmap v548: 129 pgs: 129 active+clean; 114 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.0 MiB/s wr, 390 op/s 2024-09-12T01:34:38.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:34:37 smithi104 ceph-mon[35219]: pgmap v549: 129 pgs: 129 active+clean; 114 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.1 MiB/s wr, 489 op/s 2024-09-12T01:34:38.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:34:37 smithi001 ceph-mon[30387]: pgmap v549: 129 pgs: 129 active+clean; 114 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.1 MiB/s wr, 489 op/s 2024-09-12T01:34:38.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:34:37 smithi053 ceph-mon[35158]: pgmap v549: 129 pgs: 129 active+clean; 114 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.1 MiB/s wr, 489 op/s 2024-09-12T01:34:40.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:34:39 smithi001 ceph-mon[30387]: pgmap v550: 129 pgs: 129 active+clean; 114 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 915 KiB/s wr, 408 op/s 2024-09-12T01:34:40.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:34:39 smithi053 ceph-mon[35158]: pgmap v550: 129 pgs: 129 active+clean; 114 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 915 KiB/s wr, 408 op/s 2024-09-12T01:34:40.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:34:39 smithi104 ceph-mon[35219]: pgmap v550: 129 pgs: 129 active+clean; 114 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 915 KiB/s wr, 408 op/s 2024-09-12T01:34:42.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:34:41 smithi001 ceph-mon[30387]: pgmap v551: 129 pgs: 129 active+clean; 117 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.3 MiB/s wr, 573 op/s 2024-09-12T01:34:42.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:34:41 smithi053 ceph-mon[35158]: pgmap v551: 129 pgs: 129 active+clean; 117 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.3 MiB/s wr, 573 op/s 2024-09-12T01:34:42.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:34:41 smithi104 ceph-mon[35219]: pgmap v551: 129 pgs: 129 active+clean; 117 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.3 MiB/s wr, 573 op/s 2024-09-12T01:34:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.860+0000 7f3ab9200640 1 -- 172.21.15.1:0/360932646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ab4103710 msgr2=0x7f3ab4109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.860+0000 7f3ab9200640 1 --2- 172.21.15.1:0/360932646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ab4103710 0x7f3ab4109fa0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009ca0 tx=0x7f3aac02a8c0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.860+0000 7f3ab9200640 1 -- 172.21.15.1:0/360932646 shutdown_connections 2024-09-12T01:34:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.860+0000 7f3ab9200640 1 --2- 172.21.15.1:0/360932646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ab4103710 0x7f3ab4109fa0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.860+0000 7f3ab9200640 1 --2- 172.21.15.1:0/360932646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ab4102d50 0x7f3ab41031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.860+0000 7f3ab9200640 1 --2- 172.21.15.1:0/360932646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ab4101b60 0x7f3ab4101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.860+0000 7f3ab9200640 1 -- 172.21.15.1:0/360932646 >> 172.21.15.1:0/360932646 conn(0x7f3ab40fd2e0 msgr2=0x7f3ab40ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.860+0000 7f3ab9200640 1 -- 172.21.15.1:0/360932646 shutdown_connections 2024-09-12T01:34:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.860+0000 7f3ab9200640 1 -- 172.21.15.1:0/360932646 wait complete. 2024-09-12T01:34:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.862+0000 7f3ab9200640 1 Processor -- start 2024-09-12T01:34:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.862+0000 7f3ab9200640 1 -- start start 2024-09-12T01:34:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.862+0000 7f3ab9200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ab4101b60 0x7f3ab419a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.862+0000 7f3ab9200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ab4102d50 0x7f3ab419a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.862+0000 7f3ab9200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ab4103710 0x7f3ab41a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.862+0000 7f3ab9200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab410d7c0 con 0x7f3ab4101b60 2024-09-12T01:34:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.862+0000 7f3ab9200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab410d640 con 0x7f3ab4103710 2024-09-12T01:34:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.862+0000 7f3ab9200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab410d940 con 0x7f3ab4102d50 2024-09-12T01:34:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.862+0000 7f3ab1400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ab4102d50 0x7f3ab419a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.862+0000 7f3ab2800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ab4103710 0x7f3ab41a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.862+0000 7f3ab1400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ab4102d50 0x7f3ab419a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:53016/0 (socket says 172.21.15.1:53016) 2024-09-12T01:34:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.862+0000 7f3ab1e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ab4101b60 0x7f3ab419a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.862+0000 7f3ab1400640 1 -- 172.21.15.1:0/963504373 learned_addr learned my addr 172.21.15.1:0/963504373 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:34:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.863+0000 7f3ab1400640 1 -- 172.21.15.1:0/963504373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ab4103710 msgr2=0x7f3ab41a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.863+0000 7f3ab1400640 1 --2- 172.21.15.1:0/963504373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ab4103710 0x7f3ab41a15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.863+0000 7f3ab1400640 1 -- 172.21.15.1:0/963504373 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ab4101b60 msgr2=0x7f3ab419a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.863+0000 7f3ab1400640 1 --2- 172.21.15.1:0/963504373 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ab4101b60 0x7f3ab419a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.863+0000 7f3ab1400640 1 -- 172.21.15.1:0/963504373 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ab41a1c40 con 0x7f3ab4102d50 2024-09-12T01:34:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.863+0000 7f3ab1400640 1 --2- 172.21.15.1:0/963504373 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ab4102d50 0x7f3ab419a560 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000e9f0 tx=0x7f3aa000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.863+0000 7f3aaaa00640 1 -- 172.21.15.1:0/963504373 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa000cd80 con 0x7f3ab4102d50 2024-09-12T01:34:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.863+0000 7f3aaaa00640 1 -- 172.21.15.1:0/963504373 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aa0010910 con 0x7f3ab4102d50 2024-09-12T01:34:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.864+0000 7f3ab9200640 1 -- 172.21.15.1:0/963504373 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ab41a1f30 con 0x7f3ab4102d50 2024-09-12T01:34:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.864+0000 7f3aaaa00640 1 -- 172.21.15.1:0/963504373 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0018990 con 0x7f3ab4102d50 2024-09-12T01:34:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.864+0000 7f3ab9200640 1 -- 172.21.15.1:0/963504373 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ab41a23f0 con 0x7f3ab4102d50 2024-09-12T01:34:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.867+0000 7f3abc0b6740 1 Processor -- start 2024-09-12T01:34:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.867+0000 7f3abc0b6740 1 -- start start 2024-09-12T01:34:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.867+0000 7f3aaaa00640 1 -- 172.21.15.1:0/963504373 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f3aa0010ac0 con 0x7f3ab4102d50 2024-09-12T01:34:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.867+0000 7f3abc0b6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacc98d090 0x55dacc98d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.867+0000 7f3abc0b6740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dacc98d9d0 0x55dacc992490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.867+0000 7f3abc0b6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dacc992a60 0x55dacc994f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.867+0000 7f3abc0b6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55dacc8f2100 con 0x55dacc992a60 2024-09-12T01:34:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.867+0000 7f3abc0b6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55dacc89a1b0 con 0x55dacc98d090 2024-09-12T01:34:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.867+0000 7f3abc0b6740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55dacc8ee0c0 con 0x55dacc98d9d0 2024-09-12T01:34:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.868+0000 7f3ab2800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacc98d090 0x55dacc98d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.868+0000 7f3ab1e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dacc98d9d0 0x55dacc992490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.868+0000 7f3ab2800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacc98d090 0x55dacc98d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:35330/0 (socket says 172.21.15.1:35330) 2024-09-12T01:34:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.868+0000 7f3ab2800640 1 -- 172.21.15.1:0/3549722446 learned_addr learned my addr 172.21.15.1:0/3549722446 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:34:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.868+0000 7f3ab1400640 1 --2- 172.21.15.1:0/3549722446 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dacc992a60 0x55dacc994f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:42.870 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.868+0000 7f3ab2800640 1 -- 172.21.15.1:0/3549722446 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dacc98d9d0 msgr2=0x55dacc992490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:42.870 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.868+0000 7f3ab2800640 1 --2- 172.21.15.1:0/3549722446 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dacc98d9d0 0x55dacc992490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.870 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.868+0000 7f3ab2800640 1 -- 172.21.15.1:0/3549722446 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dacc992a60 msgr2=0x55dacc994f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:42.870 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.868+0000 7f3ab2800640 1 --2- 172.21.15.1:0/3549722446 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dacc992a60 0x55dacc994f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.870 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.868+0000 7f3ab2800640 1 -- 172.21.15.1:0/3549722446 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dacc605210 con 0x55dacc98d090 2024-09-12T01:34:42.870 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.868+0000 7f3ab1400640 1 --2- 172.21.15.1:0/3549722446 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dacc992a60 0x55dacc994f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:34:42.870 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.868+0000 7f3ab2800640 1 --2- 172.21.15.1:0/3549722446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacc98d090 0x55dacc98d490 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009ca0 tx=0x7f3aac004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c1fa4003212bcae server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:42.870 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.869+0000 7f3a9f400640 1 -- 172.21.15.1:0/3549722446 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac0098c0 con 0x55dacc98d090 2024-09-12T01:34:42.870 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.869+0000 7f3a9f400640 1 -- 172.21.15.1:0/3549722446 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aac036040 con 0x55dacc98d090 2024-09-12T01:34:42.871 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.869+0000 7f3a9f400640 1 -- 172.21.15.1:0/3549722446 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac004290 con 0x55dacc98d090 2024-09-12T01:34:42.871 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.869+0000 7f3aaaa00640 1 --2- 172.21.15.1:0/963504373 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f3a980a2b80 0x7f3a980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:42.871 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.869+0000 7f3aaaa00640 1 -- 172.21.15.1:0/963504373 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f3aa00cf6f0 con 0x7f3ab4102d50 2024-09-12T01:34:42.871 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.869+0000 7f3ab1e00640 1 --2- 172.21.15.1:0/963504373 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f3a980a2b80 0x7f3a980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:42.871 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.870+0000 7f3abc0b6740 1 -- 172.21.15.1:0/3549722446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacc98d090 msgr2=0x55dacc98d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:42.871 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.870+0000 7f3abc0b6740 1 --2- 172.21.15.1:0/3549722446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacc98d090 0x55dacc98d490 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009ca0 tx=0x7f3aac004a40 comp rx=0 tx=0).stop 2024-09-12T01:34:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.870+0000 7f3abc0b6740 1 -- 172.21.15.1:0/3549722446 shutdown_connections 2024-09-12T01:34:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.870+0000 7f3abc0b6740 1 --2- 172.21.15.1:0/3549722446 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dacc992a60 0x55dacc994f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.870+0000 7f3abc0b6740 1 --2- 172.21.15.1:0/3549722446 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dacc98d9d0 0x55dacc992490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.870+0000 7f3abc0b6740 1 --2- 172.21.15.1:0/3549722446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacc98d090 0x55dacc98d490 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.870+0000 7f3abc0b6740 1 -- 172.21.15.1:0/3549722446 >> 172.21.15.1:0/3549722446 conn(0x55dacc988700 msgr2=0x55dacc988b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.870+0000 7f3abc0b6740 1 -- 172.21.15.1:0/3549722446 shutdown_connections 2024-09-12T01:34:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.870+0000 7f3abc0b6740 1 -- 172.21.15.1:0/3549722446 wait complete. 2024-09-12T01:34:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.871+0000 7f3abc0b6740 1 -- start start 2024-09-12T01:34:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.871+0000 7f3abc0b6740 1 Processor -- start 2024-09-12T01:34:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.871+0000 7f3ab1e00640 1 --2- 172.21.15.1:0/963504373 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f3a980a2b80 0x7f3a980a5040 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x55dacc98e400 tx=0x7f3aa4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.872+0000 7f3abc0b6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dacc98d090 0x55dacc997000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.872+0000 7f3abc0b6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacc98d9d0 0x55dacc997540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.872+0000 7f3abc0b6740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dacc992a60 0x55dacc99e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.872+0000 7f3abc0b6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55dacc9957f0 con 0x55dacc98d090 2024-09-12T01:34:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.872+0000 7f3abc0b6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55dacc87b090 con 0x55dacc98d9d0 2024-09-12T01:34:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.872+0000 7f3abc0b6740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55dacc995970 con 0x55dacc992a60 2024-09-12T01:34:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.872+0000 7f3abc0b6740 10 client.0 _ll_register_callbacks cb 0x55dacc8cff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:34:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.872+0000 7f3ab2800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dacc992a60 0x55dacc99e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.872+0000 7f3ab1e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dacc98d090 0x55dacc997000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.872+0000 7f3ab2800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dacc992a60 0x55dacc99e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:53032/0 (socket says 172.21.15.1:53032) 2024-09-12T01:34:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.872+0000 7f3ab2800640 1 -- 172.21.15.1:0/908570884 learned_addr learned my addr 172.21.15.1:0/908570884 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:34:42.875 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.872+0000 7f3ab1400640 1 --2- 172.21.15.1:0/908570884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacc98d9d0 0x55dacc997540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:42.875 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.872+0000 7f3ab1400640 1 -- 172.21.15.1:0/908570884 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dacc992a60 msgr2=0x55dacc99e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:42.875 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.872+0000 7f3ab1400640 1 --2- 172.21.15.1:0/908570884 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dacc992a60 0x55dacc99e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.875 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.872+0000 7f3ab1400640 1 -- 172.21.15.1:0/908570884 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dacc98d090 msgr2=0x55dacc997000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:42.875 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.872+0000 7f3ab1400640 1 --2- 172.21.15.1:0/908570884 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dacc98d090 0x55dacc997000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.875 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.872+0000 7f3ab1400640 1 -- 172.21.15.1:0/908570884 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dacc6d1620 con 0x55dacc98d9d0 2024-09-12T01:34:42.875 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.873+0000 7f3ab1400640 1 --2- 172.21.15.1:0/908570884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacc98d9d0 0x55dacc997540 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3aa00cceb0 tx=0x7f3aa0011e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f38b3ebca75e63f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:42.875 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.873+0000 7f3a93e00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-12T01:34:42.875 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.873+0000 7f3a93e00640 1 -- 172.21.15.1:0/908570884 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa00a4830 con 0x55dacc98d9d0 2024-09-12T01:34:42.875 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.873+0000 7f3a93e00640 1 -- 172.21.15.1:0/908570884 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f3aa00cfba0 con 0x55dacc98d9d0 2024-09-12T01:34:42.875 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.873+0000 7f3abc0b6740 1 -- 172.21.15.1:0/908570884 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3aac036040 con 0x55dacc98d9d0 2024-09-12T01:34:42.876 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.874+0000 7f3a93e00640 1 -- 172.21.15.1:0/908570884 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3aa00cfda0 con 0x55dacc98d9d0 2024-09-12T01:34:42.876 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.874+0000 7f3abc0b6740 10 client.25595 fetch_fsmap learned FSMap version 35 2024-09-12T01:34:42.876 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.874+0000 7f3abc0b6740 1 -- 172.21.15.1:0/908570884 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55dacc9a13a0 con 0x55dacc98d9d0 2024-09-12T01:34:42.876 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.874+0000 7f3a93e00640 1 -- 172.21.15.1:0/908570884 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aac036040 con 0x55dacc98d9d0 2024-09-12T01:34:42.876 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.874+0000 7f3a93e00640 1 -- 172.21.15.1:0/908570884 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f3aa00d2ce0 con 0x55dacc98d9d0 2024-09-12T01:34:42.876 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.875+0000 7f3abc0b6740 10 client.25595 fetch_fsmap finished waiting for FSMap version 35 2024-09-12T01:34:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.875+0000 7f3abc0b6740 10 client.25595 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:34:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.875+0000 7f3abc0b6740 20 client.25595 populate_metadata read hostname 'smithi001' 2024-09-12T01:34:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.875+0000 7f3abc0b6740 1 --2- 172.21.15.1:0/908570884 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55dacc9a2a70 0x55dacc9a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.875+0000 7f3abc0b6740 4 client.25595 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:34:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.875+0000 7f3abc0b6740 1 -- 172.21.15.1:0/908570884 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dacc8ff830 con 0x55dacc9a2a70 2024-09-12T01:34:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.875+0000 7f3ab1e00640 1 --2- 172.21.15.1:0/908570884 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55dacc9a2a70 0x55dacc9a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.876+0000 7f3ab1e00640 1 --2- 172.21.15.1:0/908570884 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55dacc9a2a70 0x55dacc9a4e80 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f454da910c632dac server_cookie=a1242d251791ae2d in_seq=0 out_seq=0 2024-09-12T01:34:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.876+0000 7f3a93e00640 10 client.25595 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:34:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.877+0000 7f3a93e00640 1 -- 172.21.15.1:0/908570884 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dacc8ff830 con 0x55dacc9a2a70 2024-09-12T01:34:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.877+0000 7f3a93e00640 10 client.25595 handle_command_reply: tid=0 2024-09-12T01:34:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.877+0000 7f3abc0b6740 1 client.25595 shutdown 2024-09-12T01:34:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.878+0000 7f3abc0b6740 1 -- 172.21.15.1:0/908570884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacc98d9d0 msgr2=0x55dacc997540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.878+0000 7f3abc0b6740 1 --2- 172.21.15.1:0/908570884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacc98d9d0 0x55dacc997540 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3aa00cceb0 tx=0x7f3aa0011e20 comp rx=0 tx=0).stop 2024-09-12T01:34:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.878+0000 7f3abc0b6740 1 -- 172.21.15.1:0/908570884 shutdown_connections 2024-09-12T01:34:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.878+0000 7f3abc0b6740 1 --2- 172.21.15.1:0/908570884 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55dacc9a2a70 0x55dacc9a4e80 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.878+0000 7f3abc0b6740 1 --2- 172.21.15.1:0/908570884 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dacc992a60 0x55dacc99e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.879+0000 7f3abc0b6740 1 --2- 172.21.15.1:0/908570884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacc98d9d0 0x55dacc997540 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.879+0000 7f3abc0b6740 1 --2- 172.21.15.1:0/908570884 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dacc98d090 0x55dacc997000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.879+0000 7f3abc0b6740 1 -- 172.21.15.1:0/908570884 >> 172.21.15.1:0/908570884 conn(0x55dacc988700 msgr2=0x55dacc989f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.879+0000 7f3a93e00640 0 client.25595 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:34:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.879+0000 7f3abc0b6740 1 -- 172.21.15.1:0/908570884 shutdown_connections 2024-09-12T01:34:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.879+0000 7f3abc0b6740 1 -- 172.21.15.1:0/908570884 wait complete. 2024-09-12T01:34:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.879+0000 7f3abc0b6740 20 client.25595 trim_cache size 0 max 16384 2024-09-12T01:34:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.896+0000 7f3abc0b6740 1 Processor -- start 2024-09-12T01:34:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.897+0000 7f3abc0b6740 1 -- start start 2024-09-12T01:34:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.897+0000 7f3abc0b6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dacc98d090 0x55dacc9a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.897+0000 7f3abc0b6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacc98d9d0 0x55dacc99e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.897+0000 7f3abc0b6740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dacc992a60 0x55dacc9895b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.897+0000 7f3abc0b6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab41ac900 con 0x55dacc98d090 2024-09-12T01:34:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.897+0000 7f3abc0b6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab41ae440 con 0x55dacc98d9d0 2024-09-12T01:34:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.897+0000 7f3abc0b6740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab41acfd0 con 0x55dacc992a60 2024-09-12T01:34:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.898+0000 7f3ab1e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dacc98d090 0x55dacc9a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.898+0000 7f3ab1e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dacc98d090 0x55dacc9a4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44282/0 (socket says 172.21.15.1:44282) 2024-09-12T01:34:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.898+0000 7f3ab1e00640 1 -- 172.21.15.1:0/4216308520 learned_addr learned my addr 172.21.15.1:0/4216308520 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:34:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.898+0000 7f3ab1400640 1 --2- 172.21.15.1:0/4216308520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacc98d9d0 0x55dacc99e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.898+0000 7f3ab2800640 1 --2- 172.21.15.1:0/4216308520 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dacc992a60 0x55dacc9895b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.898+0000 7f3ab1e00640 1 -- 172.21.15.1:0/4216308520 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dacc992a60 msgr2=0x55dacc9895b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.898+0000 7f3ab1e00640 1 --2- 172.21.15.1:0/4216308520 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dacc992a60 0x55dacc9895b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.898+0000 7f3ab1e00640 1 -- 172.21.15.1:0/4216308520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacc98d9d0 msgr2=0x55dacc99e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.898+0000 7f3ab1e00640 1 --2- 172.21.15.1:0/4216308520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacc98d9d0 0x55dacc99e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.898+0000 7f3ab1e00640 1 -- 172.21.15.1:0/4216308520 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dacc9a54d0 con 0x55dacc98d090 2024-09-12T01:34:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.898+0000 7f3ab2800640 1 --2- 172.21.15.1:0/4216308520 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dacc992a60 0x55dacc9895b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:34:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.899+0000 7f3ab1e00640 1 --2- 172.21.15.1:0/4216308520 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dacc98d090 0x55dacc9a4e60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4013010 tx=0x7f3aa401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=655ee1905adb93d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.899+0000 7f3a9ea00640 1 -- 172.21.15.1:0/4216308520 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa4003bc0 con 0x55dacc98d090 2024-09-12T01:34:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.899+0000 7f3a9ea00640 1 -- 172.21.15.1:0/4216308520 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aa4003d60 con 0x55dacc98d090 2024-09-12T01:34:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.899+0000 7f3a9ea00640 1 -- 172.21.15.1:0/4216308520 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa401f840 con 0x55dacc98d090 2024-09-12T01:34:42.901 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.900+0000 7f3abc0b6740 1 -- 172.21.15.1:0/4216308520 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dacc98d090 msgr2=0x55dacc9a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.900+0000 7f3abc0b6740 1 --2- 172.21.15.1:0/4216308520 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dacc98d090 0x55dacc9a4e60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4013010 tx=0x7f3aa401a000 comp rx=0 tx=0).stop 2024-09-12T01:34:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.900+0000 7f3abc0b6740 1 -- 172.21.15.1:0/4216308520 shutdown_connections 2024-09-12T01:34:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.900+0000 7f3abc0b6740 1 --2- 172.21.15.1:0/4216308520 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dacc992a60 0x55dacc9895b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.900+0000 7f3abc0b6740 1 --2- 172.21.15.1:0/4216308520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacc98d9d0 0x55dacc99e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.900+0000 7f3abc0b6740 1 --2- 172.21.15.1:0/4216308520 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dacc98d090 0x55dacc9a4e60 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.900+0000 7f3abc0b6740 1 -- 172.21.15.1:0/4216308520 >> 172.21.15.1:0/4216308520 conn(0x55dacc988700 msgr2=0x55dacc992220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.900+0000 7f3abc0b6740 1 -- 172.21.15.1:0/4216308520 shutdown_connections 2024-09-12T01:34:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.900+0000 7f3abc0b6740 1 -- 172.21.15.1:0/4216308520 wait complete. 2024-09-12T01:34:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.901+0000 7f3abc0b6740 1 -- start start 2024-09-12T01:34:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.901+0000 7f3abc0b6740 1 Processor -- start 2024-09-12T01:34:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.902+0000 7f3abc0b6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacc98d090 0x55dacc99bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.902+0000 7f3abc0b6740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dacc98d9d0 0x55dacc99c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.902+0000 7f3abc0b6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dacc992a60 0x55dacc99d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:42.903 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.902+0000 7f3abc0b6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab41add70 con 0x55dacc992a60 2024-09-12T01:34:42.903 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.902+0000 7f3abc0b6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab41ad6a0 con 0x55dacc98d090 2024-09-12T01:34:42.903 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.902+0000 7f3abc0b6740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab41aa8e0 con 0x55dacc98d9d0 2024-09-12T01:34:42.903 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.902+0000 7f3abc0b6740 10 client.0 _ll_register_callbacks cb 0x7f3aa000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:34:42.903 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.902+0000 7f3ab1400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dacc98d9d0 0x55dacc99c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:42.903 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.902+0000 7f3ab1e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacc98d090 0x55dacc99bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:42.903 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.902+0000 7f3ab1400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dacc98d9d0 0x55dacc99c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:53050/0 (socket says 172.21.15.1:53050) 2024-09-12T01:34:42.903 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.902+0000 7f3ab2800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dacc992a60 0x55dacc99d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:42.903 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.902+0000 7f3ab1e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacc98d090 0x55dacc99bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:35336/0 (socket says 172.21.15.1:35336) 2024-09-12T01:34:42.903 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.902+0000 7f3ab1400640 1 -- 172.21.15.1:0/894652756 learned_addr learned my addr 172.21.15.1:0/894652756 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:34:42.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.902+0000 7f3ab1400640 1 -- 172.21.15.1:0/894652756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacc98d090 msgr2=0x55dacc99bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:42.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.902+0000 7f3ab1400640 1 --2- 172.21.15.1:0/894652756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacc98d090 0x55dacc99bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.902+0000 7f3ab1400640 1 -- 172.21.15.1:0/894652756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dacc992a60 msgr2=0x55dacc99d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:42.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.902+0000 7f3ab1400640 1 --2- 172.21.15.1:0/894652756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dacc992a60 0x55dacc99d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.902+0000 7f3ab1400640 1 -- 172.21.15.1:0/894652756 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dacc99d7b0 con 0x55dacc98d9d0 2024-09-12T01:34:42.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.903+0000 7f3ab1400640 1 --2- 172.21.15.1:0/894652756 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dacc98d9d0 0x55dacc99c190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0098640 tx=0x7f3aa009bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e56a5c80be183c1a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:42.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.903+0000 7f3a93e00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-12T01:34:42.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.903+0000 7f3a93e00640 1 -- 172.21.15.1:0/894652756 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa00a8070 con 0x55dacc98d9d0 2024-09-12T01:34:42.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.903+0000 7f3abc0b6740 1 -- 172.21.15.1:0/894652756 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3aa4003d60 con 0x55dacc98d9d0 2024-09-12T01:34:42.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.903+0000 7f3a93e00640 1 -- 172.21.15.1:0/894652756 <== mon.1 v2:172.21.15.104:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f3aa0010430 con 0x55dacc98d9d0 2024-09-12T01:34:42.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.903+0000 7f3a93e00640 1 -- 172.21.15.1:0/894652756 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3aa00b2020 con 0x55dacc98d9d0 2024-09-12T01:34:42.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.903+0000 7f3abc0b6740 10 client.25600 fetch_fsmap learned FSMap version 35 2024-09-12T01:34:42.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.903+0000 7f3abc0b6740 1 -- 172.21.15.1:0/894652756 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55dacca0d800 con 0x55dacc98d9d0 2024-09-12T01:34:42.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.903+0000 7f3a93e00640 1 -- 172.21.15.1:0/894652756 <== mon.1 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aa4003d60 con 0x55dacc98d9d0 2024-09-12T01:34:42.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.904+0000 7f3a93e00640 1 -- 172.21.15.1:0/894652756 <== mon.1 v2:172.21.15.104:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f3aa00ab030 con 0x55dacc98d9d0 2024-09-12T01:34:42.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.904+0000 7f3abc0b6740 10 client.25600 fetch_fsmap finished waiting for FSMap version 35 2024-09-12T01:34:42.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.904+0000 7f3abc0b6740 10 client.25600 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:34:42.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.904+0000 7f3abc0b6740 20 client.25600 populate_metadata read hostname 'smithi001' 2024-09-12T01:34:42.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.904+0000 7f3abc0b6740 1 --2- 172.21.15.1:0/894652756 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55dacca0e230 0x55dacca10650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:42.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.904+0000 7f3abc0b6740 4 client.25600 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:34:42.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.904+0000 7f3abc0b6740 1 -- 172.21.15.1:0/894652756 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dacc8f9b80 con 0x55dacca0e230 2024-09-12T01:34:42.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.904+0000 7f3ab1e00640 1 --2- 172.21.15.1:0/894652756 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55dacca0e230 0x55dacca10650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:42.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.905+0000 7f3ab1e00640 1 --2- 172.21.15.1:0/894652756 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55dacca0e230 0x55dacca10650 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2def9911e297279f server_cookie=81330f2a7988788 in_seq=0 out_seq=0 2024-09-12T01:34:42.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.905+0000 7f3a93e00640 10 client.25600 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:34:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.920+0000 7f3a93e00640 1 -- 172.21.15.1:0/894652756 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dacc8f9b80 con 0x55dacca0e230 2024-09-12T01:34:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.920+0000 7f3a93e00640 10 client.25600 handle_command_reply: tid=0 2024-09-12T01:34:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.920+0000 7f3abc0b6740 1 client.25600 shutdown 2024-09-12T01:34:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.921+0000 7f3abc0b6740 1 -- 172.21.15.1:0/894652756 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dacc98d9d0 msgr2=0x55dacc99c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.921+0000 7f3abc0b6740 1 --2- 172.21.15.1:0/894652756 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dacc98d9d0 0x55dacc99c190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0098640 tx=0x7f3aa009bce0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.921+0000 7f3abc0b6740 1 -- 172.21.15.1:0/894652756 shutdown_connections 2024-09-12T01:34:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.921+0000 7f3abc0b6740 1 --2- 172.21.15.1:0/894652756 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55dacca0e230 0x55dacca10650 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.921+0000 7f3abc0b6740 1 --2- 172.21.15.1:0/894652756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dacc992a60 0x55dacc99d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.921+0000 7f3abc0b6740 1 --2- 172.21.15.1:0/894652756 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dacc98d9d0 0x55dacc99c190 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.921+0000 7f3abc0b6740 1 --2- 172.21.15.1:0/894652756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dacc98d090 0x55dacc99bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.921+0000 7f3abc0b6740 1 -- 172.21.15.1:0/894652756 >> 172.21.15.1:0/894652756 conn(0x55dacc988700 msgr2=0x55dacc987060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.921+0000 7f3a93e00640 0 client.25600 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:34:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.922+0000 7f3abc0b6740 1 -- 172.21.15.1:0/894652756 shutdown_connections 2024-09-12T01:34:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.922+0000 7f3abc0b6740 1 -- 172.21.15.1:0/894652756 wait complete. 2024-09-12T01:34:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.922+0000 7f3abc0b6740 20 client.25600 trim_cache size 0 max 16384 2024-09-12T01:34:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-12T01:34:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-12T01:34:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-12T01:34:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-12T01:34:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.923+0000 7f3ab9200640 1 -- 172.21.15.1:0/963504373 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f3a980a2b80 msgr2=0x7f3a980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.923+0000 7f3ab9200640 1 --2- 172.21.15.1:0/963504373 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f3a980a2b80 0x7f3a980a5040 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x55dacc98e400 tx=0x7f3aa4005f90 comp rx=0 tx=0).stop 2024-09-12T01:34:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.923+0000 7f3ab9200640 1 -- 172.21.15.1:0/963504373 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ab4102d50 msgr2=0x7f3ab419a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.923+0000 7f3ab9200640 1 --2- 172.21.15.1:0/963504373 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ab4102d50 0x7f3ab419a560 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000e9f0 tx=0x7f3aa000eeb0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.923+0000 7f3ab9200640 1 -- 172.21.15.1:0/963504373 shutdown_connections 2024-09-12T01:34:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.923+0000 7f3ab9200640 1 --2- 172.21.15.1:0/963504373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ab4103710 0x7f3ab41a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.923+0000 7f3ab9200640 1 --2- 172.21.15.1:0/963504373 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f3a980a2b80 0x7f3a980a5040 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.923+0000 7f3ab9200640 1 --2- 172.21.15.1:0/963504373 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ab4102d50 0x7f3ab419a560 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.923+0000 7f3ab9200640 1 --2- 172.21.15.1:0/963504373 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ab4101b60 0x7f3ab419a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.923+0000 7f3ab9200640 1 -- 172.21.15.1:0/963504373 >> 172.21.15.1:0/963504373 conn(0x7f3ab40fd2e0 msgr2=0x7f3ab4074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.923+0000 7f3ab9200640 1 -- 172.21.15.1:0/963504373 shutdown_connections 2024-09-12T01:34:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:42.923+0000 7f3ab9200640 1 -- 172.21.15.1:0/963504373 wait complete. 2024-09-12T01:34:44.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:34:43 smithi001 ceph-mon[30387]: pgmap v552: 129 pgs: 129 active+clean; 109 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.0 MiB/s wr, 448 op/s 2024-09-12T01:34:44.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:34:43 smithi053 ceph-mon[35158]: pgmap v552: 129 pgs: 129 active+clean; 109 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.0 MiB/s wr, 448 op/s 2024-09-12T01:34:44.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:34:43 smithi104 ceph-mon[35219]: pgmap v552: 129 pgs: 129 active+clean; 109 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.0 MiB/s wr, 448 op/s 2024-09-12T01:34:46.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:34:45 smithi001 ceph-mon[30387]: pgmap v553: 129 pgs: 129 active+clean; 112 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.1 MiB/s wr, 486 op/s 2024-09-12T01:34:46.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:34:45 smithi053 ceph-mon[35158]: pgmap v553: 129 pgs: 129 active+clean; 112 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.1 MiB/s wr, 486 op/s 2024-09-12T01:34:46.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:34:45 smithi104 ceph-mon[35219]: pgmap v553: 129 pgs: 129 active+clean; 112 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.1 MiB/s wr, 486 op/s 2024-09-12T01:34:46.645 INFO:tasks.workunit.client.0.smithi001.stdout:10000 mapread 0x33cc4e3 thru 0x33cec16 (0x2734 bytes) 2024-09-12T01:34:48.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:34:47 smithi001 ceph-mon[30387]: pgmap v554: 129 pgs: 129 active+clean; 107 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.3 MiB/s wr, 530 op/s 2024-09-12T01:34:48.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:34:47 smithi053 ceph-mon[35158]: pgmap v554: 129 pgs: 129 active+clean; 107 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.3 MiB/s wr, 530 op/s 2024-09-12T01:34:48.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:34:47 smithi104 ceph-mon[35219]: pgmap v554: 129 pgs: 129 active+clean; 107 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.3 MiB/s wr, 530 op/s 2024-09-12T01:34:50.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:34:49 smithi001 ceph-mon[30387]: pgmap v555: 129 pgs: 129 active+clean; 107 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 1.0 MiB/s wr, 394 op/s 2024-09-12T01:34:50.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:34:49 smithi053 ceph-mon[35158]: pgmap v555: 129 pgs: 129 active+clean; 107 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 1.0 MiB/s wr, 394 op/s 2024-09-12T01:34:50.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:34:49 smithi104 ceph-mon[35219]: pgmap v555: 129 pgs: 129 active+clean; 107 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 1.0 MiB/s wr, 394 op/s 2024-09-12T01:34:52.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:34:51 smithi001 ceph-mon[30387]: pgmap v556: 129 pgs: 129 active+clean; 114 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.4 MiB/s wr, 555 op/s 2024-09-12T01:34:52.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:34:51 smithi053 ceph-mon[35158]: pgmap v556: 129 pgs: 129 active+clean; 114 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.4 MiB/s wr, 555 op/s 2024-09-12T01:34:52.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:34:51 smithi104 ceph-mon[35219]: pgmap v556: 129 pgs: 129 active+clean; 114 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.4 MiB/s wr, 555 op/s 2024-09-12T01:34:54.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:34:53 smithi001 ceph-mon[30387]: pgmap v557: 129 pgs: 129 active+clean; 115 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 1.1 MiB/s wr, 424 op/s 2024-09-12T01:34:54.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:34:53 smithi053 ceph-mon[35158]: pgmap v557: 129 pgs: 129 active+clean; 115 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 1.1 MiB/s wr, 424 op/s 2024-09-12T01:34:54.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:34:53 smithi104 ceph-mon[35219]: pgmap v557: 129 pgs: 129 active+clean; 115 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 1.1 MiB/s wr, 424 op/s 2024-09-12T01:34:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.063+0000 7f992da00640 1 -- 172.21.15.1:0/426985584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9928103710 msgr2=0x7f9928109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.063+0000 7f992da00640 1 --2- 172.21.15.1:0/426985584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9928103710 0x7f9928109fa0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9920009be0 tx=0x7f992002a6d0 comp rx=0 tx=0).stop 2024-09-12T01:34:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.063+0000 7f992da00640 1 -- 172.21.15.1:0/426985584 shutdown_connections 2024-09-12T01:34:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.063+0000 7f992da00640 1 --2- 172.21.15.1:0/426985584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9928103710 0x7f9928109fa0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.063+0000 7f992da00640 1 --2- 172.21.15.1:0/426985584 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9928102d50 0x7f99281031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.063+0000 7f992da00640 1 --2- 172.21.15.1:0/426985584 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9928101b60 0x7f9928101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.063+0000 7f992da00640 1 -- 172.21.15.1:0/426985584 >> 172.21.15.1:0/426985584 conn(0x7f99280fd2e0 msgr2=0x7f99280ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.064+0000 7f992da00640 1 -- 172.21.15.1:0/426985584 shutdown_connections 2024-09-12T01:34:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.064+0000 7f992da00640 1 -- 172.21.15.1:0/426985584 wait complete. 2024-09-12T01:34:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.065+0000 7f992da00640 1 Processor -- start 2024-09-12T01:34:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.065+0000 7f992da00640 1 -- start start 2024-09-12T01:34:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.066+0000 7f992da00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9928101b60 0x7f992819e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.066+0000 7f992da00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9928102d50 0x7f992819e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.066+0000 7f992da00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9928103710 0x7f99281a5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.066+0000 7f992da00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9928076120 con 0x7f9928101b60 2024-09-12T01:34:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.066+0000 7f992da00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9928075fa0 con 0x7f9928102d50 2024-09-12T01:34:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.066+0000 7f992da00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f99280762a0 con 0x7f9928103710 2024-09-12T01:34:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.066+0000 7f9926a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9928101b60 0x7f992819e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.066+0000 7f9926a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9928101b60 0x7f992819e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59836/0 (socket says 172.21.15.1:59836) 2024-09-12T01:34:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.066+0000 7f9926a00640 1 -- 172.21.15.1:0/413347474 learned_addr learned my addr 172.21.15.1:0/413347474 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:34:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.067+0000 7f9927400640 1 --2- 172.21.15.1:0/413347474 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9928103710 0x7f99281a5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.067+0000 7f9926a00640 1 -- 172.21.15.1:0/413347474 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9928103710 msgr2=0x7f99281a5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.067+0000 7f9926a00640 1 --2- 172.21.15.1:0/413347474 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9928103710 0x7f99281a5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.067+0000 7f9926a00640 1 -- 172.21.15.1:0/413347474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9928102d50 msgr2=0x7f992819e9c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T01:34:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.067+0000 7f9926a00640 1 --2- 172.21.15.1:0/413347474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9928102d50 0x7f992819e9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.067+0000 7f9926a00640 1 -- 172.21.15.1:0/413347474 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99281a60a0 con 0x7f9928101b60 2024-09-12T01:34:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.068+0000 7f9926a00640 1 --2- 172.21.15.1:0/413347474 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9928101b60 0x7f992819e480 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f991400e9f0 tx=0x7f991400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.068+0000 7f991f400640 1 -- 172.21.15.1:0/413347474 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991400ccc0 con 0x7f9928101b60 2024-09-12T01:34:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.068+0000 7f991f400640 1 -- 172.21.15.1:0/413347474 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99140035f0 con 0x7f9928101b60 2024-09-12T01:34:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.068+0000 7f991f400640 1 -- 172.21.15.1:0/413347474 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991400ce60 con 0x7f9928101b60 2024-09-12T01:34:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.069+0000 7f992da00640 1 -- 172.21.15.1:0/413347474 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99281a6390 con 0x7f9928101b60 2024-09-12T01:34:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.069+0000 7f992da00640 1 -- 172.21.15.1:0/413347474 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99281a6890 con 0x7f9928101b60 2024-09-12T01:34:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.075+0000 7f99308c6740 1 Processor -- start 2024-09-12T01:34:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.075+0000 7f991f400640 1 -- 172.21.15.1:0/413347474 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f9914010690 con 0x7f9928101b60 2024-09-12T01:34:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.075+0000 7f99308c6740 1 -- start start 2024-09-12T01:34:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.076+0000 7f99308c6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559ea7ae6090 0x559ea7ae6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.076+0000 7f99308c6740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea7ae69d0 0x559ea7aeb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.076+0000 7f99308c6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea7aeba60 0x559ea7aedf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.076+0000 7f99308c6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559ea7a4b100 con 0x559ea7aeba60 2024-09-12T01:34:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.076+0000 7f99308c6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x559ea79f31b0 con 0x559ea7ae6090 2024-09-12T01:34:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.076+0000 7f99308c6740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559ea7a470c0 con 0x559ea7ae69d0 2024-09-12T01:34:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.076+0000 7f9927400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559ea7ae6090 0x559ea7ae6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.076+0000 7f9926000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea7aeba60 0x559ea7aedf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.076+0000 7f9927400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559ea7ae6090 0x559ea7ae6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:59670/0 (socket says 172.21.15.1:59670) 2024-09-12T01:34:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.076+0000 7f9927400640 1 -- 172.21.15.1:0/3298723268 learned_addr learned my addr 172.21.15.1:0/3298723268 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:34:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.076+0000 7f9926a00640 1 --2- 172.21.15.1:0/3298723268 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea7ae69d0 0x559ea7aeb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.077+0000 7f9926a00640 1 -- 172.21.15.1:0/3298723268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559ea7ae6090 msgr2=0x559ea7ae6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.077+0000 7f9926a00640 1 --2- 172.21.15.1:0/3298723268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559ea7ae6090 0x559ea7ae6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.077+0000 7f9926a00640 1 -- 172.21.15.1:0/3298723268 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea7aeba60 msgr2=0x559ea7aedf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.077+0000 7f9926a00640 1 --2- 172.21.15.1:0/3298723268 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea7aeba60 0x559ea7aedf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.077+0000 7f9926a00640 1 -- 172.21.15.1:0/3298723268 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ea775e210 con 0x559ea7ae69d0 2024-09-12T01:34:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.077+0000 7f991f400640 1 --2- 172.21.15.1:0/413347474 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f990c0a2d10 0x7f990c0a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.077+0000 7f9926a00640 1 --2- 172.21.15.1:0/3298723268 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea7ae69d0 0x559ea7aeb490 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f99140ccea0 tx=0x7f99140d5b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f50263da0bcd3878 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.078+0000 7f991f400640 1 -- 172.21.15.1:0/413347474 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f99140cf760 con 0x7f9928101b60 2024-09-12T01:34:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.078+0000 7f9913e00640 1 -- 172.21.15.1:0/3298723268 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9914098ef0 con 0x559ea7ae69d0 2024-09-12T01:34:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.078+0000 7f9913e00640 1 -- 172.21.15.1:0/3298723268 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f991409d350 con 0x559ea7ae69d0 2024-09-12T01:34:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.078+0000 7f9913e00640 1 -- 172.21.15.1:0/3298723268 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991409c5b0 con 0x559ea7ae69d0 2024-09-12T01:34:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.078+0000 7f9927400640 1 --2- 172.21.15.1:0/413347474 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f990c0a2d10 0x7f990c0a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.079+0000 7f99308c6740 1 -- 172.21.15.1:0/3298723268 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea7ae69d0 msgr2=0x559ea7aeb490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.079+0000 7f99308c6740 1 --2- 172.21.15.1:0/3298723268 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea7ae69d0 0x559ea7aeb490 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f99140ccea0 tx=0x7f99140d5b40 comp rx=0 tx=0).stop 2024-09-12T01:34:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.079+0000 7f9927400640 1 --2- 172.21.15.1:0/413347474 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f990c0a2d10 0x7f990c0a51d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9920004af0 tx=0x7f9920009940 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:56.080 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.080+0000 7f99308c6740 1 -- 172.21.15.1:0/3298723268 shutdown_connections 2024-09-12T01:34:56.080 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.080+0000 7f99308c6740 1 --2- 172.21.15.1:0/3298723268 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea7aeba60 0x559ea7aedf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:56.080 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.080+0000 7f99308c6740 1 --2- 172.21.15.1:0/3298723268 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea7ae69d0 0x559ea7aeb490 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:56.080 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.080+0000 7f99308c6740 1 --2- 172.21.15.1:0/3298723268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559ea7ae6090 0x559ea7ae6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:56.080 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.080+0000 7f99308c6740 1 -- 172.21.15.1:0/3298723268 >> 172.21.15.1:0/3298723268 conn(0x559ea7ae1700 msgr2=0x559ea7ae1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:56.080 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.080+0000 7f99308c6740 1 -- 172.21.15.1:0/3298723268 shutdown_connections 2024-09-12T01:34:56.080 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.080+0000 7f99308c6740 1 -- 172.21.15.1:0/3298723268 wait complete. 2024-09-12T01:34:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.080+0000 7f99308c6740 1 -- start start 2024-09-12T01:34:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.081+0000 7f99308c6740 1 Processor -- start 2024-09-12T01:34:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.081+0000 7f99308c6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559ea7ae6090 0x559ea7af0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.081+0000 7f99308c6740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea7ae69d0 0x559ea7af07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.081+0000 7f99308c6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea7aeba60 0x559ea7af7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.081+0000 7f99308c6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559ea7aeea30 con 0x559ea7aeba60 2024-09-12T01:34:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.081+0000 7f99308c6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x559ea79d4090 con 0x559ea7ae6090 2024-09-12T01:34:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.081+0000 7f99308c6740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559ea7aeebb0 con 0x559ea7ae69d0 2024-09-12T01:34:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.081+0000 7f99308c6740 10 client.0 _ll_register_callbacks cb 0x559ea7a28f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:34:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.082+0000 7f9926000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea7ae69d0 0x559ea7af07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.082+0000 7f9926a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559ea7ae6090 0x559ea7af0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.082+0000 7f9926000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea7ae69d0 0x559ea7af07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:39046/0 (socket says 172.21.15.1:39046) 2024-09-12T01:34:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.082+0000 7f9926000640 1 -- 172.21.15.1:0/4040063141 learned_addr learned my addr 172.21.15.1:0/4040063141 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:34:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.082+0000 7f9926a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559ea7ae6090 0x559ea7af0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:59674/0 (socket says 172.21.15.1:59674) 2024-09-12T01:34:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.082+0000 7f9927400640 1 --2- 172.21.15.1:0/4040063141 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea7aeba60 0x559ea7af7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.082+0000 7f9926a00640 1 -- 172.21.15.1:0/4040063141 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea7ae69d0 msgr2=0x559ea7af07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.082+0000 7f9926a00640 1 --2- 172.21.15.1:0/4040063141 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea7ae69d0 0x559ea7af07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.082+0000 7f9926a00640 1 -- 172.21.15.1:0/4040063141 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea7aeba60 msgr2=0x559ea7af7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.082+0000 7f9926a00640 1 --2- 172.21.15.1:0/4040063141 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea7aeba60 0x559ea7af7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.082+0000 7f9926a00640 1 -- 172.21.15.1:0/4040063141 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ea782a620 con 0x559ea7ae6090 2024-09-12T01:34:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.082+0000 7f9927400640 1 --2- 172.21.15.1:0/4040063141 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea7aeba60 0x559ea7af7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:34:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.082+0000 7f9926a00640 1 --2- 172.21.15.1:0/4040063141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559ea7ae6090 0x559ea7af0280 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9914009530 tx=0x7f99140d7710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1ccfe534b690235 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.082+0000 7f9910c00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-12T01:34:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.083+0000 7f9910c00640 1 -- 172.21.15.1:0/4040063141 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991409c790 con 0x559ea7ae6090 2024-09-12T01:34:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.083+0000 7f9910c00640 1 -- 172.21.15.1:0/4040063141 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f9914003b10 con 0x559ea7ae6090 2024-09-12T01:34:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.083+0000 7f99308c6740 1 -- 172.21.15.1:0/4040063141 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f991409d350 con 0x559ea7ae6090 2024-09-12T01:34:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.083+0000 7f9910c00640 1 -- 172.21.15.1:0/4040063141 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f991409ca20 con 0x559ea7ae6090 2024-09-12T01:34:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.083+0000 7f99308c6740 10 client.25610 fetch_fsmap learned FSMap version 35 2024-09-12T01:34:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.083+0000 7f99308c6740 1 -- 172.21.15.1:0/4040063141 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559ea7aeea30 con 0x559ea7ae6090 2024-09-12T01:34:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.083+0000 7f9910c00640 1 -- 172.21.15.1:0/4040063141 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f991409d350 con 0x559ea7ae6090 2024-09-12T01:34:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.083+0000 7f9910c00640 1 -- 172.21.15.1:0/4040063141 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f991409ceb0 con 0x559ea7ae6090 2024-09-12T01:34:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.083+0000 7f99308c6740 10 client.25610 fetch_fsmap finished waiting for FSMap version 35 2024-09-12T01:34:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.083+0000 7f99308c6740 10 client.25610 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:34:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.084+0000 7f99308c6740 20 client.25610 populate_metadata read hostname 'smithi001' 2024-09-12T01:34:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.084+0000 7f99308c6740 1 --2- 172.21.15.1:0/4040063141 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x559ea7afb770 0x559ea7afdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.084+0000 7f99308c6740 4 client.25610 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:34:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.084+0000 7f99308c6740 1 -- 172.21.15.1:0/4040063141 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ea7a58830 con 0x559ea7afb770 2024-09-12T01:34:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.084+0000 7f9926000640 1 --2- 172.21.15.1:0/4040063141 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x559ea7afb770 0x559ea7afdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.085+0000 7f9926000640 1 --2- 172.21.15.1:0/4040063141 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x559ea7afb770 0x559ea7afdb60 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69df4d655f2904af server_cookie=60f6bd05ca0e1fcb in_seq=0 out_seq=0 2024-09-12T01:34:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.085+0000 7f9910c00640 10 client.25610 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:34:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.085+0000 7f9910c00640 1 -- 172.21.15.1:0/4040063141 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559ea7a58830 con 0x559ea7afb770 2024-09-12T01:34:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.085+0000 7f9910c00640 10 client.25610 handle_command_reply: tid=0 2024-09-12T01:34:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.085+0000 7f99308c6740 1 client.25610 shutdown 2024-09-12T01:34:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.086+0000 7f99308c6740 1 -- 172.21.15.1:0/4040063141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559ea7ae6090 msgr2=0x559ea7af0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.086+0000 7f99308c6740 1 --2- 172.21.15.1:0/4040063141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559ea7ae6090 0x559ea7af0280 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9914009530 tx=0x7f99140d7710 comp rx=0 tx=0).stop 2024-09-12T01:34:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.086+0000 7f99308c6740 1 -- 172.21.15.1:0/4040063141 shutdown_connections 2024-09-12T01:34:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.086+0000 7f99308c6740 1 --2- 172.21.15.1:0/4040063141 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x559ea7afb770 0x559ea7afdb60 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.086+0000 7f99308c6740 1 --2- 172.21.15.1:0/4040063141 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea7aeba60 0x559ea7af7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.086+0000 7f99308c6740 1 --2- 172.21.15.1:0/4040063141 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea7ae69d0 0x559ea7af07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.086+0000 7f99308c6740 1 --2- 172.21.15.1:0/4040063141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559ea7ae6090 0x559ea7af0280 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.086+0000 7f99308c6740 1 -- 172.21.15.1:0/4040063141 >> 172.21.15.1:0/4040063141 conn(0x559ea7ae1700 msgr2=0x559ea7ae2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.086+0000 7f9910c00640 0 client.25610 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:34:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.086+0000 7f99308c6740 1 -- 172.21.15.1:0/4040063141 shutdown_connections 2024-09-12T01:34:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.086+0000 7f99308c6740 1 -- 172.21.15.1:0/4040063141 wait complete. 2024-09-12T01:34:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.086+0000 7f99308c6740 20 client.25610 trim_cache size 0 max 16384 2024-09-12T01:34:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.101+0000 7f99308c6740 1 Processor -- start 2024-09-12T01:34:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.101+0000 7f99308c6740 1 -- start start 2024-09-12T01:34:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.102+0000 7f99308c6740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea7ae6090 0x559ea7ae0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.102+0000 7f99308c6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea7ae69d0 0x559ea7ae0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.102+0000 7f99308c6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559ea7aeba60 0x559ea7af77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.102+0000 7f99308c6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f99281b0eb0 con 0x559ea7ae69d0 2024-09-12T01:34:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.102+0000 7f99308c6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f99281b29f0 con 0x559ea7aeba60 2024-09-12T01:34:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.102+0000 7f99308c6740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f99281b1580 con 0x559ea7ae6090 2024-09-12T01:34:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.102+0000 7f9926000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea7ae69d0 0x559ea7ae0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.102+0000 7f9926000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea7ae69d0 0x559ea7ae0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59870/0 (socket says 172.21.15.1:59870) 2024-09-12T01:34:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.102+0000 7f9927400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559ea7aeba60 0x559ea7af77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.102+0000 7f9926000640 1 -- 172.21.15.1:0/1637365845 learned_addr learned my addr 172.21.15.1:0/1637365845 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:34:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.102+0000 7f9926a00640 1 --2- 172.21.15.1:0/1637365845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea7ae6090 0x559ea7ae0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:56.104 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.103+0000 7f9927400640 1 -- 172.21.15.1:0/1637365845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea7ae6090 msgr2=0x559ea7ae0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:56.104 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.103+0000 7f9927400640 1 --2- 172.21.15.1:0/1637365845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea7ae6090 0x559ea7ae0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:56.104 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.103+0000 7f9927400640 1 -- 172.21.15.1:0/1637365845 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea7ae69d0 msgr2=0x559ea7ae0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:56.104 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.103+0000 7f9927400640 1 --2- 172.21.15.1:0/1637365845 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea7ae69d0 0x559ea7ae0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.103+0000 7f9927400640 1 -- 172.21.15.1:0/1637365845 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ea7ae1520 con 0x559ea7aeba60 2024-09-12T01:34:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.103+0000 7f9926000640 1 --2- 172.21.15.1:0/1637365845 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea7ae69d0 0x559ea7ae0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:34:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.103+0000 7f9926a00640 1 --2- 172.21.15.1:0/1637365845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea7ae6090 0x559ea7ae0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:34:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.103+0000 7f9927400640 1 --2- 172.21.15.1:0/1637365845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559ea7aeba60 0x559ea7af77c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9920009b30 tx=0x7f992003f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc7e44219960ecf6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.103+0000 7f9913400640 1 -- 172.21.15.1:0/1637365845 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9920051070 con 0x559ea7aeba60 2024-09-12T01:34:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.103+0000 7f9913400640 1 -- 172.21.15.1:0/1637365845 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9920031bc0 con 0x559ea7aeba60 2024-09-12T01:34:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.103+0000 7f9913400640 1 -- 172.21.15.1:0/1637365845 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9920036040 con 0x559ea7aeba60 2024-09-12T01:34:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.104+0000 7f99308c6740 1 -- 172.21.15.1:0/1637365845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559ea7aeba60 msgr2=0x559ea7af77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.104+0000 7f99308c6740 1 --2- 172.21.15.1:0/1637365845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559ea7aeba60 0x559ea7af77c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9920009b30 tx=0x7f992003f000 comp rx=0 tx=0).stop 2024-09-12T01:34:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.104+0000 7f99308c6740 1 -- 172.21.15.1:0/1637365845 shutdown_connections 2024-09-12T01:34:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.104+0000 7f99308c6740 1 --2- 172.21.15.1:0/1637365845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559ea7aeba60 0x559ea7af77c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.104+0000 7f99308c6740 1 --2- 172.21.15.1:0/1637365845 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea7ae69d0 0x559ea7ae0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.104+0000 7f99308c6740 1 --2- 172.21.15.1:0/1637365845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea7ae6090 0x559ea7ae0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.104+0000 7f99308c6740 1 -- 172.21.15.1:0/1637365845 >> 172.21.15.1:0/1637365845 conn(0x559ea7ae1700 msgr2=0x559ea7ae0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.104+0000 7f99308c6740 1 -- 172.21.15.1:0/1637365845 shutdown_connections 2024-09-12T01:34:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.104+0000 7f99308c6740 1 -- 172.21.15.1:0/1637365845 wait complete. 2024-09-12T01:34:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.105+0000 7f99308c6740 1 -- start start 2024-09-12T01:34:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.105+0000 7f99308c6740 1 Processor -- start 2024-09-12T01:34:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.105+0000 7f99308c6740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea7ae6090 0x559ea7af5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.105+0000 7f99308c6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea7ae69d0 0x559ea7af5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.105+0000 7f99308c6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559ea7aeba60 0x559ea7af6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.105+0000 7f99308c6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f99281b2320 con 0x559ea7ae69d0 2024-09-12T01:34:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.105+0000 7f99308c6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f99281b1c50 con 0x559ea7aeba60 2024-09-12T01:34:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.105+0000 7f99308c6740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f99281aee90 con 0x559ea7ae6090 2024-09-12T01:34:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.105+0000 7f99308c6740 10 client.0 _ll_register_callbacks cb 0x7f99140a0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:34:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.106+0000 7f9926a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea7ae6090 0x559ea7af5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.106+0000 7f9926a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea7ae6090 0x559ea7af5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:39066/0 (socket says 172.21.15.1:39066) 2024-09-12T01:34:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.106+0000 7f9926a00640 1 -- 172.21.15.1:0/2805289279 learned_addr learned my addr 172.21.15.1:0/2805289279 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:34:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.106+0000 7f9926000640 1 --2- 172.21.15.1:0/2805289279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea7ae69d0 0x559ea7af5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.106+0000 7f9927400640 1 --2- 172.21.15.1:0/2805289279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559ea7aeba60 0x559ea7af6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.106+0000 7f9927400640 1 -- 172.21.15.1:0/2805289279 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea7ae6090 msgr2=0x559ea7af5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.106+0000 7f9927400640 1 --2- 172.21.15.1:0/2805289279 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea7ae6090 0x559ea7af5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.106+0000 7f9927400640 1 -- 172.21.15.1:0/2805289279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea7ae69d0 msgr2=0x559ea7af5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:34:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.106+0000 7f9927400640 1 --2- 172.21.15.1:0/2805289279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea7ae69d0 0x559ea7af5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.106+0000 7f9927400640 1 -- 172.21.15.1:0/2805289279 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ea7af7260 con 0x559ea7aeba60 2024-09-12T01:34:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.106+0000 7f9926000640 1 --2- 172.21.15.1:0/2805289279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea7ae69d0 0x559ea7af5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:34:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.107+0000 7f9927400640 1 --2- 172.21.15.1:0/2805289279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559ea7aeba60 0x559ea7af6c50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9920006010 tx=0x7f9920030da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ae5064098677f71 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:34:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.107+0000 7f9910c00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-12T01:34:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.107+0000 7f9910c00640 1 -- 172.21.15.1:0/2805289279 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9920036070 con 0x559ea7aeba60 2024-09-12T01:34:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.108+0000 7f9910c00640 1 -- 172.21.15.1:0/2805289279 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f9920003970 con 0x559ea7aeba60 2024-09-12T01:34:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.108+0000 7f99308c6740 1 -- 172.21.15.1:0/2805289279 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9920031bc0 con 0x559ea7aeba60 2024-09-12T01:34:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.108+0000 7f9910c00640 1 -- 172.21.15.1:0/2805289279 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9920063050 con 0x559ea7aeba60 2024-09-12T01:34:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.108+0000 7f99308c6740 10 client.25616 fetch_fsmap learned FSMap version 35 2024-09-12T01:34:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.108+0000 7f99308c6740 1 -- 172.21.15.1:0/2805289279 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559ea7b66950 con 0x559ea7aeba60 2024-09-12T01:34:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.108+0000 7f9910c00640 1 -- 172.21.15.1:0/2805289279 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9920031bc0 con 0x559ea7aeba60 2024-09-12T01:34:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.109+0000 7f9910c00640 1 -- 172.21.15.1:0/2805289279 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f992005e030 con 0x559ea7aeba60 2024-09-12T01:34:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.109+0000 7f99308c6740 10 client.25616 fetch_fsmap finished waiting for FSMap version 35 2024-09-12T01:34:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.109+0000 7f99308c6740 10 client.25616 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:34:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.109+0000 7f99308c6740 20 client.25616 populate_metadata read hostname 'smithi001' 2024-09-12T01:34:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.109+0000 7f99308c6740 1 --2- 172.21.15.1:0/2805289279 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x559ea7b67300 0x559ea7b69720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:34:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.109+0000 7f99308c6740 4 client.25616 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:34:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.109+0000 7f99308c6740 1 -- 172.21.15.1:0/2805289279 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559ea7a52b80 con 0x559ea7b67300 2024-09-12T01:34:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.110+0000 7f9926a00640 1 --2- 172.21.15.1:0/2805289279 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x559ea7b67300 0x559ea7b69720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:34:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.110+0000 7f9926a00640 1 --2- 172.21.15.1:0/2805289279 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x559ea7b67300 0x559ea7b69720 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=362ea91a0a05ea64 server_cookie=2223cd3fb061792c in_seq=0 out_seq=0 2024-09-12T01:34:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.110+0000 7f9910c00640 10 client.25616 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:34:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.118+0000 7f9910c00640 1 -- 172.21.15.1:0/2805289279 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559ea7a52b80 con 0x559ea7b67300 2024-09-12T01:34:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.118+0000 7f9910c00640 10 client.25616 handle_command_reply: tid=0 2024-09-12T01:34:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.118+0000 7f99308c6740 1 client.25616 shutdown 2024-09-12T01:34:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.119+0000 7f99308c6740 1 -- 172.21.15.1:0/2805289279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559ea7aeba60 msgr2=0x559ea7af6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.119+0000 7f99308c6740 1 --2- 172.21.15.1:0/2805289279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559ea7aeba60 0x559ea7af6c50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9920006010 tx=0x7f9920030da0 comp rx=0 tx=0).stop 2024-09-12T01:34:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.119+0000 7f99308c6740 1 -- 172.21.15.1:0/2805289279 shutdown_connections 2024-09-12T01:34:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.119+0000 7f99308c6740 1 --2- 172.21.15.1:0/2805289279 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x559ea7b67300 0x559ea7b69720 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.119+0000 7f99308c6740 1 --2- 172.21.15.1:0/2805289279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559ea7aeba60 0x559ea7af6c50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.119+0000 7f99308c6740 1 --2- 172.21.15.1:0/2805289279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ea7ae69d0 0x559ea7af5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.119+0000 7f99308c6740 1 --2- 172.21.15.1:0/2805289279 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea7ae6090 0x559ea7af5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.119+0000 7f9910c00640 0 client.25616 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:34:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.119+0000 7f99308c6740 1 -- 172.21.15.1:0/2805289279 >> 172.21.15.1:0/2805289279 conn(0x559ea7ae1700 msgr2=0x559ea7ae0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.119+0000 7f99308c6740 1 -- 172.21.15.1:0/2805289279 shutdown_connections 2024-09-12T01:34:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.119+0000 7f99308c6740 1 -- 172.21.15.1:0/2805289279 wait complete. 2024-09-12T01:34:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.119+0000 7f99308c6740 20 client.25616 trim_cache size 0 max 16384 2024-09-12T01:34:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-12T01:34:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-12T01:34:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-12T01:34:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-12T01:34:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.120+0000 7f992da00640 1 -- 172.21.15.1:0/413347474 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f990c0a2d10 msgr2=0x7f990c0a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.120+0000 7f992da00640 1 --2- 172.21.15.1:0/413347474 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f990c0a2d10 0x7f990c0a51d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9920004af0 tx=0x7f9920009940 comp rx=0 tx=0).stop 2024-09-12T01:34:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.120+0000 7f992da00640 1 -- 172.21.15.1:0/413347474 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9928101b60 msgr2=0x7f992819e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:34:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.120+0000 7f992da00640 1 --2- 172.21.15.1:0/413347474 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9928101b60 0x7f992819e480 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f991400e9f0 tx=0x7f991400eeb0 comp rx=0 tx=0).stop 2024-09-12T01:34:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.120+0000 7f992da00640 1 -- 172.21.15.1:0/413347474 shutdown_connections 2024-09-12T01:34:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.120+0000 7f992da00640 1 --2- 172.21.15.1:0/413347474 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f990c0a2d10 0x7f990c0a51d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.120+0000 7f992da00640 1 --2- 172.21.15.1:0/413347474 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9928103710 0x7f99281a5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.120+0000 7f992da00640 1 --2- 172.21.15.1:0/413347474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9928102d50 0x7f992819e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.120+0000 7f992da00640 1 --2- 172.21.15.1:0/413347474 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9928101b60 0x7f992819e480 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:34:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.120+0000 7f992da00640 1 -- 172.21.15.1:0/413347474 >> 172.21.15.1:0/413347474 conn(0x7f99280fd2e0 msgr2=0x7f992810e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:34:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.120+0000 7f992da00640 1 -- 172.21.15.1:0/413347474 shutdown_connections 2024-09-12T01:34:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:34:56.120+0000 7f992da00640 1 -- 172.21.15.1:0/413347474 wait complete. 2024-09-12T01:34:56.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:34:55 smithi001 ceph-mon[30387]: pgmap v558: 129 pgs: 129 active+clean; 114 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.2 MiB/s wr, 466 op/s 2024-09-12T01:34:56.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:34:55 smithi053 ceph-mon[35158]: pgmap v558: 129 pgs: 129 active+clean; 114 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.2 MiB/s wr, 466 op/s 2024-09-12T01:34:56.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:34:55 smithi104 ceph-mon[35219]: pgmap v558: 129 pgs: 129 active+clean; 114 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.2 MiB/s wr, 466 op/s 2024-09-12T01:34:58.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:34:57 smithi001 ceph-mon[30387]: pgmap v559: 129 pgs: 129 active+clean; 121 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.3 MiB/s wr, 522 op/s 2024-09-12T01:34:58.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:34:57 smithi053 ceph-mon[35158]: pgmap v559: 129 pgs: 129 active+clean; 121 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.3 MiB/s wr, 522 op/s 2024-09-12T01:34:58.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:34:57 smithi104 ceph-mon[35219]: pgmap v559: 129 pgs: 129 active+clean; 121 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.3 MiB/s wr, 522 op/s 2024-09-12T01:34:59.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:34:58 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:34:59.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:34:58 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:34:59.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:34:58 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:35:00.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:34:59 smithi001 ceph-mon[30387]: pgmap v560: 129 pgs: 129 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 988 KiB/s wr, 403 op/s 2024-09-12T01:35:00.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:34:59 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:35:00.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:34:59 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:35:00.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:34:59 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:35:00.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:34:59 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:35:00.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:34:59 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:35:00.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:34:59 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:35:00.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:34:59 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:35:00.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:34:59 smithi053 ceph-mon[35158]: pgmap v560: 129 pgs: 129 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 988 KiB/s wr, 403 op/s 2024-09-12T01:35:00.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:34:59 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:35:00.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:34:59 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:35:00.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:34:59 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:35:00.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:34:59 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:35:00.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:34:59 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:35:00.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:34:59 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:35:00.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:34:59 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:35:00.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:34:59 smithi104 ceph-mon[35219]: pgmap v560: 129 pgs: 129 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 988 KiB/s wr, 403 op/s 2024-09-12T01:35:00.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:34:59 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:35:00.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:34:59 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:35:00.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:34:59 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:35:00.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:34:59 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:35:00.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:34:59 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:35:00.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:34:59 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:35:00.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:34:59 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:35:01.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:35:00 smithi001 ceph-mon[30387]: Adjusting osd_memory_target on smithi001 to 177.5M 2024-09-12T01:35:01.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:35:00 smithi001 ceph-mon[30387]: Unable to set osd_memory_target on smithi001 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-12T01:35:01.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:35:00 smithi053 ceph-mon[35158]: Adjusting osd_memory_target on smithi001 to 177.5M 2024-09-12T01:35:01.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:35:00 smithi053 ceph-mon[35158]: Unable to set osd_memory_target on smithi001 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-12T01:35:01.266 INFO:tasks.workunit.client.0.smithi001.stdout:truncating to largest ever: 0x63e5d2a 2024-09-12T01:35:01.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:35:00 smithi104 ceph-mon[35219]: Adjusting osd_memory_target on smithi001 to 177.5M 2024-09-12T01:35:01.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:35:00 smithi104 ceph-mon[35219]: Unable to set osd_memory_target on smithi001 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-12T01:35:02.199 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:35:01 smithi001 ceph-mon[30387]: pgmap v561: 129 pgs: 129 active+clean; 124 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.3 MiB/s wr, 534 op/s 2024-09-12T01:35:02.245 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:35:01 smithi053 ceph-mon[35158]: pgmap v561: 129 pgs: 129 active+clean; 124 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.3 MiB/s wr, 534 op/s 2024-09-12T01:35:02.333 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:35:01 smithi104 ceph-mon[35219]: pgmap v561: 129 pgs: 129 active+clean; 124 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.3 MiB/s wr, 534 op/s 2024-09-12T01:35:04.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:35:03 smithi001 ceph-mon[30387]: pgmap v562: 129 pgs: 129 active+clean; 124 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 1.0 MiB/s wr, 433 op/s 2024-09-12T01:35:04.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:35:03 smithi053 ceph-mon[35158]: pgmap v562: 129 pgs: 129 active+clean; 124 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 1.0 MiB/s wr, 433 op/s 2024-09-12T01:35:04.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:35:03 smithi104 ceph-mon[35219]: pgmap v562: 129 pgs: 129 active+clean; 124 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 1.0 MiB/s wr, 433 op/s 2024-09-12T01:35:06.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:35:05 smithi001 ceph-mon[30387]: pgmap v563: 129 pgs: 129 active+clean; 123 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.1 MiB/s wr, 475 op/s 2024-09-12T01:35:06.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:35:05 smithi053 ceph-mon[35158]: pgmap v563: 129 pgs: 129 active+clean; 123 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.1 MiB/s wr, 475 op/s 2024-09-12T01:35:06.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:35:05 smithi104 ceph-mon[35219]: pgmap v563: 129 pgs: 129 active+clean; 123 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.1 MiB/s wr, 475 op/s 2024-09-12T01:35:06.908 INFO:tasks.workunit.client.0.smithi001.stdout:truncating to largest ever: 0x63e9b52 2024-09-12T01:35:08.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:35:07 smithi001 ceph-mon[30387]: pgmap v564: 129 pgs: 129 active+clean; 117 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.2 MiB/s wr, 516 op/s 2024-09-12T01:35:08.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:35:07 smithi053 ceph-mon[35158]: pgmap v564: 129 pgs: 129 active+clean; 117 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.2 MiB/s wr, 516 op/s 2024-09-12T01:35:08.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:35:07 smithi104 ceph-mon[35219]: pgmap v564: 129 pgs: 129 active+clean; 117 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.2 MiB/s wr, 516 op/s 2024-09-12T01:35:09.269 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.269+0000 7fd2b1c00640 1 -- 172.21.15.1:0/1173173158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd2ac104f40 msgr2=0x7fd2ac1053c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:09.269 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.269+0000 7fd2b1c00640 1 --2- 172.21.15.1:0/1173173158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd2ac104f40 0x7fd2ac1053c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd298009900 tx=0x7fd29802a410 comp rx=0 tx=0).stop 2024-09-12T01:35:09.269 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.269+0000 7fd2b1c00640 1 -- 172.21.15.1:0/1173173158 shutdown_connections 2024-09-12T01:35:09.269 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.269+0000 7fd2b1c00640 1 --2- 172.21.15.1:0/1173173158 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2ac105900 0x7fd2ac10c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:09.269 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.269+0000 7fd2b1c00640 1 --2- 172.21.15.1:0/1173173158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd2ac104f40 0x7fd2ac1053c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:09.269 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.269+0000 7fd2b1c00640 1 --2- 172.21.15.1:0/1173173158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2ac103d50 0x7fd2ac104150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:09.269 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.269+0000 7fd2b1c00640 1 -- 172.21.15.1:0/1173173158 >> 172.21.15.1:0/1173173158 conn(0x7fd2ac0ff4b0 msgr2=0x7fd2ac1018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:35:09.269 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.269+0000 7fd2b1c00640 1 -- 172.21.15.1:0/1173173158 shutdown_connections 2024-09-12T01:35:09.270 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.269+0000 7fd2b1c00640 1 -- 172.21.15.1:0/1173173158 wait complete. 2024-09-12T01:35:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.270+0000 7fd2b1c00640 1 Processor -- start 2024-09-12T01:35:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.270+0000 7fd2b1c00640 1 -- start start 2024-09-12T01:35:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.270+0000 7fd2b1c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2ac103d50 0x7fd2ac19a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.270+0000 7fd2b1c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd2ac104f40 0x7fd2ac19a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.270+0000 7fd2b1c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2ac105900 0x7fd2ac1a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.270+0000 7fd2b1c00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ac10fa00 con 0x7fd2ac103d50 2024-09-12T01:35:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.270+0000 7fd2b1c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ac10f880 con 0x7fd2ac105900 2024-09-12T01:35:09.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.270+0000 7fd2b1c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ac10fb80 con 0x7fd2ac104f40 2024-09-12T01:35:09.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.270+0000 7fd2aaa00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2ac103d50 0x7fd2ac19a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:09.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.270+0000 7fd2aaa00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2ac103d50 0x7fd2ac19a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43480/0 (socket says 172.21.15.1:43480) 2024-09-12T01:35:09.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.271+0000 7fd2aaa00640 1 -- 172.21.15.1:0/3237380330 learned_addr learned my addr 172.21.15.1:0/3237380330 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:35:09.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.271+0000 7fd2aa000640 1 --2- 172.21.15.1:0/3237380330 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd2ac104f40 0x7fd2ac19a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:09.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.271+0000 7fd2ab400640 1 --2- 172.21.15.1:0/3237380330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2ac105900 0x7fd2ac1a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:09.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.271+0000 7fd2aaa00640 1 -- 172.21.15.1:0/3237380330 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd2ac104f40 msgr2=0x7fd2ac19a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:09.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.271+0000 7fd2aaa00640 1 --2- 172.21.15.1:0/3237380330 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd2ac104f40 0x7fd2ac19a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:09.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.271+0000 7fd2aaa00640 1 -- 172.21.15.1:0/3237380330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2ac105900 msgr2=0x7fd2ac1a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:09.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.271+0000 7fd2aaa00640 1 --2- 172.21.15.1:0/3237380330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2ac105900 0x7fd2ac1a15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:09.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.271+0000 7fd2aaa00640 1 -- 172.21.15.1:0/3237380330 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2ac1a1c50 con 0x7fd2ac103d50 2024-09-12T01:35:09.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.271+0000 7fd2aaa00640 1 --2- 172.21.15.1:0/3237380330 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2ac103d50 0x7fd2ac19a030 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd29c00e9c0 tx=0x7fd29c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:35:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.272+0000 7fd2a3400640 1 -- 172.21.15.1:0/3237380330 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29c00cde0 con 0x7fd2ac103d50 2024-09-12T01:35:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.272+0000 7fd2a3400640 1 -- 172.21.15.1:0/3237380330 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd29c010910 con 0x7fd2ac103d50 2024-09-12T01:35:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.272+0000 7fd2b1c00640 1 -- 172.21.15.1:0/3237380330 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2ac1a1f40 con 0x7fd2ac103d50 2024-09-12T01:35:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.272+0000 7fd2a3400640 1 -- 172.21.15.1:0/3237380330 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29c018a10 con 0x7fd2ac103d50 2024-09-12T01:35:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.272+0000 7fd2b1c00640 1 -- 172.21.15.1:0/3237380330 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2ac1a23b0 con 0x7fd2ac103d50 2024-09-12T01:35:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.275+0000 7fd2b4bd3740 1 Processor -- start 2024-09-12T01:35:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.275+0000 7fd2b4bd3740 1 -- start start 2024-09-12T01:35:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.275+0000 7fd2b4bd3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c70c2cc090 0x55c70c2cc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.275+0000 7fd2b4bd3740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c70c2cc9d0 0x55c70c2d1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.275+0000 7fd2b4bd3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c70c2d1a60 0x55c70c2d3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.275+0000 7fd2b4bd3740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c70c231100 con 0x55c70c2cc9d0 2024-09-12T01:35:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.275+0000 7fd2b4bd3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55c70c1d91b0 con 0x55c70c2d1a60 2024-09-12T01:35:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.275+0000 7fd2b4bd3740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c70c22d0c0 con 0x55c70c2cc090 2024-09-12T01:35:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.275+0000 7fd2a3400640 1 -- 172.21.15.1:0/3237380330 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fd29c01f020 con 0x7fd2ac103d50 2024-09-12T01:35:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.275+0000 7fd2ab400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c70c2cc090 0x55c70c2cc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.276+0000 7fd2ab400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c70c2cc090 0x55c70c2cc490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:56806/0 (socket says 172.21.15.1:56806) 2024-09-12T01:35:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.276+0000 7fd2ab400640 1 -- 172.21.15.1:0/607104804 learned_addr learned my addr 172.21.15.1:0/607104804 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:35:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.276+0000 7fd2aa000640 1 --2- 172.21.15.1:0/607104804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c70c2d1a60 0x55c70c2d3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.276+0000 7fd2aaa00640 1 --2- 172.21.15.1:0/607104804 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c70c2cc9d0 0x55c70c2d1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.276+0000 7fd2ab400640 1 -- 172.21.15.1:0/607104804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c70c2d1a60 msgr2=0x55c70c2d3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:35:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.276+0000 7fd2ab400640 1 --2- 172.21.15.1:0/607104804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c70c2d1a60 0x55c70c2d3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.276+0000 7fd2ab400640 1 -- 172.21.15.1:0/607104804 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c70c2cc9d0 msgr2=0x55c70c2d1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:35:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.276+0000 7fd2ab400640 1 --2- 172.21.15.1:0/607104804 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c70c2cc9d0 0x55c70c2d1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.276+0000 7fd2ab400640 1 -- 172.21.15.1:0/607104804 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c70bf44210 con 0x55c70c2cc090 2024-09-12T01:35:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.276+0000 7fd2aaa00640 1 --2- 172.21.15.1:0/607104804 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c70c2cc9d0 0x55c70c2d1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:35:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.276+0000 7fd2ab400640 1 --2- 172.21.15.1:0/607104804 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c70c2cc090 0x55c70c2cc490 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4009fd0 tx=0x7fd2a40118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4a62030da8179b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:35:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.277+0000 7fd297e00640 1 -- 172.21.15.1:0/607104804 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a4004d20 con 0x55c70c2cc090 2024-09-12T01:35:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.277+0000 7fd297e00640 1 -- 172.21.15.1:0/607104804 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2a4004050 con 0x55c70c2cc090 2024-09-12T01:35:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.277+0000 7fd297e00640 1 -- 172.21.15.1:0/607104804 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a4005260 con 0x55c70c2cc090 2024-09-12T01:35:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.277+0000 7fd2a3400640 1 --2- 172.21.15.1:0/3237380330 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fd2900a2c00 0x7fd2900a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.277+0000 7fd2b4bd3740 1 -- 172.21.15.1:0/607104804 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c70c2cc090 msgr2=0x55c70c2cc490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.277+0000 7fd2b4bd3740 1 --2- 172.21.15.1:0/607104804 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c70c2cc090 0x55c70c2cc490 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4009fd0 tx=0x7fd2a40118d0 comp rx=0 tx=0).stop 2024-09-12T01:35:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.277+0000 7fd2b4bd3740 1 -- 172.21.15.1:0/607104804 shutdown_connections 2024-09-12T01:35:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.277+0000 7fd2b4bd3740 1 --2- 172.21.15.1:0/607104804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c70c2d1a60 0x55c70c2d3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.277+0000 7fd2b4bd3740 1 --2- 172.21.15.1:0/607104804 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c70c2cc9d0 0x55c70c2d1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.278+0000 7fd2b4bd3740 1 --2- 172.21.15.1:0/607104804 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c70c2cc090 0x55c70c2cc490 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.278+0000 7fd2b4bd3740 1 -- 172.21.15.1:0/607104804 >> 172.21.15.1:0/607104804 conn(0x55c70c2c7700 msgr2=0x55c70c2c7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:35:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.278+0000 7fd2a3400640 1 -- 172.21.15.1:0/3237380330 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fd29c0cf510 con 0x7fd2ac103d50 2024-09-12T01:35:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.278+0000 7fd2b4bd3740 1 -- 172.21.15.1:0/607104804 shutdown_connections 2024-09-12T01:35:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.278+0000 7fd2b4bd3740 1 -- 172.21.15.1:0/607104804 wait complete. 2024-09-12T01:35:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.278+0000 7fd2aaa00640 1 --2- 172.21.15.1:0/3237380330 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fd2900a2c00 0x7fd2900a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.278+0000 7fd2b4bd3740 1 -- start start 2024-09-12T01:35:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.278+0000 7fd2aaa00640 1 --2- 172.21.15.1:0/3237380330 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fd2900a2c00 0x7fd2900a50c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd29c0ccff0 tx=0x7fd29c00efb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:35:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.279+0000 7fd2b4bd3740 1 Processor -- start 2024-09-12T01:35:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.279+0000 7fd2b4bd3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c70c2cc090 0x55c70c2d61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.279+0000 7fd2b4bd3740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c70c2cc9d0 0x55c70c2d6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.279+0000 7fd2b4bd3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c70c2d1a60 0x55c70c2dd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.279+0000 7fd2b4bd3740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c70c2d4a30 con 0x55c70c2cc9d0 2024-09-12T01:35:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.279+0000 7fd2b4bd3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55c70c1ba090 con 0x55c70c2d1a60 2024-09-12T01:35:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.279+0000 7fd2b4bd3740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c70c2d4bb0 con 0x55c70c2cc090 2024-09-12T01:35:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.279+0000 7fd2b4bd3740 10 client.0 _ll_register_callbacks cb 0x55c70c20ef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:35:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.279+0000 7fd2aa000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c70c2cc9d0 0x55c70c2d6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.279+0000 7fd2ab400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c70c2cc090 0x55c70c2d61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.279+0000 7fd2aa000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c70c2cc9d0 0x55c70c2d6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43500/0 (socket says 172.21.15.1:43500) 2024-09-12T01:35:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.279+0000 7fd2aa000640 1 -- 172.21.15.1:0/3122651663 learned_addr learned my addr 172.21.15.1:0/3122651663 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:35:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.279+0000 7fd2ab400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c70c2cc090 0x55c70c2d61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:56820/0 (socket says 172.21.15.1:56820) 2024-09-12T01:35:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.279+0000 7fd2ab400640 1 --2- 172.21.15.1:0/3122651663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c70c2d1a60 0x55c70c2dd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.280+0000 7fd2aa000640 1 -- 172.21.15.1:0/3122651663 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c70c2cc090 msgr2=0x55c70c2d61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:35:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.280+0000 7fd2aa000640 1 --2- 172.21.15.1:0/3122651663 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c70c2cc090 0x55c70c2d61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.280+0000 7fd2aa000640 1 -- 172.21.15.1:0/3122651663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c70c2d1a60 msgr2=0x55c70c2dd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:35:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.280+0000 7fd2aa000640 1 --2- 172.21.15.1:0/3122651663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c70c2d1a60 0x55c70c2dd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.280+0000 7fd2aa000640 1 -- 172.21.15.1:0/3122651663 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c70c010620 con 0x55c70c2cc9d0 2024-09-12T01:35:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.280+0000 7fd2aa000640 1 --2- 172.21.15.1:0/3122651663 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c70c2cc9d0 0x55c70c2d6730 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd2980314e0 tx=0x7fd2980043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=471bbe61c0723c38 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:35:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.280+0000 7fd294c00640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-12T01:35:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.281+0000 7fd294c00640 1 -- 172.21.15.1:0/3122651663 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd298035040 con 0x55c70c2cc9d0 2024-09-12T01:35:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.281+0000 7fd2b4bd3740 1 -- 172.21.15.1:0/3122651663 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2a4004050 con 0x55c70c2cc9d0 2024-09-12T01:35:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.281+0000 7fd294c00640 1 -- 172.21.15.1:0/3122651663 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fd298005450 con 0x55c70c2cc9d0 2024-09-12T01:35:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.281+0000 7fd294c00640 1 -- 172.21.15.1:0/3122651663 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd29804f050 con 0x55c70c2cc9d0 2024-09-12T01:35:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.281+0000 7fd2b4bd3740 10 client.15579 fetch_fsmap learned FSMap version 35 2024-09-12T01:35:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.281+0000 7fd2b4bd3740 1 -- 172.21.15.1:0/3122651663 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c70c2e0590 con 0x55c70c2cc9d0 2024-09-12T01:35:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.281+0000 7fd294c00640 1 -- 172.21.15.1:0/3122651663 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2a4004050 con 0x55c70c2cc9d0 2024-09-12T01:35:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.281+0000 7fd294c00640 1 -- 172.21.15.1:0/3122651663 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fd298031a10 con 0x55c70c2cc9d0 2024-09-12T01:35:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.281+0000 7fd2b4bd3740 10 client.15579 fetch_fsmap finished waiting for FSMap version 35 2024-09-12T01:35:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.281+0000 7fd2b4bd3740 10 client.15579 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:35:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.282+0000 7fd2b4bd3740 20 client.15579 populate_metadata read hostname 'smithi001' 2024-09-12T01:35:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.282+0000 7fd2b4bd3740 1 --2- 172.21.15.1:0/3122651663 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55c70c2e1c60 0x55c70c2e4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.282+0000 7fd2b4bd3740 4 client.15579 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:35:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.282+0000 7fd2b4bd3740 1 -- 172.21.15.1:0/3122651663 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c70c23e830 con 0x55c70c2e1c60 2024-09-12T01:35:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.282+0000 7fd2ab400640 1 --2- 172.21.15.1:0/3122651663 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55c70c2e1c60 0x55c70c2e4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.282+0000 7fd2ab400640 1 --2- 172.21.15.1:0/3122651663 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55c70c2e1c60 0x55c70c2e4050 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c1fa224b11fe76 server_cookie=8127a8e220e4c75a in_seq=0 out_seq=0 2024-09-12T01:35:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.282+0000 7fd294c00640 10 client.15579 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:35:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.283+0000 7fd294c00640 1 -- 172.21.15.1:0/3122651663 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c70c23e830 con 0x55c70c2e1c60 2024-09-12T01:35:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.283+0000 7fd294c00640 10 client.15579 handle_command_reply: tid=0 2024-09-12T01:35:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.283+0000 7fd2b4bd3740 1 client.15579 shutdown 2024-09-12T01:35:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.284+0000 7fd2b4bd3740 1 -- 172.21.15.1:0/3122651663 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c70c2cc9d0 msgr2=0x55c70c2d6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.284+0000 7fd2b4bd3740 1 --2- 172.21.15.1:0/3122651663 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c70c2cc9d0 0x55c70c2d6730 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd2980314e0 tx=0x7fd2980043b0 comp rx=0 tx=0).stop 2024-09-12T01:35:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.284+0000 7fd2b4bd3740 1 -- 172.21.15.1:0/3122651663 shutdown_connections 2024-09-12T01:35:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.284+0000 7fd2b4bd3740 1 --2- 172.21.15.1:0/3122651663 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55c70c2e1c60 0x55c70c2e4050 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.284+0000 7fd2b4bd3740 1 --2- 172.21.15.1:0/3122651663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c70c2d1a60 0x55c70c2dd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.284+0000 7fd2b4bd3740 1 --2- 172.21.15.1:0/3122651663 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c70c2cc9d0 0x55c70c2d6730 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.284+0000 7fd2b4bd3740 1 --2- 172.21.15.1:0/3122651663 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c70c2cc090 0x55c70c2d61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.284+0000 7fd294c00640 0 client.15579 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:35:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.284+0000 7fd2b4bd3740 1 -- 172.21.15.1:0/3122651663 >> 172.21.15.1:0/3122651663 conn(0x55c70c2c7700 msgr2=0x55c70c2c8f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:35:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.284+0000 7fd2b4bd3740 1 -- 172.21.15.1:0/3122651663 shutdown_connections 2024-09-12T01:35:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.284+0000 7fd2b4bd3740 1 -- 172.21.15.1:0/3122651663 wait complete. 2024-09-12T01:35:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.284+0000 7fd2b4bd3740 20 client.15579 trim_cache size 0 max 16384 2024-09-12T01:35:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.301+0000 7fd2b4bd3740 1 Processor -- start 2024-09-12T01:35:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.301+0000 7fd2b4bd3740 1 -- start start 2024-09-12T01:35:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.301+0000 7fd2b4bd3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c70c2cc090 0x55c70c2c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.301+0000 7fd2b4bd3740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c70c2cc9d0 0x55c70c2dd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.301+0000 7fd2b4bd3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c70c2d1a60 0x55c70c2c8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.301+0000 7fd2b4bd3740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ac1ac970 con 0x55c70c2cc9d0 2024-09-12T01:35:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.301+0000 7fd2b4bd3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ac1ae4b0 con 0x55c70c2cc090 2024-09-12T01:35:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.301+0000 7fd2b4bd3740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ac1ad040 con 0x55c70c2d1a60 2024-09-12T01:35:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.301+0000 7fd2aa000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c70c2cc9d0 0x55c70c2dd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.301+0000 7fd2ab400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c70c2cc090 0x55c70c2c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.301+0000 7fd2ab400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c70c2cc090 0x55c70c2c6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:37270/0 (socket says 172.21.15.1:37270) 2024-09-12T01:35:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.301+0000 7fd2ab400640 1 -- 172.21.15.1:0/1791615893 learned_addr learned my addr 172.21.15.1:0/1791615893 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:35:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.301+0000 7fd2aa000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c70c2cc9d0 0x55c70c2dd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43502/0 (socket says 172.21.15.1:43502) 2024-09-12T01:35:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.301+0000 7fd2ab400640 1 --2- 172.21.15.1:0/1791615893 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c70c2d1a60 0x55c70c2c8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:09.308 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.302+0000 7fd2aa000640 1 -- 172.21.15.1:0/1791615893 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c70c2d1a60 msgr2=0x55c70c2c8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:35:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.302+0000 7fd2aa000640 1 --2- 172.21.15.1:0/1791615893 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c70c2d1a60 0x55c70c2c8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.302+0000 7fd2aa000640 1 -- 172.21.15.1:0/1791615893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c70c2cc090 msgr2=0x55c70c2c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:35:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.302+0000 7fd2aa000640 1 --2- 172.21.15.1:0/1791615893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c70c2cc090 0x55c70c2c6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.302+0000 7fd2aa000640 1 -- 172.21.15.1:0/1791615893 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c70c2c2d60 con 0x55c70c2cc9d0 2024-09-12T01:35:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.302+0000 7fd2ab400640 1 --2- 172.21.15.1:0/1791615893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c70c2cc090 0x55c70c2c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:35:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.302+0000 7fd2aa000640 1 --2- 172.21.15.1:0/1791615893 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c70c2cc9d0 0x55c70c2dd730 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd298005ec0 tx=0x7fd29802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d06baddb591ab8a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:35:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.302+0000 7fd297400640 1 -- 172.21.15.1:0/1791615893 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd298000e60 con 0x55c70c2cc9d0 2024-09-12T01:35:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.302+0000 7fd297400640 1 -- 172.21.15.1:0/1791615893 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd298041070 con 0x55c70c2cc9d0 2024-09-12T01:35:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.302+0000 7fd297400640 1 -- 172.21.15.1:0/1791615893 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd298035070 con 0x55c70c2cc9d0 2024-09-12T01:35:09.310 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.303+0000 7fd2b4bd3740 1 -- 172.21.15.1:0/1791615893 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c70c2cc9d0 msgr2=0x55c70c2dd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:09.310 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.303+0000 7fd2b4bd3740 1 --2- 172.21.15.1:0/1791615893 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c70c2cc9d0 0x55c70c2dd730 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd298005ec0 tx=0x7fd29802a410 comp rx=0 tx=0).stop 2024-09-12T01:35:09.310 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.303+0000 7fd2b4bd3740 1 -- 172.21.15.1:0/1791615893 shutdown_connections 2024-09-12T01:35:09.310 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.303+0000 7fd2b4bd3740 1 --2- 172.21.15.1:0/1791615893 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c70c2d1a60 0x55c70c2c8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:09.310 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.303+0000 7fd2b4bd3740 1 --2- 172.21.15.1:0/1791615893 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c70c2cc9d0 0x55c70c2dd730 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:09.310 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.303+0000 7fd2b4bd3740 1 --2- 172.21.15.1:0/1791615893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c70c2cc090 0x55c70c2c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:09.310 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.303+0000 7fd2b4bd3740 1 -- 172.21.15.1:0/1791615893 >> 172.21.15.1:0/1791615893 conn(0x55c70c2c7700 msgr2=0x55c70c2c6060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:35:09.310 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.303+0000 7fd2b4bd3740 1 -- 172.21.15.1:0/1791615893 shutdown_connections 2024-09-12T01:35:09.310 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.303+0000 7fd2b4bd3740 1 -- 172.21.15.1:0/1791615893 wait complete. 2024-09-12T01:35:09.310 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.304+0000 7fd2b4bd3740 1 -- start start 2024-09-12T01:35:09.310 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.304+0000 7fd2b4bd3740 1 Processor -- start 2024-09-12T01:35:09.310 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.305+0000 7fd2b4bd3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c70c2cc090 0x55c70c2dae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:09.310 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.305+0000 7fd2b4bd3740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c70c2cc9d0 0x55c70c2db360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.305+0000 7fd2b4bd3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c70c2d1a60 0x55c70c2dc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.305+0000 7fd2b4bd3740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ac1adde0 con 0x55c70c2cc9d0 2024-09-12T01:35:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.305+0000 7fd2b4bd3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ac1ad710 con 0x55c70c2cc090 2024-09-12T01:35:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.305+0000 7fd2b4bd3740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ac1aa950 con 0x55c70c2d1a60 2024-09-12T01:35:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.305+0000 7fd2b4bd3740 10 client.0 _ll_register_callbacks cb 0x7fd298009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:35:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.305+0000 7fd2ab400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c70c2cc090 0x55c70c2dae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.305+0000 7fd2ab400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c70c2cc090 0x55c70c2dae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:37284/0 (socket says 172.21.15.1:37284) 2024-09-12T01:35:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.305+0000 7fd2ab400640 1 -- 172.21.15.1:0/1746314833 learned_addr learned my addr 172.21.15.1:0/1746314833 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:35:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.305+0000 7fd2ab400640 1 --2- 172.21.15.1:0/1746314833 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c70c2d1a60 0x55c70c2dc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.305+0000 7fd2aa000640 1 --2- 172.21.15.1:0/1746314833 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c70c2cc9d0 0x55c70c2db360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.305+0000 7fd2ab400640 1 -- 172.21.15.1:0/1746314833 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c70c2d1a60 msgr2=0x55c70c2dc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:35:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.305+0000 7fd2ab400640 1 --2- 172.21.15.1:0/1746314833 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c70c2d1a60 0x55c70c2dc370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.305+0000 7fd2ab400640 1 -- 172.21.15.1:0/1746314833 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c70c2cc9d0 msgr2=0x55c70c2db360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:35:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.305+0000 7fd2ab400640 1 --2- 172.21.15.1:0/1746314833 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c70c2cc9d0 0x55c70c2db360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.305+0000 7fd2ab400640 1 -- 172.21.15.1:0/1746314833 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c70c2dc980 con 0x55c70c2cc090 2024-09-12T01:35:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.306+0000 7fd2ab400640 1 --2- 172.21.15.1:0/1746314833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c70c2cc090 0x55c70c2dae20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4011dd0 tx=0x7fd2a4009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b974f120733c137 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:35:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.306+0000 7fd294c00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-12T01:35:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.306+0000 7fd294c00640 1 -- 172.21.15.1:0/1746314833 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a400fa50 con 0x55c70c2cc090 2024-09-12T01:35:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.306+0000 7fd2b4bd3740 1 -- 172.21.15.1:0/1746314833 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd298041070 con 0x55c70c2cc090 2024-09-12T01:35:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.306+0000 7fd294c00640 1 -- 172.21.15.1:0/1746314833 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fd2a400fbf0 con 0x55c70c2cc090 2024-09-12T01:35:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.306+0000 7fd294c00640 1 -- 172.21.15.1:0/1746314833 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd2a4045050 con 0x55c70c2cc090 2024-09-12T01:35:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.306+0000 7fd2b4bd3740 10 client.25634 fetch_fsmap learned FSMap version 35 2024-09-12T01:35:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.306+0000 7fd2b4bd3740 1 -- 172.21.15.1:0/1746314833 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c70c34c910 con 0x55c70c2cc090 2024-09-12T01:35:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.306+0000 7fd294c00640 1 -- 172.21.15.1:0/1746314833 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd298041070 con 0x55c70c2cc090 2024-09-12T01:35:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.306+0000 7fd294c00640 1 -- 172.21.15.1:0/1746314833 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fd2a4025040 con 0x55c70c2cc090 2024-09-12T01:35:09.313 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.306+0000 7fd2b4bd3740 10 client.25634 fetch_fsmap finished waiting for FSMap version 35 2024-09-12T01:35:09.313 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.306+0000 7fd2b4bd3740 10 client.25634 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:35:09.313 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.306+0000 7fd2b4bd3740 20 client.25634 populate_metadata read hostname 'smithi001' 2024-09-12T01:35:09.313 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.307+0000 7fd2b4bd3740 1 --2- 172.21.15.1:0/1746314833 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55c70c34d080 0x55c70c34f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:09.313 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.307+0000 7fd2b4bd3740 4 client.25634 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:35:09.313 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.307+0000 7fd2b4bd3740 1 -- 172.21.15.1:0/1746314833 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c70c238b80 con 0x55c70c34d080 2024-09-12T01:35:09.313 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.307+0000 7fd2aa000640 1 --2- 172.21.15.1:0/1746314833 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55c70c34d080 0x55c70c34f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:09.313 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.307+0000 7fd2aa000640 1 --2- 172.21.15.1:0/1746314833 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55c70c34d080 0x55c70c34f4a0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d47134de56ca8e38 server_cookie=cdbdf04baae17bda in_seq=0 out_seq=0 2024-09-12T01:35:09.313 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.307+0000 7fd294c00640 10 client.25634 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:35:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.315+0000 7fd294c00640 1 -- 172.21.15.1:0/1746314833 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c70c238b80 con 0x55c70c34d080 2024-09-12T01:35:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.315+0000 7fd294c00640 10 client.25634 handle_command_reply: tid=0 2024-09-12T01:35:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.315+0000 7fd2b4bd3740 1 client.25634 shutdown 2024-09-12T01:35:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.316+0000 7fd2b4bd3740 1 -- 172.21.15.1:0/1746314833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c70c2cc090 msgr2=0x55c70c2dae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.316+0000 7fd2b4bd3740 1 --2- 172.21.15.1:0/1746314833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c70c2cc090 0x55c70c2dae20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4011dd0 tx=0x7fd2a4009500 comp rx=0 tx=0).stop 2024-09-12T01:35:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.316+0000 7fd2b4bd3740 1 -- 172.21.15.1:0/1746314833 shutdown_connections 2024-09-12T01:35:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.316+0000 7fd2b4bd3740 1 --2- 172.21.15.1:0/1746314833 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55c70c34d080 0x55c70c34f4a0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.316+0000 7fd2b4bd3740 1 --2- 172.21.15.1:0/1746314833 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c70c2d1a60 0x55c70c2dc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.316+0000 7fd2b4bd3740 1 --2- 172.21.15.1:0/1746314833 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c70c2cc9d0 0x55c70c2db360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.316+0000 7fd294c00640 0 client.25634 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:35:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.316+0000 7fd2b4bd3740 1 --2- 172.21.15.1:0/1746314833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c70c2cc090 0x55c70c2dae20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.316+0000 7fd2b4bd3740 1 -- 172.21.15.1:0/1746314833 >> 172.21.15.1:0/1746314833 conn(0x55c70c2c7700 msgr2=0x55c70c2c6880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:35:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.316+0000 7fd2b4bd3740 1 -- 172.21.15.1:0/1746314833 shutdown_connections 2024-09-12T01:35:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.316+0000 7fd2b4bd3740 1 -- 172.21.15.1:0/1746314833 wait complete. 2024-09-12T01:35:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.316+0000 7fd2b4bd3740 20 client.25634 trim_cache size 0 max 16384 2024-09-12T01:35:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-12T01:35:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-12T01:35:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-12T01:35:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-12T01:35:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.317+0000 7fd2b1c00640 1 -- 172.21.15.1:0/3237380330 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fd2900a2c00 msgr2=0x7fd2900a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.317+0000 7fd2b1c00640 1 --2- 172.21.15.1:0/3237380330 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fd2900a2c00 0x7fd2900a50c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd29c0ccff0 tx=0x7fd29c00efb0 comp rx=0 tx=0).stop 2024-09-12T01:35:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.317+0000 7fd2b1c00640 1 -- 172.21.15.1:0/3237380330 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2ac103d50 msgr2=0x7fd2ac19a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.317+0000 7fd2b1c00640 1 --2- 172.21.15.1:0/3237380330 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2ac103d50 0x7fd2ac19a030 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd29c00e9c0 tx=0x7fd29c00ee80 comp rx=0 tx=0).stop 2024-09-12T01:35:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.317+0000 7fd2b1c00640 1 -- 172.21.15.1:0/3237380330 shutdown_connections 2024-09-12T01:35:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.317+0000 7fd2b1c00640 1 --2- 172.21.15.1:0/3237380330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2ac105900 0x7fd2ac1a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.317+0000 7fd2b1c00640 1 --2- 172.21.15.1:0/3237380330 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fd2900a2c00 0x7fd2900a50c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.317+0000 7fd2b1c00640 1 --2- 172.21.15.1:0/3237380330 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd2ac104f40 0x7fd2ac19a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.317+0000 7fd2b1c00640 1 --2- 172.21.15.1:0/3237380330 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2ac103d50 0x7fd2ac19a030 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.317+0000 7fd2b1c00640 1 -- 172.21.15.1:0/3237380330 >> 172.21.15.1:0/3237380330 conn(0x7fd2ac0ff4b0 msgr2=0x7fd2ac1082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:35:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.317+0000 7fd2b1c00640 1 -- 172.21.15.1:0/3237380330 shutdown_connections 2024-09-12T01:35:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:09.317+0000 7fd2b1c00640 1 -- 172.21.15.1:0/3237380330 wait complete. 2024-09-12T01:35:10.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:35:09 smithi053 ceph-mon[35158]: pgmap v565: 129 pgs: 129 active+clean; 117 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 980 KiB/s wr, 387 op/s 2024-09-12T01:35:10.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:35:09 smithi104 ceph-mon[35219]: pgmap v565: 129 pgs: 129 active+clean; 117 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 980 KiB/s wr, 387 op/s 2024-09-12T01:35:10.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:35:09 smithi001 ceph-mon[30387]: pgmap v565: 129 pgs: 129 active+clean; 117 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 980 KiB/s wr, 387 op/s 2024-09-12T01:35:11.390 INFO:tasks.workunit.client.0.smithi001.stdout:20000 read 0xa6cbf1 thru 0xa6e358 (0x1768 bytes) 2024-09-12T01:35:12.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:35:11 smithi053 ceph-mon[35158]: pgmap v566: 129 pgs: 129 active+clean; 115 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.3 MiB/s wr, 516 op/s 2024-09-12T01:35:12.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:35:11 smithi104 ceph-mon[35219]: pgmap v566: 129 pgs: 129 active+clean; 115 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.3 MiB/s wr, 516 op/s 2024-09-12T01:35:12.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:35:11 smithi001 ceph-mon[30387]: pgmap v566: 129 pgs: 129 active+clean; 115 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.3 MiB/s wr, 516 op/s 2024-09-12T01:35:14.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:35:13 smithi053 ceph-mon[35158]: pgmap v567: 129 pgs: 129 active+clean; 118 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.0 MiB/s wr, 443 op/s 2024-09-12T01:35:14.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:35:13 smithi104 ceph-mon[35219]: pgmap v567: 129 pgs: 129 active+clean; 118 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.0 MiB/s wr, 443 op/s 2024-09-12T01:35:14.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:35:13 smithi001 ceph-mon[30387]: pgmap v567: 129 pgs: 129 active+clean; 118 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.0 MiB/s wr, 443 op/s 2024-09-12T01:35:15.585 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0x63ed6b9 2024-09-12T01:35:16.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:35:15 smithi053 ceph-mon[35158]: pgmap v568: 129 pgs: 129 active+clean; 119 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.1 MiB/s wr, 457 op/s 2024-09-12T01:35:16.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:35:15 smithi104 ceph-mon[35219]: pgmap v568: 129 pgs: 129 active+clean; 119 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.1 MiB/s wr, 457 op/s 2024-09-12T01:35:16.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:35:15 smithi001 ceph-mon[30387]: pgmap v568: 129 pgs: 129 active+clean; 119 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.1 MiB/s wr, 457 op/s 2024-09-12T01:35:18.230 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:35:17 smithi001 ceph-mon[30387]: pgmap v569: 129 pgs: 129 active+clean; 114 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 1.2 MiB/s wr, 499 op/s 2024-09-12T01:35:18.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:35:17 smithi053 ceph-mon[35158]: pgmap v569: 129 pgs: 129 active+clean; 114 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 1.2 MiB/s wr, 499 op/s 2024-09-12T01:35:18.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:35:17 smithi104 ceph-mon[35219]: pgmap v569: 129 pgs: 129 active+clean; 114 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 1.2 MiB/s wr, 499 op/s 2024-09-12T01:35:20.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:35:19 smithi053 ceph-mon[35158]: pgmap v570: 129 pgs: 129 active+clean; 114 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 933 KiB/s wr, 381 op/s 2024-09-12T01:35:20.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:35:19 smithi104 ceph-mon[35219]: pgmap v570: 129 pgs: 129 active+clean; 114 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 933 KiB/s wr, 381 op/s 2024-09-12T01:35:20.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:35:19 smithi001 ceph-mon[30387]: pgmap v570: 129 pgs: 129 active+clean; 114 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 933 KiB/s wr, 381 op/s 2024-09-12T01:35:22.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:35:21 smithi053 ceph-mon[35158]: pgmap v571: 129 pgs: 129 active+clean; 111 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 1.3 MiB/s wr, 502 op/s 2024-09-12T01:35:22.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:35:21 smithi104 ceph-mon[35219]: pgmap v571: 129 pgs: 129 active+clean; 111 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 1.3 MiB/s wr, 502 op/s 2024-09-12T01:35:22.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:35:21 smithi001 ceph-mon[30387]: pgmap v571: 129 pgs: 129 active+clean; 111 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 1.3 MiB/s wr, 502 op/s 2024-09-12T01:35:22.453 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.453+0000 7f634b600640 1 -- 172.21.15.1:0/2772506851 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f63441025c0 msgr2=0x7f63441049b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:22.453 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.453+0000 7f634b600640 1 --2- 172.21.15.1:0/2772506851 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f63441025c0 0x7f63441049b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f633800fb60 tx=0x7f633802e4b0 comp rx=0 tx=0).stop 2024-09-12T01:35:22.453 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.453+0000 7f634b600640 1 -- 172.21.15.1:0/2772506851 shutdown_connections 2024-09-12T01:35:22.453 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.453+0000 7f634b600640 1 --2- 172.21.15.1:0/2772506851 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6344107820 0x7f6344109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:22.453 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.453+0000 7f634b600640 1 --2- 172.21.15.1:0/2772506851 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6344104ef0 0x7f63441072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:22.453 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.453+0000 7f634b600640 1 --2- 172.21.15.1:0/2772506851 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f63441025c0 0x7f63441049b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:22.454 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.453+0000 7f634b600640 1 -- 172.21.15.1:0/2772506851 >> 172.21.15.1:0/2772506851 conn(0x7f63440fc1c0 msgr2=0x7f63440fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:35:22.454 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.453+0000 7f634b600640 1 -- 172.21.15.1:0/2772506851 shutdown_connections 2024-09-12T01:35:22.454 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.453+0000 7f634b600640 1 -- 172.21.15.1:0/2772506851 wait complete. 2024-09-12T01:35:22.454 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.455+0000 7f634b600640 1 Processor -- start 2024-09-12T01:35:22.454 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.455+0000 7f634b600640 1 -- start start 2024-09-12T01:35:22.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.455+0000 7f634b600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63441025c0 0x7f6344195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:22.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.455+0000 7f634b600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6344104ef0 0x7f6344196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:22.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.455+0000 7f634b600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6344107820 0x7f634419d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:22.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.455+0000 7f634b600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f634410b650 con 0x7f6344104ef0 2024-09-12T01:35:22.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.455+0000 7f634b600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f634410b4d0 con 0x7f63441025c0 2024-09-12T01:35:22.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.455+0000 7f634b600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f634410b7d0 con 0x7f6344107820 2024-09-12T01:35:22.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.455+0000 7f6348c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63441025c0 0x7f6344195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:22.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.455+0000 7f6348c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63441025c0 0x7f6344195d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:48912/0 (socket says 172.21.15.1:48912) 2024-09-12T01:35:22.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.456+0000 7f6348c00640 1 -- 172.21.15.1:0/270869493 learned_addr learned my addr 172.21.15.1:0/270869493 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:35:22.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.456+0000 7f6343e00640 1 --2- 172.21.15.1:0/270869493 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6344104ef0 0x7f6344196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:22.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.456+0000 7f6349600640 1 --2- 172.21.15.1:0/270869493 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6344107820 0x7f634419d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:22.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.456+0000 7f6348c00640 1 -- 172.21.15.1:0/270869493 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6344107820 msgr2=0x7f634419d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:22.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.456+0000 7f6348c00640 1 --2- 172.21.15.1:0/270869493 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6344107820 0x7f634419d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:22.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.456+0000 7f6348c00640 1 -- 172.21.15.1:0/270869493 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6344104ef0 msgr2=0x7f6344196260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:22.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.456+0000 7f6348c00640 1 --2- 172.21.15.1:0/270869493 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6344104ef0 0x7f6344196260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:22.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.456+0000 7f6348c00640 1 -- 172.21.15.1:0/270869493 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f634419d930 con 0x7f63441025c0 2024-09-12T01:35:22.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.456+0000 7f6349600640 1 --2- 172.21.15.1:0/270869493 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6344107820 0x7f634419d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:35:22.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.456+0000 7f6348c00640 1 --2- 172.21.15.1:0/270869493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63441025c0 0x7f6344195d20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f633800fc80 tx=0x7f63380052f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:35:22.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.457+0000 7f6341600640 1 -- 172.21.15.1:0/270869493 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6338044070 con 0x7f63441025c0 2024-09-12T01:35:22.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.457+0000 7f634b600640 1 -- 172.21.15.1:0/270869493 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f634419dbc0 con 0x7f63441025c0 2024-09-12T01:35:22.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.457+0000 7f6341600640 1 -- 172.21.15.1:0/270869493 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63380358e0 con 0x7f63441025c0 2024-09-12T01:35:22.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.457+0000 7f6341600640 1 -- 172.21.15.1:0/270869493 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6338004ad0 con 0x7f63441025c0 2024-09-12T01:35:22.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.457+0000 7f634b600640 1 -- 172.21.15.1:0/270869493 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f634419e100 con 0x7f63441025c0 2024-09-12T01:35:22.460 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.460+0000 7f6341600640 1 -- 172.21.15.1:0/270869493 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f6338047070 con 0x7f63441025c0 2024-09-12T01:35:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.462+0000 7f634e64f740 1 Processor -- start 2024-09-12T01:35:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.462+0000 7f634e64f740 1 -- start start 2024-09-12T01:35:22.463 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.462+0000 7f634e64f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579f70ba090 0x5579f70ba490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:22.463 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.462+0000 7f6341600640 1 --2- 172.21.15.1:0/270869493 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f632c0a2c00 0x7f632c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:22.463 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.462+0000 7f634e64f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5579f70ba9d0 0x5579f70bf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:22.464 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.462+0000 7f634e64f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579f70bfa60 0x5579f70c1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:22.464 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.462+0000 7f634e64f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5579f701f100 con 0x5579f70ba9d0 2024-09-12T01:35:22.464 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.462+0000 7f634e64f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5579f6fc71b0 con 0x5579f70ba090 2024-09-12T01:35:22.464 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.462+0000 7f634e64f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5579f701b0c0 con 0x5579f70bfa60 2024-09-12T01:35:22.464 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.462+0000 7f6341600640 1 -- 172.21.15.1:0/270869493 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f63380f4420 con 0x7f63441025c0 2024-09-12T01:35:22.464 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.462+0000 7f6349600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579f70ba090 0x5579f70ba490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:22.464 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.462+0000 7f6349600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579f70ba090 0x5579f70ba490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:48928/0 (socket says 172.21.15.1:48928) 2024-09-12T01:35:22.464 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.462+0000 7f6349600640 1 -- 172.21.15.1:0/924109742 learned_addr learned my addr 172.21.15.1:0/924109742 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:35:22.464 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.462+0000 7f6348c00640 1 --2- 172.21.15.1:0/270869493 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f632c0a2c00 0x7f632c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:22.464 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.462+0000 7f6343e00640 1 --2- 172.21.15.1:0/924109742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5579f70ba9d0 0x5579f70bf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:22.464 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.462+0000 7f6349600640 1 --2- 172.21.15.1:0/924109742 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579f70bfa60 0x5579f70c1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:22.464 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.463+0000 7f6349600640 1 -- 172.21.15.1:0/924109742 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579f70bfa60 msgr2=0x5579f70c1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:35:22.464 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.463+0000 7f6349600640 1 --2- 172.21.15.1:0/924109742 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579f70bfa60 0x5579f70c1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:22.465 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.463+0000 7f6349600640 1 -- 172.21.15.1:0/924109742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5579f70ba9d0 msgr2=0x5579f70bf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:35:22.465 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.463+0000 7f6349600640 1 --2- 172.21.15.1:0/924109742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5579f70ba9d0 0x5579f70bf490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:22.465 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.463+0000 7f6349600640 1 -- 172.21.15.1:0/924109742 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579f6d32210 con 0x5579f70ba090 2024-09-12T01:35:22.465 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.463+0000 7f6343e00640 1 --2- 172.21.15.1:0/924109742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5579f70ba9d0 0x5579f70bf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:35:22.465 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.463+0000 7f6348c00640 1 --2- 172.21.15.1:0/270869493 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f632c0a2c00 0x7f632c0a50c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6338005820 tx=0x7f633802e4b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:35:22.465 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.463+0000 7f6349600640 1 --2- 172.21.15.1:0/924109742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579f70ba090 0x5579f70ba490 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f633c00e470 tx=0x7f633c0095e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4640c08a713fd632 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:35:22.465 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.463+0000 7f6335c00640 1 -- 172.21.15.1:0/924109742 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633c013030 con 0x5579f70ba090 2024-09-12T01:35:22.465 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.463+0000 7f6335c00640 1 -- 172.21.15.1:0/924109742 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f633c016050 con 0x5579f70ba090 2024-09-12T01:35:22.465 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.463+0000 7f6335c00640 1 -- 172.21.15.1:0/924109742 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633c018070 con 0x5579f70ba090 2024-09-12T01:35:22.468 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.464+0000 7f634e64f740 1 -- 172.21.15.1:0/924109742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579f70ba090 msgr2=0x5579f70ba490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:22.468 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.464+0000 7f634e64f740 1 --2- 172.21.15.1:0/924109742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579f70ba090 0x5579f70ba490 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f633c00e470 tx=0x7f633c0095e0 comp rx=0 tx=0).stop 2024-09-12T01:35:22.468 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.465+0000 7f634e64f740 1 -- 172.21.15.1:0/924109742 shutdown_connections 2024-09-12T01:35:22.468 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.465+0000 7f634e64f740 1 --2- 172.21.15.1:0/924109742 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579f70bfa60 0x5579f70c1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:22.468 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.465+0000 7f634e64f740 1 --2- 172.21.15.1:0/924109742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5579f70ba9d0 0x5579f70bf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:22.468 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.465+0000 7f634e64f740 1 --2- 172.21.15.1:0/924109742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579f70ba090 0x5579f70ba490 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:22.468 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.465+0000 7f634e64f740 1 -- 172.21.15.1:0/924109742 >> 172.21.15.1:0/924109742 conn(0x5579f70b5700 msgr2=0x5579f70b5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:35:22.468 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.465+0000 7f634e64f740 1 -- 172.21.15.1:0/924109742 shutdown_connections 2024-09-12T01:35:22.468 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.465+0000 7f634e64f740 1 -- 172.21.15.1:0/924109742 wait complete. 2024-09-12T01:35:22.468 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.465+0000 7f634e64f740 1 -- start start 2024-09-12T01:35:22.469 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.466+0000 7f634e64f740 1 Processor -- start 2024-09-12T01:35:22.469 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.466+0000 7f634e64f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579f70ba090 0x5579f70c4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:22.469 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.466+0000 7f634e64f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579f70ba9d0 0x5579f70c47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:22.469 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.466+0000 7f634e64f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5579f70bfa60 0x5579f70cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:22.469 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.466+0000 7f634e64f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5579f70c2a30 con 0x5579f70bfa60 2024-09-12T01:35:22.469 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.466+0000 7f634e64f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5579f6fa8090 con 0x5579f70ba9d0 2024-09-12T01:35:22.469 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.466+0000 7f634e64f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5579f70c2bb0 con 0x5579f70ba090 2024-09-12T01:35:22.469 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.466+0000 7f634e64f740 10 client.0 _ll_register_callbacks cb 0x5579f6ffcf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:35:22.469 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.466+0000 7f6343e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579f70ba9d0 0x5579f70c47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:22.469 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.466+0000 7f6349600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579f70ba090 0x5579f70c4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:22.469 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.467+0000 7f6343e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579f70ba9d0 0x5579f70c47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:48944/0 (socket says 172.21.15.1:48944) 2024-09-12T01:35:22.469 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.467+0000 7f6343e00640 1 -- 172.21.15.1:0/3847057641 learned_addr learned my addr 172.21.15.1:0/3847057641 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:35:22.469 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.467+0000 7f6349600640 1 --2- 172.21.15.1:0/3847057641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5579f70bfa60 0x5579f70cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:22.469 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.467+0000 7f6343e00640 1 -- 172.21.15.1:0/3847057641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579f70ba090 msgr2=0x5579f70c4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:35:22.470 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.467+0000 7f6343e00640 1 --2- 172.21.15.1:0/3847057641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579f70ba090 0x5579f70c4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:22.470 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.467+0000 7f6343e00640 1 -- 172.21.15.1:0/3847057641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5579f70bfa60 msgr2=0x5579f70cb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:35:22.470 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.467+0000 7f6343e00640 1 --2- 172.21.15.1:0/3847057641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5579f70bfa60 0x5579f70cb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:22.470 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.467+0000 7f6343e00640 1 -- 172.21.15.1:0/3847057641 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579f6dfe620 con 0x5579f70ba9d0 2024-09-12T01:35:22.470 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.467+0000 7f6343e00640 1 --2- 172.21.15.1:0/3847057641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579f70ba9d0 0x5579f70c47b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f633000b990 tx=0x7f633000be50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=294fd5cfcb8182f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:35:22.470 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.467+0000 7f632a000640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-12T01:35:22.470 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.467+0000 7f632a000640 1 -- 172.21.15.1:0/3847057641 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6330011440 con 0x5579f70ba9d0 2024-09-12T01:35:22.470 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.467+0000 7f634e64f740 1 -- 172.21.15.1:0/3847057641 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f633c016050 con 0x5579f70ba9d0 2024-09-12T01:35:22.470 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.467+0000 7f632a000640 1 -- 172.21.15.1:0/3847057641 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f63300115e0 con 0x5579f70ba9d0 2024-09-12T01:35:22.470 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.468+0000 7f632a000640 1 -- 172.21.15.1:0/3847057641 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f63300117e0 con 0x5579f70ba9d0 2024-09-12T01:35:22.470 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.468+0000 7f634e64f740 10 client.25646 fetch_fsmap learned FSMap version 35 2024-09-12T01:35:22.470 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.468+0000 7f634e64f740 1 -- 172.21.15.1:0/3847057641 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5579f70ce6b0 con 0x5579f70ba9d0 2024-09-12T01:35:22.470 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.468+0000 7f632a000640 1 -- 172.21.15.1:0/3847057641 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f633c016050 con 0x5579f70ba9d0 2024-09-12T01:35:22.473 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.468+0000 7f632a000640 1 -- 172.21.15.1:0/3847057641 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f633001b9f0 con 0x5579f70ba9d0 2024-09-12T01:35:22.473 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.469+0000 7f634e64f740 10 client.25646 fetch_fsmap finished waiting for FSMap version 35 2024-09-12T01:35:22.473 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.469+0000 7f634e64f740 10 client.25646 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:35:22.473 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.469+0000 7f634e64f740 20 client.25646 populate_metadata read hostname 'smithi001' 2024-09-12T01:35:22.473 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.469+0000 7f634e64f740 1 --2- 172.21.15.1:0/3847057641 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5579f70cfdf0 0x5579f70d2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:22.473 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.469+0000 7f634e64f740 4 client.25646 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:35:22.473 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.469+0000 7f634e64f740 1 -- 172.21.15.1:0/3847057641 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579f702c830 con 0x5579f70cfdf0 2024-09-12T01:35:22.473 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.469+0000 7f6349600640 1 --2- 172.21.15.1:0/3847057641 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5579f70cfdf0 0x5579f70d2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:22.473 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.470+0000 7f6349600640 1 --2- 172.21.15.1:0/3847057641 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5579f70cfdf0 0x5579f70d2230 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57c67cf0f150d14c server_cookie=d042e5b26ee42336 in_seq=0 out_seq=0 2024-09-12T01:35:22.474 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.470+0000 7f632a000640 10 client.25646 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:35:22.474 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.471+0000 7f632a000640 1 -- 172.21.15.1:0/3847057641 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5579f702c830 con 0x5579f70cfdf0 2024-09-12T01:35:22.474 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.471+0000 7f632a000640 10 client.25646 handle_command_reply: tid=0 2024-09-12T01:35:22.474 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.471+0000 7f634e64f740 1 client.25646 shutdown 2024-09-12T01:35:22.474 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.471+0000 7f634e64f740 1 -- 172.21.15.1:0/3847057641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579f70ba9d0 msgr2=0x5579f70c47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:22.474 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.471+0000 7f634e64f740 1 --2- 172.21.15.1:0/3847057641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579f70ba9d0 0x5579f70c47b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f633000b990 tx=0x7f633000be50 comp rx=0 tx=0).stop 2024-09-12T01:35:22.474 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.471+0000 7f634e64f740 1 -- 172.21.15.1:0/3847057641 shutdown_connections 2024-09-12T01:35:22.474 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.471+0000 7f634e64f740 1 --2- 172.21.15.1:0/3847057641 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5579f70cfdf0 0x5579f70d2230 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:22.474 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.471+0000 7f634e64f740 1 --2- 172.21.15.1:0/3847057641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5579f70bfa60 0x5579f70cb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:22.474 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.471+0000 7f634e64f740 1 --2- 172.21.15.1:0/3847057641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579f70ba9d0 0x5579f70c47b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:22.474 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.471+0000 7f632a000640 0 client.25646 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:35:22.474 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.471+0000 7f634e64f740 1 --2- 172.21.15.1:0/3847057641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579f70ba090 0x5579f70c4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:22.474 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.471+0000 7f634e64f740 1 -- 172.21.15.1:0/3847057641 >> 172.21.15.1:0/3847057641 conn(0x5579f70b5700 msgr2=0x5579f70b6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:35:22.474 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.471+0000 7f634e64f740 1 -- 172.21.15.1:0/3847057641 shutdown_connections 2024-09-12T01:35:22.475 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.471+0000 7f634e64f740 1 -- 172.21.15.1:0/3847057641 wait complete. 2024-09-12T01:35:22.475 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.471+0000 7f634e64f740 20 client.25646 trim_cache size 0 max 16384 2024-09-12T01:35:22.496 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.496+0000 7f634e64f740 1 Processor -- start 2024-09-12T01:35:22.496 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.497+0000 7f634e64f740 1 -- start start 2024-09-12T01:35:22.497 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.497+0000 7f634e64f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579f70b5700 0x5579f70b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:22.497 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.497+0000 7f634e64f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5579f70ba090 0x5579f70cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:22.497 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.497+0000 7f634e64f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579f70ba9d0 0x5579f70b7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:22.497 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.497+0000 7f634e64f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f63441a8630 con 0x5579f70ba090 2024-09-12T01:35:22.497 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.497+0000 7f634e64f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f63441aa170 con 0x5579f70b5700 2024-09-12T01:35:22.497 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.497+0000 7f634e64f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f63441a8d00 con 0x5579f70ba9d0 2024-09-12T01:35:22.501 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.498+0000 7f6349600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579f70ba9d0 0x5579f70b7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:22.501 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.498+0000 7f6343e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5579f70ba090 0x5579f70cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:22.501 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.498+0000 7f6349600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579f70ba9d0 0x5579f70b7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:40662/0 (socket says 172.21.15.1:40662) 2024-09-12T01:35:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.498+0000 7f6349600640 1 -- 172.21.15.1:0/4287643600 learned_addr learned my addr 172.21.15.1:0/4287643600 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:35:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.498+0000 7f6349600640 1 --2- 172.21.15.1:0/4287643600 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579f70b5700 0x5579f70b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.498+0000 7f6349600640 1 -- 172.21.15.1:0/4287643600 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579f70b5700 msgr2=0x5579f70b6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:35:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.498+0000 7f6349600640 1 --2- 172.21.15.1:0/4287643600 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579f70b5700 0x5579f70b6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.498+0000 7f6349600640 1 -- 172.21.15.1:0/4287643600 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5579f70ba090 msgr2=0x5579f70cb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:35:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.498+0000 7f6349600640 1 --2- 172.21.15.1:0/4287643600 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5579f70ba090 0x5579f70cb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.499+0000 7f6349600640 1 -- 172.21.15.1:0/4287643600 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579f70b5520 con 0x5579f70ba9d0 2024-09-12T01:35:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.499+0000 7f6343e00640 1 --2- 172.21.15.1:0/4287643600 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5579f70ba090 0x5579f70cb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:35:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.499+0000 7f6349600640 1 --2- 172.21.15.1:0/4287643600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579f70ba9d0 0x5579f70b7030 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f633c00e440 tx=0x7f633c00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae62aa7ce5c5ff33 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:35:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.499+0000 7f6335200640 1 -- 172.21.15.1:0/4287643600 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633c018030 con 0x5579f70ba9d0 2024-09-12T01:35:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.500+0000 7f6335200640 1 -- 172.21.15.1:0/4287643600 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f633c016070 con 0x5579f70ba9d0 2024-09-12T01:35:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.500+0000 7f6335200640 1 -- 172.21.15.1:0/4287643600 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633c028040 con 0x5579f70ba9d0 2024-09-12T01:35:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.502+0000 7f634e64f740 1 -- 172.21.15.1:0/4287643600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579f70ba9d0 msgr2=0x5579f70b7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.502+0000 7f634e64f740 1 --2- 172.21.15.1:0/4287643600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579f70ba9d0 0x5579f70b7030 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f633c00e440 tx=0x7f633c00f690 comp rx=0 tx=0).stop 2024-09-12T01:35:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.503+0000 7f634e64f740 1 -- 172.21.15.1:0/4287643600 shutdown_connections 2024-09-12T01:35:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.503+0000 7f634e64f740 1 --2- 172.21.15.1:0/4287643600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579f70ba9d0 0x5579f70b7030 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.503+0000 7f634e64f740 1 --2- 172.21.15.1:0/4287643600 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5579f70ba090 0x5579f70cb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.503+0000 7f634e64f740 1 --2- 172.21.15.1:0/4287643600 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579f70b5700 0x5579f70b6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.503+0000 7f634e64f740 1 -- 172.21.15.1:0/4287643600 >> 172.21.15.1:0/4287643600 conn(0x5579f70bf5f0 msgr2=0x5579f70b4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:35:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.504+0000 7f634e64f740 1 -- 172.21.15.1:0/4287643600 shutdown_connections 2024-09-12T01:35:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.504+0000 7f634e64f740 1 -- 172.21.15.1:0/4287643600 wait complete. 2024-09-12T01:35:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.504+0000 7f634e64f740 1 -- start start 2024-09-12T01:35:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.505+0000 7f634e64f740 1 Processor -- start 2024-09-12T01:35:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.505+0000 7f634e64f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5579f70b5700 0x5579f70c9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.505+0000 7f634e64f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579f70ba090 0x5579f70c99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.506+0000 7f634e64f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579f70ba9d0 0x5579f70ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.506+0000 7f634e64f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f63441a9aa0 con 0x5579f70b5700 2024-09-12T01:35:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.506+0000 7f634e64f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f63441a93d0 con 0x5579f70ba090 2024-09-12T01:35:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.506+0000 7f634e64f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f63441a6610 con 0x5579f70ba9d0 2024-09-12T01:35:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.506+0000 7f634e64f740 10 client.0 _ll_register_callbacks cb 0x7f63300042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:35:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.506+0000 7f6349600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5579f70b5700 0x5579f70c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.507+0000 7f6349600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5579f70b5700 0x5579f70c9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48124/0 (socket says 172.21.15.1:48124) 2024-09-12T01:35:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.507+0000 7f6349600640 1 -- 172.21.15.1:0/3320514196 learned_addr learned my addr 172.21.15.1:0/3320514196 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:35:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.507+0000 7f6349600640 1 --2- 172.21.15.1:0/3320514196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579f70ba9d0 0x5579f70ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.507+0000 7f6343e00640 1 --2- 172.21.15.1:0/3320514196 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579f70ba090 0x5579f70c99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.507+0000 7f6349600640 1 -- 172.21.15.1:0/3320514196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579f70ba9d0 msgr2=0x5579f70ca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:35:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.507+0000 7f6349600640 1 --2- 172.21.15.1:0/3320514196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579f70ba9d0 0x5579f70ca9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.507+0000 7f6349600640 1 -- 172.21.15.1:0/3320514196 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579f70ba090 msgr2=0x5579f70c99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:35:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.507+0000 7f6349600640 1 --2- 172.21.15.1:0/3320514196 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579f70ba090 0x5579f70c99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.507+0000 7f6349600640 1 -- 172.21.15.1:0/3320514196 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579f70cafc0 con 0x5579f70b5700 2024-09-12T01:35:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.508+0000 7f6349600640 1 --2- 172.21.15.1:0/3320514196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5579f70b5700 0x5579f70c9460 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f633c00f5e0 tx=0x7f633c009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fab09af0d65ab4f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:35:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.508+0000 7f632a000640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-12T01:35:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.508+0000 7f632a000640 1 -- 172.21.15.1:0/3320514196 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633c032070 con 0x5579f70b5700 2024-09-12T01:35:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.508+0000 7f632a000640 1 -- 172.21.15.1:0/3320514196 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f633c0050b0 con 0x5579f70b5700 2024-09-12T01:35:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.508+0000 7f634e64f740 1 -- 172.21.15.1:0/3320514196 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f633c016070 con 0x5579f70b5700 2024-09-12T01:35:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.509+0000 7f632a000640 1 -- 172.21.15.1:0/3320514196 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f633c040050 con 0x5579f70b5700 2024-09-12T01:35:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.511+0000 7f634e64f740 10 client.15603 fetch_fsmap learned FSMap version 35 2024-09-12T01:35:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.511+0000 7f634e64f740 1 -- 172.21.15.1:0/3320514196 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5579f7154910 con 0x5579f70b5700 2024-09-12T01:35:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.511+0000 7f632a000640 1 -- 172.21.15.1:0/3320514196 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f633c016070 con 0x5579f70b5700 2024-09-12T01:35:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.511+0000 7f632a000640 1 -- 172.21.15.1:0/3320514196 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f633c043020 con 0x5579f70b5700 2024-09-12T01:35:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.511+0000 7f634e64f740 10 client.15603 fetch_fsmap finished waiting for FSMap version 35 2024-09-12T01:35:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.511+0000 7f634e64f740 10 client.15603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:35:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.511+0000 7f634e64f740 20 client.15603 populate_metadata read hostname 'smithi001' 2024-09-12T01:35:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.511+0000 7f634e64f740 1 --2- 172.21.15.1:0/3320514196 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5579f70bfa60 0x5579f7157070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.511+0000 7f634e64f740 4 client.15603 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:35:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.511+0000 7f634e64f740 1 -- 172.21.15.1:0/3320514196 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5579f7026b80 con 0x5579f70bfa60 2024-09-12T01:35:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.512+0000 7f6343e00640 1 --2- 172.21.15.1:0/3320514196 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5579f70bfa60 0x5579f7157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.512+0000 7f632a000640 10 client.15603 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:35:22.514 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.514+0000 7f6343e00640 1 --2- 172.21.15.1:0/3320514196 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5579f70bfa60 0x5579f7157070 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b203fbfaff1b0b58 server_cookie=5e9587b2dbab3a5c in_seq=0 out_seq=0 2024-09-12T01:35:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.523+0000 7f632a000640 1 -- 172.21.15.1:0/3320514196 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5579f7026b80 con 0x5579f70bfa60 2024-09-12T01:35:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.523+0000 7f632a000640 10 client.15603 handle_command_reply: tid=0 2024-09-12T01:35:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.523+0000 7f634e64f740 1 client.15603 shutdown 2024-09-12T01:35:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.524+0000 7f634e64f740 1 -- 172.21.15.1:0/3320514196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5579f70b5700 msgr2=0x5579f70c9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.524+0000 7f634e64f740 1 --2- 172.21.15.1:0/3320514196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5579f70b5700 0x5579f70c9460 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f633c00f5e0 tx=0x7f633c009860 comp rx=0 tx=0).stop 2024-09-12T01:35:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.524+0000 7f634e64f740 1 -- 172.21.15.1:0/3320514196 shutdown_connections 2024-09-12T01:35:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.524+0000 7f634e64f740 1 --2- 172.21.15.1:0/3320514196 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5579f70bfa60 0x5579f7157070 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.524+0000 7f634e64f740 1 --2- 172.21.15.1:0/3320514196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579f70ba9d0 0x5579f70ca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.524+0000 7f634e64f740 1 --2- 172.21.15.1:0/3320514196 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5579f70ba090 0x5579f70c99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.524+0000 7f634e64f740 1 --2- 172.21.15.1:0/3320514196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5579f70b5700 0x5579f70c9460 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.524+0000 7f634e64f740 1 -- 172.21.15.1:0/3320514196 >> 172.21.15.1:0/3320514196 conn(0x5579f70bf5f0 msgr2=0x5579f70b5ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:35:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.524+0000 7f632a000640 0 client.15603 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:35:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.524+0000 7f634e64f740 1 -- 172.21.15.1:0/3320514196 shutdown_connections 2024-09-12T01:35:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.524+0000 7f634e64f740 1 -- 172.21.15.1:0/3320514196 wait complete. 2024-09-12T01:35:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.524+0000 7f634e64f740 20 client.15603 trim_cache size 0 max 16384 2024-09-12T01:35:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-12T01:35:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-12T01:35:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-12T01:35:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-12T01:35:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.525+0000 7f634b600640 1 -- 172.21.15.1:0/270869493 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f632c0a2c00 msgr2=0x7f632c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.525+0000 7f634b600640 1 --2- 172.21.15.1:0/270869493 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f632c0a2c00 0x7f632c0a50c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6338005820 tx=0x7f633802e4b0 comp rx=0 tx=0).stop 2024-09-12T01:35:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.525+0000 7f634b600640 1 -- 172.21.15.1:0/270869493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63441025c0 msgr2=0x7f6344195d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.525+0000 7f634b600640 1 --2- 172.21.15.1:0/270869493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63441025c0 0x7f6344195d20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f633800fc80 tx=0x7f63380052f0 comp rx=0 tx=0).stop 2024-09-12T01:35:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.525+0000 7f634b600640 1 -- 172.21.15.1:0/270869493 shutdown_connections 2024-09-12T01:35:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.525+0000 7f634b600640 1 --2- 172.21.15.1:0/270869493 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f632c0a2c00 0x7f632c0a50c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.525+0000 7f634b600640 1 --2- 172.21.15.1:0/270869493 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6344107820 0x7f634419d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.525+0000 7f634b600640 1 --2- 172.21.15.1:0/270869493 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6344104ef0 0x7f6344196260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.525+0000 7f634b600640 1 --2- 172.21.15.1:0/270869493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63441025c0 0x7f6344195d20 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.525+0000 7f634b600640 1 -- 172.21.15.1:0/270869493 >> 172.21.15.1:0/270869493 conn(0x7f63440fc1c0 msgr2=0x7f6344102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:35:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.525+0000 7f634b600640 1 -- 172.21.15.1:0/270869493 shutdown_connections 2024-09-12T01:35:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:22.525+0000 7f634b600640 1 -- 172.21.15.1:0/270869493 wait complete. 2024-09-12T01:35:24.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:35:23 smithi053 ceph-mon[35158]: pgmap v572: 129 pgs: 129 active+clean; 111 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 1.1 MiB/s wr, 428 op/s 2024-09-12T01:35:24.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:35:23 smithi104 ceph-mon[35219]: pgmap v572: 129 pgs: 129 active+clean; 111 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 1.1 MiB/s wr, 428 op/s 2024-09-12T01:35:24.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:35:23 smithi001 ceph-mon[30387]: pgmap v572: 129 pgs: 129 active+clean; 111 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 1.1 MiB/s wr, 428 op/s 2024-09-12T01:35:26.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:35:25 smithi053 ceph-mon[35158]: pgmap v573: 129 pgs: 129 active+clean; 112 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 1.2 MiB/s wr, 435 op/s 2024-09-12T01:35:26.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:35:25 smithi104 ceph-mon[35219]: pgmap v573: 129 pgs: 129 active+clean; 112 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 1.2 MiB/s wr, 435 op/s 2024-09-12T01:35:26.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:35:25 smithi001 ceph-mon[30387]: pgmap v573: 129 pgs: 129 active+clean; 112 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 1.2 MiB/s wr, 435 op/s 2024-09-12T01:35:28.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:35:27 smithi053 ceph-mon[35158]: pgmap v574: 129 pgs: 129 active+clean; 120 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 1.3 MiB/s wr, 470 op/s 2024-09-12T01:35:28.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:35:27 smithi104 ceph-mon[35219]: pgmap v574: 129 pgs: 129 active+clean; 120 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 1.3 MiB/s wr, 470 op/s 2024-09-12T01:35:28.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:35:27 smithi001 ceph-mon[30387]: pgmap v574: 129 pgs: 129 active+clean; 120 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 1.3 MiB/s wr, 470 op/s 2024-09-12T01:35:30.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:35:29 smithi053 ceph-mon[35158]: pgmap v575: 129 pgs: 129 active+clean; 120 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 1009 KiB/s wr, 353 op/s 2024-09-12T01:35:30.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:35:29 smithi104 ceph-mon[35219]: pgmap v575: 129 pgs: 129 active+clean; 120 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 1009 KiB/s wr, 353 op/s 2024-09-12T01:35:30.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:35:29 smithi001 ceph-mon[30387]: pgmap v575: 129 pgs: 129 active+clean; 120 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 1009 KiB/s wr, 353 op/s 2024-09-12T01:35:32.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:35:31 smithi053 ceph-mon[35158]: pgmap v576: 129 pgs: 129 active+clean; 119 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 1.3 MiB/s wr, 481 op/s 2024-09-12T01:35:32.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:35:31 smithi104 ceph-mon[35219]: pgmap v576: 129 pgs: 129 active+clean; 119 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 1.3 MiB/s wr, 481 op/s 2024-09-12T01:35:32.429 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:35:31 smithi001 ceph-mon[30387]: pgmap v576: 129 pgs: 129 active+clean; 119 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 1.3 MiB/s wr, 481 op/s 2024-09-12T01:35:34.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:35:33 smithi053 ceph-mon[35158]: pgmap v577: 129 pgs: 129 active+clean; 118 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 1001 KiB/s wr, 421 op/s 2024-09-12T01:35:34.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:35:33 smithi104 ceph-mon[35219]: pgmap v577: 129 pgs: 129 active+clean; 118 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 1001 KiB/s wr, 421 op/s 2024-09-12T01:35:34.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:35:33 smithi001 ceph-mon[30387]: pgmap v577: 129 pgs: 129 active+clean; 118 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 1001 KiB/s wr, 421 op/s 2024-09-12T01:35:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.668+0000 7f44aaa00640 1 -- 172.21.15.1:0/2488577943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44a4105760 msgr2=0x7f44a410bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.668+0000 7f44aaa00640 1 --2- 172.21.15.1:0/2488577943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44a4105760 0x7f44a410bff0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4498009900 tx=0x7f449802a400 comp rx=0 tx=0).stop 2024-09-12T01:35:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.668+0000 7f44aaa00640 1 -- 172.21.15.1:0/2488577943 shutdown_connections 2024-09-12T01:35:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.668+0000 7f44aaa00640 1 --2- 172.21.15.1:0/2488577943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44a4105760 0x7f44a410bff0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.668+0000 7f44aaa00640 1 --2- 172.21.15.1:0/2488577943 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44a4104da0 0x7f44a4105220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.668+0000 7f44aaa00640 1 --2- 172.21.15.1:0/2488577943 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44a4103bb0 0x7f44a4103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.668+0000 7f44aaa00640 1 -- 172.21.15.1:0/2488577943 >> 172.21.15.1:0/2488577943 conn(0x7f44a40ff310 msgr2=0x7f44a4101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:35:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.668+0000 7f44aaa00640 1 -- 172.21.15.1:0/2488577943 shutdown_connections 2024-09-12T01:35:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.668+0000 7f44aaa00640 1 -- 172.21.15.1:0/2488577943 wait complete. 2024-09-12T01:35:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.669+0000 7f44aaa00640 1 Processor -- start 2024-09-12T01:35:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.670+0000 7f44aaa00640 1 -- start start 2024-09-12T01:35:35.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.671+0000 7f44aaa00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44a4103bb0 0x7f44a4110d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:35.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.671+0000 7f44aaa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44a4104da0 0x7f44a4111250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:35.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.671+0000 7f44aaa00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44a4105760 0x7f44a41162b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:35.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.671+0000 7f44a9600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44a4103bb0 0x7f44a4110d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:35.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.671+0000 7f44aaa00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f44a4077990 con 0x7f44a4105760 2024-09-12T01:35:35.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.671+0000 7f44a9600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44a4103bb0 0x7f44a4110d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:46084/0 (socket says 172.21.15.1:46084) 2024-09-12T01:35:35.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.671+0000 7f44a9600640 1 -- 172.21.15.1:0/201623381 learned_addr learned my addr 172.21.15.1:0/201623381 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:35:35.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.671+0000 7f44aaa00640 1 -- 172.21.15.1:0/201623381 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f44a4077810 con 0x7f44a4104da0 2024-09-12T01:35:35.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.671+0000 7f44aaa00640 1 -- 172.21.15.1:0/201623381 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f44a4077b10 con 0x7f44a4103bb0 2024-09-12T01:35:35.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.671+0000 7f44a8c00640 1 --2- 172.21.15.1:0/201623381 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44a4104da0 0x7f44a4111250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:35.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.671+0000 7f44aa000640 1 --2- 172.21.15.1:0/201623381 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44a4105760 0x7f44a41162b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.672+0000 7f44a9600640 1 -- 172.21.15.1:0/201623381 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44a4104da0 msgr2=0x7f44a4111250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.672+0000 7f44a9600640 1 --2- 172.21.15.1:0/201623381 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44a4104da0 0x7f44a4111250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.672+0000 7f44a9600640 1 -- 172.21.15.1:0/201623381 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44a4105760 msgr2=0x7f44a41162b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.672+0000 7f44a9600640 1 --2- 172.21.15.1:0/201623381 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44a4105760 0x7f44a41162b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.672+0000 7f44a9600640 1 -- 172.21.15.1:0/201623381 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a4116920 con 0x7f44a4103bb0 2024-09-12T01:35:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.672+0000 7f44aa000640 1 --2- 172.21.15.1:0/201623381 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44a4105760 0x7f44a41162b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:35:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.673+0000 7f44a9600640 1 --2- 172.21.15.1:0/201623381 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44a4103bb0 0x7f44a4110d10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f449c00eb70 tx=0x7f449c00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:35:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.673+0000 7f44a2000640 1 -- 172.21.15.1:0/201623381 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449c003b70 con 0x7f44a4103bb0 2024-09-12T01:35:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.673+0000 7f44a2000640 1 -- 172.21.15.1:0/201623381 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f449c00ee70 con 0x7f44a4103bb0 2024-09-12T01:35:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.673+0000 7f44aaa00640 1 -- 172.21.15.1:0/201623381 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44a41aea10 con 0x7f44a4103bb0 2024-09-12T01:35:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.673+0000 7f44a2000640 1 -- 172.21.15.1:0/201623381 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449c010830 con 0x7f44a4103bb0 2024-09-12T01:35:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.673+0000 7f44aaa00640 1 -- 172.21.15.1:0/201623381 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44a41aeed0 con 0x7f44a4103bb0 2024-09-12T01:35:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.678+0000 7f44aeec5740 1 Processor -- start 2024-09-12T01:35:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.678+0000 7f44aeec5740 1 -- start start 2024-09-12T01:35:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.678+0000 7f44aeec5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdbcc9260 0x55ecdbcc9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.678+0000 7f44aeec5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ecdbcc9ba0 0x55ecdbcce660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.678+0000 7f44aeec5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ecdbccec30 0x55ecdbcd10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.678+0000 7f44aeec5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ecdbc2e100 con 0x55ecdbcc9ba0 2024-09-12T01:35:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.678+0000 7f44aeec5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ecdbbd61b0 con 0x55ecdbcc9260 2024-09-12T01:35:35.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.678+0000 7f44aeec5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ecdbc2a0c0 con 0x55ecdbccec30 2024-09-12T01:35:35.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.679+0000 7f44a2000640 1 -- 172.21.15.1:0/201623381 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f449c004020 con 0x7f44a4103bb0 2024-09-12T01:35:35.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.679+0000 7f44aa000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdbcc9260 0x55ecdbcc9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:35.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.680+0000 7f44aa000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdbcc9260 0x55ecdbcc9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:41210/0 (socket says 172.21.15.1:41210) 2024-09-12T01:35:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.680+0000 7f44aa000640 1 -- 172.21.15.1:0/4225449605 learned_addr learned my addr 172.21.15.1:0/4225449605 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:35:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.680+0000 7f44a8c00640 1 --2- 172.21.15.1:0/4225449605 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ecdbccec30 0x55ecdbcd10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.680+0000 7f44a8c00640 1 -- 172.21.15.1:0/4225449605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdbcc9260 msgr2=0x55ecdbcc9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:35:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.680+0000 7f44a8c00640 1 --2- 172.21.15.1:0/4225449605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdbcc9260 0x55ecdbcc9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.680+0000 7f44a8c00640 1 -- 172.21.15.1:0/4225449605 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ecdbcc9ba0 msgr2=0x55ecdbcce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:35:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.680+0000 7f44a8c00640 1 --2- 172.21.15.1:0/4225449605 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ecdbcc9ba0 0x55ecdbcce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.680+0000 7f44a8c00640 1 -- 172.21.15.1:0/4225449605 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecdb941210 con 0x55ecdbccec30 2024-09-12T01:35:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.681+0000 7f44a8c00640 1 --2- 172.21.15.1:0/4225449605 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ecdbccec30 0x55ecdbcd10f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f44900058c0 tx=0x7f449000f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55844cd5eb91162d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:35:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.681+0000 7f4496600640 1 -- 172.21.15.1:0/4225449605 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4490004020 con 0x55ecdbccec30 2024-09-12T01:35:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.681+0000 7f4496600640 1 -- 172.21.15.1:0/4225449605 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44900041c0 con 0x55ecdbccec30 2024-09-12T01:35:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.682+0000 7f4496600640 1 -- 172.21.15.1:0/4225449605 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4490018790 con 0x55ecdbccec30 2024-09-12T01:35:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.682+0000 7f44a2000640 1 --2- 172.21.15.1:0/201623381 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f448c0a2cd0 0x7f448c0a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.682+0000 7f44a2000640 1 -- 172.21.15.1:0/201623381 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f449c0cfab0 con 0x7f44a4103bb0 2024-09-12T01:35:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.682+0000 7f44aeec5740 1 -- 172.21.15.1:0/4225449605 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ecdbccec30 msgr2=0x55ecdbcd10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.683+0000 7f44aeec5740 1 --2- 172.21.15.1:0/4225449605 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ecdbccec30 0x55ecdbcd10f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f44900058c0 tx=0x7f449000f3c0 comp rx=0 tx=0).stop 2024-09-12T01:35:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.683+0000 7f44aa000640 1 --2- 172.21.15.1:0/201623381 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f448c0a2cd0 0x7f448c0a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.683+0000 7f44aa000640 1 --2- 172.21.15.1:0/201623381 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f448c0a2cd0 0x7f448c0a5190 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4498031440 tx=0x7f44980315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:35:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.684+0000 7f44aeec5740 1 -- 172.21.15.1:0/4225449605 shutdown_connections 2024-09-12T01:35:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.684+0000 7f44aeec5740 1 --2- 172.21.15.1:0/4225449605 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ecdbccec30 0x55ecdbcd10f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.684+0000 7f44aeec5740 1 --2- 172.21.15.1:0/4225449605 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ecdbcc9ba0 0x55ecdbcce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.684+0000 7f44aeec5740 1 --2- 172.21.15.1:0/4225449605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdbcc9260 0x55ecdbcc9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.684+0000 7f44aeec5740 1 -- 172.21.15.1:0/4225449605 >> 172.21.15.1:0/4225449605 conn(0x55ecdbcc48d0 msgr2=0x55ecdbcc4cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:35:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.684+0000 7f44aeec5740 1 -- 172.21.15.1:0/4225449605 shutdown_connections 2024-09-12T01:35:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.684+0000 7f44aeec5740 1 -- 172.21.15.1:0/4225449605 wait complete. 2024-09-12T01:35:35.686 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.684+0000 7f44aeec5740 1 -- start start 2024-09-12T01:35:35.686 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.685+0000 7f44aeec5740 1 Processor -- start 2024-09-12T01:35:35.686 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.685+0000 7f44aeec5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdbcc9260 0x55ecdbcd3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:35.686 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.685+0000 7f44aeec5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ecdbcc9ba0 0x55ecdbcd39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:35.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.685+0000 7f44aeec5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ecdbccec30 0x55ecdbcda9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:35.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.685+0000 7f44aeec5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ecdbcd1c00 con 0x55ecdbcc9ba0 2024-09-12T01:35:35.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.685+0000 7f44aeec5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ecdbbb7090 con 0x55ecdbcc9260 2024-09-12T01:35:35.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.685+0000 7f44aeec5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ecdbcd1d80 con 0x55ecdbccec30 2024-09-12T01:35:35.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.685+0000 7f44aeec5740 10 client.0 _ll_register_callbacks cb 0x55ecdbc0bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:35:35.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.685+0000 7f44a9600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdbcc9260 0x55ecdbcd3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:35.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.685+0000 7f44a9600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdbcc9260 0x55ecdbcd3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:41212/0 (socket says 172.21.15.1:41212) 2024-09-12T01:35:35.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.685+0000 7f44a9600640 1 -- 172.21.15.1:0/2491206254 learned_addr learned my addr 172.21.15.1:0/2491206254 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:35:35.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.686+0000 7f44aa000640 1 --2- 172.21.15.1:0/2491206254 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ecdbccec30 0x55ecdbcda9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:35.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.686+0000 7f44a9600640 1 -- 172.21.15.1:0/2491206254 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ecdbccec30 msgr2=0x55ecdbcda9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:35:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.686+0000 7f44a9600640 1 --2- 172.21.15.1:0/2491206254 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ecdbccec30 0x55ecdbcda9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.686+0000 7f44a9600640 1 -- 172.21.15.1:0/2491206254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ecdbcc9ba0 msgr2=0x55ecdbcd39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:35:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.686+0000 7f44a8c00640 1 --2- 172.21.15.1:0/2491206254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ecdbcc9ba0 0x55ecdbcd39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.686+0000 7f44a9600640 1 --2- 172.21.15.1:0/2491206254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ecdbcc9ba0 0x55ecdbcd39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.686+0000 7f44a9600640 1 -- 172.21.15.1:0/2491206254 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecdba0d620 con 0x55ecdbcc9260 2024-09-12T01:35:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.686+0000 7f44a8c00640 1 --2- 172.21.15.1:0/2491206254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ecdbcc9ba0 0x55ecdbcd39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:35:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.686+0000 7f44a9600640 1 --2- 172.21.15.1:0/2491206254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdbcc9260 0x55ecdbcd3470 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f449c098380 tx=0x7f449c0981c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a1ba76592a9ad0c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:35:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.686+0000 7f448aa00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-12T01:35:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.686+0000 7f448aa00640 1 -- 172.21.15.1:0/2491206254 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449c09b880 con 0x55ecdbcc9260 2024-09-12T01:35:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.686+0000 7f44aeec5740 1 -- 172.21.15.1:0/2491206254 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44900041c0 con 0x55ecdbcc9260 2024-09-12T01:35:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.686+0000 7f448aa00640 1 -- 172.21.15.1:0/2491206254 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f449c0a4700 con 0x55ecdbcc9260 2024-09-12T01:35:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.687+0000 7f448aa00640 1 -- 172.21.15.1:0/2491206254 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f449c09c620 con 0x55ecdbcc9260 2024-09-12T01:35:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.687+0000 7f44aeec5740 10 client.25664 fetch_fsmap learned FSMap version 35 2024-09-12T01:35:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.687+0000 7f44aeec5740 1 -- 172.21.15.1:0/2491206254 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ecdbcdd900 con 0x55ecdbcc9260 2024-09-12T01:35:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.687+0000 7f448aa00640 1 -- 172.21.15.1:0/2491206254 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44900041c0 con 0x55ecdbcc9260 2024-09-12T01:35:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.687+0000 7f448aa00640 1 -- 172.21.15.1:0/2491206254 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f449c09f030 con 0x55ecdbcc9260 2024-09-12T01:35:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.687+0000 7f44aeec5740 10 client.25664 fetch_fsmap finished waiting for FSMap version 35 2024-09-12T01:35:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.687+0000 7f44aeec5740 10 client.25664 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:35:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.687+0000 7f44aeec5740 20 client.25664 populate_metadata read hostname 'smithi001' 2024-09-12T01:35:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.687+0000 7f44aeec5740 1 --2- 172.21.15.1:0/2491206254 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55ecdbcdef30 0x55ecdbce1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.687+0000 7f44aeec5740 4 client.25664 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:35:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.687+0000 7f44aeec5740 1 -- 172.21.15.1:0/2491206254 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ecdbc3b830 con 0x55ecdbcdef30 2024-09-12T01:35:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.688+0000 7f44a8c00640 1 --2- 172.21.15.1:0/2491206254 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55ecdbcdef30 0x55ecdbce1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.688+0000 7f44a8c00640 1 --2- 172.21.15.1:0/2491206254 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55ecdbcdef30 0x55ecdbce1320 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcc335a8f4e7984 server_cookie=ebfa79d2e2c1f42f in_seq=0 out_seq=0 2024-09-12T01:35:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.688+0000 7f448aa00640 10 client.25664 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:35:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.689+0000 7f448aa00640 1 -- 172.21.15.1:0/2491206254 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ecdbc3b830 con 0x55ecdbcdef30 2024-09-12T01:35:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.689+0000 7f448aa00640 10 client.25664 handle_command_reply: tid=0 2024-09-12T01:35:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.689+0000 7f44aeec5740 1 client.25664 shutdown 2024-09-12T01:35:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.689+0000 7f44aeec5740 1 -- 172.21.15.1:0/2491206254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdbcc9260 msgr2=0x55ecdbcd3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.689+0000 7f44aeec5740 1 --2- 172.21.15.1:0/2491206254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdbcc9260 0x55ecdbcd3470 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f449c098380 tx=0x7f449c0981c0 comp rx=0 tx=0).stop 2024-09-12T01:35:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.689+0000 7f44aeec5740 1 -- 172.21.15.1:0/2491206254 shutdown_connections 2024-09-12T01:35:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.689+0000 7f44aeec5740 1 --2- 172.21.15.1:0/2491206254 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55ecdbcdef30 0x55ecdbce1320 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.689+0000 7f44aeec5740 1 --2- 172.21.15.1:0/2491206254 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ecdbccec30 0x55ecdbcda9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.689+0000 7f44aeec5740 1 --2- 172.21.15.1:0/2491206254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ecdbcc9ba0 0x55ecdbcd39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.689+0000 7f448aa00640 0 client.25664 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:35:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.689+0000 7f44aeec5740 1 --2- 172.21.15.1:0/2491206254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdbcc9260 0x55ecdbcd3470 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.689+0000 7f44aeec5740 1 -- 172.21.15.1:0/2491206254 >> 172.21.15.1:0/2491206254 conn(0x55ecdbcc48d0 msgr2=0x55ecdbcc60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:35:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.690+0000 7f44aeec5740 1 -- 172.21.15.1:0/2491206254 shutdown_connections 2024-09-12T01:35:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.690+0000 7f44aeec5740 1 -- 172.21.15.1:0/2491206254 wait complete. 2024-09-12T01:35:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.690+0000 7f44aeec5740 20 client.25664 trim_cache size 0 max 16384 2024-09-12T01:35:35.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.707+0000 7f44aeec5740 1 Processor -- start 2024-09-12T01:35:35.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.707+0000 7f44aeec5740 1 -- start start 2024-09-12T01:35:35.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.707+0000 7f44aeec5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ecdbcc9260 0x55ecdbcd3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:35.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.707+0000 7f44aeec5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdbcc9ba0 0x55ecdbcda9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:35.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.707+0000 7f44aeec5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ecdbccec30 0x55ecdbcc5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:35.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.707+0000 7f44aeec5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f44a41b92f0 con 0x55ecdbcc9260 2024-09-12T01:35:35.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.707+0000 7f44aeec5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f44a41bae30 con 0x55ecdbcc9ba0 2024-09-12T01:35:35.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.707+0000 7f44aeec5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f44a41b99c0 con 0x55ecdbccec30 2024-09-12T01:35:35.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.708+0000 7f44a8c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdbcc9ba0 0x55ecdbcda9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:35.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.708+0000 7f44aa000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ecdbccec30 0x55ecdbcc5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:35.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.708+0000 7f44a8c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdbcc9ba0 0x55ecdbcda9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:41222/0 (socket says 172.21.15.1:41222) 2024-09-12T01:35:35.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.708+0000 7f44a8c00640 1 -- 172.21.15.1:0/3499275807 learned_addr learned my addr 172.21.15.1:0/3499275807 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:35:35.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.708+0000 7f44aa000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ecdbccec30 0x55ecdbcc5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:46104/0 (socket says 172.21.15.1:46104) 2024-09-12T01:35:35.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.708+0000 7f44a9600640 1 --2- 172.21.15.1:0/3499275807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ecdbcc9260 0x55ecdbcd3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:35.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.708+0000 7f44a8c00640 1 -- 172.21.15.1:0/3499275807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ecdbccec30 msgr2=0x55ecdbcc5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:35:35.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.708+0000 7f44a8c00640 1 --2- 172.21.15.1:0/3499275807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ecdbccec30 0x55ecdbcc5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:35.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.708+0000 7f44a8c00640 1 -- 172.21.15.1:0/3499275807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ecdbcc9260 msgr2=0x55ecdbcd3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:35:35.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.708+0000 7f44a8c00640 1 --2- 172.21.15.1:0/3499275807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ecdbcc9260 0x55ecdbcd3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:35.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.708+0000 7f44a8c00640 1 -- 172.21.15.1:0/3499275807 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecdbcc46f0 con 0x55ecdbcc9ba0 2024-09-12T01:35:35.712 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.708+0000 7f44a8c00640 1 --2- 172.21.15.1:0/3499275807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdbcc9ba0 0x55ecdbcda9b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f449000b010 tx=0x7f449000a5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26db20e5dd2d9dfe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:35:35.712 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.709+0000 7f4495c00640 1 -- 172.21.15.1:0/3499275807 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449000fdd0 con 0x55ecdbcc9ba0 2024-09-12T01:35:35.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.709+0000 7f4495c00640 1 -- 172.21.15.1:0/3499275807 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4490029070 con 0x55ecdbcc9ba0 2024-09-12T01:35:35.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.709+0000 7f4495c00640 1 -- 172.21.15.1:0/3499275807 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44900049d0 con 0x55ecdbcc9ba0 2024-09-12T01:35:35.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.709+0000 7f44aeec5740 1 -- 172.21.15.1:0/3499275807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdbcc9ba0 msgr2=0x55ecdbcda9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:35.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.709+0000 7f44aeec5740 1 --2- 172.21.15.1:0/3499275807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdbcc9ba0 0x55ecdbcda9b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f449000b010 tx=0x7f449000a5b0 comp rx=0 tx=0).stop 2024-09-12T01:35:35.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.710+0000 7f44aeec5740 1 -- 172.21.15.1:0/3499275807 shutdown_connections 2024-09-12T01:35:35.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.710+0000 7f44aeec5740 1 --2- 172.21.15.1:0/3499275807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ecdbccec30 0x55ecdbcc5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:35.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.710+0000 7f44aeec5740 1 --2- 172.21.15.1:0/3499275807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdbcc9ba0 0x55ecdbcda9b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:35.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.710+0000 7f44aeec5740 1 --2- 172.21.15.1:0/3499275807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ecdbcc9260 0x55ecdbcd3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:35.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.710+0000 7f44aeec5740 1 -- 172.21.15.1:0/3499275807 >> 172.21.15.1:0/3499275807 conn(0x55ecdbcc48d0 msgr2=0x55ecdbcd3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:35:35.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.710+0000 7f44aeec5740 1 -- 172.21.15.1:0/3499275807 shutdown_connections 2024-09-12T01:35:35.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.710+0000 7f44aeec5740 1 -- 172.21.15.1:0/3499275807 wait complete. 2024-09-12T01:35:35.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.710+0000 7f44aeec5740 1 -- start start 2024-09-12T01:35:35.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.710+0000 7f44aeec5740 1 Processor -- start 2024-09-12T01:35:35.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.711+0000 7f44aeec5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ecdbcc9260 0x55ecdbcd7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:35.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.711+0000 7f44aeec5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ecdbcc9ba0 0x55ecdbcd8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:35.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.711+0000 7f44aeec5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdbccec30 0x55ecdbcd90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:35.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.711+0000 7f44aeec5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f44a41ba760 con 0x55ecdbcc9260 2024-09-12T01:35:35.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.711+0000 7f44aeec5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f44a41ba090 con 0x55ecdbccec30 2024-09-12T01:35:35.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.711+0000 7f44aeec5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f44a41b72d0 con 0x55ecdbcc9ba0 2024-09-12T01:35:35.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.711+0000 7f44aeec5740 10 client.0 _ll_register_callbacks cb 0x7f449c0bc050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:35:35.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.711+0000 7f44a9600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ecdbcc9260 0x55ecdbcd7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:35.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.711+0000 7f44aa000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdbccec30 0x55ecdbcd90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:35.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.711+0000 7f44a8c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ecdbcc9ba0 0x55ecdbcd8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:35.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.711+0000 7f44a9600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ecdbcc9260 0x55ecdbcd7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56408/0 (socket says 172.21.15.1:56408) 2024-09-12T01:35:35.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.711+0000 7f44a9600640 1 -- 172.21.15.1:0/2761366265 learned_addr learned my addr 172.21.15.1:0/2761366265 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:35:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.711+0000 7f44aa000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdbccec30 0x55ecdbcd90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:41234/0 (socket says 172.21.15.1:41234) 2024-09-12T01:35:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.711+0000 7f44aa000640 1 -- 172.21.15.1:0/2761366265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ecdbcc9ba0 msgr2=0x55ecdbcd8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:35:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.711+0000 7f44aa000640 1 --2- 172.21.15.1:0/2761366265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ecdbcc9ba0 0x55ecdbcd8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.711+0000 7f44aa000640 1 -- 172.21.15.1:0/2761366265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ecdbcc9260 msgr2=0x55ecdbcd7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:35:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.711+0000 7f44aa000640 1 --2- 172.21.15.1:0/2761366265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ecdbcc9260 0x55ecdbcd7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.711+0000 7f44aa000640 1 -- 172.21.15.1:0/2761366265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecdbcd96b0 con 0x55ecdbccec30 2024-09-12T01:35:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.711+0000 7f44a8c00640 1 --2- 172.21.15.1:0/2761366265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ecdbcc9ba0 0x55ecdbcd8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:35:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.711+0000 7f44a9600640 1 --2- 172.21.15.1:0/2761366265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ecdbcc9260 0x55ecdbcd7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:35:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.712+0000 7f44aa000640 1 --2- 172.21.15.1:0/2761366265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdbccec30 0x55ecdbcd90a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f44980096c0 tx=0x7f4498009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=108338b86bf5216c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:35:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.712+0000 7f448aa00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-12T01:35:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.712+0000 7f448aa00640 1 -- 172.21.15.1:0/2761366265 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4498035040 con 0x55ecdbccec30 2024-09-12T01:35:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.712+0000 7f44aeec5740 1 -- 172.21.15.1:0/2761366265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4490029070 con 0x55ecdbccec30 2024-09-12T01:35:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.712+0000 7f448aa00640 1 -- 172.21.15.1:0/2761366265 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f44980328c0 con 0x55ecdbccec30 2024-09-12T01:35:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.712+0000 7f448aa00640 1 -- 172.21.15.1:0/2761366265 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4498032ac0 con 0x55ecdbccec30 2024-09-12T01:35:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.712+0000 7f44aeec5740 10 client.25670 fetch_fsmap learned FSMap version 35 2024-09-12T01:35:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.712+0000 7f44aeec5740 1 -- 172.21.15.1:0/2761366265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ecdbcdaa80 con 0x55ecdbccec30 2024-09-12T01:35:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.713+0000 7f448aa00640 1 -- 172.21.15.1:0/2761366265 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4490029070 con 0x55ecdbccec30 2024-09-12T01:35:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.713+0000 7f448aa00640 1 -- 172.21.15.1:0/2761366265 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f4498052020 con 0x55ecdbccec30 2024-09-12T01:35:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.713+0000 7f44aeec5740 10 client.25670 fetch_fsmap finished waiting for FSMap version 35 2024-09-12T01:35:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.713+0000 7f44aeec5740 10 client.25670 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:35:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.713+0000 7f44aeec5740 20 client.25670 populate_metadata read hostname 'smithi001' 2024-09-12T01:35:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.713+0000 7f44aeec5740 1 --2- 172.21.15.1:0/2761366265 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55ecdbd4a120 0x55ecdbd4c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.713+0000 7f44aeec5740 4 client.25670 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:35:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.713+0000 7f44aeec5740 1 -- 172.21.15.1:0/2761366265 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ecdbc35b80 con 0x55ecdbd4a120 2024-09-12T01:35:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.713+0000 7f44a9600640 1 --2- 172.21.15.1:0/2761366265 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55ecdbd4a120 0x55ecdbd4c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.714+0000 7f44a9600640 1 --2- 172.21.15.1:0/2761366265 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55ecdbd4a120 0x55ecdbd4c540 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8dda5fc5c04fb4e server_cookie=cd40b8a47334f014 in_seq=0 out_seq=0 2024-09-12T01:35:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.714+0000 7f448aa00640 10 client.25670 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:35:35.722 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.722+0000 7f448aa00640 1 -- 172.21.15.1:0/2761366265 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ecdbc35b80 con 0x55ecdbd4a120 2024-09-12T01:35:35.722 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.722+0000 7f448aa00640 10 client.25670 handle_command_reply: tid=0 2024-09-12T01:35:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.722+0000 7f44aeec5740 1 client.25670 shutdown 2024-09-12T01:35:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.722+0000 7f44aeec5740 1 -- 172.21.15.1:0/2761366265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdbccec30 msgr2=0x55ecdbcd90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.722+0000 7f44aeec5740 1 --2- 172.21.15.1:0/2761366265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdbccec30 0x55ecdbcd90a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f44980096c0 tx=0x7f4498009260 comp rx=0 tx=0).stop 2024-09-12T01:35:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.722+0000 7f44aeec5740 1 -- 172.21.15.1:0/2761366265 shutdown_connections 2024-09-12T01:35:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.722+0000 7f44aeec5740 1 --2- 172.21.15.1:0/2761366265 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55ecdbd4a120 0x55ecdbd4c540 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.722+0000 7f44aeec5740 1 --2- 172.21.15.1:0/2761366265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdbccec30 0x55ecdbcd90a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.722+0000 7f44aeec5740 1 --2- 172.21.15.1:0/2761366265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ecdbcc9ba0 0x55ecdbcd8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.722+0000 7f448aa00640 0 client.25670 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:35:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.722+0000 7f44aeec5740 1 --2- 172.21.15.1:0/2761366265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ecdbcc9260 0x55ecdbcd7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.722+0000 7f44aeec5740 1 -- 172.21.15.1:0/2761366265 >> 172.21.15.1:0/2761366265 conn(0x55ecdbcc48d0 msgr2=0x55ecdbcc31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:35:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.723+0000 7f44aeec5740 1 -- 172.21.15.1:0/2761366265 shutdown_connections 2024-09-12T01:35:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.723+0000 7f44aeec5740 1 -- 172.21.15.1:0/2761366265 wait complete. 2024-09-12T01:35:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.723+0000 7f44aeec5740 20 client.25670 trim_cache size 0 max 16384 2024-09-12T01:35:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-12T01:35:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-12T01:35:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-12T01:35:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-12T01:35:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.723+0000 7f44aaa00640 1 -- 172.21.15.1:0/201623381 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f448c0a2cd0 msgr2=0x7f448c0a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.723+0000 7f44aaa00640 1 --2- 172.21.15.1:0/201623381 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f448c0a2cd0 0x7f448c0a5190 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4498031440 tx=0x7f44980315b0 comp rx=0 tx=0).stop 2024-09-12T01:35:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.723+0000 7f44aaa00640 1 -- 172.21.15.1:0/201623381 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44a4103bb0 msgr2=0x7f44a4110d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.723+0000 7f44aaa00640 1 --2- 172.21.15.1:0/201623381 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44a4103bb0 0x7f44a4110d10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f449c00eb70 tx=0x7f449c00c6a0 comp rx=0 tx=0).stop 2024-09-12T01:35:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.724+0000 7f44aaa00640 1 -- 172.21.15.1:0/201623381 shutdown_connections 2024-09-12T01:35:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.724+0000 7f44aaa00640 1 --2- 172.21.15.1:0/201623381 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44a4105760 0x7f44a41162b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.724+0000 7f44aaa00640 1 --2- 172.21.15.1:0/201623381 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44a4104da0 0x7f44a4111250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.724+0000 7f44aaa00640 1 --2- 172.21.15.1:0/201623381 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f448c0a2cd0 0x7f448c0a5190 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.724+0000 7f44aaa00640 1 --2- 172.21.15.1:0/201623381 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44a4103bb0 0x7f44a4110d10 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.724+0000 7f44aaa00640 1 -- 172.21.15.1:0/201623381 >> 172.21.15.1:0/201623381 conn(0x7f44a40ff310 msgr2=0x7f44a4105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:35:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.724+0000 7f44aaa00640 1 -- 172.21.15.1:0/201623381 shutdown_connections 2024-09-12T01:35:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:35.724+0000 7f44aaa00640 1 -- 172.21.15.1:0/201623381 wait complete. 2024-09-12T01:35:36.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:35:35 smithi053 ceph-mon[35158]: pgmap v578: 129 pgs: 129 active+clean; 120 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 1.0 MiB/s wr, 435 op/s 2024-09-12T01:35:36.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:35:35 smithi104 ceph-mon[35219]: pgmap v578: 129 pgs: 129 active+clean; 120 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 1.0 MiB/s wr, 435 op/s 2024-09-12T01:35:36.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:35:35 smithi001 ceph-mon[30387]: pgmap v578: 129 pgs: 129 active+clean; 120 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 1.0 MiB/s wr, 435 op/s 2024-09-12T01:35:38.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:35:37 smithi053 ceph-mon[35158]: pgmap v579: 129 pgs: 129 active+clean; 128 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.1 MiB/s wr, 485 op/s 2024-09-12T01:35:38.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:35:37 smithi104 ceph-mon[35219]: pgmap v579: 129 pgs: 129 active+clean; 128 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.1 MiB/s wr, 485 op/s 2024-09-12T01:35:38.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:35:37 smithi001 ceph-mon[30387]: pgmap v579: 129 pgs: 129 active+clean; 128 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.1 MiB/s wr, 485 op/s 2024-09-12T01:35:39.262 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0x63ef9ee 2024-09-12T01:35:40.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:35:39 smithi053 ceph-mon[35158]: pgmap v580: 129 pgs: 129 active+clean; 128 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 842 KiB/s wr, 377 op/s 2024-09-12T01:35:40.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:35:39 smithi104 ceph-mon[35219]: pgmap v580: 129 pgs: 129 active+clean; 128 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 842 KiB/s wr, 377 op/s 2024-09-12T01:35:40.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:35:39 smithi001 ceph-mon[30387]: pgmap v580: 129 pgs: 129 active+clean; 128 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 842 KiB/s wr, 377 op/s 2024-09-12T01:35:42.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:35:41 smithi053 ceph-mon[35158]: pgmap v581: 129 pgs: 129 active+clean; 120 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.1 MiB/s wr, 498 op/s 2024-09-12T01:35:42.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:35:41 smithi104 ceph-mon[35219]: pgmap v581: 129 pgs: 129 active+clean; 120 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.1 MiB/s wr, 498 op/s 2024-09-12T01:35:42.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:35:41 smithi001 ceph-mon[30387]: pgmap v581: 129 pgs: 129 active+clean; 120 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.1 MiB/s wr, 498 op/s 2024-09-12T01:35:44.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:35:43 smithi104 ceph-mon[35219]: pgmap v582: 129 pgs: 129 active+clean; 120 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 997 KiB/s wr, 429 op/s 2024-09-12T01:35:44.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:35:43 smithi001 ceph-mon[30387]: pgmap v582: 129 pgs: 129 active+clean; 120 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 997 KiB/s wr, 429 op/s 2024-09-12T01:35:44.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:35:43 smithi053 ceph-mon[35158]: pgmap v582: 129 pgs: 129 active+clean; 120 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 997 KiB/s wr, 429 op/s 2024-09-12T01:35:46.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:35:46 smithi104 ceph-mon[35219]: pgmap v583: 129 pgs: 129 active+clean; 124 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.0 MiB/s wr, 436 op/s 2024-09-12T01:35:46.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:35:46 smithi001 ceph-mon[30387]: pgmap v583: 129 pgs: 129 active+clean; 124 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.0 MiB/s wr, 436 op/s 2024-09-12T01:35:46.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:35:46 smithi053 ceph-mon[35158]: pgmap v583: 129 pgs: 129 active+clean; 124 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.0 MiB/s wr, 436 op/s 2024-09-12T01:35:48.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:35:48 smithi104 ceph-mon[35219]: pgmap v584: 129 pgs: 129 active+clean; 125 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.2 MiB/s wr, 480 op/s 2024-09-12T01:35:48.430 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:35:48 smithi001 ceph-mon[30387]: pgmap v584: 129 pgs: 129 active+clean; 125 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.2 MiB/s wr, 480 op/s 2024-09-12T01:35:48.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:35:48 smithi053 ceph-mon[35158]: pgmap v584: 129 pgs: 129 active+clean; 125 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.2 MiB/s wr, 480 op/s 2024-09-12T01:35:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.877+0000 7ff6d8600640 1 -- 172.21.15.1:0/2559072544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6d0078790 msgr2=0x7ff6d0079030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.877+0000 7ff6d8600640 1 --2- 172.21.15.1:0/2559072544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6d0078790 0x7ff6d0079030 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8009ca0 tx=0x7ff6c802a8d0 comp rx=0 tx=0).stop 2024-09-12T01:35:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.877+0000 7ff6d8600640 1 -- 172.21.15.1:0/2559072544 shutdown_connections 2024-09-12T01:35:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.877+0000 7ff6d8600640 1 --2- 172.21.15.1:0/2559072544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6d0078790 0x7ff6d0079030 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.877+0000 7ff6d8600640 1 --2- 172.21.15.1:0/2559072544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6d0077df0 0x7ff6d0078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:48.879 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.877+0000 7ff6d8600640 1 --2- 172.21.15.1:0/2559072544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6d0076bb0 0x7ff6d0076fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:48.879 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.877+0000 7ff6d8600640 1 -- 172.21.15.1:0/2559072544 >> 172.21.15.1:0/2559072544 conn(0x7ff6d00ff4b0 msgr2=0x7ff6d01018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:35:48.879 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.878+0000 7ff6d8600640 1 -- 172.21.15.1:0/2559072544 shutdown_connections 2024-09-12T01:35:48.879 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.878+0000 7ff6d8600640 1 -- 172.21.15.1:0/2559072544 wait complete. 2024-09-12T01:35:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.879+0000 7ff6d8600640 1 Processor -- start 2024-09-12T01:35:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.879+0000 7ff6d8600640 1 -- start start 2024-09-12T01:35:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.880+0000 7ff6d8600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6d0076bb0 0x7ff6d019e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.880+0000 7ff6d8600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6d0077df0 0x7ff6d019e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.880+0000 7ff6d8600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6d0078790 0x7ff6d01a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.880+0000 7ff6d5200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6d0077df0 0x7ff6d019e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.880+0000 7ff6d5200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6d0077df0 0x7ff6d019e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:52376/0 (socket says 172.21.15.1:52376) 2024-09-12T01:35:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.880+0000 7ff6d5200640 1 -- 172.21.15.1:0/1279795918 learned_addr learned my addr 172.21.15.1:0/1279795918 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:35:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.880+0000 7ff6d8600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d0113e30 con 0x7ff6d0078790 2024-09-12T01:35:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.880+0000 7ff6d8600640 1 -- 172.21.15.1:0/1279795918 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d0113cb0 con 0x7ff6d0076bb0 2024-09-12T01:35:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.880+0000 7ff6d8600640 1 -- 172.21.15.1:0/1279795918 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d0113fb0 con 0x7ff6d0077df0 2024-09-12T01:35:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.880+0000 7ff6d5c00640 1 --2- 172.21.15.1:0/1279795918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6d0076bb0 0x7ff6d019e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.881+0000 7ff6d5200640 1 -- 172.21.15.1:0/1279795918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6d0076bb0 msgr2=0x7ff6d019e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.881+0000 7ff6d5200640 1 --2- 172.21.15.1:0/1279795918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6d0076bb0 0x7ff6d019e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.881+0000 7ff6d5200640 1 -- 172.21.15.1:0/1279795918 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6d0078790 msgr2=0x7ff6d01a59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.881+0000 7ff6d5200640 1 --2- 172.21.15.1:0/1279795918 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6d0078790 0x7ff6d01a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.881+0000 7ff6d5200640 1 -- 172.21.15.1:0/1279795918 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6d01a6030 con 0x7ff6d0077df0 2024-09-12T01:35:48.882 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.881+0000 7ff6d5200640 1 --2- 172.21.15.1:0/1279795918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6d0077df0 0x7ff6d019e950 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff6c400e9f0 tx=0x7ff6c400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:35:48.883 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.882+0000 7ff6ce000640 1 -- 172.21.15.1:0/1279795918 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c400cd80 con 0x7ff6d0077df0 2024-09-12T01:35:48.883 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.882+0000 7ff6ce000640 1 -- 172.21.15.1:0/1279795918 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6c4010910 con 0x7ff6d0077df0 2024-09-12T01:35:48.883 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.882+0000 7ff6ce000640 1 -- 172.21.15.1:0/1279795918 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4018950 con 0x7ff6d0077df0 2024-09-12T01:35:48.883 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.882+0000 7ff6d8600640 1 -- 172.21.15.1:0/1279795918 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6d01a6320 con 0x7ff6d0077df0 2024-09-12T01:35:48.883 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.882+0000 7ff6d8600640 1 -- 172.21.15.1:0/1279795918 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6d01a6820 con 0x7ff6d0077df0 2024-09-12T01:35:48.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.886+0000 7ff6ce000640 1 -- 172.21.15.1:0/1279795918 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ff6c4018b50 con 0x7ff6d0077df0 2024-09-12T01:35:48.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.887+0000 7ff6db642740 1 Processor -- start 2024-09-12T01:35:48.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.887+0000 7ff6db642740 1 -- start start 2024-09-12T01:35:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.887+0000 7ff6db642740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e747133090 0x55e747133490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.887+0000 7ff6db642740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7471339d0 0x55e747138490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.887+0000 7ff6db642740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e747138a60 0x55e74713af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.887+0000 7ff6db642740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e747098100 con 0x55e747133090 2024-09-12T01:35:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.887+0000 7ff6db642740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55e7470401b0 con 0x55e7471339d0 2024-09-12T01:35:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.887+0000 7ff6db642740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e7470940c0 con 0x55e747138a60 2024-09-12T01:35:48.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.888+0000 7ff6d5200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e747138a60 0x55e74713af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:48.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.888+0000 7ff6d6600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e747133090 0x55e747133490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.888+0000 7ff6d5200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e747138a60 0x55e74713af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:52388/0 (socket says 172.21.15.1:52388) 2024-09-12T01:35:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.888+0000 7ff6d6600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e747133090 0x55e747133490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51858/0 (socket says 172.21.15.1:51858) 2024-09-12T01:35:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.888+0000 7ff6d6600640 1 -- 172.21.15.1:0/2966726944 learned_addr learned my addr 172.21.15.1:0/2966726944 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:35:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.888+0000 7ff6d5c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7471339d0 0x55e747138490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.888+0000 7ff6ce000640 1 --2- 172.21.15.1:0/1279795918 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff6b80a2c00 0x7ff6b80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.888+0000 7ff6d5200640 1 -- 172.21.15.1:0/2966726944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7471339d0 msgr2=0x55e747138490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:35:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.888+0000 7ff6d5200640 1 --2- 172.21.15.1:0/2966726944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7471339d0 0x55e747138490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.888+0000 7ff6d5200640 1 -- 172.21.15.1:0/2966726944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e747133090 msgr2=0x55e747133490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:35:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.888+0000 7ff6d5200640 1 --2- 172.21.15.1:0/2966726944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e747133090 0x55e747133490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.888+0000 7ff6d5200640 1 -- 172.21.15.1:0/2966726944 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e746dab210 con 0x55e747138a60 2024-09-12T01:35:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.888+0000 7ff6ce000640 1 -- 172.21.15.1:0/1279795918 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7ff6c40cf990 con 0x7ff6d0077df0 2024-09-12T01:35:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.888+0000 7ff6d6600640 1 --2- 172.21.15.1:0/1279795918 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff6b80a2c00 0x7ff6b80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.889+0000 7ff6d5200640 1 --2- 172.21.15.1:0/2966726944 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e747138a60 0x55e74713af20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff6c40cd040 tx=0x7ff6c4023000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f37ffb90e0bdb6d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:35:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.889+0000 7ff6c2a00640 1 -- 172.21.15.1:0/2966726944 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c409b940 con 0x55e747138a60 2024-09-12T01:35:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.889+0000 7ff6c2a00640 1 -- 172.21.15.1:0/2966726944 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6c409c500 con 0x55e747138a60 2024-09-12T01:35:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.889+0000 7ff6c2a00640 1 -- 172.21.15.1:0/2966726944 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4014490 con 0x55e747138a60 2024-09-12T01:35:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.890+0000 7ff6db642740 1 -- 172.21.15.1:0/2966726944 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e747138a60 msgr2=0x55e74713af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.890+0000 7ff6db642740 1 --2- 172.21.15.1:0/2966726944 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e747138a60 0x55e74713af20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff6c40cd040 tx=0x7ff6c4023000 comp rx=0 tx=0).stop 2024-09-12T01:35:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.890+0000 7ff6d6600640 1 --2- 172.21.15.1:0/1279795918 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff6b80a2c00 0x7ff6b80a50c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8009430 tx=0x7ff6c80095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:35:48.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.890+0000 7ff6db642740 1 -- 172.21.15.1:0/2966726944 shutdown_connections 2024-09-12T01:35:48.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.890+0000 7ff6db642740 1 --2- 172.21.15.1:0/2966726944 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e747138a60 0x55e74713af20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:48.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.890+0000 7ff6db642740 1 --2- 172.21.15.1:0/2966726944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7471339d0 0x55e747138490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:48.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.890+0000 7ff6db642740 1 --2- 172.21.15.1:0/2966726944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e747133090 0x55e747133490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:48.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.890+0000 7ff6db642740 1 -- 172.21.15.1:0/2966726944 >> 172.21.15.1:0/2966726944 conn(0x55e74712e700 msgr2=0x55e74712eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:35:48.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.890+0000 7ff6db642740 1 -- 172.21.15.1:0/2966726944 shutdown_connections 2024-09-12T01:35:48.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.890+0000 7ff6db642740 1 -- 172.21.15.1:0/2966726944 wait complete. 2024-09-12T01:35:48.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.890+0000 7ff6db642740 1 -- start start 2024-09-12T01:35:48.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.891+0000 7ff6db642740 1 Processor -- start 2024-09-12T01:35:48.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.891+0000 7ff6db642740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e747133090 0x55e74713d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:48.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.891+0000 7ff6db642740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7471339d0 0x55e74713d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:48.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.891+0000 7ff6db642740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e747138a60 0x55e747144710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:48.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.891+0000 7ff6db642740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e74713b9d0 con 0x55e747133090 2024-09-12T01:35:48.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.891+0000 7ff6db642740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55e747021090 con 0x55e7471339d0 2024-09-12T01:35:48.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.891+0000 7ff6db642740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e74713bb50 con 0x55e747138a60 2024-09-12T01:35:48.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.891+0000 7ff6db642740 10 client.0 _ll_register_callbacks cb 0x55e747075f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:35:48.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.891+0000 7ff6d5c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e747133090 0x55e74713d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:48.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.891+0000 7ff6d5c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e747133090 0x55e74713d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51868/0 (socket says 172.21.15.1:51868) 2024-09-12T01:35:48.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.891+0000 7ff6d5c00640 1 -- 172.21.15.1:0/2306714742 learned_addr learned my addr 172.21.15.1:0/2306714742 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:35:48.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.891+0000 7ff6d5200640 1 --2- 172.21.15.1:0/2306714742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7471339d0 0x55e74713d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:48.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.892+0000 7ff6d6600640 1 --2- 172.21.15.1:0/2306714742 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e747138a60 0x55e747144710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:48.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.892+0000 7ff6d5c00640 1 -- 172.21.15.1:0/2306714742 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e747138a60 msgr2=0x55e747144710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:35:48.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.892+0000 7ff6d5c00640 1 --2- 172.21.15.1:0/2306714742 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e747138a60 0x55e747144710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:48.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.892+0000 7ff6d5c00640 1 -- 172.21.15.1:0/2306714742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7471339d0 msgr2=0x55e74713d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:35:48.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.892+0000 7ff6d5c00640 1 --2- 172.21.15.1:0/2306714742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7471339d0 0x55e74713d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:48.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.892+0000 7ff6d5c00640 1 -- 172.21.15.1:0/2306714742 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e746e77620 con 0x55e747133090 2024-09-12T01:35:48.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.892+0000 7ff6d5200640 1 --2- 172.21.15.1:0/2306714742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7471339d0 0x55e74713d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:35:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.892+0000 7ff6d5c00640 1 --2- 172.21.15.1:0/2306714742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e747133090 0x55e74713d190 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc011740 tx=0x7ff6bc011c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6138deee3485e6c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:35:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.892+0000 7ff6b7400640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-12T01:35:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.892+0000 7ff6b7400640 1 -- 172.21.15.1:0/2306714742 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc01e060 con 0x55e747133090 2024-09-12T01:35:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.892+0000 7ff6b7400640 1 -- 172.21.15.1:0/2306714742 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7ff6bc00fdb0 con 0x55e747133090 2024-09-12T01:35:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.892+0000 7ff6db642740 1 -- 172.21.15.1:0/2306714742 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6c409c500 con 0x55e747133090 2024-09-12T01:35:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.892+0000 7ff6b7400640 1 -- 172.21.15.1:0/2306714742 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff6bc025050 con 0x55e747133090 2024-09-12T01:35:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.892+0000 7ff6db642740 10 client.15618 fetch_fsmap learned FSMap version 35 2024-09-12T01:35:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.892+0000 7ff6db642740 1 -- 172.21.15.1:0/2306714742 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e747147620 con 0x55e747133090 2024-09-12T01:35:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.893+0000 7ff6b7400640 1 -- 172.21.15.1:0/2306714742 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6c409c500 con 0x55e747133090 2024-09-12T01:35:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.893+0000 7ff6b7400640 1 -- 172.21.15.1:0/2306714742 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7ff6bc018030 con 0x55e747133090 2024-09-12T01:35:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.893+0000 7ff6db642740 10 client.15618 fetch_fsmap finished waiting for FSMap version 35 2024-09-12T01:35:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.893+0000 7ff6db642740 10 client.15618 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:35:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.893+0000 7ff6db642740 20 client.15618 populate_metadata read hostname 'smithi001' 2024-09-12T01:35:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.893+0000 7ff6db642740 1 --2- 172.21.15.1:0/2306714742 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55e747148cf0 0x55e74714b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.894+0000 7ff6db642740 4 client.15618 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:35:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.894+0000 7ff6db642740 1 -- 172.21.15.1:0/2306714742 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7470a5830 con 0x55e747148cf0 2024-09-12T01:35:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.894+0000 7ff6d5200640 1 --2- 172.21.15.1:0/2306714742 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55e747148cf0 0x55e74714b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.894+0000 7ff6d5200640 1 --2- 172.21.15.1:0/2306714742 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55e747148cf0 0x55e74714b160 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6518470c25615d69 server_cookie=aaa8bb6dacb547e in_seq=0 out_seq=0 2024-09-12T01:35:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.894+0000 7ff6b7400640 10 client.15618 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:35:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.895+0000 7ff6b7400640 1 -- 172.21.15.1:0/2306714742 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e7470a5830 con 0x55e747148cf0 2024-09-12T01:35:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.895+0000 7ff6b7400640 10 client.15618 handle_command_reply: tid=0 2024-09-12T01:35:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.895+0000 7ff6db642740 1 client.15618 shutdown 2024-09-12T01:35:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.895+0000 7ff6db642740 1 -- 172.21.15.1:0/2306714742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e747133090 msgr2=0x55e74713d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.895+0000 7ff6db642740 1 --2- 172.21.15.1:0/2306714742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e747133090 0x55e74713d190 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc011740 tx=0x7ff6bc011c00 comp rx=0 tx=0).stop 2024-09-12T01:35:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.895+0000 7ff6db642740 1 -- 172.21.15.1:0/2306714742 shutdown_connections 2024-09-12T01:35:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.895+0000 7ff6db642740 1 --2- 172.21.15.1:0/2306714742 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55e747148cf0 0x55e74714b160 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.895+0000 7ff6db642740 1 --2- 172.21.15.1:0/2306714742 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e747138a60 0x55e747144710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.895+0000 7ff6db642740 1 --2- 172.21.15.1:0/2306714742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7471339d0 0x55e74713d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.895+0000 7ff6db642740 1 --2- 172.21.15.1:0/2306714742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e747133090 0x55e74713d190 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:48.901 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.895+0000 7ff6b7400640 0 client.15618 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:35:48.901 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.895+0000 7ff6db642740 1 -- 172.21.15.1:0/2306714742 >> 172.21.15.1:0/2306714742 conn(0x55e74712e700 msgr2=0x55e74712ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:35:48.901 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.896+0000 7ff6db642740 1 -- 172.21.15.1:0/2306714742 shutdown_connections 2024-09-12T01:35:48.901 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.896+0000 7ff6db642740 1 -- 172.21.15.1:0/2306714742 wait complete. 2024-09-12T01:35:48.901 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.896+0000 7ff6db642740 20 client.15618 trim_cache size 0 max 16384 2024-09-12T01:35:48.918 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.918+0000 7ff6db642740 1 Processor -- start 2024-09-12T01:35:48.919 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.919+0000 7ff6db642740 1 -- start start 2024-09-12T01:35:48.919 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.919+0000 7ff6db642740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e747133090 0x55e7471296a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:48.919 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.919+0000 7ff6db642740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e7471339d0 0x55e747144630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:48.919 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.919+0000 7ff6db642740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e747138a60 0x55e74712faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:48.919 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.919+0000 7ff6db642740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d01b0de0 con 0x55e7471339d0 2024-09-12T01:35:48.919 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.919+0000 7ff6db642740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d01b2920 con 0x55e747138a60 2024-09-12T01:35:48.919 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.919+0000 7ff6db642740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d01b14b0 con 0x55e747133090 2024-09-12T01:35:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.919+0000 7ff6d5200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e7471339d0 0x55e747144630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.919+0000 7ff6d6600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e747138a60 0x55e74712faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.919+0000 7ff6d6600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e747138a60 0x55e74712faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:35852/0 (socket says 172.21.15.1:35852) 2024-09-12T01:35:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.919+0000 7ff6d6600640 1 -- 172.21.15.1:0/2807429020 learned_addr learned my addr 172.21.15.1:0/2807429020 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:35:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.919+0000 7ff6d5c00640 1 --2- 172.21.15.1:0/2807429020 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e747133090 0x55e7471296a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.920+0000 7ff6d6600640 1 -- 172.21.15.1:0/2807429020 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e747133090 msgr2=0x55e7471296a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:35:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.920+0000 7ff6d6600640 1 --2- 172.21.15.1:0/2807429020 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e747133090 0x55e7471296a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.920+0000 7ff6d6600640 1 -- 172.21.15.1:0/2807429020 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e7471339d0 msgr2=0x55e747144630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:35:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.920+0000 7ff6d6600640 1 --2- 172.21.15.1:0/2807429020 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e7471339d0 0x55e747144630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.920+0000 7ff6d6600640 1 -- 172.21.15.1:0/2807429020 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e74712d740 con 0x55e747138a60 2024-09-12T01:35:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.920+0000 7ff6d5200640 1 --2- 172.21.15.1:0/2807429020 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e7471339d0 0x55e747144630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:35:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.920+0000 7ff6d6600640 1 --2- 172.21.15.1:0/2807429020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e747138a60 0x55e74712faf0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8004af0 tx=0x7ff6c8036000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d6f9fc561080880 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:35:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.920+0000 7ff6c2000640 1 -- 172.21.15.1:0/2807429020 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c8004b40 con 0x55e747138a60 2024-09-12T01:35:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.920+0000 7ff6c2000640 1 -- 172.21.15.1:0/2807429020 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6c8036070 con 0x55e747138a60 2024-09-12T01:35:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.920+0000 7ff6c2000640 1 -- 172.21.15.1:0/2807429020 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c802f620 con 0x55e747138a60 2024-09-12T01:35:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.921+0000 7ff6db642740 1 -- 172.21.15.1:0/2807429020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e747138a60 msgr2=0x55e74712faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.921+0000 7ff6db642740 1 --2- 172.21.15.1:0/2807429020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e747138a60 0x55e74712faf0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8004af0 tx=0x7ff6c8036000 comp rx=0 tx=0).stop 2024-09-12T01:35:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.921+0000 7ff6db642740 1 -- 172.21.15.1:0/2807429020 shutdown_connections 2024-09-12T01:35:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.921+0000 7ff6db642740 1 --2- 172.21.15.1:0/2807429020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e747138a60 0x55e74712faf0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.921+0000 7ff6db642740 1 --2- 172.21.15.1:0/2807429020 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e7471339d0 0x55e747144630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.921+0000 7ff6db642740 1 --2- 172.21.15.1:0/2807429020 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e747133090 0x55e7471296a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.921+0000 7ff6db642740 1 -- 172.21.15.1:0/2807429020 >> 172.21.15.1:0/2807429020 conn(0x55e74712e700 msgr2=0x55e74713c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:35:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.921+0000 7ff6db642740 1 -- 172.21.15.1:0/2807429020 shutdown_connections 2024-09-12T01:35:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.921+0000 7ff6db642740 1 -- 172.21.15.1:0/2807429020 wait complete. 2024-09-12T01:35:48.926 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.922+0000 7ff6db642740 1 -- start start 2024-09-12T01:35:48.926 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.922+0000 7ff6db642740 1 Processor -- start 2024-09-12T01:35:48.926 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.922+0000 7ff6db642740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e747133090 0x55e7471414a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:48.926 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.922+0000 7ff6db642740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7471339d0 0x55e7471419e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:48.926 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.922+0000 7ff6db642740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e747138a60 0x55e7471b1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:48.926 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.922+0000 7ff6db642740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d01b2250 con 0x55e747133090 2024-09-12T01:35:48.926 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.922+0000 7ff6db642740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d01b1b80 con 0x55e7471339d0 2024-09-12T01:35:48.926 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.922+0000 7ff6db642740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d01aedc0 con 0x55e747138a60 2024-09-12T01:35:48.926 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.922+0000 7ff6db642740 10 client.0 _ll_register_callbacks cb 0x7ff6bc00c690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:35:48.927 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.923+0000 7ff6d5200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7471339d0 0x55e7471419e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:48.927 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.923+0000 7ff6d5c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e747133090 0x55e7471414a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:48.927 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.923+0000 7ff6d5200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7471339d0 0x55e7471419e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:35858/0 (socket says 172.21.15.1:35858) 2024-09-12T01:35:48.927 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.923+0000 7ff6d5200640 1 -- 172.21.15.1:0/1505862062 learned_addr learned my addr 172.21.15.1:0/1505862062 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:35:48.927 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.923+0000 7ff6d6600640 1 --2- 172.21.15.1:0/1505862062 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e747138a60 0x55e7471b1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:48.927 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.923+0000 7ff6d5200640 1 -- 172.21.15.1:0/1505862062 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e747138a60 msgr2=0x55e7471b1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:35:48.927 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.923+0000 7ff6d5200640 1 --2- 172.21.15.1:0/1505862062 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e747138a60 0x55e7471b1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:48.927 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.923+0000 7ff6d5200640 1 -- 172.21.15.1:0/1505862062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e747133090 msgr2=0x55e7471414a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:35:48.927 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.923+0000 7ff6d5200640 1 --2- 172.21.15.1:0/1505862062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e747133090 0x55e7471414a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:48.927 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.923+0000 7ff6d5200640 1 -- 172.21.15.1:0/1505862062 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7471b1f70 con 0x55e7471339d0 2024-09-12T01:35:48.928 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.923+0000 7ff6d5200640 1 --2- 172.21.15.1:0/1505862062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7471339d0 0x55e7471419e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff6c401a010 tx=0x7ff6c401a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80d62370520ade12 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:35:48.928 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.923+0000 7ff6b7400640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-12T01:35:48.928 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.923+0000 7ff6b7400640 1 -- 172.21.15.1:0/1505862062 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4024120 con 0x55e7471339d0 2024-09-12T01:35:48.928 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.924+0000 7ff6b7400640 1 -- 172.21.15.1:0/1505862062 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7ff6c40b0020 con 0x55e7471339d0 2024-09-12T01:35:48.928 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.924+0000 7ff6db642740 1 -- 172.21.15.1:0/1505862062 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6c8036070 con 0x55e7471339d0 2024-09-12T01:35:48.928 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.924+0000 7ff6b7400640 1 -- 172.21.15.1:0/1505862062 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff6c409b760 con 0x55e7471339d0 2024-09-12T01:35:48.928 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.924+0000 7ff6db642740 10 client.25688 fetch_fsmap learned FSMap version 35 2024-09-12T01:35:48.928 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.924+0000 7ff6db642740 1 -- 172.21.15.1:0/1505862062 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e7471b3220 con 0x55e7471339d0 2024-09-12T01:35:48.928 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.924+0000 7ff6b7400640 1 -- 172.21.15.1:0/1505862062 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6c8036070 con 0x55e7471339d0 2024-09-12T01:35:48.928 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.924+0000 7ff6b7400640 1 -- 172.21.15.1:0/1505862062 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7ff6c40cd120 con 0x55e7471339d0 2024-09-12T01:35:48.928 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.924+0000 7ff6db642740 10 client.25688 fetch_fsmap finished waiting for FSMap version 35 2024-09-12T01:35:48.928 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.924+0000 7ff6db642740 10 client.25688 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:35:48.928 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.924+0000 7ff6db642740 20 client.25688 populate_metadata read hostname 'smithi001' 2024-09-12T01:35:48.929 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.924+0000 7ff6db642740 1 --2- 172.21.15.1:0/1505862062 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55e7471b37b0 0x55e7471b5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:35:48.929 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.924+0000 7ff6db642740 4 client.25688 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:35:48.929 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.924+0000 7ff6db642740 1 -- 172.21.15.1:0/1505862062 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e747129d80 con 0x55e7471b37b0 2024-09-12T01:35:48.929 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.925+0000 7ff6d5c00640 1 --2- 172.21.15.1:0/1505862062 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55e7471b37b0 0x55e7471b5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:35:48.929 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.925+0000 7ff6d5c00640 1 --2- 172.21.15.1:0/1505862062 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55e7471b37b0 0x55e7471b5ba0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce24ec11fb7cd725 server_cookie=d8ab6f895ba508fc in_seq=0 out_seq=0 2024-09-12T01:35:48.929 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.925+0000 7ff6b7400640 10 client.25688 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:35:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.936+0000 7ff6b7400640 1 -- 172.21.15.1:0/1505862062 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e747129d80 con 0x55e7471b37b0 2024-09-12T01:35:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.936+0000 7ff6b7400640 10 client.25688 handle_command_reply: tid=0 2024-09-12T01:35:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.936+0000 7ff6db642740 1 client.25688 shutdown 2024-09-12T01:35:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.936+0000 7ff6db642740 1 -- 172.21.15.1:0/1505862062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7471339d0 msgr2=0x55e7471419e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.936+0000 7ff6db642740 1 --2- 172.21.15.1:0/1505862062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7471339d0 0x55e7471419e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff6c401a010 tx=0x7ff6c401a040 comp rx=0 tx=0).stop 2024-09-12T01:35:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.936+0000 7ff6db642740 1 -- 172.21.15.1:0/1505862062 shutdown_connections 2024-09-12T01:35:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.936+0000 7ff6db642740 1 --2- 172.21.15.1:0/1505862062 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55e7471b37b0 0x55e7471b5ba0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.936+0000 7ff6db642740 1 --2- 172.21.15.1:0/1505862062 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e747138a60 0x55e7471b1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.936+0000 7ff6db642740 1 --2- 172.21.15.1:0/1505862062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7471339d0 0x55e7471419e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.936+0000 7ff6b7400640 0 client.25688 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:35:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.936+0000 7ff6db642740 1 --2- 172.21.15.1:0/1505862062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e747133090 0x55e7471414a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.936+0000 7ff6db642740 1 -- 172.21.15.1:0/1505862062 >> 172.21.15.1:0/1505862062 conn(0x55e74712e700 msgr2=0x55e74713cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:35:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.936+0000 7ff6db642740 1 -- 172.21.15.1:0/1505862062 shutdown_connections 2024-09-12T01:35:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.936+0000 7ff6db642740 1 -- 172.21.15.1:0/1505862062 wait complete. 2024-09-12T01:35:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.936+0000 7ff6db642740 20 client.25688 trim_cache size 0 max 16384 2024-09-12T01:35:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-12T01:35:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-12T01:35:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-12T01:35:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-12T01:35:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.937+0000 7ff6d8600640 1 -- 172.21.15.1:0/1279795918 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff6b80a2c00 msgr2=0x7ff6b80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.937+0000 7ff6d8600640 1 --2- 172.21.15.1:0/1279795918 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff6b80a2c00 0x7ff6b80a50c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8009430 tx=0x7ff6c80095c0 comp rx=0 tx=0).stop 2024-09-12T01:35:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.937+0000 7ff6d8600640 1 -- 172.21.15.1:0/1279795918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6d0077df0 msgr2=0x7ff6d019e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:35:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.937+0000 7ff6d8600640 1 --2- 172.21.15.1:0/1279795918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6d0077df0 0x7ff6d019e950 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff6c400e9f0 tx=0x7ff6c400eeb0 comp rx=0 tx=0).stop 2024-09-12T01:35:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.937+0000 7ff6d8600640 1 -- 172.21.15.1:0/1279795918 shutdown_connections 2024-09-12T01:35:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.937+0000 7ff6d8600640 1 --2- 172.21.15.1:0/1279795918 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6d0078790 0x7ff6d01a59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.937+0000 7ff6d8600640 1 --2- 172.21.15.1:0/1279795918 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff6b80a2c00 0x7ff6b80a50c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.937+0000 7ff6d8600640 1 --2- 172.21.15.1:0/1279795918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6d0077df0 0x7ff6d019e950 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.937+0000 7ff6d8600640 1 --2- 172.21.15.1:0/1279795918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6d0076bb0 0x7ff6d019e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:35:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.937+0000 7ff6d8600640 1 -- 172.21.15.1:0/1279795918 >> 172.21.15.1:0/1279795918 conn(0x7ff6d00ff4b0 msgr2=0x7ff6d0100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:35:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.937+0000 7ff6d8600640 1 -- 172.21.15.1:0/1279795918 shutdown_connections 2024-09-12T01:35:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:35:48.937+0000 7ff6d8600640 1 -- 172.21.15.1:0/1279795918 wait complete. 2024-09-12T01:35:50.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:35:50 smithi104 ceph-mon[35219]: pgmap v585: 129 pgs: 129 active+clean; 125 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 903 KiB/s wr, 366 op/s 2024-09-12T01:35:50.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:35:50 smithi001 ceph-mon[30387]: pgmap v585: 129 pgs: 129 active+clean; 125 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 903 KiB/s wr, 366 op/s 2024-09-12T01:35:50.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:35:50 smithi053 ceph-mon[35158]: pgmap v585: 129 pgs: 129 active+clean; 125 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 903 KiB/s wr, 366 op/s 2024-09-12T01:35:52.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:35:52 smithi104 ceph-mon[35219]: pgmap v586: 129 pgs: 129 active+clean; 121 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 1.2 MiB/s wr, 491 op/s 2024-09-12T01:35:52.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:35:52 smithi001 ceph-mon[30387]: pgmap v586: 129 pgs: 129 active+clean; 121 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 1.2 MiB/s wr, 491 op/s 2024-09-12T01:35:52.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:35:52 smithi053 ceph-mon[35158]: pgmap v586: 129 pgs: 129 active+clean; 121 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 1.2 MiB/s wr, 491 op/s 2024-09-12T01:35:54.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:35:54 smithi104 ceph-mon[35219]: pgmap v587: 129 pgs: 129 active+clean; 120 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 1002 KiB/s wr, 428 op/s 2024-09-12T01:35:54.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:35:54 smithi001 ceph-mon[30387]: pgmap v587: 129 pgs: 129 active+clean; 120 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 1002 KiB/s wr, 428 op/s 2024-09-12T01:35:54.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:35:54 smithi053 ceph-mon[35158]: pgmap v587: 129 pgs: 129 active+clean; 120 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 1002 KiB/s wr, 428 op/s 2024-09-12T01:35:56.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:35:56 smithi104 ceph-mon[35219]: pgmap v588: 129 pgs: 129 active+clean; 120 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 1.1 MiB/s wr, 441 op/s 2024-09-12T01:35:56.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:35:56 smithi001 ceph-mon[30387]: pgmap v588: 129 pgs: 129 active+clean; 120 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 1.1 MiB/s wr, 441 op/s 2024-09-12T01:35:56.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:35:56 smithi053 ceph-mon[35158]: pgmap v588: 129 pgs: 129 active+clean; 120 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 1.1 MiB/s wr, 441 op/s 2024-09-12T01:35:58.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:35:58 smithi104 ceph-mon[35219]: pgmap v589: 129 pgs: 129 active+clean; 120 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.2 MiB/s wr, 488 op/s 2024-09-12T01:35:58.429 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:35:58 smithi001 ceph-mon[30387]: pgmap v589: 129 pgs: 129 active+clean; 120 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.2 MiB/s wr, 488 op/s 2024-09-12T01:35:58.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:35:58 smithi053 ceph-mon[35158]: pgmap v589: 129 pgs: 129 active+clean; 120 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.2 MiB/s wr, 488 op/s 2024-09-12T01:36:00.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:00 smithi104 ceph-mon[35219]: pgmap v590: 129 pgs: 129 active+clean; 120 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 910 KiB/s wr, 374 op/s 2024-09-12T01:36:00.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:00 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:36:00.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:00 smithi001 ceph-mon[30387]: pgmap v590: 129 pgs: 129 active+clean; 120 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 910 KiB/s wr, 374 op/s 2024-09-12T01:36:00.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:00 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:36:00.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:00 smithi053 ceph-mon[35158]: pgmap v590: 129 pgs: 129 active+clean; 120 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 910 KiB/s wr, 374 op/s 2024-09-12T01:36:00.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:00 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:36:01.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:01 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:36:01.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:01 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:36:01.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:01 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:36:01.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:01 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:36:01.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:01 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:36:01.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:01 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:36:01.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:01 smithi104 ceph-mon[35219]: Adjusting osd_memory_target on smithi104 to 176.0M 2024-09-12T01:36:01.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:01 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:36:01.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:01 smithi104 ceph-mon[35219]: Unable to set osd_memory_target on smithi104 to 184589619: error parsing value: Value '184589619' is below minimum 939524096 2024-09-12T01:36:01.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:01 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:36:01.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:01 smithi104 ceph-mon[35219]: Adjusting osd_memory_target on smithi053 to 177.0M 2024-09-12T01:36:01.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:01 smithi104 ceph-mon[35219]: Unable to set osd_memory_target on smithi053 to 185699942: error parsing value: Value '185699942' is below minimum 939524096 2024-09-12T01:36:01.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:01 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:36:01.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:01 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:36:01.326 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:01 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:36:01.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:01 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:36:01.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:01 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:36:01.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:01 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:36:01.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:01 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:36:01.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:01 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:36:01.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:01 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:36:01.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:01 smithi001 ceph-mon[30387]: Adjusting osd_memory_target on smithi104 to 176.0M 2024-09-12T01:36:01.429 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:01 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:36:01.429 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:01 smithi001 ceph-mon[30387]: Unable to set osd_memory_target on smithi104 to 184589619: error parsing value: Value '184589619' is below minimum 939524096 2024-09-12T01:36:01.429 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:01 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:36:01.429 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:01 smithi001 ceph-mon[30387]: Adjusting osd_memory_target on smithi053 to 177.0M 2024-09-12T01:36:01.429 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:01 smithi001 ceph-mon[30387]: Unable to set osd_memory_target on smithi053 to 185699942: error parsing value: Value '185699942' is below minimum 939524096 2024-09-12T01:36:01.429 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:01 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:36:01.429 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:01 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:36:01.429 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:01 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:36:01.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:01 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:36:01.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:01 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:36:01.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:01 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:36:01.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:01 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:36:01.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:01 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:36:01.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:01 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:36:01.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:01 smithi053 ceph-mon[35158]: Adjusting osd_memory_target on smithi104 to 176.0M 2024-09-12T01:36:01.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:01 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:36:01.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:01 smithi053 ceph-mon[35158]: Unable to set osd_memory_target on smithi104 to 184589619: error parsing value: Value '184589619' is below minimum 939524096 2024-09-12T01:36:01.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:01 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:36:01.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:01 smithi053 ceph-mon[35158]: Adjusting osd_memory_target on smithi053 to 177.0M 2024-09-12T01:36:01.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:01 smithi053 ceph-mon[35158]: Unable to set osd_memory_target on smithi053 to 185699942: error parsing value: Value '185699942' is below minimum 939524096 2024-09-12T01:36:01.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:01 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:36:01.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:01 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:36:01.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:01 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:36:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.092+0000 7ff84f000640 1 -- 172.21.15.1:0/1043239475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff848101310 msgr2=0x7ff848101790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.092+0000 7ff84f000640 1 --2- 172.21.15.1:0/1043239475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff848101310 0x7ff848101790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff834009900 tx=0x7ff83402a3f0 comp rx=0 tx=0).stop 2024-09-12T01:36:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.093+0000 7ff84f000640 1 -- 172.21.15.1:0/1043239475 shutdown_connections 2024-09-12T01:36:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.093+0000 7ff84f000640 1 --2- 172.21.15.1:0/1043239475 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff848106010 0x7ff848108400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.093+0000 7ff84f000640 1 --2- 172.21.15.1:0/1043239475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff848101310 0x7ff848101790 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.093+0000 7ff84f000640 1 --2- 172.21.15.1:0/1043239475 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff848068d30 0x7ff848100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.093+0000 7ff84f000640 1 -- 172.21.15.1:0/1043239475 >> 172.21.15.1:0/1043239475 conn(0x7ff8480fb340 msgr2=0x7ff8480fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:36:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.093+0000 7ff84f000640 1 -- 172.21.15.1:0/1043239475 shutdown_connections 2024-09-12T01:36:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.093+0000 7ff84f000640 1 -- 172.21.15.1:0/1043239475 wait complete. 2024-09-12T01:36:02.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.094+0000 7ff84f000640 1 Processor -- start 2024-09-12T01:36:02.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.094+0000 7ff84f000640 1 -- start start 2024-09-12T01:36:02.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.095+0000 7ff84f000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff848068d30 0x7ff848197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:02.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.095+0000 7ff84f000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff848101310 0x7ff8481981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:02.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.095+0000 7ff84f000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff848106010 0x7ff84819f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:02.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.095+0000 7ff84f000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff84810d6c0 con 0x7ff848068d30 2024-09-12T01:36:02.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.095+0000 7ff84f000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff84810d540 con 0x7ff848106010 2024-09-12T01:36:02.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.095+0000 7ff84f000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff84810d840 con 0x7ff848101310 2024-09-12T01:36:02.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.095+0000 7ff84dc00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff848068d30 0x7ff848197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.095+0000 7ff84dc00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff848068d30 0x7ff848197c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46014/0 (socket says 172.21.15.1:46014) 2024-09-12T01:36:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.095+0000 7ff84dc00640 1 -- 172.21.15.1:0/2935902955 learned_addr learned my addr 172.21.15.1:0/2935902955 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:36:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.095+0000 7ff84d200640 1 --2- 172.21.15.1:0/2935902955 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff848101310 0x7ff8481981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.095+0000 7ff84e600640 1 --2- 172.21.15.1:0/2935902955 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff848106010 0x7ff84819f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.095+0000 7ff84dc00640 1 -- 172.21.15.1:0/2935902955 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff848101310 msgr2=0x7ff8481981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.095+0000 7ff84dc00640 1 --2- 172.21.15.1:0/2935902955 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff848101310 0x7ff8481981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.095+0000 7ff84dc00640 1 -- 172.21.15.1:0/2935902955 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff848106010 msgr2=0x7ff84819f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.095+0000 7ff84dc00640 1 --2- 172.21.15.1:0/2935902955 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff848106010 0x7ff84819f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.095+0000 7ff84dc00640 1 -- 172.21.15.1:0/2935902955 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff84819f8b0 con 0x7ff848068d30 2024-09-12T01:36:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.096+0000 7ff84dc00640 1 --2- 172.21.15.1:0/2935902955 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff848068d30 0x7ff848197c70 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff83c00e9c0 tx=0x7ff83c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:36:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.096+0000 7ff846000640 1 -- 172.21.15.1:0/2935902955 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83c00cd80 con 0x7ff848068d30 2024-09-12T01:36:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.096+0000 7ff84f000640 1 -- 172.21.15.1:0/2935902955 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff84819fba0 con 0x7ff848068d30 2024-09-12T01:36:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.096+0000 7ff846000640 1 -- 172.21.15.1:0/2935902955 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff83c0035f0 con 0x7ff848068d30 2024-09-12T01:36:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.096+0000 7ff846000640 1 -- 172.21.15.1:0/2935902955 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83c010690 con 0x7ff848068d30 2024-09-12T01:36:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.096+0000 7ff84f000640 1 -- 172.21.15.1:0/2935902955 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8481a00c0 con 0x7ff848068d30 2024-09-12T01:36:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.099+0000 7ff8533de740 1 Processor -- start 2024-09-12T01:36:02.100 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.100+0000 7ff8533de740 1 -- start start 2024-09-12T01:36:02.100 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.100+0000 7ff846000640 1 -- 172.21.15.1:0/2935902955 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ff83c0037a0 con 0x7ff848068d30 2024-09-12T01:36:02.100 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.100+0000 7ff8533de740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef6f3ab260 0x55ef6f3ab660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:02.100 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.100+0000 7ff8533de740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef6f3abba0 0x55ef6f3baf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:02.100 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.100+0000 7ff8533de740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef6f3bb480 0x55ef6f3bd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:02.100 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.100+0000 7ff8533de740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ef6f312100 con 0x55ef6f3ab260 2024-09-12T01:36:02.100 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.100+0000 7ff8533de740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ef6f2ba1b0 con 0x55ef6f3abba0 2024-09-12T01:36:02.100 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.100+0000 7ff8533de740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ef6f30e0c0 con 0x55ef6f3bb480 2024-09-12T01:36:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.100+0000 7ff84e600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef6f3ab260 0x55ef6f3ab660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.100+0000 7ff84dc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef6f3abba0 0x55ef6f3baf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.100+0000 7ff84e600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef6f3ab260 0x55ef6f3ab660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46024/0 (socket says 172.21.15.1:46024) 2024-09-12T01:36:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.100+0000 7ff84e600640 1 -- 172.21.15.1:0/640249989 learned_addr learned my addr 172.21.15.1:0/640249989 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:36:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.100+0000 7ff84d200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef6f3bb480 0x55ef6f3bd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.101+0000 7ff84e600640 1 -- 172.21.15.1:0/640249989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef6f3bb480 msgr2=0x55ef6f3bd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:36:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.101+0000 7ff84e600640 1 --2- 172.21.15.1:0/640249989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef6f3bb480 0x55ef6f3bd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.101+0000 7ff84e600640 1 -- 172.21.15.1:0/640249989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef6f3abba0 msgr2=0x55ef6f3baf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:36:02.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.101+0000 7ff84e600640 1 --2- 172.21.15.1:0/640249989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef6f3abba0 0x55ef6f3baf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:02.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.101+0000 7ff84e600640 1 -- 172.21.15.1:0/640249989 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef6f025210 con 0x55ef6f3ab260 2024-09-12T01:36:02.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.101+0000 7ff84d200640 1 --2- 172.21.15.1:0/640249989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef6f3bb480 0x55ef6f3bd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:36:02.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.101+0000 7ff84e600640 1 --2- 172.21.15.1:0/640249989 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef6f3ab260 0x55ef6f3ab660 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff840009fd0 tx=0x7ff8400118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef68760f289f950b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:36:02.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.101+0000 7ff83aa00640 1 -- 172.21.15.1:0/640249989 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff840004d20 con 0x55ef6f3ab260 2024-09-12T01:36:02.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.101+0000 7ff83aa00640 1 -- 172.21.15.1:0/640249989 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff840004050 con 0x55ef6f3ab260 2024-09-12T01:36:02.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.101+0000 7ff83aa00640 1 -- 172.21.15.1:0/640249989 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff840005260 con 0x55ef6f3ab260 2024-09-12T01:36:02.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.102+0000 7ff8533de740 1 -- 172.21.15.1:0/640249989 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef6f3ab260 msgr2=0x55ef6f3ab660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:02.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.102+0000 7ff8533de740 1 --2- 172.21.15.1:0/640249989 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef6f3ab260 0x55ef6f3ab660 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff840009fd0 tx=0x7ff8400118d0 comp rx=0 tx=0).stop 2024-09-12T01:36:02.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.102+0000 7ff8533de740 1 -- 172.21.15.1:0/640249989 shutdown_connections 2024-09-12T01:36:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.102+0000 7ff8533de740 1 --2- 172.21.15.1:0/640249989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef6f3bb480 0x55ef6f3bd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.102+0000 7ff8533de740 1 --2- 172.21.15.1:0/640249989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef6f3abba0 0x55ef6f3baf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.102+0000 7ff8533de740 1 --2- 172.21.15.1:0/640249989 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef6f3ab260 0x55ef6f3ab660 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.102+0000 7ff8533de740 1 -- 172.21.15.1:0/640249989 >> 172.21.15.1:0/640249989 conn(0x55ef6f3a8850 msgr2=0x55ef6f3a8c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:36:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.102+0000 7ff846000640 1 --2- 172.21.15.1:0/2935902955 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff8300a2b70 0x7ff8300a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.102+0000 7ff8533de740 1 -- 172.21.15.1:0/640249989 shutdown_connections 2024-09-12T01:36:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.102+0000 7ff846000640 1 -- 172.21.15.1:0/2935902955 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7ff83c0cf790 con 0x7ff848068d30 2024-09-12T01:36:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.102+0000 7ff8533de740 1 -- 172.21.15.1:0/640249989 wait complete. 2024-09-12T01:36:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.103+0000 7ff84e600640 1 --2- 172.21.15.1:0/2935902955 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff8300a2b70 0x7ff8300a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.103+0000 7ff8533de740 1 -- start start 2024-09-12T01:36:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.103+0000 7ff8533de740 1 Processor -- start 2024-09-12T01:36:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.103+0000 7ff84e600640 1 --2- 172.21.15.1:0/2935902955 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff8300a2b70 0x7ff8300a5030 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff840011dd0 tx=0x7ff840009d60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:36:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.104+0000 7ff8533de740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef6f3ab260 0x55ef6f3bfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.104+0000 7ff8533de740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef6f3abba0 0x55ef6f3c00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:02.109 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.104+0000 7ff8533de740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef6f3bb480 0x55ef6f3c7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:02.109 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.104+0000 7ff8533de740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ef6f3be400 con 0x55ef6f3bb480 2024-09-12T01:36:02.109 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.104+0000 7ff8533de740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ef6f29b090 con 0x55ef6f3abba0 2024-09-12T01:36:02.109 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.104+0000 7ff8533de740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ef6f3be580 con 0x55ef6f3ab260 2024-09-12T01:36:02.109 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.104+0000 7ff8533de740 10 client.0 _ll_register_callbacks cb 0x55ef6f2eff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:36:02.109 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.104+0000 7ff84e600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef6f3bb480 0x55ef6f3c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:02.109 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.104+0000 7ff84dc00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef6f3ab260 0x55ef6f3bfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:02.109 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.104+0000 7ff84e600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef6f3bb480 0x55ef6f3c7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46032/0 (socket says 172.21.15.1:46032) 2024-09-12T01:36:02.109 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.104+0000 7ff84e600640 1 -- 172.21.15.1:0/3581268844 learned_addr learned my addr 172.21.15.1:0/3581268844 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:36:02.109 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.104+0000 7ff84d200640 1 --2- 172.21.15.1:0/3581268844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef6f3abba0 0x55ef6f3c00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:02.109 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.104+0000 7ff84dc00640 1 -- 172.21.15.1:0/3581268844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef6f3abba0 msgr2=0x55ef6f3c00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:36:02.109 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.104+0000 7ff84dc00640 1 --2- 172.21.15.1:0/3581268844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef6f3abba0 0x55ef6f3c00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:02.109 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.104+0000 7ff84dc00640 1 -- 172.21.15.1:0/3581268844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef6f3bb480 msgr2=0x55ef6f3c7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:36:02.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.104+0000 7ff84dc00640 1 --2- 172.21.15.1:0/3581268844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef6f3bb480 0x55ef6f3c7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:02.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.104+0000 7ff84dc00640 1 -- 172.21.15.1:0/3581268844 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef6f0f1620 con 0x55ef6f3ab260 2024-09-12T01:36:02.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.104+0000 7ff84d200640 1 --2- 172.21.15.1:0/3581268844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef6f3abba0 0x55ef6f3c00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:36:02.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.104+0000 7ff84e600640 1 --2- 172.21.15.1:0/3581268844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef6f3bb480 0x55ef6f3c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:36:02.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.105+0000 7ff84dc00640 1 --2- 172.21.15.1:0/3581268844 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef6f3ab260 0x55ef6f3bfbb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff83c0ccf50 tx=0x7ff83c0978a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b69ee6a01a18330 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:36:02.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.105+0000 7ff82f400640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-12T01:36:02.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.105+0000 7ff82f400640 1 -- 172.21.15.1:0/3581268844 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83c09b5c0 con 0x55ef6f3ab260 2024-09-12T01:36:02.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.105+0000 7ff8533de740 1 -- 172.21.15.1:0/3581268844 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff840004050 con 0x55ef6f3ab260 2024-09-12T01:36:02.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.105+0000 7ff82f400640 1 -- 172.21.15.1:0/3581268844 <== mon.1 v2:172.21.15.104:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7ff83c09b0b0 con 0x55ef6f3ab260 2024-09-12T01:36:02.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.105+0000 7ff82f400640 1 -- 172.21.15.1:0/3581268844 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff83c0b4e60 con 0x55ef6f3ab260 2024-09-12T01:36:02.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.105+0000 7ff8533de740 10 client.25699 fetch_fsmap learned FSMap version 35 2024-09-12T01:36:02.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.105+0000 7ff8533de740 1 -- 172.21.15.1:0/3581268844 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ef6f3c9fe0 con 0x55ef6f3ab260 2024-09-12T01:36:02.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.105+0000 7ff82f400640 1 -- 172.21.15.1:0/3581268844 <== mon.1 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff840004050 con 0x55ef6f3ab260 2024-09-12T01:36:02.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.106+0000 7ff82f400640 1 -- 172.21.15.1:0/3581268844 <== mon.1 v2:172.21.15.104:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7ff83c09ab20 con 0x55ef6f3ab260 2024-09-12T01:36:02.111 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.106+0000 7ff8533de740 10 client.25699 fetch_fsmap finished waiting for FSMap version 35 2024-09-12T01:36:02.111 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.106+0000 7ff8533de740 10 client.25699 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:36:02.111 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.106+0000 7ff8533de740 20 client.25699 populate_metadata read hostname 'smithi001' 2024-09-12T01:36:02.111 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.106+0000 7ff8533de740 1 --2- 172.21.15.1:0/3581268844 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55ef6f3cb6b0 0x55ef6f3cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:02.111 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.106+0000 7ff8533de740 4 client.25699 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:36:02.111 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.106+0000 7ff8533de740 1 -- 172.21.15.1:0/3581268844 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef6f31f830 con 0x55ef6f3cb6b0 2024-09-12T01:36:02.111 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.106+0000 7ff84d200640 1 --2- 172.21.15.1:0/3581268844 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55ef6f3cb6b0 0x55ef6f3cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.107+0000 7ff84d200640 1 --2- 172.21.15.1:0/3581268844 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55ef6f3cb6b0 0x55ef6f3cdac0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cda28a559dc920a2 server_cookie=6ada4b22978e1552 in_seq=0 out_seq=0 2024-09-12T01:36:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.107+0000 7ff82f400640 10 client.25699 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:36:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.107+0000 7ff82f400640 1 -- 172.21.15.1:0/3581268844 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ef6f31f830 con 0x55ef6f3cb6b0 2024-09-12T01:36:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.107+0000 7ff82f400640 10 client.25699 handle_command_reply: tid=0 2024-09-12T01:36:02.113 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.107+0000 7ff8533de740 1 client.25699 shutdown 2024-09-12T01:36:02.113 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.108+0000 7ff8533de740 1 -- 172.21.15.1:0/3581268844 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef6f3ab260 msgr2=0x55ef6f3bfbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:02.113 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.108+0000 7ff8533de740 1 --2- 172.21.15.1:0/3581268844 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef6f3ab260 0x55ef6f3bfbb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff83c0ccf50 tx=0x7ff83c0978a0 comp rx=0 tx=0).stop 2024-09-12T01:36:02.113 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.108+0000 7ff8533de740 1 -- 172.21.15.1:0/3581268844 shutdown_connections 2024-09-12T01:36:02.113 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.108+0000 7ff8533de740 1 --2- 172.21.15.1:0/3581268844 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55ef6f3cb6b0 0x55ef6f3cdac0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:02.113 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.108+0000 7ff8533de740 1 --2- 172.21.15.1:0/3581268844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef6f3bb480 0x55ef6f3c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:02.113 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.108+0000 7ff8533de740 1 --2- 172.21.15.1:0/3581268844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef6f3abba0 0x55ef6f3c00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:02.113 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.108+0000 7ff8533de740 1 --2- 172.21.15.1:0/3581268844 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef6f3ab260 0x55ef6f3bfbb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:02.113 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.108+0000 7ff8533de740 1 -- 172.21.15.1:0/3581268844 >> 172.21.15.1:0/3581268844 conn(0x55ef6f3a8850 msgr2=0x55ef6f3aa060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:36:02.113 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.108+0000 7ff82f400640 0 client.25699 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:36:02.113 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.108+0000 7ff8533de740 1 -- 172.21.15.1:0/3581268844 shutdown_connections 2024-09-12T01:36:02.113 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.108+0000 7ff8533de740 1 -- 172.21.15.1:0/3581268844 wait complete. 2024-09-12T01:36:02.113 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.108+0000 7ff8533de740 20 client.25699 trim_cache size 0 max 16384 2024-09-12T01:36:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.139+0000 7ff8533de740 1 Processor -- start 2024-09-12T01:36:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.139+0000 7ff8533de740 1 -- start start 2024-09-12T01:36:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.139+0000 7ff8533de740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef6f3ab260 0x55ef6f3be950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.139+0000 7ff8533de740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef6f3abba0 0x55ef6f3c70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.139+0000 7ff8533de740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef6f3bb480 0x55ef6f3a71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.139+0000 7ff8533de740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff8481a8400 con 0x55ef6f3abba0 2024-09-12T01:36:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.139+0000 7ff8533de740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff8481a9f40 con 0x55ef6f3ab260 2024-09-12T01:36:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.139+0000 7ff8533de740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff8481a8ad0 con 0x55ef6f3bb480 2024-09-12T01:36:02.141 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.140+0000 7ff84dc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef6f3ab260 0x55ef6f3be950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:02.141 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.140+0000 7ff84e600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef6f3bb480 0x55ef6f3a71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:02.141 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.140+0000 7ff84dc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef6f3ab260 0x55ef6f3be950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:56292/0 (socket says 172.21.15.1:56292) 2024-09-12T01:36:02.141 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.140+0000 7ff84dc00640 1 -- 172.21.15.1:0/3110451681 learned_addr learned my addr 172.21.15.1:0/3110451681 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:36:02.141 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.140+0000 7ff84d200640 1 --2- 172.21.15.1:0/3110451681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef6f3abba0 0x55ef6f3c70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:02.141 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.140+0000 7ff84e600640 1 -- 172.21.15.1:0/3110451681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef6f3ab260 msgr2=0x55ef6f3be950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:36:02.141 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.140+0000 7ff84e600640 1 --2- 172.21.15.1:0/3110451681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef6f3ab260 0x55ef6f3be950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:02.141 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.140+0000 7ff84e600640 1 -- 172.21.15.1:0/3110451681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef6f3abba0 msgr2=0x55ef6f3c70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:36:02.141 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.140+0000 7ff84e600640 1 --2- 172.21.15.1:0/3110451681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef6f3abba0 0x55ef6f3c70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:02.141 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.140+0000 7ff84e600640 1 -- 172.21.15.1:0/3110451681 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef6f3ab640 con 0x55ef6f3bb480 2024-09-12T01:36:02.141 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.140+0000 7ff84e600640 1 --2- 172.21.15.1:0/3110451681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef6f3bb480 0x55ef6f3a71f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff840003ce0 tx=0x7ff840021000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b11b32e51ceaa80d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:36:02.141 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.141+0000 7ff83a000640 1 -- 172.21.15.1:0/3110451681 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84002b040 con 0x55ef6f3bb480 2024-09-12T01:36:02.141 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.141+0000 7ff83a000640 1 -- 172.21.15.1:0/3110451681 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff840036070 con 0x55ef6f3bb480 2024-09-12T01:36:02.142 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.141+0000 7ff83a000640 1 -- 172.21.15.1:0/3110451681 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84000f740 con 0x55ef6f3bb480 2024-09-12T01:36:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.142+0000 7ff8533de740 1 -- 172.21.15.1:0/3110451681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef6f3bb480 msgr2=0x55ef6f3a71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.142+0000 7ff8533de740 1 --2- 172.21.15.1:0/3110451681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef6f3bb480 0x55ef6f3a71f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff840003ce0 tx=0x7ff840021000 comp rx=0 tx=0).stop 2024-09-12T01:36:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.142+0000 7ff8533de740 1 -- 172.21.15.1:0/3110451681 shutdown_connections 2024-09-12T01:36:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.142+0000 7ff8533de740 1 --2- 172.21.15.1:0/3110451681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef6f3bb480 0x55ef6f3a71f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.142+0000 7ff8533de740 1 --2- 172.21.15.1:0/3110451681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef6f3abba0 0x55ef6f3c70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.142+0000 7ff8533de740 1 --2- 172.21.15.1:0/3110451681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef6f3ab260 0x55ef6f3be950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.142+0000 7ff8533de740 1 -- 172.21.15.1:0/3110451681 >> 172.21.15.1:0/3110451681 conn(0x55ef6f3a8850 msgr2=0x55ef6f3bfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:36:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.142+0000 7ff8533de740 1 -- 172.21.15.1:0/3110451681 shutdown_connections 2024-09-12T01:36:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.142+0000 7ff8533de740 1 -- 172.21.15.1:0/3110451681 wait complete. 2024-09-12T01:36:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.142+0000 7ff8533de740 1 -- start start 2024-09-12T01:36:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.142+0000 7ff8533de740 1 Processor -- start 2024-09-12T01:36:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.143+0000 7ff8533de740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef6f3ab260 0x55ef6f3c4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.143+0000 7ff8533de740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef6f3abba0 0x55ef6f3c4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.143+0000 7ff8533de740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef6f3bb480 0x55ef6f3c5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.143+0000 7ff8533de740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff8481a9870 con 0x55ef6f3abba0 2024-09-12T01:36:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.143+0000 7ff8533de740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff8481a91a0 con 0x55ef6f3ab260 2024-09-12T01:36:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.143+0000 7ff8533de740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff8481a63e0 con 0x55ef6f3bb480 2024-09-12T01:36:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.143+0000 7ff8533de740 10 client.0 _ll_register_callbacks cb 0x7ff83c0d5240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:36:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.143+0000 7ff84dc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef6f3ab260 0x55ef6f3c4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.143+0000 7ff84dc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef6f3ab260 0x55ef6f3c4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:56304/0 (socket says 172.21.15.1:56304) 2024-09-12T01:36:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.143+0000 7ff84d200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef6f3abba0 0x55ef6f3c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.143+0000 7ff84dc00640 1 -- 172.21.15.1:0/1227644308 learned_addr learned my addr 172.21.15.1:0/1227644308 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:36:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.143+0000 7ff84e600640 1 --2- 172.21.15.1:0/1227644308 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef6f3bb480 0x55ef6f3c5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.144+0000 7ff84dc00640 1 -- 172.21.15.1:0/1227644308 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef6f3bb480 msgr2=0x55ef6f3c5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:36:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.144+0000 7ff84dc00640 1 --2- 172.21.15.1:0/1227644308 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef6f3bb480 0x55ef6f3c5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.144+0000 7ff84dc00640 1 -- 172.21.15.1:0/1227644308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef6f3abba0 msgr2=0x55ef6f3c4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:36:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.144+0000 7ff84dc00640 1 --2- 172.21.15.1:0/1227644308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef6f3abba0 0x55ef6f3c4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.144+0000 7ff84dc00640 1 -- 172.21.15.1:0/1227644308 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef6f3c5c40 con 0x55ef6f3ab260 2024-09-12T01:36:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.144+0000 7ff84dc00640 1 --2- 172.21.15.1:0/1227644308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef6f3ab260 0x55ef6f3c4130 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff83c0ccf50 tx=0x7ff83c003d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9952af7b21b55504 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:36:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.144+0000 7ff82f400640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-12T01:36:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.144+0000 7ff82f400640 1 -- 172.21.15.1:0/1227644308 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83c09ba30 con 0x55ef6f3ab260 2024-09-12T01:36:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.144+0000 7ff82f400640 1 -- 172.21.15.1:0/1227644308 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7ff83c0ba330 con 0x55ef6f3ab260 2024-09-12T01:36:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.144+0000 7ff8533de740 1 -- 172.21.15.1:0/1227644308 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff840036070 con 0x55ef6f3ab260 2024-09-12T01:36:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.144+0000 7ff82f400640 1 -- 172.21.15.1:0/1227644308 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff83c09c360 con 0x55ef6f3ab260 2024-09-12T01:36:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.144+0000 7ff8533de740 10 client.25706 fetch_fsmap learned FSMap version 35 2024-09-12T01:36:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.144+0000 7ff8533de740 1 -- 172.21.15.1:0/1227644308 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ef6f3c6f50 con 0x55ef6f3ab260 2024-09-12T01:36:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.145+0000 7ff82f400640 1 -- 172.21.15.1:0/1227644308 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff840036070 con 0x55ef6f3ab260 2024-09-12T01:36:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.145+0000 7ff82f400640 1 -- 172.21.15.1:0/1227644308 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7ff83c0a11a0 con 0x55ef6f3ab260 2024-09-12T01:36:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.145+0000 7ff8533de740 10 client.25706 fetch_fsmap finished waiting for FSMap version 35 2024-09-12T01:36:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.145+0000 7ff8533de740 10 client.25706 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:36:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.145+0000 7ff8533de740 20 client.25706 populate_metadata read hostname 'smithi001' 2024-09-12T01:36:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.145+0000 7ff8533de740 1 --2- 172.21.15.1:0/1227644308 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55ef6f436610 0x55ef6f438a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:02.148 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.145+0000 7ff8533de740 4 client.25706 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:36:02.148 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.145+0000 7ff8533de740 1 -- 172.21.15.1:0/1227644308 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ef6f319b80 con 0x55ef6f436610 2024-09-12T01:36:02.148 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.145+0000 7ff84d200640 1 --2- 172.21.15.1:0/1227644308 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55ef6f436610 0x55ef6f438a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:02.148 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.146+0000 7ff82f400640 10 client.25706 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:36:02.148 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.146+0000 7ff84d200640 1 --2- 172.21.15.1:0/1227644308 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55ef6f436610 0x55ef6f438a30 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61f2b90d84dd131e server_cookie=189dcc825d444020 in_seq=0 out_seq=0 2024-09-12T01:36:02.158 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.158+0000 7ff82f400640 1 -- 172.21.15.1:0/1227644308 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ef6f319b80 con 0x55ef6f436610 2024-09-12T01:36:02.158 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.158+0000 7ff82f400640 10 client.25706 handle_command_reply: tid=0 2024-09-12T01:36:02.158 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.158+0000 7ff8533de740 1 client.25706 shutdown 2024-09-12T01:36:02.158 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.158+0000 7ff8533de740 1 -- 172.21.15.1:0/1227644308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef6f3ab260 msgr2=0x55ef6f3c4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:02.158 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.158+0000 7ff8533de740 1 --2- 172.21.15.1:0/1227644308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef6f3ab260 0x55ef6f3c4130 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff83c0ccf50 tx=0x7ff83c003d60 comp rx=0 tx=0).stop 2024-09-12T01:36:02.158 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.158+0000 7ff8533de740 1 -- 172.21.15.1:0/1227644308 shutdown_connections 2024-09-12T01:36:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.158+0000 7ff8533de740 1 --2- 172.21.15.1:0/1227644308 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55ef6f436610 0x55ef6f438a30 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.158+0000 7ff8533de740 1 --2- 172.21.15.1:0/1227644308 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef6f3bb480 0x55ef6f3c5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.158+0000 7ff8533de740 1 --2- 172.21.15.1:0/1227644308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef6f3abba0 0x55ef6f3c4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.158+0000 7ff82f400640 0 client.25706 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:36:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.158+0000 7ff8533de740 1 --2- 172.21.15.1:0/1227644308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef6f3ab260 0x55ef6f3c4130 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.158+0000 7ff8533de740 1 -- 172.21.15.1:0/1227644308 >> 172.21.15.1:0/1227644308 conn(0x55ef6f3a8850 msgr2=0x55ef6f3a9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:36:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.158+0000 7ff8533de740 1 -- 172.21.15.1:0/1227644308 shutdown_connections 2024-09-12T01:36:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.158+0000 7ff8533de740 1 -- 172.21.15.1:0/1227644308 wait complete. 2024-09-12T01:36:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.158+0000 7ff8533de740 20 client.25706 trim_cache size 0 max 16384 2024-09-12T01:36:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-12T01:36:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-12T01:36:02.160 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-12T01:36:02.160 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-12T01:36:02.160 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.159+0000 7ff84f000640 1 -- 172.21.15.1:0/2935902955 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff8300a2b70 msgr2=0x7ff8300a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:02.160 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.159+0000 7ff84f000640 1 --2- 172.21.15.1:0/2935902955 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff8300a2b70 0x7ff8300a5030 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff840011dd0 tx=0x7ff840009d60 comp rx=0 tx=0).stop 2024-09-12T01:36:02.160 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.159+0000 7ff84f000640 1 -- 172.21.15.1:0/2935902955 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff848068d30 msgr2=0x7ff848197c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:02.160 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.159+0000 7ff84f000640 1 --2- 172.21.15.1:0/2935902955 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff848068d30 0x7ff848197c70 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff83c00e9c0 tx=0x7ff83c00ee80 comp rx=0 tx=0).stop 2024-09-12T01:36:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.159+0000 7ff84f000640 1 -- 172.21.15.1:0/2935902955 shutdown_connections 2024-09-12T01:36:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.159+0000 7ff84f000640 1 --2- 172.21.15.1:0/2935902955 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff848106010 0x7ff84819f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.159+0000 7ff84f000640 1 --2- 172.21.15.1:0/2935902955 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ff8300a2b70 0x7ff8300a5030 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.159+0000 7ff84f000640 1 --2- 172.21.15.1:0/2935902955 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff848101310 0x7ff8481981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.159+0000 7ff84f000640 1 --2- 172.21.15.1:0/2935902955 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff848068d30 0x7ff848197c70 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.159+0000 7ff84f000640 1 -- 172.21.15.1:0/2935902955 >> 172.21.15.1:0/2935902955 conn(0x7ff8480fb340 msgr2=0x7ff848101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:36:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.159+0000 7ff84f000640 1 -- 172.21.15.1:0/2935902955 shutdown_connections 2024-09-12T01:36:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:02.159+0000 7ff84f000640 1 -- 172.21.15.1:0/2935902955 wait complete. 2024-09-12T01:36:02.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:02 smithi104 ceph-mon[35219]: pgmap v591: 129 pgs: 129 active+clean; 116 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.3 MiB/s wr, 535 op/s 2024-09-12T01:36:02.390 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:02 smithi053 ceph-mon[35158]: pgmap v591: 129 pgs: 129 active+clean; 116 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.3 MiB/s wr, 535 op/s 2024-09-12T01:36:02.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:02 smithi001 ceph-mon[30387]: pgmap v591: 129 pgs: 129 active+clean; 116 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.3 MiB/s wr, 535 op/s 2024-09-12T01:36:04.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:04 smithi104 ceph-mon[35219]: pgmap v592: 129 pgs: 129 active+clean; 115 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 1.1 MiB/s wr, 443 op/s 2024-09-12T01:36:04.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:04 smithi001 ceph-mon[30387]: pgmap v592: 129 pgs: 129 active+clean; 115 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 1.1 MiB/s wr, 443 op/s 2024-09-12T01:36:04.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:04 smithi053 ceph-mon[35158]: pgmap v592: 129 pgs: 129 active+clean; 115 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 1.1 MiB/s wr, 443 op/s 2024-09-12T01:36:06.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:06 smithi104 ceph-mon[35219]: pgmap v593: 129 pgs: 129 active+clean; 118 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.2 MiB/s wr, 457 op/s 2024-09-12T01:36:06.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:06 smithi001 ceph-mon[30387]: pgmap v593: 129 pgs: 129 active+clean; 118 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.2 MiB/s wr, 457 op/s 2024-09-12T01:36:06.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:06 smithi053 ceph-mon[35158]: pgmap v593: 129 pgs: 129 active+clean; 118 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.2 MiB/s wr, 457 op/s 2024-09-12T01:36:08.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:08 smithi104 ceph-mon[35219]: pgmap v594: 129 pgs: 129 active+clean; 118 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.2 MiB/s wr, 500 op/s 2024-09-12T01:36:08.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:08 smithi001 ceph-mon[30387]: pgmap v594: 129 pgs: 129 active+clean; 118 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.2 MiB/s wr, 500 op/s 2024-09-12T01:36:08.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:08 smithi053 ceph-mon[35158]: pgmap v594: 129 pgs: 129 active+clean; 118 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.2 MiB/s wr, 500 op/s 2024-09-12T01:36:10.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:10 smithi104 ceph-mon[35219]: pgmap v595: 129 pgs: 129 active+clean; 118 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 968 KiB/s wr, 384 op/s 2024-09-12T01:36:10.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:10 smithi001 ceph-mon[30387]: pgmap v595: 129 pgs: 129 active+clean; 118 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 968 KiB/s wr, 384 op/s 2024-09-12T01:36:10.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:10 smithi053 ceph-mon[35158]: pgmap v595: 129 pgs: 129 active+clean; 118 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 968 KiB/s wr, 384 op/s 2024-09-12T01:36:12.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:12 smithi104 ceph-mon[35219]: pgmap v596: 129 pgs: 129 active+clean; 119 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.3 MiB/s wr, 538 op/s 2024-09-12T01:36:12.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:12 smithi001 ceph-mon[30387]: pgmap v596: 129 pgs: 129 active+clean; 119 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.3 MiB/s wr, 538 op/s 2024-09-12T01:36:12.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:12 smithi053 ceph-mon[35158]: pgmap v596: 129 pgs: 129 active+clean; 119 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.3 MiB/s wr, 538 op/s 2024-09-12T01:36:14.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:14 smithi104 ceph-mon[35219]: pgmap v597: 129 pgs: 129 active+clean; 123 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 948 KiB/s wr, 411 op/s 2024-09-12T01:36:14.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:14 smithi001 ceph-mon[30387]: pgmap v597: 129 pgs: 129 active+clean; 123 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 948 KiB/s wr, 411 op/s 2024-09-12T01:36:14.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:14 smithi053 ceph-mon[35158]: pgmap v597: 129 pgs: 129 active+clean; 123 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 948 KiB/s wr, 411 op/s 2024-09-12T01:36:15.321 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.321+0000 7fde9b800640 1 -- 172.21.15.1:0/1476378419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde94104f90 msgr2=0x7fde94107380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:15.321 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.321+0000 7fde9b800640 1 --2- 172.21.15.1:0/1476378419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde94104f90 0x7fde94107380 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fde80009900 tx=0x7fde8002a3f0 comp rx=0 tx=0).stop 2024-09-12T01:36:15.321 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.321+0000 7fde9b800640 1 -- 172.21.15.1:0/1476378419 shutdown_connections 2024-09-12T01:36:15.322 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.321+0000 7fde9b800640 1 --2- 172.21.15.1:0/1476378419 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde941078c0 0x7fde94109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:15.322 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.321+0000 7fde9b800640 1 --2- 172.21.15.1:0/1476378419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde94104f90 0x7fde94107380 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:15.322 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.321+0000 7fde9b800640 1 --2- 172.21.15.1:0/1476378419 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde94102660 0x7fde94104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:15.322 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.321+0000 7fde9b800640 1 -- 172.21.15.1:0/1476378419 >> 172.21.15.1:0/1476378419 conn(0x7fde940fc1c0 msgr2=0x7fde940fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:36:15.322 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.321+0000 7fde9b800640 1 -- 172.21.15.1:0/1476378419 shutdown_connections 2024-09-12T01:36:15.322 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.321+0000 7fde9b800640 1 -- 172.21.15.1:0/1476378419 wait complete. 2024-09-12T01:36:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.322+0000 7fde9b800640 1 Processor -- start 2024-09-12T01:36:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.322+0000 7fde9b800640 1 -- start start 2024-09-12T01:36:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.323+0000 7fde9b800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde94102660 0x7fde9419a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.323+0000 7fde9b800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde94104f90 0x7fde9419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.323+0000 7fde9b800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde941078c0 0x7fde941a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.323+0000 7fde9b800640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fde9410d820 con 0x7fde94102660 2024-09-12T01:36:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.323+0000 7fde9b800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fde9410d6a0 con 0x7fde94104f90 2024-09-12T01:36:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.323+0000 7fde9b800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fde9410d9a0 con 0x7fde941078c0 2024-09-12T01:36:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.323+0000 7fde99800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde941078c0 0x7fde941a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.323+0000 7fde93e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde94104f90 0x7fde9419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.323+0000 7fde99800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde941078c0 0x7fde941a1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:37758/0 (socket says 172.21.15.1:37758) 2024-09-12T01:36:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.323+0000 7fde98e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde94102660 0x7fde9419a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.323+0000 7fde99800640 1 -- 172.21.15.1:0/3658179982 learned_addr learned my addr 172.21.15.1:0/3658179982 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:36:15.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.323+0000 7fde93e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde94104f90 0x7fde9419a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:35664/0 (socket says 172.21.15.1:35664) 2024-09-12T01:36:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.324+0000 7fde99800640 1 -- 172.21.15.1:0/3658179982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde94104f90 msgr2=0x7fde9419a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.324+0000 7fde99800640 1 --2- 172.21.15.1:0/3658179982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde94104f90 0x7fde9419a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.324+0000 7fde99800640 1 -- 172.21.15.1:0/3658179982 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde94102660 msgr2=0x7fde9419a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.324+0000 7fde99800640 1 --2- 172.21.15.1:0/3658179982 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde94102660 0x7fde9419a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.324+0000 7fde99800640 1 -- 172.21.15.1:0/3658179982 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde941a1d00 con 0x7fde941078c0 2024-09-12T01:36:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.324+0000 7fde99800640 1 --2- 172.21.15.1:0/3658179982 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde941078c0 0x7fde941a1690 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fde8c00dac0 tx=0x7fde8c00df80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:36:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.324+0000 7fde91600640 1 -- 172.21.15.1:0/3658179982 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c017070 con 0x7fde941078c0 2024-09-12T01:36:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.324+0000 7fde9b800640 1 -- 172.21.15.1:0/3658179982 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde941a1ff0 con 0x7fde941078c0 2024-09-12T01:36:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.324+0000 7fde91600640 1 -- 172.21.15.1:0/3658179982 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde8c00be30 con 0x7fde941078c0 2024-09-12T01:36:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.324+0000 7fde91600640 1 -- 172.21.15.1:0/3658179982 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c00f890 con 0x7fde941078c0 2024-09-12T01:36:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.324+0000 7fde9b800640 1 -- 172.21.15.1:0/3658179982 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde941a24b0 con 0x7fde941078c0 2024-09-12T01:36:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.328+0000 7fde91600640 1 -- 172.21.15.1:0/3658179982 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fde8c00fb70 con 0x7fde941078c0 2024-09-12T01:36:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.329+0000 7fde9e83b740 1 Processor -- start 2024-09-12T01:36:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.330+0000 7fde91600640 1 --2- 172.21.15.1:0/3658179982 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fde7c0a2c00 0x7fde7c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.330+0000 7fde9e83b740 1 -- start start 2024-09-12T01:36:15.331 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.330+0000 7fde91600640 1 -- 172.21.15.1:0/3658179982 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fde8c0cf250 con 0x7fde941078c0 2024-09-12T01:36:15.332 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.330+0000 7fde9e83b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628c9659090 0x5628c9659490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:15.332 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.330+0000 7fde9e83b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628c96599d0 0x5628c965e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:15.332 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.330+0000 7fde93e00640 1 --2- 172.21.15.1:0/3658179982 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fde7c0a2c00 0x7fde7c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:15.332 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.330+0000 7fde9e83b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628c965ea60 0x5628c9660f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:15.332 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.330+0000 7fde9e83b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5628c95be100 con 0x5628c96599d0 2024-09-12T01:36:15.332 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.330+0000 7fde9e83b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5628c95661b0 con 0x5628c9659090 2024-09-12T01:36:15.332 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.330+0000 7fde9e83b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5628c95ba0c0 con 0x5628c965ea60 2024-09-12T01:36:15.332 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.330+0000 7fde93e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628c96599d0 0x5628c965e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:15.332 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.330+0000 7fde93e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628c96599d0 0x5628c965e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53378/0 (socket says 172.21.15.1:53378) 2024-09-12T01:36:15.332 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.330+0000 7fde93e00640 1 -- 172.21.15.1:0/1368855588 learned_addr learned my addr 172.21.15.1:0/1368855588 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:36:15.332 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.331+0000 7fde98e00640 1 --2- 172.21.15.1:0/1368855588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628c9659090 0x5628c9659490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:15.332 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.331+0000 7fde99800640 1 --2- 172.21.15.1:0/1368855588 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628c965ea60 0x5628c9660f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.331+0000 7fde99800640 1 -- 172.21.15.1:0/1368855588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628c9659090 msgr2=0x5628c9659490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:36:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.331+0000 7fde99800640 1 --2- 172.21.15.1:0/1368855588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628c9659090 0x5628c9659490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.331+0000 7fde99800640 1 -- 172.21.15.1:0/1368855588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628c96599d0 msgr2=0x5628c965e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:36:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.331+0000 7fde99800640 1 --2- 172.21.15.1:0/1368855588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628c96599d0 0x5628c965e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.331+0000 7fde99800640 1 -- 172.21.15.1:0/1368855588 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628c92d1210 con 0x5628c965ea60 2024-09-12T01:36:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.331+0000 7fde98e00640 1 --2- 172.21.15.1:0/1368855588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628c9659090 0x5628c9659490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:36:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.331+0000 7fde93e00640 1 --2- 172.21.15.1:0/1368855588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628c96599d0 0x5628c965e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:36:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.331+0000 7fde93e00640 1 --2- 172.21.15.1:0/3658179982 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fde7c0a2c00 0x7fde7c0a50c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fde80031440 tx=0x7fde80004040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:36:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.331+0000 7fde99800640 1 --2- 172.21.15.1:0/1368855588 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628c965ea60 0x5628c9660f20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fde8c00b6f0 tx=0x7fde8c09b770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cffac12f6f73ef4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:36:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.331+0000 7fde86000640 1 -- 172.21.15.1:0/1368855588 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c020070 con 0x5628c965ea60 2024-09-12T01:36:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.332+0000 7fde86000640 1 -- 172.21.15.1:0/1368855588 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde8c099a70 con 0x5628c965ea60 2024-09-12T01:36:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.332+0000 7fde86000640 1 -- 172.21.15.1:0/1368855588 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c09abc0 con 0x5628c965ea60 2024-09-12T01:36:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.332+0000 7fde9e83b740 1 -- 172.21.15.1:0/1368855588 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628c965ea60 msgr2=0x5628c9660f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.332+0000 7fde9e83b740 1 --2- 172.21.15.1:0/1368855588 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628c965ea60 0x5628c9660f20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fde8c00b6f0 tx=0x7fde8c09b770 comp rx=0 tx=0).stop 2024-09-12T01:36:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.333+0000 7fde9e83b740 1 -- 172.21.15.1:0/1368855588 shutdown_connections 2024-09-12T01:36:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.333+0000 7fde9e83b740 1 --2- 172.21.15.1:0/1368855588 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628c965ea60 0x5628c9660f20 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.333+0000 7fde9e83b740 1 --2- 172.21.15.1:0/1368855588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628c96599d0 0x5628c965e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.333+0000 7fde9e83b740 1 --2- 172.21.15.1:0/1368855588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628c9659090 0x5628c9659490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.333+0000 7fde9e83b740 1 -- 172.21.15.1:0/1368855588 >> 172.21.15.1:0/1368855588 conn(0x5628c9654700 msgr2=0x5628c9654b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:36:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.333+0000 7fde9e83b740 1 -- 172.21.15.1:0/1368855588 shutdown_connections 2024-09-12T01:36:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.333+0000 7fde9e83b740 1 -- 172.21.15.1:0/1368855588 wait complete. 2024-09-12T01:36:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.333+0000 7fde9e83b740 1 -- start start 2024-09-12T01:36:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.333+0000 7fde9e83b740 1 Processor -- start 2024-09-12T01:36:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.334+0000 7fde9e83b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628c9659090 0x5628c9663330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:15.342 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.334+0000 7fde9e83b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628c96599d0 0x5628c9663870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:15.342 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.334+0000 7fde9e83b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628c965ea60 0x5628c966a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:15.342 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.334+0000 7fde9e83b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5628c96619d0 con 0x5628c9659090 2024-09-12T01:36:15.342 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.334+0000 7fde9e83b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5628c9547090 con 0x5628c965ea60 2024-09-12T01:36:15.342 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.334+0000 7fde9e83b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5628c9661b50 con 0x5628c96599d0 2024-09-12T01:36:15.342 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.334+0000 7fde9e83b740 10 client.0 _ll_register_callbacks cb 0x5628c959bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:36:15.342 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.334+0000 7fde98e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628c9659090 0x5628c9663330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:15.342 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.334+0000 7fde99800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628c965ea60 0x5628c966a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:15.343 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.334+0000 7fde99800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628c965ea60 0x5628c966a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:35688/0 (socket says 172.21.15.1:35688) 2024-09-12T01:36:15.343 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.334+0000 7fde99800640 1 -- 172.21.15.1:0/2213491184 learned_addr learned my addr 172.21.15.1:0/2213491184 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:36:15.343 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.334+0000 7fde93e00640 1 --2- 172.21.15.1:0/2213491184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628c96599d0 0x5628c9663870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:15.343 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.334+0000 7fde99800640 1 -- 172.21.15.1:0/2213491184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628c96599d0 msgr2=0x5628c9663870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:36:15.343 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.334+0000 7fde99800640 1 --2- 172.21.15.1:0/2213491184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628c96599d0 0x5628c9663870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:15.343 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.335+0000 7fde99800640 1 -- 172.21.15.1:0/2213491184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628c9659090 msgr2=0x5628c9663330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:36:15.344 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.335+0000 7fde99800640 1 --2- 172.21.15.1:0/2213491184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628c9659090 0x5628c9663330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:15.344 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.335+0000 7fde99800640 1 -- 172.21.15.1:0/2213491184 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628c939d620 con 0x5628c965ea60 2024-09-12T01:36:15.344 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.335+0000 7fde99800640 1 --2- 172.21.15.1:0/2213491184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628c965ea60 0x5628c966a840 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fde8c012040 tx=0x7fde8c010e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de285f0ce0529520 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:36:15.344 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.335+0000 7fde7aa00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-12T01:36:15.344 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.335+0000 7fde7aa00640 1 -- 172.21.15.1:0/2213491184 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c014030 con 0x5628c965ea60 2024-09-12T01:36:15.344 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.335+0000 7fde7aa00640 1 -- 172.21.15.1:0/2213491184 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fde8c097e50 con 0x5628c965ea60 2024-09-12T01:36:15.344 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.335+0000 7fde9e83b740 1 -- 172.21.15.1:0/2213491184 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde8c099a70 con 0x5628c965ea60 2024-09-12T01:36:15.345 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.335+0000 7fde7aa00640 1 -- 172.21.15.1:0/2213491184 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fde8c020050 con 0x5628c965ea60 2024-09-12T01:36:15.345 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.335+0000 7fde9e83b740 10 client.25718 fetch_fsmap learned FSMap version 35 2024-09-12T01:36:15.345 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.335+0000 7fde9e83b740 1 -- 172.21.15.1:0/2213491184 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5628c966d630 con 0x5628c965ea60 2024-09-12T01:36:15.345 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.335+0000 7fde7aa00640 1 -- 172.21.15.1:0/2213491184 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde8c099a70 con 0x5628c965ea60 2024-09-12T01:36:15.345 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.336+0000 7fde7aa00640 1 -- 172.21.15.1:0/2213491184 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fde8c09a630 con 0x5628c965ea60 2024-09-12T01:36:15.345 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.336+0000 7fde9e83b740 10 client.25718 fetch_fsmap finished waiting for FSMap version 35 2024-09-12T01:36:15.345 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.336+0000 7fde9e83b740 10 client.25718 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:36:15.345 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.336+0000 7fde9e83b740 20 client.25718 populate_metadata read hostname 'smithi001' 2024-09-12T01:36:15.345 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.336+0000 7fde9e83b740 1 --2- 172.21.15.1:0/2213491184 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5628c966e980 0x5628c9670d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:15.346 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.336+0000 7fde9e83b740 4 client.25718 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:36:15.346 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.336+0000 7fde9e83b740 1 -- 172.21.15.1:0/2213491184 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628c95cb830 con 0x5628c966e980 2024-09-12T01:36:15.346 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.336+0000 7fde98e00640 1 --2- 172.21.15.1:0/2213491184 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5628c966e980 0x5628c9670d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:15.346 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.337+0000 7fde98e00640 1 --2- 172.21.15.1:0/2213491184 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5628c966e980 0x5628c9670d70 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d301415ea2f6cba server_cookie=47690b42e521da37 in_seq=0 out_seq=0 2024-09-12T01:36:15.346 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.337+0000 7fde7aa00640 10 client.25718 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:36:15.347 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.337+0000 7fde7aa00640 1 -- 172.21.15.1:0/2213491184 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5628c95cb830 con 0x5628c966e980 2024-09-12T01:36:15.347 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.337+0000 7fde7aa00640 10 client.25718 handle_command_reply: tid=0 2024-09-12T01:36:15.347 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.337+0000 7fde9e83b740 1 client.25718 shutdown 2024-09-12T01:36:15.347 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.338+0000 7fde9e83b740 1 -- 172.21.15.1:0/2213491184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628c965ea60 msgr2=0x5628c966a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:15.347 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.338+0000 7fde9e83b740 1 --2- 172.21.15.1:0/2213491184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628c965ea60 0x5628c966a840 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fde8c012040 tx=0x7fde8c010e50 comp rx=0 tx=0).stop 2024-09-12T01:36:15.347 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.338+0000 7fde9e83b740 1 -- 172.21.15.1:0/2213491184 shutdown_connections 2024-09-12T01:36:15.347 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.338+0000 7fde9e83b740 1 --2- 172.21.15.1:0/2213491184 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5628c966e980 0x5628c9670d70 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:15.347 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.338+0000 7fde9e83b740 1 --2- 172.21.15.1:0/2213491184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628c965ea60 0x5628c966a840 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:15.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.338+0000 7fde9e83b740 1 --2- 172.21.15.1:0/2213491184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628c96599d0 0x5628c9663870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:15.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.338+0000 7fde7aa00640 0 client.25718 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:36:15.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.338+0000 7fde9e83b740 1 --2- 172.21.15.1:0/2213491184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628c9659090 0x5628c9663330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:15.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.338+0000 7fde9e83b740 1 -- 172.21.15.1:0/2213491184 >> 172.21.15.1:0/2213491184 conn(0x5628c9654700 msgr2=0x5628c9655f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:36:15.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.338+0000 7fde9e83b740 1 -- 172.21.15.1:0/2213491184 shutdown_connections 2024-09-12T01:36:15.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.338+0000 7fde9e83b740 1 -- 172.21.15.1:0/2213491184 wait complete. 2024-09-12T01:36:15.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.338+0000 7fde9e83b740 20 client.25718 trim_cache size 0 max 16384 2024-09-12T01:36:15.356 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.355+0000 7fde9e83b740 1 Processor -- start 2024-09-12T01:36:15.356 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.355+0000 7fde9e83b740 1 -- start start 2024-09-12T01:36:15.356 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.355+0000 7fde9e83b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628c9659090 0x5628c9653770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:15.356 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.355+0000 7fde9e83b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628c96599d0 0x5628c966a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:15.356 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.355+0000 7fde9e83b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628c965ea60 0x5628c96754d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:15.356 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.355+0000 7fde9e83b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fde941aca10 con 0x5628c9659090 2024-09-12T01:36:15.357 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.355+0000 7fde9e83b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fde941ae550 con 0x5628c96599d0 2024-09-12T01:36:15.357 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.355+0000 7fde9e83b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fde941ad0e0 con 0x5628c965ea60 2024-09-12T01:36:15.357 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.356+0000 7fde99800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628c965ea60 0x5628c96754d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:15.357 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.356+0000 7fde98e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628c9659090 0x5628c9653770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:15.357 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.356+0000 7fde93e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628c96599d0 0x5628c966a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:15.357 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.356+0000 7fde99800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628c965ea60 0x5628c96754d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:37772/0 (socket says 172.21.15.1:37772) 2024-09-12T01:36:15.357 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.356+0000 7fde99800640 1 -- 172.21.15.1:0/1454126082 learned_addr learned my addr 172.21.15.1:0/1454126082 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:36:15.357 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.356+0000 7fde99800640 1 -- 172.21.15.1:0/1454126082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628c96599d0 msgr2=0x5628c966a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:36:15.357 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.356+0000 7fde99800640 1 --2- 172.21.15.1:0/1454126082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628c96599d0 0x5628c966a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:15.357 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.356+0000 7fde99800640 1 -- 172.21.15.1:0/1454126082 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628c9659090 msgr2=0x5628c9653770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:36:15.357 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.356+0000 7fde99800640 1 --2- 172.21.15.1:0/1454126082 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628c9659090 0x5628c9653770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:15.357 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.356+0000 7fde99800640 1 -- 172.21.15.1:0/1454126082 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628c964fd60 con 0x5628c965ea60 2024-09-12T01:36:15.357 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.356+0000 7fde99800640 1 --2- 172.21.15.1:0/1454126082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628c965ea60 0x5628c96754d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fde8c099a20 tx=0x7fde8c0cc9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5509b48c8c65757 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:36:15.358 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.356+0000 7fde85600640 1 -- 172.21.15.1:0/1454126082 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c014070 con 0x5628c965ea60 2024-09-12T01:36:15.358 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.356+0000 7fde85600640 1 -- 172.21.15.1:0/1454126082 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde8c0b4070 con 0x5628c965ea60 2024-09-12T01:36:15.358 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.356+0000 7fde85600640 1 -- 172.21.15.1:0/1454126082 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c097de0 con 0x5628c965ea60 2024-09-12T01:36:15.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.357+0000 7fde9e83b740 1 -- 172.21.15.1:0/1454126082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628c965ea60 msgr2=0x5628c96754d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.357+0000 7fde9e83b740 1 --2- 172.21.15.1:0/1454126082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628c965ea60 0x5628c96754d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fde8c099a20 tx=0x7fde8c0cc9e0 comp rx=0 tx=0).stop 2024-09-12T01:36:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.357+0000 7fde9e83b740 1 -- 172.21.15.1:0/1454126082 shutdown_connections 2024-09-12T01:36:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.357+0000 7fde9e83b740 1 --2- 172.21.15.1:0/1454126082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628c965ea60 0x5628c96754d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.357+0000 7fde9e83b740 1 --2- 172.21.15.1:0/1454126082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628c96599d0 0x5628c966a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.357+0000 7fde9e83b740 1 --2- 172.21.15.1:0/1454126082 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628c9659090 0x5628c9653770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.357+0000 7fde9e83b740 1 -- 172.21.15.1:0/1454126082 >> 172.21.15.1:0/1454126082 conn(0x5628c9654700 msgr2=0x5628c9653060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:36:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.357+0000 7fde9e83b740 1 -- 172.21.15.1:0/1454126082 shutdown_connections 2024-09-12T01:36:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.358+0000 7fde9e83b740 1 -- 172.21.15.1:0/1454126082 wait complete. 2024-09-12T01:36:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.358+0000 7fde9e83b740 1 -- start start 2024-09-12T01:36:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.358+0000 7fde9e83b740 1 Processor -- start 2024-09-12T01:36:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.358+0000 7fde9e83b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628c9659090 0x5628c9667fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.358+0000 7fde9e83b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628c96599d0 0x5628c96684e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.358+0000 7fde9e83b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628c965ea60 0x5628c96694a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.358+0000 7fde9e83b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fde941ade80 con 0x5628c965ea60 2024-09-12T01:36:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.358+0000 7fde9e83b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fde941ad7b0 con 0x5628c96599d0 2024-09-12T01:36:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.358+0000 7fde9e83b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fde941aa9f0 con 0x5628c9659090 2024-09-12T01:36:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.358+0000 7fde9e83b740 10 client.0 _ll_register_callbacks cb 0x7fde80009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:36:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.359+0000 7fde93e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628c96599d0 0x5628c96684e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.359+0000 7fde93e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628c96599d0 0x5628c96684e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:35704/0 (socket says 172.21.15.1:35704) 2024-09-12T01:36:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.359+0000 7fde93e00640 1 -- 172.21.15.1:0/3033479679 learned_addr learned my addr 172.21.15.1:0/3033479679 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:36:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.359+0000 7fde98e00640 1 --2- 172.21.15.1:0/3033479679 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628c9659090 0x5628c9667fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.359+0000 7fde99800640 1 --2- 172.21.15.1:0/3033479679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628c965ea60 0x5628c96694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.359+0000 7fde93e00640 1 -- 172.21.15.1:0/3033479679 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628c9659090 msgr2=0x5628c9667fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:36:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.359+0000 7fde93e00640 1 --2- 172.21.15.1:0/3033479679 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628c9659090 0x5628c9667fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.359+0000 7fde93e00640 1 -- 172.21.15.1:0/3033479679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628c965ea60 msgr2=0x5628c96694a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:36:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.359+0000 7fde93e00640 1 --2- 172.21.15.1:0/3033479679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628c965ea60 0x5628c96694a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:15.363 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.359+0000 7fde93e00640 1 -- 172.21.15.1:0/3033479679 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628c9669ab0 con 0x5628c96599d0 2024-09-12T01:36:15.363 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.359+0000 7fde93e00640 1 --2- 172.21.15.1:0/3033479679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628c96599d0 0x5628c96684e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fde9406a4d0 tx=0x7fde8002a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65c1c3a79122f58a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:36:15.363 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.359+0000 7fde7aa00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-12T01:36:15.363 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.360+0000 7fde7aa00640 1 -- 172.21.15.1:0/3033479679 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde80004dd0 con 0x5628c96599d0 2024-09-12T01:36:15.363 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.360+0000 7fde9e83b740 1 -- 172.21.15.1:0/3033479679 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde8c0b4070 con 0x5628c96599d0 2024-09-12T01:36:15.363 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.360+0000 7fde7aa00640 1 -- 172.21.15.1:0/3033479679 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fde800316b0 con 0x5628c96599d0 2024-09-12T01:36:15.363 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.360+0000 7fde7aa00640 1 -- 172.21.15.1:0/3033479679 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fde800318b0 con 0x5628c96599d0 2024-09-12T01:36:15.363 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.360+0000 7fde9e83b740 10 client.25724 fetch_fsmap learned FSMap version 35 2024-09-12T01:36:15.363 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.360+0000 7fde9e83b740 1 -- 172.21.15.1:0/3033479679 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5628c96d9630 con 0x5628c96599d0 2024-09-12T01:36:15.363 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.360+0000 7fde7aa00640 1 -- 172.21.15.1:0/3033479679 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde8c0b4070 con 0x5628c96599d0 2024-09-12T01:36:15.363 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.361+0000 7fde7aa00640 1 -- 172.21.15.1:0/3033479679 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fde80003cc0 con 0x5628c96599d0 2024-09-12T01:36:15.364 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.361+0000 7fde9e83b740 10 client.25724 fetch_fsmap finished waiting for FSMap version 35 2024-09-12T01:36:15.364 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.361+0000 7fde9e83b740 10 client.25724 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:36:15.364 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.361+0000 7fde9e83b740 20 client.25724 populate_metadata read hostname 'smithi001' 2024-09-12T01:36:15.364 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.361+0000 7fde9e83b740 1 --2- 172.21.15.1:0/3033479679 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5628c96d9f80 0x5628c96dc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:15.364 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.361+0000 7fde9e83b740 4 client.25724 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:36:15.364 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.361+0000 7fde9e83b740 1 -- 172.21.15.1:0/3033479679 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5628c95c5b80 con 0x5628c96d9f80 2024-09-12T01:36:15.364 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.361+0000 7fde98e00640 1 --2- 172.21.15.1:0/3033479679 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5628c96d9f80 0x5628c96dc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:15.365 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.362+0000 7fde98e00640 1 --2- 172.21.15.1:0/3033479679 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5628c96d9f80 0x5628c96dc370 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb39fe1debd02f9f server_cookie=3fe52a8bbd04b54f in_seq=0 out_seq=0 2024-09-12T01:36:15.365 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.362+0000 7fde7aa00640 10 client.25724 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:36:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.372+0000 7fde7aa00640 1 -- 172.21.15.1:0/3033479679 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5628c95c5b80 con 0x5628c96d9f80 2024-09-12T01:36:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.372+0000 7fde7aa00640 10 client.25724 handle_command_reply: tid=0 2024-09-12T01:36:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.372+0000 7fde9e83b740 1 client.25724 shutdown 2024-09-12T01:36:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.372+0000 7fde9e83b740 1 -- 172.21.15.1:0/3033479679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628c96599d0 msgr2=0x5628c96684e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.372+0000 7fde9e83b740 1 --2- 172.21.15.1:0/3033479679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628c96599d0 0x5628c96684e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fde9406a4d0 tx=0x7fde8002a3f0 comp rx=0 tx=0).stop 2024-09-12T01:36:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.372+0000 7fde9e83b740 1 -- 172.21.15.1:0/3033479679 shutdown_connections 2024-09-12T01:36:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.372+0000 7fde9e83b740 1 --2- 172.21.15.1:0/3033479679 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5628c96d9f80 0x5628c96dc370 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.372+0000 7fde9e83b740 1 --2- 172.21.15.1:0/3033479679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628c965ea60 0x5628c96694a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.372+0000 7fde7aa00640 0 client.25724 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:36:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.372+0000 7fde9e83b740 1 --2- 172.21.15.1:0/3033479679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628c96599d0 0x5628c96684e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.372+0000 7fde9e83b740 1 --2- 172.21.15.1:0/3033479679 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5628c9659090 0x5628c9667fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.372+0000 7fde9e83b740 1 -- 172.21.15.1:0/3033479679 >> 172.21.15.1:0/3033479679 conn(0x5628c9654700 msgr2=0x5628c9653880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:36:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.372+0000 7fde9e83b740 1 -- 172.21.15.1:0/3033479679 shutdown_connections 2024-09-12T01:36:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.372+0000 7fde9e83b740 1 -- 172.21.15.1:0/3033479679 wait complete. 2024-09-12T01:36:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.373+0000 7fde9e83b740 20 client.25724 trim_cache size 0 max 16384 2024-09-12T01:36:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-12T01:36:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-12T01:36:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-12T01:36:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-12T01:36:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.373+0000 7fde9b800640 1 -- 172.21.15.1:0/3658179982 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fde7c0a2c00 msgr2=0x7fde7c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.373+0000 7fde9b800640 1 --2- 172.21.15.1:0/3658179982 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fde7c0a2c00 0x7fde7c0a50c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fde80031440 tx=0x7fde80004040 comp rx=0 tx=0).stop 2024-09-12T01:36:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.373+0000 7fde9b800640 1 -- 172.21.15.1:0/3658179982 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde941078c0 msgr2=0x7fde941a1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.373+0000 7fde9b800640 1 --2- 172.21.15.1:0/3658179982 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde941078c0 0x7fde941a1690 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fde8c00dac0 tx=0x7fde8c00df80 comp rx=0 tx=0).stop 2024-09-12T01:36:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.373+0000 7fde9b800640 1 -- 172.21.15.1:0/3658179982 shutdown_connections 2024-09-12T01:36:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.373+0000 7fde9b800640 1 --2- 172.21.15.1:0/3658179982 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fde7c0a2c00 0x7fde7c0a50c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.373+0000 7fde9b800640 1 --2- 172.21.15.1:0/3658179982 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde941078c0 0x7fde941a1690 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.373+0000 7fde9b800640 1 --2- 172.21.15.1:0/3658179982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde94104f90 0x7fde9419a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.373+0000 7fde9b800640 1 --2- 172.21.15.1:0/3658179982 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde94102660 0x7fde9419a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.373+0000 7fde9b800640 1 -- 172.21.15.1:0/3658179982 >> 172.21.15.1:0/3658179982 conn(0x7fde940fc1c0 msgr2=0x7fde94101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:36:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.374+0000 7fde9b800640 1 -- 172.21.15.1:0/3658179982 shutdown_connections 2024-09-12T01:36:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:15.374+0000 7fde9b800640 1 -- 172.21.15.1:0/3658179982 wait complete. 2024-09-12T01:36:16.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:16 smithi104 ceph-mon[35219]: pgmap v598: 129 pgs: 129 active+clean; 125 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1.1 MiB/s wr, 446 op/s 2024-09-12T01:36:16.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:16 smithi001 ceph-mon[30387]: pgmap v598: 129 pgs: 129 active+clean; 125 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1.1 MiB/s wr, 446 op/s 2024-09-12T01:36:16.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:16 smithi053 ceph-mon[35158]: pgmap v598: 129 pgs: 129 active+clean; 125 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1.1 MiB/s wr, 446 op/s 2024-09-12T01:36:18.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:18 smithi104 ceph-mon[35219]: pgmap v599: 129 pgs: 129 active+clean; 122 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 1.2 MiB/s wr, 481 op/s 2024-09-12T01:36:18.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:18 smithi001 ceph-mon[30387]: pgmap v599: 129 pgs: 129 active+clean; 122 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 1.2 MiB/s wr, 481 op/s 2024-09-12T01:36:18.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:18 smithi053 ceph-mon[35158]: pgmap v599: 129 pgs: 129 active+clean; 122 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 1.2 MiB/s wr, 481 op/s 2024-09-12T01:36:20.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:20 smithi104 ceph-mon[35219]: pgmap v600: 129 pgs: 129 active+clean; 122 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 903 KiB/s wr, 366 op/s 2024-09-12T01:36:20.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:20 smithi001 ceph-mon[30387]: pgmap v600: 129 pgs: 129 active+clean; 122 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 903 KiB/s wr, 366 op/s 2024-09-12T01:36:20.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:20 smithi053 ceph-mon[35158]: pgmap v600: 129 pgs: 129 active+clean; 122 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 903 KiB/s wr, 366 op/s 2024-09-12T01:36:22.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:22 smithi104 ceph-mon[35219]: pgmap v601: 129 pgs: 129 active+clean; 120 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1.3 MiB/s wr, 514 op/s 2024-09-12T01:36:22.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:22 smithi001 ceph-mon[30387]: pgmap v601: 129 pgs: 129 active+clean; 120 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1.3 MiB/s wr, 514 op/s 2024-09-12T01:36:22.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:22 smithi053 ceph-mon[35158]: pgmap v601: 129 pgs: 129 active+clean; 120 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1.3 MiB/s wr, 514 op/s 2024-09-12T01:36:24.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:24 smithi104 ceph-mon[35219]: pgmap v602: 129 pgs: 129 active+clean; 123 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 1017 KiB/s wr, 391 op/s 2024-09-12T01:36:24.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:24 smithi001 ceph-mon[30387]: pgmap v602: 129 pgs: 129 active+clean; 123 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 1017 KiB/s wr, 391 op/s 2024-09-12T01:36:24.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:24 smithi053 ceph-mon[35158]: pgmap v602: 129 pgs: 129 active+clean; 123 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 1017 KiB/s wr, 391 op/s 2024-09-12T01:36:26.324 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:26 smithi104 ceph-mon[35219]: pgmap v603: 129 pgs: 129 active+clean; 126 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1.1 MiB/s wr, 426 op/s 2024-09-12T01:36:26.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:26 smithi001 ceph-mon[30387]: pgmap v603: 129 pgs: 129 active+clean; 126 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1.1 MiB/s wr, 426 op/s 2024-09-12T01:36:26.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:26 smithi053 ceph-mon[35158]: pgmap v603: 129 pgs: 129 active+clean; 126 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1.1 MiB/s wr, 426 op/s 2024-09-12T01:36:28.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:28 smithi104 ceph-mon[35219]: pgmap v604: 129 pgs: 129 active+clean; 119 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.2 MiB/s wr, 476 op/s 2024-09-12T01:36:28.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:28 smithi001 ceph-mon[30387]: pgmap v604: 129 pgs: 129 active+clean; 119 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.2 MiB/s wr, 476 op/s 2024-09-12T01:36:28.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:28 smithi053 ceph-mon[35158]: pgmap v604: 129 pgs: 129 active+clean; 119 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.2 MiB/s wr, 476 op/s 2024-09-12T01:36:28.510 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.510+0000 7fb642000640 1 -- 172.21.15.1:0/394692808 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb63c100cf0 msgr2=0x7fb63c101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:28.510 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.510+0000 7fb642000640 1 --2- 172.21.15.1:0/394692808 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb63c100cf0 0x7fb63c101150 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb628009900 tx=0x7fb62802a410 comp rx=0 tx=0).stop 2024-09-12T01:36:28.511 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.510+0000 7fb642000640 1 -- 172.21.15.1:0/394692808 shutdown_connections 2024-09-12T01:36:28.511 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.510+0000 7fb642000640 1 --2- 172.21.15.1:0/394692808 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb63c101690 0x7fb63c101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:28.511 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.510+0000 7fb642000640 1 --2- 172.21.15.1:0/394692808 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb63c100cf0 0x7fb63c101150 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:28.511 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.510+0000 7fb642000640 1 --2- 172.21.15.1:0/394692808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb63c1003b0 0x7fb63c1007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:28.511 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.510+0000 7fb642000640 1 -- 172.21.15.1:0/394692808 >> 172.21.15.1:0/394692808 conn(0x7fb63c0fc1c0 msgr2=0x7fb63c0fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:36:28.511 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.510+0000 7fb642000640 1 -- 172.21.15.1:0/394692808 shutdown_connections 2024-09-12T01:36:28.511 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.511+0000 7fb642000640 1 -- 172.21.15.1:0/394692808 wait complete. 2024-09-12T01:36:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.512+0000 7fb642000640 1 Processor -- start 2024-09-12T01:36:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.512+0000 7fb642000640 1 -- start start 2024-09-12T01:36:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.512+0000 7fb642000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb63c1003b0 0x7fb63c19a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.512+0000 7fb642000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb63c100cf0 0x7fb63c19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.512+0000 7fb642000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb63c101690 0x7fb63c1a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.512+0000 7fb642000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb63c10fa00 con 0x7fb63c100cf0 2024-09-12T01:36:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.512+0000 7fb642000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb63c10f880 con 0x7fb63c1003b0 2024-09-12T01:36:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.512+0000 7fb642000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb63c10fb80 con 0x7fb63c101690 2024-09-12T01:36:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.512+0000 7fb63b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb63c1003b0 0x7fb63c19a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.512+0000 7fb63b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb63c1003b0 0x7fb63c19a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:51246/0 (socket says 172.21.15.1:51246) 2024-09-12T01:36:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.512+0000 7fb63b400640 1 -- 172.21.15.1:0/1767959848 learned_addr learned my addr 172.21.15.1:0/1767959848 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:36:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.512+0000 7fb63be00640 1 --2- 172.21.15.1:0/1767959848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb63c101690 0x7fb63c1a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.513+0000 7fb63aa00640 1 --2- 172.21.15.1:0/1767959848 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb63c100cf0 0x7fb63c19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.513+0000 7fb63b400640 1 -- 172.21.15.1:0/1767959848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb63c101690 msgr2=0x7fb63c1a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.513+0000 7fb63b400640 1 --2- 172.21.15.1:0/1767959848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb63c101690 0x7fb63c1a1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.513+0000 7fb63b400640 1 -- 172.21.15.1:0/1767959848 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb63c100cf0 msgr2=0x7fb63c19a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.513+0000 7fb63b400640 1 --2- 172.21.15.1:0/1767959848 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb63c100cf0 0x7fb63c19a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.513+0000 7fb63b400640 1 -- 172.21.15.1:0/1767959848 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb63c1a1cd0 con 0x7fb63c1003b0 2024-09-12T01:36:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.513+0000 7fb63b400640 1 --2- 172.21.15.1:0/1767959848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb63c1003b0 0x7fb63c19a0b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb62c00e9f0 tx=0x7fb62c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:36:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.513+0000 7fb633e00640 1 -- 172.21.15.1:0/1767959848 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62c00cd80 con 0x7fb63c1003b0 2024-09-12T01:36:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.514+0000 7fb633e00640 1 -- 172.21.15.1:0/1767959848 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb62c005500 con 0x7fb63c1003b0 2024-09-12T01:36:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.514+0000 7fb633e00640 1 -- 172.21.15.1:0/1767959848 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62c010690 con 0x7fb63c1003b0 2024-09-12T01:36:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.514+0000 7fb642000640 1 -- 172.21.15.1:0/1767959848 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb63c1a1fc0 con 0x7fb63c1003b0 2024-09-12T01:36:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.514+0000 7fb642000640 1 -- 172.21.15.1:0/1767959848 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb63c1a23d0 con 0x7fb63c1003b0 2024-09-12T01:36:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.517+0000 7fb633e00640 1 -- 172.21.15.1:0/1767959848 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fb62c0046c0 con 0x7fb63c1003b0 2024-09-12T01:36:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.518+0000 7fb633e00640 1 --2- 172.21.15.1:0/1767959848 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fb6200a2c00 0x7fb6200a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.519+0000 7fb633e00640 1 -- 172.21.15.1:0/1767959848 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fb62c0cf920 con 0x7fb63c1003b0 2024-09-12T01:36:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.519+0000 7fb644e34740 1 Processor -- start 2024-09-12T01:36:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.519+0000 7fb63aa00640 1 --2- 172.21.15.1:0/1767959848 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fb6200a2c00 0x7fb6200a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.519+0000 7fb644e34740 1 -- start start 2024-09-12T01:36:28.520 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.519+0000 7fb644e34740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591596b3090 0x5591596b3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:28.520 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.519+0000 7fb644e34740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5591596b39d0 0x5591596b8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.519+0000 7fb644e34740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591596b8a60 0x5591596baf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.519+0000 7fb644e34740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559159618100 con 0x5591596b3090 2024-09-12T01:36:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.519+0000 7fb644e34740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5591595c01b0 con 0x5591596b39d0 2024-09-12T01:36:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.519+0000 7fb644e34740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5591596140c0 con 0x5591596b8a60 2024-09-12T01:36:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.520+0000 7fb63b400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591596b3090 0x5591596b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.520+0000 7fb63be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591596b8a60 0x5591596baf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.520+0000 7fb63be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591596b8a60 0x5591596baf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:37810/0 (socket says 172.21.15.1:37810) 2024-09-12T01:36:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.520+0000 7fb63be00640 1 -- 172.21.15.1:0/2091750044 learned_addr learned my addr 172.21.15.1:0/2091750044 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:36:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.520+0000 7fb63aa00640 1 --2- 172.21.15.1:0/1767959848 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fb6200a2c00 0x7fb6200a50c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb628006010 tx=0x7fb628000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:36:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.520+0000 7fb63aa00640 1 --2- 172.21.15.1:0/2091750044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5591596b39d0 0x5591596b8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.520+0000 7fb63be00640 1 -- 172.21.15.1:0/2091750044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5591596b39d0 msgr2=0x5591596b8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:36:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.520+0000 7fb63be00640 1 --2- 172.21.15.1:0/2091750044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5591596b39d0 0x5591596b8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.520+0000 7fb63be00640 1 -- 172.21.15.1:0/2091750044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591596b3090 msgr2=0x5591596b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:36:28.522 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.520+0000 7fb63be00640 1 --2- 172.21.15.1:0/2091750044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591596b3090 0x5591596b3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:28.522 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.520+0000 7fb63be00640 1 -- 172.21.15.1:0/2091750044 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55915932b210 con 0x5591596b8a60 2024-09-12T01:36:28.522 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.520+0000 7fb63be00640 1 --2- 172.21.15.1:0/2091750044 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591596b8a60 0x5591596baf20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb634009fa0 tx=0x7fb63400f7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b66853381cd91cc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:36:28.522 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.521+0000 7fb627e00640 1 -- 172.21.15.1:0/2091750044 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb634015070 con 0x5591596b8a60 2024-09-12T01:36:28.522 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.521+0000 7fb627e00640 1 -- 172.21.15.1:0/2091750044 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb63401ab40 con 0x5591596b8a60 2024-09-12T01:36:28.522 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.521+0000 7fb627e00640 1 -- 172.21.15.1:0/2091750044 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb634011df0 con 0x5591596b8a60 2024-09-12T01:36:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.521+0000 7fb644e34740 1 -- 172.21.15.1:0/2091750044 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591596b8a60 msgr2=0x5591596baf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.521+0000 7fb644e34740 1 --2- 172.21.15.1:0/2091750044 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591596b8a60 0x5591596baf20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb634009fa0 tx=0x7fb63400f7f0 comp rx=0 tx=0).stop 2024-09-12T01:36:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.522+0000 7fb644e34740 1 -- 172.21.15.1:0/2091750044 shutdown_connections 2024-09-12T01:36:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.522+0000 7fb644e34740 1 --2- 172.21.15.1:0/2091750044 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591596b8a60 0x5591596baf20 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.522+0000 7fb644e34740 1 --2- 172.21.15.1:0/2091750044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5591596b39d0 0x5591596b8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.522+0000 7fb644e34740 1 --2- 172.21.15.1:0/2091750044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591596b3090 0x5591596b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.522+0000 7fb644e34740 1 -- 172.21.15.1:0/2091750044 >> 172.21.15.1:0/2091750044 conn(0x5591596ae700 msgr2=0x5591596aeb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:36:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.522+0000 7fb644e34740 1 -- 172.21.15.1:0/2091750044 shutdown_connections 2024-09-12T01:36:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.522+0000 7fb644e34740 1 -- 172.21.15.1:0/2091750044 wait complete. 2024-09-12T01:36:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.522+0000 7fb644e34740 1 -- start start 2024-09-12T01:36:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.522+0000 7fb644e34740 1 Processor -- start 2024-09-12T01:36:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.523+0000 7fb644e34740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591596b3090 0x5591596bd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.523+0000 7fb644e34740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591596b39d0 0x5591596bd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.523+0000 7fb644e34740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5591596b8a60 0x5591596c4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.523+0000 7fb644e34740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5591596bba30 con 0x5591596b39d0 2024-09-12T01:36:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.523+0000 7fb644e34740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5591595a1090 con 0x5591596b8a60 2024-09-12T01:36:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.523+0000 7fb644e34740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5591596bbbb0 con 0x5591596b3090 2024-09-12T01:36:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.523+0000 7fb644e34740 10 client.0 _ll_register_callbacks cb 0x5591595f5f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:36:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.523+0000 7fb63b400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591596b3090 0x5591596bd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.523+0000 7fb63b400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591596b3090 0x5591596bd1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:37816/0 (socket says 172.21.15.1:37816) 2024-09-12T01:36:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.523+0000 7fb63b400640 1 -- 172.21.15.1:0/938201861 learned_addr learned my addr 172.21.15.1:0/938201861 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:36:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.523+0000 7fb63aa00640 1 --2- 172.21.15.1:0/938201861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591596b39d0 0x5591596bd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.523+0000 7fb63be00640 1 --2- 172.21.15.1:0/938201861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5591596b8a60 0x5591596c4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.523+0000 7fb63b400640 1 -- 172.21.15.1:0/938201861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5591596b8a60 msgr2=0x5591596c4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:36:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.523+0000 7fb63b400640 1 --2- 172.21.15.1:0/938201861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5591596b8a60 0x5591596c4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.523+0000 7fb63b400640 1 -- 172.21.15.1:0/938201861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591596b39d0 msgr2=0x5591596bd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:36:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.523+0000 7fb63b400640 1 --2- 172.21.15.1:0/938201861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591596b39d0 0x5591596bd720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.523+0000 7fb63b400640 1 -- 172.21.15.1:0/938201861 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591593f7620 con 0x5591596b3090 2024-09-12T01:36:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.524+0000 7fb63b400640 1 --2- 172.21.15.1:0/938201861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591596b3090 0x5591596bd1e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb62c00e9c0 tx=0x7fb62c09c5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30fec76709b00e54 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:36:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.524+0000 7fb624c00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-12T01:36:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.524+0000 7fb624c00640 1 -- 172.21.15.1:0/938201861 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62c0b9060 con 0x5591596b3090 2024-09-12T01:36:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.524+0000 7fb624c00640 1 -- 172.21.15.1:0/938201861 <== mon.1 v2:172.21.15.104:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fb62c0a4570 con 0x5591596b3090 2024-09-12T01:36:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.524+0000 7fb644e34740 1 -- 172.21.15.1:0/938201861 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb63401ab40 con 0x5591596b3090 2024-09-12T01:36:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.524+0000 7fb624c00640 1 -- 172.21.15.1:0/938201861 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb62c0ab050 con 0x5591596b3090 2024-09-12T01:36:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.524+0000 7fb644e34740 10 client.25735 fetch_fsmap learned FSMap version 35 2024-09-12T01:36:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.524+0000 7fb644e34740 1 -- 172.21.15.1:0/938201861 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5591596c7620 con 0x5591596b3090 2024-09-12T01:36:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.524+0000 7fb624c00640 1 -- 172.21.15.1:0/938201861 <== mon.1 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb63401ab40 con 0x5591596b3090 2024-09-12T01:36:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.525+0000 7fb624c00640 1 -- 172.21.15.1:0/938201861 <== mon.1 v2:172.21.15.104:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fb62c09ac50 con 0x5591596b3090 2024-09-12T01:36:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.525+0000 7fb644e34740 10 client.25735 fetch_fsmap finished waiting for FSMap version 35 2024-09-12T01:36:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.525+0000 7fb644e34740 10 client.25735 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:36:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.525+0000 7fb644e34740 20 client.25735 populate_metadata read hostname 'smithi001' 2024-09-12T01:36:28.529 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.525+0000 7fb644e34740 1 --2- 172.21.15.1:0/938201861 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5591596c8ca0 0x5591596cb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:28.529 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.525+0000 7fb644e34740 4 client.25735 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:36:28.529 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.525+0000 7fb644e34740 1 -- 172.21.15.1:0/938201861 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559159625830 con 0x5591596c8ca0 2024-09-12T01:36:28.529 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.525+0000 7fb63aa00640 1 --2- 172.21.15.1:0/938201861 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5591596c8ca0 0x5591596cb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:28.529 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.526+0000 7fb63aa00640 1 --2- 172.21.15.1:0/938201861 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5591596c8ca0 0x5591596cb090 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4aeca2723f889e67 server_cookie=bed1f68419d9e0ba in_seq=0 out_seq=0 2024-09-12T01:36:28.529 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.526+0000 7fb624c00640 10 client.25735 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:36:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.526+0000 7fb624c00640 1 -- 172.21.15.1:0/938201861 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559159625830 con 0x5591596c8ca0 2024-09-12T01:36:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.526+0000 7fb624c00640 10 client.25735 handle_command_reply: tid=0 2024-09-12T01:36:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.526+0000 7fb644e34740 1 client.25735 shutdown 2024-09-12T01:36:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.527+0000 7fb644e34740 1 -- 172.21.15.1:0/938201861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591596b3090 msgr2=0x5591596bd1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.527+0000 7fb644e34740 1 --2- 172.21.15.1:0/938201861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591596b3090 0x5591596bd1e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb62c00e9c0 tx=0x7fb62c09c5d0 comp rx=0 tx=0).stop 2024-09-12T01:36:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.527+0000 7fb644e34740 1 -- 172.21.15.1:0/938201861 shutdown_connections 2024-09-12T01:36:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.527+0000 7fb644e34740 1 --2- 172.21.15.1:0/938201861 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x5591596c8ca0 0x5591596cb090 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.527+0000 7fb644e34740 1 --2- 172.21.15.1:0/938201861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5591596b8a60 0x5591596c4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.527+0000 7fb644e34740 1 --2- 172.21.15.1:0/938201861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591596b39d0 0x5591596bd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.527+0000 7fb644e34740 1 --2- 172.21.15.1:0/938201861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591596b3090 0x5591596bd1e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.527+0000 7fb624c00640 0 client.25735 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:36:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.527+0000 7fb644e34740 1 -- 172.21.15.1:0/938201861 >> 172.21.15.1:0/938201861 conn(0x5591596ae700 msgr2=0x5591596aff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:36:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.527+0000 7fb644e34740 1 -- 172.21.15.1:0/938201861 shutdown_connections 2024-09-12T01:36:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.527+0000 7fb644e34740 1 -- 172.21.15.1:0/938201861 wait complete. 2024-09-12T01:36:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.527+0000 7fb644e34740 20 client.25735 trim_cache size 0 max 16384 2024-09-12T01:36:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.546+0000 7fb644e34740 1 Processor -- start 2024-09-12T01:36:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.546+0000 7fb644e34740 1 -- start start 2024-09-12T01:36:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.546+0000 7fb644e34740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5591596b3090 0x5591596ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.546+0000 7fb644e34740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591596b39d0 0x5591596c4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.546+0000 7fb644e34740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591596b8a60 0x5591596cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.546+0000 7fb644e34740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb63c1ac990 con 0x5591596b8a60 2024-09-12T01:36:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.546+0000 7fb644e34740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb63c1ae4d0 con 0x5591596b3090 2024-09-12T01:36:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.546+0000 7fb644e34740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb63c1ad060 con 0x5591596b39d0 2024-09-12T01:36:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.546+0000 7fb63be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591596b8a60 0x5591596cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.546+0000 7fb63b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5591596b3090 0x5591596ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.546+0000 7fb63b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5591596b3090 0x5591596ad770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:51276/0 (socket says 172.21.15.1:51276) 2024-09-12T01:36:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.546+0000 7fb63b400640 1 -- 172.21.15.1:0/2572833192 learned_addr learned my addr 172.21.15.1:0/2572833192 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:36:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.547+0000 7fb63be00640 1 -- 172.21.15.1:0/2572833192 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591596b39d0 msgr2=0x5591596c4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:36:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.547+0000 7fb63aa00640 1 --2- 172.21.15.1:0/2572833192 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591596b39d0 0x5591596c4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.547+0000 7fb63be00640 1 --2- 172.21.15.1:0/2572833192 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591596b39d0 0x5591596c4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.547+0000 7fb63be00640 1 -- 172.21.15.1:0/2572833192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5591596b3090 msgr2=0x5591596ad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:36:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.547+0000 7fb63be00640 1 --2- 172.21.15.1:0/2572833192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5591596b3090 0x5591596ad770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.547+0000 7fb63be00640 1 -- 172.21.15.1:0/2572833192 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591596a9d60 con 0x5591596b8a60 2024-09-12T01:36:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.547+0000 7fb63b400640 1 --2- 172.21.15.1:0/2572833192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5591596b3090 0x5591596ad770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:36:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.547+0000 7fb63aa00640 1 --2- 172.21.15.1:0/2572833192 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591596b39d0 0x5591596c4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:36:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.547+0000 7fb63be00640 1 --2- 172.21.15.1:0/2572833192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591596b8a60 0x5591596cf970 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb634006010 tx=0x7fb634009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94503bafedbeb48d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:36:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.547+0000 7fb627400640 1 -- 172.21.15.1:0/2572833192 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb634029070 con 0x5591596b8a60 2024-09-12T01:36:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.547+0000 7fb627400640 1 -- 172.21.15.1:0/2572833192 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb63401a940 con 0x5591596b8a60 2024-09-12T01:36:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.547+0000 7fb627400640 1 -- 172.21.15.1:0/2572833192 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb634015040 con 0x5591596b8a60 2024-09-12T01:36:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.548+0000 7fb644e34740 1 -- 172.21.15.1:0/2572833192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591596b8a60 msgr2=0x5591596cf970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.548+0000 7fb644e34740 1 --2- 172.21.15.1:0/2572833192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591596b8a60 0x5591596cf970 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb634006010 tx=0x7fb634009440 comp rx=0 tx=0).stop 2024-09-12T01:36:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.548+0000 7fb644e34740 1 -- 172.21.15.1:0/2572833192 shutdown_connections 2024-09-12T01:36:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.548+0000 7fb644e34740 1 --2- 172.21.15.1:0/2572833192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591596b8a60 0x5591596cf970 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.548+0000 7fb644e34740 1 --2- 172.21.15.1:0/2572833192 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591596b39d0 0x5591596c4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.548+0000 7fb644e34740 1 --2- 172.21.15.1:0/2572833192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5591596b3090 0x5591596ad770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.548+0000 7fb644e34740 1 -- 172.21.15.1:0/2572833192 >> 172.21.15.1:0/2572833192 conn(0x5591596ae700 msgr2=0x5591596ad060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:36:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.548+0000 7fb644e34740 1 -- 172.21.15.1:0/2572833192 shutdown_connections 2024-09-12T01:36:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.548+0000 7fb644e34740 1 -- 172.21.15.1:0/2572833192 wait complete. 2024-09-12T01:36:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.549+0000 7fb644e34740 1 -- start start 2024-09-12T01:36:28.553 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.549+0000 7fb644e34740 1 Processor -- start 2024-09-12T01:36:28.553 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.549+0000 7fb644e34740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5591596b3090 0x5591596c26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:28.553 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.549+0000 7fb644e34740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591596b39d0 0x5591596c2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:28.553 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.549+0000 7fb644e34740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591596b8a60 0x5591596c3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:28.554 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.549+0000 7fb644e34740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb63c1ade00 con 0x5591596b8a60 2024-09-12T01:36:28.554 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.549+0000 7fb644e34740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb63c1ad730 con 0x5591596b3090 2024-09-12T01:36:28.554 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.549+0000 7fb644e34740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb63c1aa970 con 0x5591596b39d0 2024-09-12T01:36:28.554 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.549+0000 7fb644e34740 10 client.0 _ll_register_callbacks cb 0x7fb62c09bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:36:28.554 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.550+0000 7fb63b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5591596b3090 0x5591596c26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:28.554 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.550+0000 7fb63b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5591596b3090 0x5591596c26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:51278/0 (socket says 172.21.15.1:51278) 2024-09-12T01:36:28.554 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.550+0000 7fb63b400640 1 -- 172.21.15.1:0/943137380 learned_addr learned my addr 172.21.15.1:0/943137380 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:36:28.554 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.550+0000 7fb63be00640 1 --2- 172.21.15.1:0/943137380 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591596b8a60 0x5591596c3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:28.554 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.550+0000 7fb63aa00640 1 --2- 172.21.15.1:0/943137380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591596b39d0 0x5591596c2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:28.554 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.550+0000 7fb63b400640 1 -- 172.21.15.1:0/943137380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591596b39d0 msgr2=0x5591596c2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:36:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.550+0000 7fb63b400640 1 --2- 172.21.15.1:0/943137380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591596b39d0 0x5591596c2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.550+0000 7fb63b400640 1 -- 172.21.15.1:0/943137380 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591596b8a60 msgr2=0x5591596c3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:36:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.550+0000 7fb63b400640 1 --2- 172.21.15.1:0/943137380 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591596b8a60 0x5591596c3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.550+0000 7fb63b400640 1 -- 172.21.15.1:0/943137380 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591596c4250 con 0x5591596b3090 2024-09-12T01:36:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.550+0000 7fb63be00640 1 --2- 172.21.15.1:0/943137380 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591596b8a60 0x5591596c3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:36:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.550+0000 7fb63aa00640 1 --2- 172.21.15.1:0/943137380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591596b39d0 0x5591596c2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:36:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.550+0000 7fb63b400640 1 --2- 172.21.15.1:0/943137380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5591596b3090 0x5591596c26f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb62c09ba90 tx=0x7fb62c005760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=324a8bf6af2c4fa3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:36:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.550+0000 7fb624c00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-12T01:36:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.550+0000 7fb624c00640 1 -- 172.21.15.1:0/943137380 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62c09b3e0 con 0x5591596b3090 2024-09-12T01:36:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.550+0000 7fb644e34740 1 -- 172.21.15.1:0/943137380 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb63401a940 con 0x5591596b3090 2024-09-12T01:36:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.550+0000 7fb624c00640 1 -- 172.21.15.1:0/943137380 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fb62c0a4570 con 0x5591596b3090 2024-09-12T01:36:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.551+0000 7fb624c00640 1 -- 172.21.15.1:0/943137380 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb62c08e050 con 0x5591596b3090 2024-09-12T01:36:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.551+0000 7fb644e34740 10 client.25742 fetch_fsmap learned FSMap version 35 2024-09-12T01:36:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.551+0000 7fb644e34740 1 -- 172.21.15.1:0/943137380 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5591597340b0 con 0x5591596b3090 2024-09-12T01:36:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.551+0000 7fb624c00640 1 -- 172.21.15.1:0/943137380 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb63401a940 con 0x5591596b3090 2024-09-12T01:36:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.551+0000 7fb624c00640 1 -- 172.21.15.1:0/943137380 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fb62c0ae070 con 0x5591596b3090 2024-09-12T01:36:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.551+0000 7fb644e34740 10 client.25742 fetch_fsmap finished waiting for FSMap version 35 2024-09-12T01:36:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.551+0000 7fb644e34740 10 client.25742 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:36:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.551+0000 7fb644e34740 20 client.25742 populate_metadata read hostname 'smithi001' 2024-09-12T01:36:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.551+0000 7fb644e34740 1 --2- 172.21.15.1:0/943137380 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x559159734a00 0x559159736df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.551+0000 7fb644e34740 4 client.25742 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:36:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.551+0000 7fb644e34740 1 -- 172.21.15.1:0/943137380 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55915961fb80 con 0x559159734a00 2024-09-12T01:36:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.551+0000 7fb63aa00640 1 --2- 172.21.15.1:0/943137380 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x559159734a00 0x559159736df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.552+0000 7fb63aa00640 1 --2- 172.21.15.1:0/943137380 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x559159734a00 0x559159736df0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c46a3b30230205d6 server_cookie=cd607a71bc8edc76 in_seq=0 out_seq=0 2024-09-12T01:36:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.552+0000 7fb624c00640 10 client.25742 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:36:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.561+0000 7fb624c00640 1 -- 172.21.15.1:0/943137380 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55915961fb80 con 0x559159734a00 2024-09-12T01:36:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.561+0000 7fb624c00640 10 client.25742 handle_command_reply: tid=0 2024-09-12T01:36:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.561+0000 7fb644e34740 1 client.25742 shutdown 2024-09-12T01:36:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.561+0000 7fb644e34740 1 -- 172.21.15.1:0/943137380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5591596b3090 msgr2=0x5591596c26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.561+0000 7fb644e34740 1 --2- 172.21.15.1:0/943137380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5591596b3090 0x5591596c26f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb62c09ba90 tx=0x7fb62c005760 comp rx=0 tx=0).stop 2024-09-12T01:36:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.561+0000 7fb644e34740 1 -- 172.21.15.1:0/943137380 shutdown_connections 2024-09-12T01:36:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.561+0000 7fb644e34740 1 --2- 172.21.15.1:0/943137380 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x559159734a00 0x559159736df0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.561+0000 7fb644e34740 1 --2- 172.21.15.1:0/943137380 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591596b8a60 0x5591596c3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.561+0000 7fb644e34740 1 --2- 172.21.15.1:0/943137380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591596b39d0 0x5591596c2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.561+0000 7fb644e34740 1 --2- 172.21.15.1:0/943137380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5591596b3090 0x5591596c26f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.561+0000 7fb624c00640 0 client.25742 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:36:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.561+0000 7fb644e34740 1 -- 172.21.15.1:0/943137380 >> 172.21.15.1:0/943137380 conn(0x5591596ae700 msgr2=0x5591596ad880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:36:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.561+0000 7fb644e34740 1 -- 172.21.15.1:0/943137380 shutdown_connections 2024-09-12T01:36:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.561+0000 7fb644e34740 1 -- 172.21.15.1:0/943137380 wait complete. 2024-09-12T01:36:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.561+0000 7fb644e34740 20 client.25742 trim_cache size 0 max 16384 2024-09-12T01:36:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-12T01:36:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-12T01:36:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-12T01:36:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-12T01:36:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.562+0000 7fb642000640 1 -- 172.21.15.1:0/1767959848 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fb6200a2c00 msgr2=0x7fb6200a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.562+0000 7fb642000640 1 --2- 172.21.15.1:0/1767959848 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fb6200a2c00 0x7fb6200a50c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb628006010 tx=0x7fb628000bc0 comp rx=0 tx=0).stop 2024-09-12T01:36:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.562+0000 7fb642000640 1 -- 172.21.15.1:0/1767959848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb63c1003b0 msgr2=0x7fb63c19a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.562+0000 7fb642000640 1 --2- 172.21.15.1:0/1767959848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb63c1003b0 0x7fb63c19a0b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb62c00e9f0 tx=0x7fb62c00edf0 comp rx=0 tx=0).stop 2024-09-12T01:36:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.562+0000 7fb642000640 1 -- 172.21.15.1:0/1767959848 shutdown_connections 2024-09-12T01:36:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.562+0000 7fb642000640 1 --2- 172.21.15.1:0/1767959848 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fb6200a2c00 0x7fb6200a50c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.562+0000 7fb642000640 1 --2- 172.21.15.1:0/1767959848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb63c101690 0x7fb63c1a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.562+0000 7fb642000640 1 --2- 172.21.15.1:0/1767959848 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb63c100cf0 0x7fb63c19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.562+0000 7fb642000640 1 --2- 172.21.15.1:0/1767959848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb63c1003b0 0x7fb63c19a0b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.562+0000 7fb642000640 1 -- 172.21.15.1:0/1767959848 >> 172.21.15.1:0/1767959848 conn(0x7fb63c0fc1c0 msgr2=0x7fb63c0fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:36:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.562+0000 7fb642000640 1 -- 172.21.15.1:0/1767959848 shutdown_connections 2024-09-12T01:36:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:28.562+0000 7fb642000640 1 -- 172.21.15.1:0/1767959848 wait complete. 2024-09-12T01:36:29.109 INFO:tasks.workunit.client.0.smithi001.stdout:All 50000 operations completed A-OK! 2024-09-12T01:36:29.141 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-12T01:36:29.142 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-12T01:36:30.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:30 smithi001 ceph-mon[30387]: pgmap v605: 129 pgs: 129 active+clean; 119 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 921 KiB/s wr, 367 op/s 2024-09-12T01:36:30.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:30 smithi053 ceph-mon[35158]: pgmap v605: 129 pgs: 129 active+clean; 119 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 921 KiB/s wr, 367 op/s 2024-09-12T01:36:30.575 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:30 smithi104 ceph-mon[35219]: pgmap v605: 129 pgs: 129 active+clean; 119 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 921 KiB/s wr, 367 op/s 2024-09-12T01:36:32.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:32 smithi001 ceph-mon[30387]: pgmap v606: 129 pgs: 129 active+clean; 120 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.3 MiB/s wr, 497 op/s 2024-09-12T01:36:32.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:32 smithi053 ceph-mon[35158]: pgmap v606: 129 pgs: 129 active+clean; 120 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.3 MiB/s wr, 497 op/s 2024-09-12T01:36:32.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:32 smithi104 ceph-mon[35219]: pgmap v606: 129 pgs: 129 active+clean; 120 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.3 MiB/s wr, 497 op/s 2024-09-12T01:36:34.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:34 smithi001 ceph-mon[30387]: pgmap v607: 129 pgs: 129 active+clean; 116 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 1.0 MiB/s wr, 375 op/s 2024-09-12T01:36:34.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:34 smithi053 ceph-mon[35158]: pgmap v607: 129 pgs: 129 active+clean; 116 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 1.0 MiB/s wr, 375 op/s 2024-09-12T01:36:34.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:34 smithi104 ceph-mon[35219]: pgmap v607: 129 pgs: 129 active+clean; 116 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 1.0 MiB/s wr, 375 op/s 2024-09-12T01:36:36.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:36 smithi001 ceph-mon[30387]: pgmap v608: 129 pgs: 129 active+clean; 113 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 1.3 MiB/s wr, 388 op/s 2024-09-12T01:36:36.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:36 smithi053 ceph-mon[35158]: pgmap v608: 129 pgs: 129 active+clean; 113 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 1.3 MiB/s wr, 388 op/s 2024-09-12T01:36:36.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:36 smithi104 ceph-mon[35219]: pgmap v608: 129 pgs: 129 active+clean; 113 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 1.3 MiB/s wr, 388 op/s 2024-09-12T01:36:38.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:38 smithi001 ceph-mon[30387]: pgmap v609: 129 pgs: 129 active+clean; 106 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.1 MiB/s wr, 440 op/s 2024-09-12T01:36:38.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:38 smithi053 ceph-mon[35158]: pgmap v609: 129 pgs: 129 active+clean; 106 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.1 MiB/s wr, 440 op/s 2024-09-12T01:36:38.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:38 smithi104 ceph-mon[35219]: pgmap v609: 129 pgs: 129 active+clean; 106 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.1 MiB/s wr, 440 op/s 2024-09-12T01:36:40.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:40 smithi001 ceph-mon[30387]: pgmap v610: 129 pgs: 129 active+clean; 110 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.2 MiB/s wr, 343 op/s 2024-09-12T01:36:40.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:40 smithi053 ceph-mon[35158]: pgmap v610: 129 pgs: 129 active+clean; 110 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.2 MiB/s wr, 343 op/s 2024-09-12T01:36:40.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:40 smithi104 ceph-mon[35219]: pgmap v610: 129 pgs: 129 active+clean; 110 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.2 MiB/s wr, 343 op/s 2024-09-12T01:36:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.694+0000 7f2a26c00640 1 -- 172.21.15.1:0/3929141332 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a20102ba0 msgr2=0x7f2a20103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.694+0000 7f2a26c00640 1 --2- 172.21.15.1:0/3929141332 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a20102ba0 0x7f2a20103020 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c009900 tx=0x7f2a0c02a3f0 comp rx=0 tx=0).stop 2024-09-12T01:36:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.694+0000 7f2a26c00640 1 -- 172.21.15.1:0/3929141332 shutdown_connections 2024-09-12T01:36:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.694+0000 7f2a26c00640 1 --2- 172.21.15.1:0/3929141332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a20103560 0x7f2a20109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.694+0000 7f2a26c00640 1 --2- 172.21.15.1:0/3929141332 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a20102ba0 0x7f2a20103020 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.694+0000 7f2a26c00640 1 --2- 172.21.15.1:0/3929141332 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a201019b0 0x7f2a20101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.694+0000 7f2a26c00640 1 -- 172.21.15.1:0/3929141332 >> 172.21.15.1:0/3929141332 conn(0x7f2a200fd130 msgr2=0x7f2a200ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:36:41.707 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.706+0000 7f2a26c00640 1 -- 172.21.15.1:0/3929141332 shutdown_connections 2024-09-12T01:36:41.707 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.706+0000 7f2a26c00640 1 -- 172.21.15.1:0/3929141332 wait complete. 2024-09-12T01:36:41.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.707+0000 7f2a26c00640 1 Processor -- start 2024-09-12T01:36:41.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.707+0000 7f2a26c00640 1 -- start start 2024-09-12T01:36:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.708+0000 7f2a26c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a201019b0 0x7f2a20199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.708+0000 7f2a26c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a20102ba0 0x7f2a2019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.708+0000 7f2a26c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a20103560 0x7f2a201a1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.708+0000 7f2a26c00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2010d680 con 0x7f2a20103560 2024-09-12T01:36:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.708+0000 7f2a26c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2010d500 con 0x7f2a201019b0 2024-09-12T01:36:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.708+0000 7f2a26c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2010d800 con 0x7f2a20102ba0 2024-09-12T01:36:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.708+0000 7f2a25800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a201019b0 0x7f2a20199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.708+0000 7f2a25800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a201019b0 0x7f2a20199ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:34954/0 (socket says 172.21.15.1:34954) 2024-09-12T01:36:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.708+0000 7f2a26200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a20103560 0x7f2a201a1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.708+0000 7f2a25800640 1 -- 172.21.15.1:0/2409992364 learned_addr learned my addr 172.21.15.1:0/2409992364 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:36:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.708+0000 7f2a24e00640 1 --2- 172.21.15.1:0/2409992364 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a20102ba0 0x7f2a2019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.708+0000 7f2a25800640 1 -- 172.21.15.1:0/2409992364 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a20102ba0 msgr2=0x7f2a2019a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.708+0000 7f2a25800640 1 --2- 172.21.15.1:0/2409992364 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a20102ba0 0x7f2a2019a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.708+0000 7f2a25800640 1 -- 172.21.15.1:0/2409992364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a20103560 msgr2=0x7f2a201a1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.708+0000 7f2a25800640 1 --2- 172.21.15.1:0/2409992364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a20103560 0x7f2a201a1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.708+0000 7f2a25800640 1 -- 172.21.15.1:0/2409992364 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a201a1ae0 con 0x7f2a201019b0 2024-09-12T01:36:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.708+0000 7f2a24e00640 1 --2- 172.21.15.1:0/2409992364 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a20102ba0 0x7f2a2019a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:36:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.708+0000 7f2a26200640 1 --2- 172.21.15.1:0/2409992364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a20103560 0x7f2a201a1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:36:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.709+0000 7f2a25800640 1 --2- 172.21.15.1:0/2409992364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a201019b0 0x7f2a20199ec0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2a1400d950 tx=0x7f2a1400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:36:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.709+0000 7f2a1e000640 1 -- 172.21.15.1:0/2409992364 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a14017020 con 0x7f2a201019b0 2024-09-12T01:36:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.709+0000 7f2a1e000640 1 -- 172.21.15.1:0/2409992364 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a14009c30 con 0x7f2a201019b0 2024-09-12T01:36:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.709+0000 7f2a26c00640 1 -- 172.21.15.1:0/2409992364 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a201a1dd0 con 0x7f2a201019b0 2024-09-12T01:36:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.709+0000 7f2a1e000640 1 -- 172.21.15.1:0/2409992364 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a1400f740 con 0x7f2a201019b0 2024-09-12T01:36:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.709+0000 7f2a26c00640 1 -- 172.21.15.1:0/2409992364 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a201a2230 con 0x7f2a201019b0 2024-09-12T01:36:41.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.712+0000 7f2a2afc4740 1 Processor -- start 2024-09-12T01:36:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.712+0000 7f2a2afc4740 1 -- start start 2024-09-12T01:36:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.712+0000 7f2a1e000640 1 -- 172.21.15.1:0/2409992364 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f2a1401a020 con 0x7f2a201019b0 2024-09-12T01:36:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.712+0000 7f2a2afc4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5d3c3b260 0x55f5d3c3b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.712+0000 7f2a2afc4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5d3c3bba0 0x55f5d3c40660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.713+0000 7f2a2afc4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5d3c40c30 0x55f5d3c430f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.713+0000 7f2a2afc4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f5d3ba0100 con 0x55f5d3c3bba0 2024-09-12T01:36:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.713+0000 7f2a2afc4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55f5d3b481b0 con 0x55f5d3c40c30 2024-09-12T01:36:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.713+0000 7f2a2afc4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f5d3b9c0c0 con 0x55f5d3c3b260 2024-09-12T01:36:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.713+0000 7f2a26200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5d3c3b260 0x55f5d3c3b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.713+0000 7f2a26200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5d3c3b260 0x55f5d3c3b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:53492/0 (socket says 172.21.15.1:53492) 2024-09-12T01:36:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.713+0000 7f2a26200640 1 -- 172.21.15.1:0/2060171512 learned_addr learned my addr 172.21.15.1:0/2060171512 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:36:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.713+0000 7f2a24e00640 1 --2- 172.21.15.1:0/2060171512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5d3c40c30 0x55f5d3c430f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.713+0000 7f2a25800640 1 --2- 172.21.15.1:0/2060171512 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5d3c3bba0 0x55f5d3c40660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.713+0000 7f2a26200640 1 -- 172.21.15.1:0/2060171512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5d3c40c30 msgr2=0x55f5d3c430f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:36:41.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.713+0000 7f2a26200640 1 --2- 172.21.15.1:0/2060171512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5d3c40c30 0x55f5d3c430f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:41.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.713+0000 7f2a26200640 1 -- 172.21.15.1:0/2060171512 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5d3c3bba0 msgr2=0x55f5d3c40660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:36:41.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.713+0000 7f2a26200640 1 --2- 172.21.15.1:0/2060171512 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5d3c3bba0 0x55f5d3c40660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:41.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.713+0000 7f2a26200640 1 -- 172.21.15.1:0/2060171512 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5d38b3210 con 0x55f5d3c3b260 2024-09-12T01:36:41.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.714+0000 7f2a26200640 1 --2- 172.21.15.1:0/2060171512 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5d3c3b260 0x55f5d3c3b660 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2a18011790 tx=0x7f2a18011c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20a44a01815ccdb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:36:41.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.714+0000 7f2a12a00640 1 -- 172.21.15.1:0/2060171512 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a18003c30 con 0x55f5d3c3b260 2024-09-12T01:36:41.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.714+0000 7f2a12a00640 1 -- 172.21.15.1:0/2060171512 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a1800fe80 con 0x55f5d3c3b260 2024-09-12T01:36:41.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.714+0000 7f2a12a00640 1 -- 172.21.15.1:0/2060171512 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a180193e0 con 0x55f5d3c3b260 2024-09-12T01:36:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.714+0000 7f2a1e000640 1 --2- 172.21.15.1:0/2409992364 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f2a080a2b70 0x7f2a080a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.714+0000 7f2a1e000640 1 -- 172.21.15.1:0/2409992364 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f2a140ced50 con 0x7f2a201019b0 2024-09-12T01:36:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.714+0000 7f2a25800640 1 --2- 172.21.15.1:0/2409992364 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f2a080a2b70 0x7f2a080a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.714+0000 7f2a2afc4740 1 -- 172.21.15.1:0/2060171512 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5d3c3b260 msgr2=0x55f5d3c3b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.714+0000 7f2a2afc4740 1 --2- 172.21.15.1:0/2060171512 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5d3c3b260 0x55f5d3c3b660 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2a18011790 tx=0x7f2a18011c50 comp rx=0 tx=0).stop 2024-09-12T01:36:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.715+0000 7f2a2afc4740 1 -- 172.21.15.1:0/2060171512 shutdown_connections 2024-09-12T01:36:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.715+0000 7f2a2afc4740 1 --2- 172.21.15.1:0/2060171512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5d3c40c30 0x55f5d3c430f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.715+0000 7f2a2afc4740 1 --2- 172.21.15.1:0/2060171512 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5d3c3bba0 0x55f5d3c40660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.715+0000 7f2a2afc4740 1 --2- 172.21.15.1:0/2060171512 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5d3c3b260 0x55f5d3c3b660 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.715+0000 7f2a2afc4740 1 -- 172.21.15.1:0/2060171512 >> 172.21.15.1:0/2060171512 conn(0x55f5d3c368d0 msgr2=0x55f5d3c36cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:36:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.715+0000 7f2a2afc4740 1 -- 172.21.15.1:0/2060171512 shutdown_connections 2024-09-12T01:36:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.715+0000 7f2a2afc4740 1 -- 172.21.15.1:0/2060171512 wait complete. 2024-09-12T01:36:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.715+0000 7f2a2afc4740 1 -- start start 2024-09-12T01:36:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.715+0000 7f2a25800640 1 --2- 172.21.15.1:0/2409992364 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f2a080a2b70 0x7f2a080a5030 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2a140cc830 tx=0x7f2a1400bf60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:36:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.715+0000 7f2a2afc4740 1 Processor -- start 2024-09-12T01:36:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.716+0000 7f2a2afc4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5d3c3b260 0x55f5d3c453e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.716+0000 7f2a2afc4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5d3c3bba0 0x55f5d3c45920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.716+0000 7f2a2afc4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5d3c40c30 0x55f5d3c4c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.716+0000 7f2a2afc4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f5d3c43c00 con 0x55f5d3c3b260 2024-09-12T01:36:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.716+0000 7f2a2afc4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55f5d3b29090 con 0x55f5d3c40c30 2024-09-12T01:36:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.716+0000 7f2a2afc4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f5d3c43d80 con 0x55f5d3c3bba0 2024-09-12T01:36:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.716+0000 7f2a2afc4740 10 client.0 _ll_register_callbacks cb 0x55f5d3b7df30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:36:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.716+0000 7f2a24e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5d3c3bba0 0x55f5d3c45920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.716+0000 7f2a24e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5d3c3bba0 0x55f5d3c45920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:53506/0 (socket says 172.21.15.1:53506) 2024-09-12T01:36:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.716+0000 7f2a24e00640 1 -- 172.21.15.1:0/3367591157 learned_addr learned my addr 172.21.15.1:0/3367591157 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:36:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.716+0000 7f2a26200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5d3c3b260 0x55f5d3c453e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.716+0000 7f2a26200640 1 --2- 172.21.15.1:0/3367591157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5d3c40c30 0x55f5d3c4c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.717+0000 7f2a24e00640 1 -- 172.21.15.1:0/3367591157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5d3c40c30 msgr2=0x55f5d3c4c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:36:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.717+0000 7f2a24e00640 1 --2- 172.21.15.1:0/3367591157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5d3c40c30 0x55f5d3c4c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.717+0000 7f2a24e00640 1 -- 172.21.15.1:0/3367591157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5d3c3b260 msgr2=0x55f5d3c453e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:36:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.717+0000 7f2a24e00640 1 --2- 172.21.15.1:0/3367591157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5d3c3b260 0x55f5d3c453e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.717+0000 7f2a24e00640 1 -- 172.21.15.1:0/3367591157 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5d397f620 con 0x55f5d3c3bba0 2024-09-12T01:36:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.717+0000 7f2a26200640 1 --2- 172.21.15.1:0/3367591157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5d3c3b260 0x55f5d3c453e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:36:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.717+0000 7f2a24e00640 1 --2- 172.21.15.1:0/3367591157 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5d3c3bba0 0x55f5d3c45920 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c006010 tx=0x7f2a0c005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=361b746fb085b32d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:36:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.717+0000 7f2a07400640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-12T01:36:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.717+0000 7f2a07400640 1 -- 172.21.15.1:0/3367591157 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c035040 con 0x55f5d3c3bba0 2024-09-12T01:36:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.717+0000 7f2a2afc4740 1 -- 172.21.15.1:0/3367591157 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f5d3bad830 con 0x55f5d3c3bba0 2024-09-12T01:36:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.717+0000 7f2a07400640 1 -- 172.21.15.1:0/3367591157 <== mon.1 v2:172.21.15.104:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f2a0c004970 con 0x55f5d3c3bba0 2024-09-12T01:36:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.718+0000 7f2a07400640 1 -- 172.21.15.1:0/3367591157 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2a0c04f050 con 0x55f5d3c3bba0 2024-09-12T01:36:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.718+0000 7f2a2afc4740 10 client.25750 fetch_fsmap learned FSMap version 35 2024-09-12T01:36:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.718+0000 7f2a2afc4740 1 -- 172.21.15.1:0/3367591157 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f5d3c4f810 con 0x55f5d3c3bba0 2024-09-12T01:36:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.718+0000 7f2a07400640 1 -- 172.21.15.1:0/3367591157 <== mon.1 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f5d3bad830 con 0x55f5d3c3bba0 2024-09-12T01:36:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.718+0000 7f2a07400640 1 -- 172.21.15.1:0/3367591157 <== mon.1 v2:172.21.15.104:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f2a0c044020 con 0x55f5d3c3bba0 2024-09-12T01:36:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.718+0000 7f2a2afc4740 10 client.25750 fetch_fsmap finished waiting for FSMap version 35 2024-09-12T01:36:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.718+0000 7f2a2afc4740 10 client.25750 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:36:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.718+0000 7f2a2afc4740 20 client.25750 populate_metadata read hostname 'smithi001' 2024-09-12T01:36:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.718+0000 7f2a2afc4740 1 --2- 172.21.15.1:0/3367591157 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55f5d3c50e90 0x55f5d3c532e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.718+0000 7f2a2afc4740 4 client.25750 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:36:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.718+0000 7f2a2afc4740 1 -- 172.21.15.1:0/3367591157 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5d3ba7b80 con 0x55f5d3c50e90 2024-09-12T01:36:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.719+0000 7f2a26200640 1 --2- 172.21.15.1:0/3367591157 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55f5d3c50e90 0x55f5d3c532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:41.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.719+0000 7f2a26200640 1 --2- 172.21.15.1:0/3367591157 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55f5d3c50e90 0x55f5d3c532e0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a33d3c5b859f62d server_cookie=7ef3fd8d8d48bf84 in_seq=0 out_seq=0 2024-09-12T01:36:41.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.719+0000 7f2a07400640 10 client.25750 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:36:41.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.720+0000 7f2a07400640 1 -- 172.21.15.1:0/3367591157 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f5d3ba7b80 con 0x55f5d3c50e90 2024-09-12T01:36:41.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.720+0000 7f2a07400640 10 client.25750 handle_command_reply: tid=0 2024-09-12T01:36:41.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.720+0000 7f2a2afc4740 1 client.25750 shutdown 2024-09-12T01:36:41.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.720+0000 7f2a2afc4740 1 -- 172.21.15.1:0/3367591157 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5d3c3bba0 msgr2=0x55f5d3c45920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:41.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.720+0000 7f2a2afc4740 1 --2- 172.21.15.1:0/3367591157 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5d3c3bba0 0x55f5d3c45920 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c006010 tx=0x7f2a0c005270 comp rx=0 tx=0).stop 2024-09-12T01:36:41.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.720+0000 7f2a2afc4740 1 -- 172.21.15.1:0/3367591157 shutdown_connections 2024-09-12T01:36:41.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.720+0000 7f2a2afc4740 1 --2- 172.21.15.1:0/3367591157 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55f5d3c50e90 0x55f5d3c532e0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:41.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.720+0000 7f2a2afc4740 1 --2- 172.21.15.1:0/3367591157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5d3c40c30 0x55f5d3c4c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:41.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.720+0000 7f2a07400640 0 client.25750 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:36:41.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.720+0000 7f2a2afc4740 1 --2- 172.21.15.1:0/3367591157 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5d3c3bba0 0x55f5d3c45920 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:41.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.720+0000 7f2a2afc4740 1 --2- 172.21.15.1:0/3367591157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5d3c3b260 0x55f5d3c453e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:41.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.720+0000 7f2a2afc4740 1 -- 172.21.15.1:0/3367591157 >> 172.21.15.1:0/3367591157 conn(0x55f5d3c368d0 msgr2=0x55f5d3c380e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:36:41.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.720+0000 7f2a2afc4740 1 -- 172.21.15.1:0/3367591157 shutdown_connections 2024-09-12T01:36:41.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.720+0000 7f2a2afc4740 1 -- 172.21.15.1:0/3367591157 wait complete. 2024-09-12T01:36:41.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.720+0000 7f2a2afc4740 20 client.25750 trim_cache size 0 max 16384 2024-09-12T01:36:41.739 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.738+0000 7f2a2afc4740 1 Processor -- start 2024-09-12T01:36:41.740 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.738+0000 7f2a2afc4740 1 -- start start 2024-09-12T01:36:41.740 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.738+0000 7f2a2afc4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5d3c3b260 0x55f5d3c36cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:41.740 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.738+0000 7f2a2afc4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5d3c3bba0 0x55f5d3c4c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:41.740 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.738+0000 7f2a2afc4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5d3c40c30 0x55f5d3c371f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:41.740 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.738+0000 7f2a2afc4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2a201ac790 con 0x55f5d3c3b260 2024-09-12T01:36:41.740 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.738+0000 7f2a2afc4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2a201ae2d0 con 0x55f5d3c3bba0 2024-09-12T01:36:41.740 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.738+0000 7f2a2afc4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2a201ace60 con 0x55f5d3c40c30 2024-09-12T01:36:41.740 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.739+0000 7f2a24e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5d3c3bba0 0x55f5d3c4c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:41.740 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.739+0000 7f2a24e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5d3c3bba0 0x55f5d3c4c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:34980/0 (socket says 172.21.15.1:34980) 2024-09-12T01:36:41.740 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.739+0000 7f2a24e00640 1 -- 172.21.15.1:0/259562619 learned_addr learned my addr 172.21.15.1:0/259562619 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:36:41.740 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.739+0000 7f2a26200640 1 --2- 172.21.15.1:0/259562619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5d3c3b260 0x55f5d3c36cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:41.742 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.739+0000 7f2a24e00640 1 -- 172.21.15.1:0/259562619 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5d3c40c30 msgr2=0x55f5d3c371f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:36:41.742 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.739+0000 7f2a24e00640 1 --2- 172.21.15.1:0/259562619 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5d3c40c30 0x55f5d3c371f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:41.742 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.739+0000 7f2a24e00640 1 -- 172.21.15.1:0/259562619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5d3c3b260 msgr2=0x55f5d3c36cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:36:41.742 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.739+0000 7f2a24e00640 1 --2- 172.21.15.1:0/259562619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5d3c3b260 0x55f5d3c36cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:41.742 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.739+0000 7f2a24e00640 1 -- 172.21.15.1:0/259562619 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5d3c366f0 con 0x55f5d3c3bba0 2024-09-12T01:36:41.742 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.739+0000 7f2a24e00640 1 --2- 172.21.15.1:0/259562619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5d3c3bba0 0x55f5d3c4c920 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c000c00 tx=0x7f2a0c003a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91a1984fc9aba6f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:36:41.742 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.740+0000 7f2a12000640 1 -- 172.21.15.1:0/259562619 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c035030 con 0x55f5d3c3bba0 2024-09-12T01:36:41.742 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.740+0000 7f2a12000640 1 -- 172.21.15.1:0/259562619 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a0c003e00 con 0x55f5d3c3bba0 2024-09-12T01:36:41.742 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.740+0000 7f2a12000640 1 -- 172.21.15.1:0/259562619 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c031910 con 0x55f5d3c3bba0 2024-09-12T01:36:41.742 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.740+0000 7f2a2afc4740 1 -- 172.21.15.1:0/259562619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5d3c3bba0 msgr2=0x55f5d3c4c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:41.742 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.740+0000 7f2a2afc4740 1 --2- 172.21.15.1:0/259562619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5d3c3bba0 0x55f5d3c4c920 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c000c00 tx=0x7f2a0c003a20 comp rx=0 tx=0).stop 2024-09-12T01:36:41.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.740+0000 7f2a2afc4740 1 -- 172.21.15.1:0/259562619 shutdown_connections 2024-09-12T01:36:41.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.740+0000 7f2a2afc4740 1 --2- 172.21.15.1:0/259562619 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5d3c40c30 0x55f5d3c371f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:41.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.740+0000 7f2a2afc4740 1 --2- 172.21.15.1:0/259562619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5d3c3bba0 0x55f5d3c4c920 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:41.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.740+0000 7f2a2afc4740 1 --2- 172.21.15.1:0/259562619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5d3c3b260 0x55f5d3c36cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:41.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.740+0000 7f2a2afc4740 1 -- 172.21.15.1:0/259562619 >> 172.21.15.1:0/259562619 conn(0x55f5d3c368d0 msgr2=0x55f5d3c3cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:36:41.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.741+0000 7f2a2afc4740 1 -- 172.21.15.1:0/259562619 shutdown_connections 2024-09-12T01:36:41.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.741+0000 7f2a2afc4740 1 -- 172.21.15.1:0/259562619 wait complete. 2024-09-12T01:36:41.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.741+0000 7f2a2afc4740 1 -- start start 2024-09-12T01:36:41.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.741+0000 7f2a2afc4740 1 Processor -- start 2024-09-12T01:36:41.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.741+0000 7f2a2afc4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5d3c3b260 0x55f5d3c4c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:41.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.741+0000 7f2a2afc4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5d3c3bba0 0x55f5d3c493e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:41.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.741+0000 7f2a2afc4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5d3c40c30 0x55f5d3c49960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:41.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.741+0000 7f2a2afc4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2a201adc00 con 0x55f5d3c3b260 2024-09-12T01:36:41.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.741+0000 7f2a2afc4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2a201ad530 con 0x55f5d3c3bba0 2024-09-12T01:36:41.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.742+0000 7f2a2afc4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2a201aa770 con 0x55f5d3c40c30 2024-09-12T01:36:41.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.742+0000 7f2a2afc4740 10 client.0 _ll_register_callbacks cb 0x7f2a0c005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:36:41.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.742+0000 7f2a26200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5d3c3b260 0x55f5d3c4c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:41.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.742+0000 7f2a24e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5d3c3bba0 0x55f5d3c493e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:41.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.742+0000 7f2a26200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5d3c3b260 0x55f5d3c4c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40746/0 (socket says 172.21.15.1:40746) 2024-09-12T01:36:41.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.742+0000 7f2a26200640 1 -- 172.21.15.1:0/3299420650 learned_addr learned my addr 172.21.15.1:0/3299420650 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:36:41.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.742+0000 7f2a26200640 1 --2- 172.21.15.1:0/3299420650 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5d3c40c30 0x55f5d3c49960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:41.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.742+0000 7f2a24e00640 1 -- 172.21.15.1:0/3299420650 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5d3c40c30 msgr2=0x55f5d3c49960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:36:41.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.742+0000 7f2a24e00640 1 --2- 172.21.15.1:0/3299420650 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5d3c40c30 0x55f5d3c49960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:41.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.742+0000 7f2a24e00640 1 -- 172.21.15.1:0/3299420650 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5d3c3b260 msgr2=0x55f5d3c4c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:36:41.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.742+0000 7f2a24e00640 1 --2- 172.21.15.1:0/3299420650 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5d3c3b260 0x55f5d3c4c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.742+0000 7f2a24e00640 1 -- 172.21.15.1:0/3299420650 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5d3cb9c00 con 0x55f5d3c3bba0 2024-09-12T01:36:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.742+0000 7f2a24e00640 1 --2- 172.21.15.1:0/3299420650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5d3c3bba0 0x55f5d3c493e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x55f5d3c535c0 tx=0x7f2a0c005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b3ed1a46572216d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:36:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.742+0000 7f2a07400640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-12T01:36:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.743+0000 7f2a07400640 1 -- 172.21.15.1:0/3299420650 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c035040 con 0x55f5d3c3bba0 2024-09-12T01:36:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.743+0000 7f2a07400640 1 -- 172.21.15.1:0/3299420650 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f2a0c004980 con 0x55f5d3c3bba0 2024-09-12T01:36:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.743+0000 7f2a2afc4740 1 -- 172.21.15.1:0/3299420650 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a0c003e00 con 0x55f5d3c3bba0 2024-09-12T01:36:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.743+0000 7f2a07400640 1 -- 172.21.15.1:0/3299420650 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2a0c064050 con 0x55f5d3c3bba0 2024-09-12T01:36:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.743+0000 7f2a2afc4740 10 client.25760 fetch_fsmap learned FSMap version 35 2024-09-12T01:36:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.743+0000 7f2a2afc4740 1 -- 172.21.15.1:0/3299420650 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f5d3cbaf10 con 0x55f5d3c3bba0 2024-09-12T01:36:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.743+0000 7f2a07400640 1 -- 172.21.15.1:0/3299420650 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a0c003e00 con 0x55f5d3c3bba0 2024-09-12T01:36:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.743+0000 7f2a07400640 1 -- 172.21.15.1:0/3299420650 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f2a0c05a030 con 0x55f5d3c3bba0 2024-09-12T01:36:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.743+0000 7f2a2afc4740 10 client.25760 fetch_fsmap finished waiting for FSMap version 35 2024-09-12T01:36:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.743+0000 7f2a2afc4740 10 client.25760 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:36:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.743+0000 7f2a2afc4740 20 client.25760 populate_metadata read hostname 'smithi001' 2024-09-12T01:36:41.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.743+0000 7f2a2afc4740 1 --2- 172.21.15.1:0/3299420650 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55f5d3cbb830 0x55f5d3cbdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:41.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.743+0000 7f2a2afc4740 4 client.25760 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:36:41.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.743+0000 7f2a2afc4740 1 -- 172.21.15.1:0/3299420650 --> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f5d3b247c0 con 0x55f5d3cbb830 2024-09-12T01:36:41.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.744+0000 7f2a26200640 1 --2- 172.21.15.1:0/3299420650 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55f5d3cbb830 0x55f5d3cbdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:41.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.744+0000 7f2a26200640 1 --2- 172.21.15.1:0/3299420650 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55f5d3cbb830 0x55f5d3cbdc50 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0885eaf2fe2a409 server_cookie=9d67afe80eff3fb6 in_seq=0 out_seq=0 2024-09-12T01:36:41.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.744+0000 7f2a07400640 10 client.25760 ms_handle_connect on v2:172.21.15.53:6838/4019389301 2024-09-12T01:36:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.753+0000 7f2a07400640 1 -- 172.21.15.1:0/3299420650 <== mds.0 v2:172.21.15.53:6838/4019389301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f5d3b247c0 con 0x55f5d3cbb830 2024-09-12T01:36:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.753+0000 7f2a07400640 10 client.25760 handle_command_reply: tid=0 2024-09-12T01:36:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.753+0000 7f2a2afc4740 1 client.25760 shutdown 2024-09-12T01:36:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.754+0000 7f2a2afc4740 1 -- 172.21.15.1:0/3299420650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5d3c3bba0 msgr2=0x55f5d3c493e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.754+0000 7f2a2afc4740 1 --2- 172.21.15.1:0/3299420650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5d3c3bba0 0x55f5d3c493e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x55f5d3c535c0 tx=0x7f2a0c005c00 comp rx=0 tx=0).stop 2024-09-12T01:36:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.754+0000 7f2a2afc4740 1 -- 172.21.15.1:0/3299420650 shutdown_connections 2024-09-12T01:36:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.754+0000 7f2a2afc4740 1 --2- 172.21.15.1:0/3299420650 >> [v2:172.21.15.53:6838/4019389301,v1:172.21.15.53:6839/4019389301] conn(0x55f5d3cbb830 0x55f5d3cbdc50 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.754+0000 7f2a2afc4740 1 --2- 172.21.15.1:0/3299420650 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5d3c40c30 0x55f5d3c49960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.754+0000 7f2a2afc4740 1 --2- 172.21.15.1:0/3299420650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5d3c3bba0 0x55f5d3c493e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.754+0000 7f2a2afc4740 1 --2- 172.21.15.1:0/3299420650 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5d3c3b260 0x55f5d3c4c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.754+0000 7f2a07400640 0 client.25760 ms_handle_reset on v2:172.21.15.53:6838/4019389301 2024-09-12T01:36:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.754+0000 7f2a2afc4740 1 -- 172.21.15.1:0/3299420650 >> 172.21.15.1:0/3299420650 conn(0x55f5d3c368d0 msgr2=0x55f5d3c351f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:36:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.754+0000 7f2a2afc4740 1 -- 172.21.15.1:0/3299420650 shutdown_connections 2024-09-12T01:36:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.754+0000 7f2a2afc4740 1 -- 172.21.15.1:0/3299420650 wait complete. 2024-09-12T01:36:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.754+0000 7f2a2afc4740 20 client.25760 trim_cache size 0 max 16384 2024-09-12T01:36:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-12T01:36:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-12T01:36:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-12T01:36:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-12T01:36:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.755+0000 7f2a26c00640 1 -- 172.21.15.1:0/2409992364 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f2a080a2b70 msgr2=0x7f2a080a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.755+0000 7f2a26c00640 1 --2- 172.21.15.1:0/2409992364 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f2a080a2b70 0x7f2a080a5030 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2a140cc830 tx=0x7f2a1400bf60 comp rx=0 tx=0).stop 2024-09-12T01:36:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.755+0000 7f2a26c00640 1 -- 172.21.15.1:0/2409992364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a201019b0 msgr2=0x7f2a20199ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.755+0000 7f2a26c00640 1 --2- 172.21.15.1:0/2409992364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a201019b0 0x7f2a20199ec0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2a1400d950 tx=0x7f2a1400de10 comp rx=0 tx=0).stop 2024-09-12T01:36:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.755+0000 7f2a26c00640 1 -- 172.21.15.1:0/2409992364 shutdown_connections 2024-09-12T01:36:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.755+0000 7f2a26c00640 1 --2- 172.21.15.1:0/2409992364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a20103560 0x7f2a201a1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.755+0000 7f2a26c00640 1 --2- 172.21.15.1:0/2409992364 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f2a080a2b70 0x7f2a080a5030 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.755+0000 7f2a26c00640 1 --2- 172.21.15.1:0/2409992364 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a20102ba0 0x7f2a2019a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.755+0000 7f2a26c00640 1 --2- 172.21.15.1:0/2409992364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a201019b0 0x7f2a20199ec0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.755+0000 7f2a26c00640 1 -- 172.21.15.1:0/2409992364 >> 172.21.15.1:0/2409992364 conn(0x7f2a200fd130 msgr2=0x7f2a200ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:36:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.755+0000 7f2a26c00640 1 -- 172.21.15.1:0/2409992364 shutdown_connections 2024-09-12T01:36:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-12T01:36:41.755+0000 7f2a26c00640 1 -- 172.21.15.1:0/2409992364 wait complete. 2024-09-12T01:36:42.679 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:42 smithi001 ceph-mon[30387]: pgmap v611: 129 pgs: 129 active+clean; 112 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.5 MiB/s wr, 463 op/s 2024-09-12T01:36:42.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:42 smithi053 ceph-mon[35158]: pgmap v611: 129 pgs: 129 active+clean; 112 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.5 MiB/s wr, 463 op/s 2024-09-12T01:36:42.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:42 smithi104 ceph-mon[35219]: pgmap v611: 129 pgs: 129 active+clean; 112 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.5 MiB/s wr, 463 op/s 2024-09-12T01:36:43.032 INFO:tasks.workunit:Stopping ['suites/fsx.sh'] on client.0... 2024-09-12T01:36:43.032 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-12T01:36:44.154 DEBUG:teuthology.parallel:result is None 2024-09-12T01:36:44.155 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-12T01:36:44.220 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-12T01:36:44.220 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-12T01:36:44.231 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi001.front.sepia.ceph.com 2024-09-12T01:36:44.243 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-12T01:36:44.252 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T01:36:44.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.443+0000 7fabd2200640 1 -- 172.21.15.1:0/1421313930 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabcc102c10 msgr2=0x7fabcc103090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:44.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.443+0000 7fabd2200640 1 --2- 172.21.15.1:0/1421313930 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabcc102c10 0x7fabcc103090 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fabb8009900 tx=0x7fabb802a3f0 comp rx=0 tx=0).stop 2024-09-12T01:36:44.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.443+0000 7fabd2200640 1 -- 172.21.15.1:0/1421313930 shutdown_connections 2024-09-12T01:36:44.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.443+0000 7fabd2200640 1 --2- 172.21.15.1:0/1421313930 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabcc1035d0 0x7fabcc109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:44.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.443+0000 7fabd2200640 1 --2- 172.21.15.1:0/1421313930 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabcc102c10 0x7fabcc103090 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:44.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.443+0000 7fabd2200640 1 --2- 172.21.15.1:0/1421313930 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabcc101a20 0x7fabcc101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:44.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.443+0000 7fabd2200640 1 -- 172.21.15.1:0/1421313930 >> 172.21.15.1:0/1421313930 conn(0x7fabcc0fd180 msgr2=0x7fabcc0ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:36:44.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.443+0000 7fabd2200640 1 -- 172.21.15.1:0/1421313930 shutdown_connections 2024-09-12T01:36:44.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.443+0000 7fabd2200640 1 -- 172.21.15.1:0/1421313930 wait complete. 2024-09-12T01:36:44.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.444+0000 7fabd2200640 1 Processor -- start 2024-09-12T01:36:44.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.444+0000 7fabd2200640 1 -- start start 2024-09-12T01:36:44.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.444+0000 7fabd2200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabcc101a20 0x7fabcc199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:44.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.444+0000 7fabd2200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabcc102c10 0x7fabcc19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:44.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.444+0000 7fabd2200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabcc1035d0 0x7fabcc1a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:44.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.444+0000 7fabd2200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fabcc10d6f0 con 0x7fabcc101a20 2024-09-12T01:36:44.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.444+0000 7fabd2200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fabcc10d570 con 0x7fabcc1035d0 2024-09-12T01:36:44.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.444+0000 7fabd2200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fabcc10d870 con 0x7fabcc102c10 2024-09-12T01:36:44.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.444+0000 7fabd0e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabcc101a20 0x7fabcc199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:44.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.445+0000 7fabd0e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabcc101a20 0x7fabcc199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40762/0 (socket says 172.21.15.1:40762) 2024-09-12T01:36:44.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.445+0000 7fabd0e00640 1 -- 172.21.15.1:0/3712358525 learned_addr learned my addr 172.21.15.1:0/3712358525 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:36:44.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.445+0000 7fabd1800640 1 --2- 172.21.15.1:0/3712358525 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabcc1035d0 0x7fabcc1a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:44.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.445+0000 7fabcbe00640 1 --2- 172.21.15.1:0/3712358525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabcc102c10 0x7fabcc19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:44.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.445+0000 7fabd0e00640 1 -- 172.21.15.1:0/3712358525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabcc102c10 msgr2=0x7fabcc19a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:44.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.445+0000 7fabd0e00640 1 --2- 172.21.15.1:0/3712358525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabcc102c10 0x7fabcc19a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:44.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.445+0000 7fabd0e00640 1 -- 172.21.15.1:0/3712358525 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabcc1035d0 msgr2=0x7fabcc1a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:44.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.445+0000 7fabd0e00640 1 --2- 172.21.15.1:0/3712358525 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabcc1035d0 0x7fabcc1a1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:44.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.445+0000 7fabd0e00640 1 -- 172.21.15.1:0/3712358525 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabcc1a1b70 con 0x7fabcc101a20 2024-09-12T01:36:44.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.445+0000 7fabcbe00640 1 --2- 172.21.15.1:0/3712358525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabcc102c10 0x7fabcc19a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:36:44.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.445+0000 7fabd0e00640 1 --2- 172.21.15.1:0/3712358525 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabcc101a20 0x7fabcc199f50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fabc000e9c0 tx=0x7fabc000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:36:44.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.445+0000 7fabc9600640 1 -- 172.21.15.1:0/3712358525 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc000cd80 con 0x7fabcc101a20 2024-09-12T01:36:44.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.446+0000 7fabc9600640 1 -- 172.21.15.1:0/3712358525 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabc00035f0 con 0x7fabcc101a20 2024-09-12T01:36:44.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.446+0000 7fabc9600640 1 -- 172.21.15.1:0/3712358525 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc0010690 con 0x7fabcc101a20 2024-09-12T01:36:44.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.446+0000 7fabd2200640 1 -- 172.21.15.1:0/3712358525 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabcc1a1e60 con 0x7fabcc101a20 2024-09-12T01:36:44.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.446+0000 7fabd2200640 1 -- 172.21.15.1:0/3712358525 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabcc1a2270 con 0x7fabcc101a20 2024-09-12T01:36:44.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.447+0000 7fabd2200640 1 -- 172.21.15.1:0/3712358525 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabcc1a2500 con 0x7fabcc101a20 2024-09-12T01:36:44.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.449+0000 7fabc9600640 1 -- 172.21.15.1:0/3712358525 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fabc00037a0 con 0x7fabcc101a20 2024-09-12T01:36:44.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.450+0000 7fabc9600640 1 --2- 172.21.15.1:0/3712358525 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fabb40a2b80 0x7fabb40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:44.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.450+0000 7fabc9600640 1 -- 172.21.15.1:0/3712358525 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fabc00cf830 con 0x7fabcc101a20 2024-09-12T01:36:44.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.450+0000 7fabcbe00640 1 --2- 172.21.15.1:0/3712358525 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fabb40a2b80 0x7fabb40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:44.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.451+0000 7fabcbe00640 1 --2- 172.21.15.1:0/3712358525 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fabb40a2b80 0x7fabb40a5040 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fabb8006010 tx=0x7fabb8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:36:44.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.454+0000 7fabc9600640 1 -- 172.21.15.1:0/3712358525 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fabc009c3a0 con 0x7fabcc101a20 2024-09-12T01:36:44.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.643+0000 7fabd2200640 1 -- 172.21.15.1:0/3712358525 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fabcc1a2770 con 0x7fabcc101a20 2024-09-12T01:36:44.768 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:44 smithi053 ceph-mon[35158]: pgmap v612: 129 pgs: 129 active+clean; 116 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 369 op/s 2024-09-12T01:36:44.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.644+0000 7fabc9600640 1 -- 172.21.15.1:0/3712358525 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+24751 (secure 0 0 0) 0x7fabc009bca0 con 0x7fabcc101a20 2024-09-12T01:36:44.769 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:36:44.769 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":98,"fsid":"c0e4f180-70a4-11ef-bcea-c7b262605968","created":"2024-09-12T01:18:19.337406+0000","modified":"2024-09-12T01:28:17.407774+0000","last_up_change":"2024-09-12T01:25:50.601477+0000","last_in_change":"2024-09-12T01:25:27.674600+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:21:41.204242+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:26:43.066475+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:26:46.112771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T01:26:56.040130+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b9a396ae-bf97-462f-9ef9-b10ef6b9d337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6803","nonce":1125335136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6805","nonce":1125335136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6809","nonce":1125335136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6807","nonce":1125335136}]},"public_addr":"172.21.15.1:6803/1125335136","cluster_addr":"172.21.15.1:6805/1125335136","heartbeat_back_addr":"172.21.15.1:6809/1125335136","heartbeat_front_addr":"172.21.15.1:6807/1125335136","state":["exists","up"]},{"osd":1,"uuid":"dfe7a8a3-3700-4789-ac64-88660de9e0d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6801","nonce":3131049165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6803","nonce":3131049165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6807","nonce":3131049165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6805","nonce":3131049165}]},"public_addr":"172.21.15.53:6801/3131049165","cluster_addr":"172.21.15.53:6803/3131049165","heartbeat_back_addr":"172.21.15.53:6807/3131049165","heartbeat_front_addr":"172.21.15.53:6805/3131049165","state":["exists","up"]},{"osd":2,"uuid":"5d9d3603-ba7c-4115-9267-12abcb20d55b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6801","nonce":1133285636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6803","nonce":1133285636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6807","nonce":1133285636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6805","nonce":1133285636}]},"public_addr":"172.21.15.104:6801/1133285636","cluster_addr":"172.21.15.104:6803/1133285636","heartbeat_back_addr":"172.21.15.104:6807/1133285636","heartbeat_front_addr":"172.21.15.104:6805/1133285636","state":["exists","up"]},{"osd":3,"uuid":"1e063c66-41b4-4db1-a964-28e122ce628e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6811","nonce":1247468803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6813","nonce":1247468803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6817","nonce":1247468803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6815","nonce":1247468803}]},"public_addr":"172.21.15.1:6811/1247468803","cluster_addr":"172.21.15.1:6813/1247468803","heartbeat_back_addr":"172.21.15.1:6817/1247468803","heartbeat_front_addr":"172.21.15.1:6815/1247468803","state":["exists","up"]},{"osd":4,"uuid":"fe89ece0-2171-45d5-a06d-e400f77233e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6809","nonce":2673413639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6811","nonce":2673413639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6815","nonce":2673413639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6813","nonce":2673413639}]},"public_addr":"172.21.15.53:6809/2673413639","cluster_addr":"172.21.15.53:6811/2673413639","heartbeat_back_addr":"172.21.15.53:6815/2673413639","heartbeat_front_addr":"172.21.15.53:6813/2673413639","state":["exists","up"]},{"osd":5,"uuid":"daab795c-9942-4d16-aa34-8bba30a4b99b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6809","nonce":3131498790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6811","nonce":3131498790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6815","nonce":3131498790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6813","nonce":3131498790}]},"public_addr":"172.21.15.104:6809/3131498790","cluster_addr":"172.21.15.104:6811/3131498790","heartbeat_back_addr":"172.21.15.104:6815/3131498790","heartbeat_front_addr":"172.21.15.104:6813/3131498790","state":["exists","up"]},{"osd":6,"uuid":"f31d8633-e435-499b-9d46-0aa11a2481cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6819","nonce":3413067918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6821","nonce":3413067918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6825","nonce":3413067918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6823","nonce":3413067918}]},"public_addr":"172.21.15.1:6819/3413067918","cluster_addr":"172.21.15.1:6821/3413067918","heartbeat_back_addr":"172.21.15.1:6825/3413067918","heartbeat_front_addr":"172.21.15.1:6823/3413067918","state":["exists","up"]},{"osd":7,"uuid":"e64da49e-d31e-4801-b6cc-634a233b4761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6817","nonce":2841484136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6819","nonce":2841484136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6823","nonce":2841484136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6821","nonce":2841484136}]},"public_addr":"172.21.15.53:6817/2841484136","cluster_addr":"172.21.15.53:6819/2841484136","heartbeat_back_addr":"172.21.15.53:6823/2841484136","heartbeat_front_addr":"172.21.15.53:6821/2841484136","state":["exists","up"]},{"osd":8,"uuid":"fdfb5a42-dca7-4874-b479-176b4a191e21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6817","nonce":941453547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6819","nonce":941453547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6823","nonce":941453547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6821","nonce":941453547}]},"public_addr":"172.21.15.104:6817/941453547","cluster_addr":"172.21.15.104:6819/941453547","heartbeat_back_addr":"172.21.15.104:6823/941453547","heartbeat_front_addr":"172.21.15.104:6821/941453547","state":["exists","up"]},{"osd":9,"uuid":"7f81537a-cae5-4bb8-aec2-4eb07deb7958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6827","nonce":958287586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6829","nonce":958287586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6833","nonce":958287586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6831","nonce":958287586}]},"public_addr":"172.21.15.1:6827/958287586","cluster_addr":"172.21.15.1:6829/958287586","heartbeat_back_addr":"172.21.15.1:6833/958287586","heartbeat_front_addr":"172.21.15.1:6831/958287586","state":["exists","up"]},{"osd":10,"uuid":"e3c9c37d-46b9-4585-990d-69c447453bd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6825","nonce":3357909498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6827","nonce":3357909498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6831","nonce":3357909498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6829","nonce":3357909498}]},"public_addr":"172.21.15.53:6825/3357909498","cluster_addr":"172.21.15.53:6827/3357909498","heartbeat_back_addr":"172.21.15.53:6831/3357909498","heartbeat_front_addr":"172.21.15.53:6829/3357909498","state":["exists","up"]},{"osd":11,"uuid":"474b5345-f2ae-4e0e-a197-53ab11f58c04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6825","nonce":3381427987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6827","nonce":3381427987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6831","nonce":3381427987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6829","nonce":3381427987}]},"public_addr":"172.21.15.104:6825/3381427987","cluster_addr":"172.21.15.104:6827/3381427987","heartbeat_back_addr":"172.21.15.104:6831/3381427987","heartbeat_front_addr":"172.21.15.104:6829/3381427987","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:20:46.903181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:21:12.735556+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:21:37.868739+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:22:09.141228+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:22:36.438563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:23:02.220948+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:23:33.505881+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:24:00.982176+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:24:26.010444+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:24:56.711121+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:25:23.846495+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:25:48.815427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1373984726":"2024-09-13T01:19:14.580637+0000","172.21.15.1:0/1429963718":"2024-09-13T01:19:14.580637+0000","172.21.15.1:6801/4101281615":"2024-09-13T01:19:14.580637+0000","172.21.15.1:6800/4101281615":"2024-09-13T01:19:14.580637+0000","172.21.15.1:0/2564759586":"2024-09-13T01:19:14.580637+0000","172.21.15.1:6801/2935968453":"2024-09-13T01:18:45.882682+0000","172.21.15.1:6800/2935968453":"2024-09-13T01:18:45.882682+0000","172.21.15.1:0/1459848813":"2024-09-13T01:18:45.882682+0000","172.21.15.1:0/578605922":"2024-09-13T01:18:45.882682+0000","172.21.15.1:0/738035035":"2024-09-13T01:18:45.882682+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:36:44.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.647+0000 7fabd2200640 1 -- 172.21.15.1:0/3712358525 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fabb40a2b80 msgr2=0x7fabb40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:44.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.647+0000 7fabd2200640 1 --2- 172.21.15.1:0/3712358525 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fabb40a2b80 0x7fabb40a5040 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fabb8006010 tx=0x7fabb8000bc0 comp rx=0 tx=0).stop 2024-09-12T01:36:44.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.647+0000 7fabd2200640 1 -- 172.21.15.1:0/3712358525 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabcc101a20 msgr2=0x7fabcc199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:44.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.647+0000 7fabd2200640 1 --2- 172.21.15.1:0/3712358525 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabcc101a20 0x7fabcc199f50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fabc000e9c0 tx=0x7fabc000ee80 comp rx=0 tx=0).stop 2024-09-12T01:36:44.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.647+0000 7fabd2200640 1 -- 172.21.15.1:0/3712358525 shutdown_connections 2024-09-12T01:36:44.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.647+0000 7fabd2200640 1 --2- 172.21.15.1:0/3712358525 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabcc1035d0 0x7fabcc1a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:44.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.647+0000 7fabd2200640 1 --2- 172.21.15.1:0/3712358525 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fabb40a2b80 0x7fabb40a5040 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:44.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.647+0000 7fabd2200640 1 --2- 172.21.15.1:0/3712358525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabcc102c10 0x7fabcc19a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:44.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.647+0000 7fabd2200640 1 --2- 172.21.15.1:0/3712358525 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabcc101a20 0x7fabcc199f50 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:44.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.647+0000 7fabd2200640 1 -- 172.21.15.1:0/3712358525 >> 172.21.15.1:0/3712358525 conn(0x7fabcc0fd180 msgr2=0x7fabcc074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:36:44.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.647+0000 7fabd2200640 1 -- 172.21.15.1:0/3712358525 shutdown_connections 2024-09-12T01:36:44.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.647+0000 7fabd2200640 1 -- 172.21.15.1:0/3712358525 wait complete. 2024-09-12T01:36:44.773 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T01:21:41.204242+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-12T01:26:43.066475+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-12T01:26:46.112771+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-12T01:26:56.040130+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-12T01:36:44.773 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-12T01:36:44.774 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:44 smithi001 ceph-mon[30387]: pgmap v612: 129 pgs: 129 active+clean; 116 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 369 op/s 2024-09-12T01:36:44.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:44 smithi104 ceph-mon[35219]: pgmap v612: 129 pgs: 129 active+clean; 116 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 369 op/s 2024-09-12T01:36:44.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.935+0000 7f99b7200640 1 -- 172.21.15.1:0/3049178430 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99b8076230 msgr2=0x7f99b8074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:44.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.935+0000 7f99b7200640 1 --2- 172.21.15.1:0/3049178430 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99b8076230 0x7f99b8074680 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f99a8009860 tx=0x7f99a802a340 comp rx=0 tx=0).stop 2024-09-12T01:36:44.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.935+0000 7f99b7200640 1 -- 172.21.15.1:0/3049178430 shutdown_connections 2024-09-12T01:36:44.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.935+0000 7f99b7200640 1 --2- 172.21.15.1:0/3049178430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99b810df00 0x7f99b81102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:44.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.935+0000 7f99b7200640 1 --2- 172.21.15.1:0/3049178430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f99b8074c50 0x7f99b80750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:44.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.935+0000 7f99b7200640 1 --2- 172.21.15.1:0/3049178430 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99b8076230 0x7f99b8074680 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:44.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.935+0000 7f99b7200640 1 -- 172.21.15.1:0/3049178430 >> 172.21.15.1:0/3049178430 conn(0x7f99b80fd0d0 msgr2=0x7f99b80ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:36:44.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.935+0000 7f99b7200640 1 -- 172.21.15.1:0/3049178430 shutdown_connections 2024-09-12T01:36:44.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.935+0000 7f99b7200640 1 -- 172.21.15.1:0/3049178430 wait complete. 2024-09-12T01:36:44.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.936+0000 7f99b7200640 1 Processor -- start 2024-09-12T01:36:44.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.936+0000 7f99b7200640 1 -- start start 2024-09-12T01:36:44.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.937+0000 7f99b7200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f99b8074c50 0x7f99b819e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:44.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.937+0000 7f99b7200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99b8076230 0x7f99b819e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:44.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.937+0000 7f99b7200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99b810df00 0x7f99b81a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:44.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.937+0000 7f99b7200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f99b8113bb0 con 0x7f99b8074c50 2024-09-12T01:36:44.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.937+0000 7f99b7200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f99b8113a30 con 0x7f99b810df00 2024-09-12T01:36:44.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.937+0000 7f99b7200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f99b8113d30 con 0x7f99b8076230 2024-09-12T01:36:44.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.937+0000 7f99b5e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f99b8074c50 0x7f99b819e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:44.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.937+0000 7f99ad400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99b8076230 0x7f99b819e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:44.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.937+0000 7f99ad400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99b8076230 0x7f99b819e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:53544/0 (socket says 172.21.15.1:53544) 2024-09-12T01:36:44.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.937+0000 7f99ad400640 1 -- 172.21.15.1:0/1863383375 learned_addr learned my addr 172.21.15.1:0/1863383375 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:36:44.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.937+0000 7f99b6800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99b810df00 0x7f99b81a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:44.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.938+0000 7f99ad400640 1 -- 172.21.15.1:0/1863383375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99b810df00 msgr2=0x7f99b81a5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:44.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.938+0000 7f99ad400640 1 --2- 172.21.15.1:0/1863383375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99b810df00 0x7f99b81a5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:44.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.938+0000 7f99ad400640 1 -- 172.21.15.1:0/1863383375 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f99b8074c50 msgr2=0x7f99b819e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:44.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.938+0000 7f99ad400640 1 --2- 172.21.15.1:0/1863383375 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f99b8074c50 0x7f99b819e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:44.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.938+0000 7f99ad400640 1 -- 172.21.15.1:0/1863383375 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99b81a5eb0 con 0x7f99b8076230 2024-09-12T01:36:44.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.938+0000 7f99ad400640 1 --2- 172.21.15.1:0/1863383375 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99b8076230 0x7f99b819e7d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f99a000e9c0 tx=0x7f99a000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:36:44.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.938+0000 7f99af400640 1 -- 172.21.15.1:0/1863383375 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a000ccc0 con 0x7f99b8076230 2024-09-12T01:36:44.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.938+0000 7f99af400640 1 -- 172.21.15.1:0/1863383375 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99a0005500 con 0x7f99b8076230 2024-09-12T01:36:44.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.938+0000 7f99b7200640 1 -- 172.21.15.1:0/1863383375 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99b81a61a0 con 0x7f99b8076230 2024-09-12T01:36:44.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.938+0000 7f99af400640 1 -- 172.21.15.1:0/1863383375 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a0010430 con 0x7f99b8076230 2024-09-12T01:36:44.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.938+0000 7f99b7200640 1 -- 172.21.15.1:0/1863383375 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99b81a6740 con 0x7f99b8076230 2024-09-12T01:36:44.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.939+0000 7f99b7200640 1 -- 172.21.15.1:0/1863383375 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99b81038a0 con 0x7f99b8076230 2024-09-12T01:36:44.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.942+0000 7f99af400640 1 -- 172.21.15.1:0/1863383375 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f99a00046c0 con 0x7f99b8076230 2024-09-12T01:36:44.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.943+0000 7f99af400640 1 --2- 172.21.15.1:0/1863383375 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f999c0a2c10 0x7f999c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:44.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.943+0000 7f99af400640 1 -- 172.21.15.1:0/1863383375 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f99a00cf870 con 0x7f99b8076230 2024-09-12T01:36:44.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.943+0000 7f99b5e00640 1 --2- 172.21.15.1:0/1863383375 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f999c0a2c10 0x7f999c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:44.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.944+0000 7f99b5e00640 1 --2- 172.21.15.1:0/1863383375 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f999c0a2c10 0x7f999c0a50d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f99a802a930 tx=0x7f99a8030570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:36:44.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:44.946+0000 7f99af400640 1 -- 172.21.15.1:0/1863383375 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f99a00541b0 con 0x7f99b8076230 2024-09-12T01:36:45.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.129+0000 7f99b7200640 1 -- 172.21.15.1:0/1863383375 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f99b802da10 con 0x7f99b8076230 2024-09-12T01:36:45.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.131+0000 7f99af400640 1 -- 172.21.15.1:0/1863383375 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) ==== 93+0+10 (secure 0 0 0) 0x7f99a009c3e0 con 0x7f99b8076230 2024-09-12T01:36:45.131 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-09-12T01:36:45.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.132+0000 7f99b7200640 1 -- 172.21.15.1:0/1863383375 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f999c0a2c10 msgr2=0x7f999c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:45.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.132+0000 7f99b7200640 1 --2- 172.21.15.1:0/1863383375 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f999c0a2c10 0x7f999c0a50d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f99a802a930 tx=0x7f99a8030570 comp rx=0 tx=0).stop 2024-09-12T01:36:45.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.132+0000 7f99b7200640 1 -- 172.21.15.1:0/1863383375 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99b8076230 msgr2=0x7f99b819e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:45.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.132+0000 7f99b7200640 1 --2- 172.21.15.1:0/1863383375 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99b8076230 0x7f99b819e7d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f99a000e9c0 tx=0x7f99a000ee80 comp rx=0 tx=0).stop 2024-09-12T01:36:45.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.132+0000 7f99b7200640 1 -- 172.21.15.1:0/1863383375 shutdown_connections 2024-09-12T01:36:45.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.132+0000 7f99b7200640 1 --2- 172.21.15.1:0/1863383375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99b810df00 0x7f99b81a5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:45.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.132+0000 7f99b7200640 1 --2- 172.21.15.1:0/1863383375 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f999c0a2c10 0x7f999c0a50d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:45.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.132+0000 7f99b7200640 1 --2- 172.21.15.1:0/1863383375 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99b8076230 0x7f99b819e7d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:45.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.132+0000 7f99b7200640 1 --2- 172.21.15.1:0/1863383375 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f99b8074c50 0x7f99b819e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:45.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.132+0000 7f99b7200640 1 -- 172.21.15.1:0/1863383375 >> 172.21.15.1:0/1863383375 conn(0x7f99b80fd0d0 msgr2=0x7f99b80ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:36:45.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.132+0000 7f99b7200640 1 -- 172.21.15.1:0/1863383375 shutdown_connections 2024-09-12T01:36:45.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.132+0000 7f99b7200640 1 -- 172.21.15.1:0/1863383375 wait complete. 2024-09-12T01:36:45.149 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-12T01:36:45.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.311+0000 7fcc64e00640 1 -- 172.21.15.1:0/4122389298 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc60103d40 msgr2=0x7fcc60104140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:45.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.311+0000 7fcc64e00640 1 --2- 172.21.15.1:0/4122389298 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc60103d40 0x7fcc60104140 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fcc50009ca0 tx=0x7fcc5002a8c0 comp rx=0 tx=0).stop 2024-09-12T01:36:45.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.311+0000 7fcc64e00640 1 -- 172.21.15.1:0/4122389298 shutdown_connections 2024-09-12T01:36:45.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.311+0000 7fcc64e00640 1 --2- 172.21.15.1:0/4122389298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc601058f0 0x7fcc6010c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:45.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.311+0000 7fcc64e00640 1 --2- 172.21.15.1:0/4122389298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc60104f30 0x7fcc601053b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:45.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.311+0000 7fcc64e00640 1 --2- 172.21.15.1:0/4122389298 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc60103d40 0x7fcc60104140 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:45.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.311+0000 7fcc64e00640 1 -- 172.21.15.1:0/4122389298 >> 172.21.15.1:0/4122389298 conn(0x7fcc600ff4a0 msgr2=0x7fcc601018e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:36:45.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.312+0000 7fcc64e00640 1 -- 172.21.15.1:0/4122389298 shutdown_connections 2024-09-12T01:36:45.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.312+0000 7fcc64e00640 1 -- 172.21.15.1:0/4122389298 wait complete. 2024-09-12T01:36:45.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.312+0000 7fcc64e00640 1 Processor -- start 2024-09-12T01:36:45.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.313+0000 7fcc64e00640 1 -- start start 2024-09-12T01:36:45.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.313+0000 7fcc64e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc60103d40 0x7fcc6019a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:45.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.313+0000 7fcc64e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc60104f30 0x7fcc6019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:45.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.313+0000 7fcc64e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc601058f0 0x7fcc601a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:45.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.313+0000 7fcc64e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6010f9c0 con 0x7fcc60103d40 2024-09-12T01:36:45.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.313+0000 7fcc64e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6010f840 con 0x7fcc601058f0 2024-09-12T01:36:45.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.313+0000 7fcc64e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6010fb40 con 0x7fcc60104f30 2024-09-12T01:36:45.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.313+0000 7fcc5de00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc60103d40 0x7fcc6019a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:45.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.313+0000 7fcc5de00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc60103d40 0x7fcc6019a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40792/0 (socket says 172.21.15.1:40792) 2024-09-12T01:36:45.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.313+0000 7fcc5de00640 1 -- 172.21.15.1:0/3615942135 learned_addr learned my addr 172.21.15.1:0/3615942135 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:36:45.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.313+0000 7fcc5e800640 1 --2- 172.21.15.1:0/3615942135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc601058f0 0x7fcc601a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:45.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.313+0000 7fcc55400640 1 --2- 172.21.15.1:0/3615942135 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc60104f30 0x7fcc6019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:45.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.314+0000 7fcc5de00640 1 -- 172.21.15.1:0/3615942135 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc60104f30 msgr2=0x7fcc6019a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:45.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.314+0000 7fcc5de00640 1 --2- 172.21.15.1:0/3615942135 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc60104f30 0x7fcc6019a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:45.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.314+0000 7fcc5de00640 1 -- 172.21.15.1:0/3615942135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc601058f0 msgr2=0x7fcc601a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:45.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.314+0000 7fcc5de00640 1 --2- 172.21.15.1:0/3615942135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc601058f0 0x7fcc601a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:45.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.314+0000 7fcc5de00640 1 -- 172.21.15.1:0/3615942135 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc601a1cc0 con 0x7fcc60103d40 2024-09-12T01:36:45.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.314+0000 7fcc5de00640 1 --2- 172.21.15.1:0/3615942135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc60103d40 0x7fcc6019a0a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcc50009bf0 tx=0x7fcc50031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:36:45.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.314+0000 7fcc57400640 1 -- 172.21.15.1:0/3615942135 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5002fa20 con 0x7fcc60103d40 2024-09-12T01:36:45.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.314+0000 7fcc57400640 1 -- 172.21.15.1:0/3615942135 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc5002fbc0 con 0x7fcc60103d40 2024-09-12T01:36:45.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.314+0000 7fcc57400640 1 -- 172.21.15.1:0/3615942135 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc50046750 con 0x7fcc60103d40 2024-09-12T01:36:45.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.314+0000 7fcc64e00640 1 -- 172.21.15.1:0/3615942135 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc601a1f50 con 0x7fcc60103d40 2024-09-12T01:36:45.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.314+0000 7fcc64e00640 1 -- 172.21.15.1:0/3615942135 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc601a2360 con 0x7fcc60103d40 2024-09-12T01:36:45.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.316+0000 7fcc64e00640 1 -- 172.21.15.1:0/3615942135 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc6010b1d0 con 0x7fcc60103d40 2024-09-12T01:36:45.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.317+0000 7fcc57400640 1 -- 172.21.15.1:0/3615942135 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fcc5002fd70 con 0x7fcc60103d40 2024-09-12T01:36:45.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.318+0000 7fcc57400640 1 --2- 172.21.15.1:0/3615942135 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fcc440a2b90 0x7fcc440a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:45.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.319+0000 7fcc57400640 1 -- 172.21.15.1:0/3615942135 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fcc500f0750 con 0x7fcc60103d40 2024-09-12T01:36:45.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.319+0000 7fcc55400640 1 --2- 172.21.15.1:0/3615942135 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fcc440a2b90 0x7fcc440a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:45.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.319+0000 7fcc55400640 1 --2- 172.21.15.1:0/3615942135 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fcc440a2b90 0x7fcc440a5050 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcc6019b5b0 tx=0x7fcc48005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:36:45.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.323+0000 7fcc57400640 1 -- 172.21.15.1:0/3615942135 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcc500bd2c0 con 0x7fcc60103d40 2024-09-12T01:36:45.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.516+0000 7fcc64e00640 1 -- 172.21.15.1:0/3615942135 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcc60062ef0 con 0x7fcc60103d40 2024-09-12T01:36:45.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.517+0000 7fcc57400640 1 -- 172.21.15.1:0/3615942135 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v98) ==== 104+0+11 (secure 0 0 0) 0x7fcc500bcbc0 con 0x7fcc60103d40 2024-09-12T01:36:45.518 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-09-12T01:36:45.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.520+0000 7fcc64e00640 1 -- 172.21.15.1:0/3615942135 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fcc440a2b90 msgr2=0x7fcc440a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:45.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.520+0000 7fcc64e00640 1 --2- 172.21.15.1:0/3615942135 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fcc440a2b90 0x7fcc440a5050 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcc6019b5b0 tx=0x7fcc48005f90 comp rx=0 tx=0).stop 2024-09-12T01:36:45.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.520+0000 7fcc64e00640 1 -- 172.21.15.1:0/3615942135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc60103d40 msgr2=0x7fcc6019a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:45.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.520+0000 7fcc64e00640 1 --2- 172.21.15.1:0/3615942135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc60103d40 0x7fcc6019a0a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcc50009bf0 tx=0x7fcc50031ee0 comp rx=0 tx=0).stop 2024-09-12T01:36:45.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.520+0000 7fcc64e00640 1 -- 172.21.15.1:0/3615942135 shutdown_connections 2024-09-12T01:36:45.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.520+0000 7fcc64e00640 1 --2- 172.21.15.1:0/3615942135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc601058f0 0x7fcc601a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:45.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.520+0000 7fcc64e00640 1 --2- 172.21.15.1:0/3615942135 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fcc440a2b90 0x7fcc440a5050 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:45.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.520+0000 7fcc64e00640 1 --2- 172.21.15.1:0/3615942135 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc60104f30 0x7fcc6019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:45.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.520+0000 7fcc64e00640 1 --2- 172.21.15.1:0/3615942135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc60103d40 0x7fcc6019a0a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:45.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.520+0000 7fcc64e00640 1 -- 172.21.15.1:0/3615942135 >> 172.21.15.1:0/3615942135 conn(0x7fcc600ff4a0 msgr2=0x7fcc601083a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:36:45.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.520+0000 7fcc64e00640 1 -- 172.21.15.1:0/3615942135 shutdown_connections 2024-09-12T01:36:45.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.520+0000 7fcc64e00640 1 -- 172.21.15.1:0/3615942135 wait complete. 2024-09-12T01:36:45.536 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-12T01:36:45.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.699+0000 7f907de00640 1 -- 172.21.15.1:0/4003810290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9078103b50 msgr2=0x7f9078103f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:45.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.699+0000 7f907de00640 1 --2- 172.21.15.1:0/4003810290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9078103b50 0x7f9078103f50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f906c009900 tx=0x7f906c02a3f0 comp rx=0 tx=0).stop 2024-09-12T01:36:45.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.699+0000 7f907de00640 1 -- 172.21.15.1:0/4003810290 shutdown_connections 2024-09-12T01:36:45.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.699+0000 7f907de00640 1 --2- 172.21.15.1:0/4003810290 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9078105700 0x7f907810bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:45.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.699+0000 7f907de00640 1 --2- 172.21.15.1:0/4003810290 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9078104d40 0x7f90781051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:45.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.699+0000 7f907de00640 1 --2- 172.21.15.1:0/4003810290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9078103b50 0x7f9078103f50 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:45.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.699+0000 7f907de00640 1 -- 172.21.15.1:0/4003810290 >> 172.21.15.1:0/4003810290 conn(0x7f90780ff2b0 msgr2=0x7f90781016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:36:45.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.699+0000 7f907de00640 1 -- 172.21.15.1:0/4003810290 shutdown_connections 2024-09-12T01:36:45.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.699+0000 7f907de00640 1 -- 172.21.15.1:0/4003810290 wait complete. 2024-09-12T01:36:45.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.700+0000 7f907de00640 1 Processor -- start 2024-09-12T01:36:45.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.700+0000 7f907de00640 1 -- start start 2024-09-12T01:36:45.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.700+0000 7f907de00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9078103b50 0x7f9078199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:45.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.700+0000 7f907de00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9078104d40 0x7f907819a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:45.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.700+0000 7f907de00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9078105700 0x7f90781a1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:45.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.700+0000 7f907de00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f907810f820 con 0x7f9078105700 2024-09-12T01:36:45.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.700+0000 7f907de00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f907810f6a0 con 0x7f9078104d40 2024-09-12T01:36:45.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.700+0000 7f907de00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f907810f9a0 con 0x7f9078103b50 2024-09-12T01:36:45.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.700+0000 7f907d400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9078105700 0x7f90781a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:45.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.700+0000 7f907ca00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9078103b50 0x7f9078199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:45.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.700+0000 7f9077e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9078104d40 0x7f907819a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:45.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.701+0000 7f907ca00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9078103b50 0x7f9078199ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.1:53572/0 (socket says 172.21.15.1:53572) 2024-09-12T01:36:45.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.701+0000 7f907ca00640 1 -- 172.21.15.1:0/4115488039 learned_addr learned my addr 172.21.15.1:0/4115488039 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:36:45.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.701+0000 7f9077e00640 1 -- 172.21.15.1:0/4115488039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9078103b50 msgr2=0x7f9078199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:45.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.701+0000 7f9077e00640 1 --2- 172.21.15.1:0/4115488039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9078103b50 0x7f9078199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:45.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.701+0000 7f9077e00640 1 -- 172.21.15.1:0/4115488039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9078105700 msgr2=0x7f90781a1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:45.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.701+0000 7f9077e00640 1 --2- 172.21.15.1:0/4115488039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9078105700 0x7f90781a1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:45.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.701+0000 7f9077e00640 1 -- 172.21.15.1:0/4115488039 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90781a1b00 con 0x7f9078104d40 2024-09-12T01:36:45.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.701+0000 7f907d400640 1 --2- 172.21.15.1:0/4115488039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9078105700 0x7f90781a1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:36:45.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.701+0000 7f9077e00640 1 --2- 172.21.15.1:0/4115488039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9078104d40 0x7f907819a420 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f906400e9f0 tx=0x7f906400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:36:45.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.701+0000 7f9075600640 1 -- 172.21.15.1:0/4115488039 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906400ccc0 con 0x7f9078104d40 2024-09-12T01:36:45.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.702+0000 7f907de00640 1 -- 172.21.15.1:0/4115488039 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90781a1df0 con 0x7f9078104d40 2024-09-12T01:36:45.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.702+0000 7f9075600640 1 -- 172.21.15.1:0/4115488039 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9064010910 con 0x7f9078104d40 2024-09-12T01:36:45.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.702+0000 7f9075600640 1 -- 172.21.15.1:0/4115488039 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9064018960 con 0x7f9078104d40 2024-09-12T01:36:45.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.702+0000 7f907de00640 1 -- 172.21.15.1:0/4115488039 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90781a22b0 con 0x7f9078104d40 2024-09-12T01:36:45.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.703+0000 7f907de00640 1 -- 172.21.15.1:0/4115488039 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f907810afe0 con 0x7f9078104d40 2024-09-12T01:36:45.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.705+0000 7f9075600640 1 -- 172.21.15.1:0/4115488039 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f9064010ac0 con 0x7f9078104d40 2024-09-12T01:36:45.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.708+0000 7f9075600640 1 --2- 172.21.15.1:0/4115488039 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f90600a2b90 0x7f90600a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:45.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.708+0000 7f9075600640 1 -- 172.21.15.1:0/4115488039 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f90640cfc80 con 0x7f9078104d40 2024-09-12T01:36:45.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.708+0000 7f907ca00640 1 --2- 172.21.15.1:0/4115488039 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f90600a2b90 0x7f90600a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:45.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.708+0000 7f907ca00640 1 --2- 172.21.15.1:0/4115488039 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f90600a2b90 0x7f90600a5050 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f906c000c00 tx=0x7f906c0315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:36:45.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.708+0000 7f9075600640 1 -- 172.21.15.1:0/4115488039 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f906409c1a0 con 0x7f9078104d40 2024-09-12T01:36:45.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:45 smithi053 ceph-mon[35158]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T01:36:45.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:45 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/3712358525' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:36:45.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:45 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/1863383375' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T01:36:45.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:45 smithi104 ceph-mon[35219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T01:36:45.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:45 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/3712358525' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:36:45.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:45 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/1863383375' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T01:36:45.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:45 smithi001 ceph-mon[30387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T01:36:45.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:45 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/3712358525' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:36:45.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:45 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/1863383375' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T01:36:45.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.968+0000 7f907de00640 1 -- 172.21.15.1:0/4115488039 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f907802da10 con 0x7f9078104d40 2024-09-12T01:36:45.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.968+0000 7f9075600640 1 -- 172.21.15.1:0/4115488039 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v98) ==== 100+0+11 (secure 0 0 0) 0x7f90640a04a0 con 0x7f9078104d40 2024-09-12T01:36:45.969 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-09-12T01:36:45.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.971+0000 7f907de00640 1 -- 172.21.15.1:0/4115488039 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f90600a2b90 msgr2=0x7f90600a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:45.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.971+0000 7f907de00640 1 --2- 172.21.15.1:0/4115488039 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f90600a2b90 0x7f90600a5050 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f906c000c00 tx=0x7f906c0315b0 comp rx=0 tx=0).stop 2024-09-12T01:36:45.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.971+0000 7f907de00640 1 -- 172.21.15.1:0/4115488039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9078104d40 msgr2=0x7f907819a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:45.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.971+0000 7f907de00640 1 --2- 172.21.15.1:0/4115488039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9078104d40 0x7f907819a420 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f906400e9f0 tx=0x7f906400eeb0 comp rx=0 tx=0).stop 2024-09-12T01:36:45.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.971+0000 7f907de00640 1 -- 172.21.15.1:0/4115488039 shutdown_connections 2024-09-12T01:36:45.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.971+0000 7f907de00640 1 --2- 172.21.15.1:0/4115488039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9078105700 0x7f90781a1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:45.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.971+0000 7f907de00640 1 --2- 172.21.15.1:0/4115488039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9078104d40 0x7f907819a420 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:45.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.971+0000 7f907de00640 1 --2- 172.21.15.1:0/4115488039 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f90600a2b90 0x7f90600a5050 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:45.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.971+0000 7f907de00640 1 --2- 172.21.15.1:0/4115488039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9078103b50 0x7f9078199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:45.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.971+0000 7f907de00640 1 -- 172.21.15.1:0/4115488039 >> 172.21.15.1:0/4115488039 conn(0x7f90780ff2b0 msgr2=0x7f90781081b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:36:45.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.971+0000 7f907de00640 1 -- 172.21.15.1:0/4115488039 shutdown_connections 2024-09-12T01:36:45.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:45.971+0000 7f907de00640 1 -- 172.21.15.1:0/4115488039 wait complete. 2024-09-12T01:36:45.988 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-12T01:36:46.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.149+0000 7f34de000640 1 -- 172.21.15.1:0/2536560986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34d8105900 msgr2=0x7f34d810c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:46.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.149+0000 7f34de000640 1 --2- 172.21.15.1:0/2536560986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34d8105900 0x7f34d810c190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f34d0009ca0 tx=0x7f34d002a8c0 comp rx=0 tx=0).stop 2024-09-12T01:36:46.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.149+0000 7f34de000640 1 -- 172.21.15.1:0/2536560986 shutdown_connections 2024-09-12T01:36:46.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.149+0000 7f34de000640 1 --2- 172.21.15.1:0/2536560986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34d8105900 0x7f34d810c190 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:46.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.149+0000 7f34de000640 1 --2- 172.21.15.1:0/2536560986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34d8104f40 0x7f34d81053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:46.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.149+0000 7f34de000640 1 --2- 172.21.15.1:0/2536560986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34d8103d50 0x7f34d8104150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:46.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.149+0000 7f34de000640 1 -- 172.21.15.1:0/2536560986 >> 172.21.15.1:0/2536560986 conn(0x7f34d80ff4b0 msgr2=0x7f34d81018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:36:46.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.149+0000 7f34de000640 1 -- 172.21.15.1:0/2536560986 shutdown_connections 2024-09-12T01:36:46.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.149+0000 7f34de000640 1 -- 172.21.15.1:0/2536560986 wait complete. 2024-09-12T01:36:46.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.150+0000 7f34de000640 1 Processor -- start 2024-09-12T01:36:46.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.150+0000 7f34de000640 1 -- start start 2024-09-12T01:36:46.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.150+0000 7f34de000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34d8103d50 0x7f34d8199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:46.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.150+0000 7f34de000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34d8104f40 0x7f34d819a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:46.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.150+0000 7f34de000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34d8105900 0x7f34d81a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:46.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.150+0000 7f34de000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f34d810fa00 con 0x7f34d8105900 2024-09-12T01:36:46.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.150+0000 7f34de000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f34d810f880 con 0x7f34d8103d50 2024-09-12T01:36:46.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.150+0000 7f34de000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f34d810fb80 con 0x7f34d8104f40 2024-09-12T01:36:46.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.151+0000 7f34d7e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34d8105900 0x7f34d81a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:46.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.151+0000 7f34d7e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34d8105900 0x7f34d81a1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40824/0 (socket says 172.21.15.1:40824) 2024-09-12T01:36:46.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.151+0000 7f34d7e00640 1 -- 172.21.15.1:0/3779144598 learned_addr learned my addr 172.21.15.1:0/3779144598 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:36:46.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.151+0000 7f34d7400640 1 --2- 172.21.15.1:0/3779144598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34d8103d50 0x7f34d8199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:46.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.151+0000 7f34cfe00640 1 --2- 172.21.15.1:0/3779144598 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34d8104f40 0x7f34d819a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:46.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.151+0000 7f34d7400640 1 -- 172.21.15.1:0/3779144598 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34d8104f40 msgr2=0x7f34d819a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:46.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.151+0000 7f34d7400640 1 --2- 172.21.15.1:0/3779144598 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34d8104f40 0x7f34d819a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:46.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.151+0000 7f34d7400640 1 -- 172.21.15.1:0/3779144598 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34d8105900 msgr2=0x7f34d81a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:46.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.151+0000 7f34d7400640 1 --2- 172.21.15.1:0/3779144598 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34d8105900 0x7f34d81a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:46.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.151+0000 7f34d7400640 1 -- 172.21.15.1:0/3779144598 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34d81a1bf0 con 0x7f34d8103d50 2024-09-12T01:36:46.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.151+0000 7f34d7e00640 1 --2- 172.21.15.1:0/3779144598 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34d8105900 0x7f34d81a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:36:46.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.151+0000 7f34d7400640 1 --2- 172.21.15.1:0/3779144598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34d8103d50 0x7f34d8199fd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f34c800d6a0 tx=0x7f34c800db60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:36:46.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.152+0000 7f34d4c00640 1 -- 172.21.15.1:0/3779144598 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c8017070 con 0x7f34d8103d50 2024-09-12T01:36:46.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.152+0000 7f34d4c00640 1 -- 172.21.15.1:0/3779144598 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34c8009d00 con 0x7f34d8103d50 2024-09-12T01:36:46.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.152+0000 7f34de000640 1 -- 172.21.15.1:0/3779144598 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34d81a1ee0 con 0x7f34d8103d50 2024-09-12T01:36:46.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.152+0000 7f34d4c00640 1 -- 172.21.15.1:0/3779144598 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c8003c60 con 0x7f34d8103d50 2024-09-12T01:36:46.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.152+0000 7f34de000640 1 -- 172.21.15.1:0/3779144598 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34d81a23a0 con 0x7f34d8103d50 2024-09-12T01:36:46.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.153+0000 7f34de000640 1 -- 172.21.15.1:0/3779144598 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34d810b070 con 0x7f34d8103d50 2024-09-12T01:36:46.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.155+0000 7f34d4c00640 1 -- 172.21.15.1:0/3779144598 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f34c8009820 con 0x7f34d8103d50 2024-09-12T01:36:46.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.156+0000 7f34d4c00640 1 --2- 172.21.15.1:0/3779144598 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f34bc0a2c10 0x7f34bc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:46.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.156+0000 7f34d4c00640 1 -- 172.21.15.1:0/3779144598 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f34c80cec60 con 0x7f34d8103d50 2024-09-12T01:36:46.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.156+0000 7f34cfe00640 1 --2- 172.21.15.1:0/3779144598 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f34bc0a2c10 0x7f34bc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:46.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.157+0000 7f34cfe00640 1 --2- 172.21.15.1:0/3779144598 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f34bc0a2c10 0x7f34bc0a50d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f34d819b4e0 tx=0x7f34c0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:36:46.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.158+0000 7f34d4c00640 1 -- 172.21.15.1:0/3779144598 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f34c809b200 con 0x7f34d8103d50 2024-09-12T01:36:46.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.344+0000 7f34de000640 1 -- 172.21.15.1:0/3779144598 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f34d802d7c0 con 0x7f34d8103d50 2024-09-12T01:36:46.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.345+0000 7f34d4c00640 1 -- 172.21.15.1:0/3779144598 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v98) ==== 103+0+11 (secure 0 0 0) 0x7f34c809f500 con 0x7f34d8103d50 2024-09-12T01:36:46.345 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-09-12T01:36:46.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.347+0000 7f34de000640 1 -- 172.21.15.1:0/3779144598 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f34bc0a2c10 msgr2=0x7f34bc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:46.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.347+0000 7f34de000640 1 --2- 172.21.15.1:0/3779144598 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f34bc0a2c10 0x7f34bc0a50d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f34d819b4e0 tx=0x7f34c0005f90 comp rx=0 tx=0).stop 2024-09-12T01:36:46.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.347+0000 7f34de000640 1 -- 172.21.15.1:0/3779144598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34d8103d50 msgr2=0x7f34d8199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:46.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.347+0000 7f34de000640 1 --2- 172.21.15.1:0/3779144598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34d8103d50 0x7f34d8199fd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f34c800d6a0 tx=0x7f34c800db60 comp rx=0 tx=0).stop 2024-09-12T01:36:46.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.347+0000 7f34de000640 1 -- 172.21.15.1:0/3779144598 shutdown_connections 2024-09-12T01:36:46.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.347+0000 7f34de000640 1 --2- 172.21.15.1:0/3779144598 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34d8105900 0x7f34d81a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:46.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.347+0000 7f34de000640 1 --2- 172.21.15.1:0/3779144598 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f34bc0a2c10 0x7f34bc0a50d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:46.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.347+0000 7f34de000640 1 --2- 172.21.15.1:0/3779144598 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34d8104f40 0x7f34d819a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:46.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.347+0000 7f34de000640 1 --2- 172.21.15.1:0/3779144598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34d8103d50 0x7f34d8199fd0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:46.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.347+0000 7f34de000640 1 -- 172.21.15.1:0/3779144598 >> 172.21.15.1:0/3779144598 conn(0x7f34d80ff4b0 msgr2=0x7f34d8108240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:36:46.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.347+0000 7f34de000640 1 -- 172.21.15.1:0/3779144598 shutdown_connections 2024-09-12T01:36:46.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.347+0000 7f34de000640 1 -- 172.21.15.1:0/3779144598 wait complete. 2024-09-12T01:36:46.363 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-12T01:36:46.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.522+0000 7f1c77e00640 1 -- 172.21.15.1:0/3575743050 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c78103830 msgr2=0x7f1c78103cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:46.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.522+0000 7f1c77e00640 1 --2- 172.21.15.1:0/3575743050 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c78103830 0x7f1c78103cb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1c64009900 tx=0x7f1c6402a410 comp rx=0 tx=0).stop 2024-09-12T01:36:46.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.522+0000 7f1c77e00640 1 -- 172.21.15.1:0/3575743050 shutdown_connections 2024-09-12T01:36:46.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.522+0000 7f1c77e00640 1 --2- 172.21.15.1:0/3575743050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c781041f0 0x7f1c7810c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:46.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.522+0000 7f1c77e00640 1 --2- 172.21.15.1:0/3575743050 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c78103830 0x7f1c78103cb0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:46.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.522+0000 7f1c77e00640 1 --2- 172.21.15.1:0/3575743050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c78105300 0x7f1c78105700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:46.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.522+0000 7f1c77e00640 1 -- 172.21.15.1:0/3575743050 >> 172.21.15.1:0/3575743050 conn(0x7f1c780ff380 msgr2=0x7f1c781017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:36:46.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.522+0000 7f1c77e00640 1 -- 172.21.15.1:0/3575743050 shutdown_connections 2024-09-12T01:36:46.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.522+0000 7f1c77e00640 1 -- 172.21.15.1:0/3575743050 wait complete. 2024-09-12T01:36:46.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.523+0000 7f1c77e00640 1 Processor -- start 2024-09-12T01:36:46.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.523+0000 7f1c77e00640 1 -- start start 2024-09-12T01:36:46.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.523+0000 7f1c77e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c78103830 0x7f1c7819a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:46.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.523+0000 7f1c77e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c781041f0 0x7f1c7819a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:46.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.524+0000 7f1c77e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c78105300 0x7f1c781a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:46.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.524+0000 7f1c77e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7810f9f0 con 0x7f1c78103830 2024-09-12T01:36:46.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.524+0000 7f1c77e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7810f870 con 0x7f1c781041f0 2024-09-12T01:36:46.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.524+0000 7f1c77e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7810fb70 con 0x7f1c78105300 2024-09-12T01:36:46.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.524+0000 7f1c76a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c78103830 0x7f1c7819a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:46.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.524+0000 7f1c76a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c78103830 0x7f1c7819a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40852/0 (socket says 172.21.15.1:40852) 2024-09-12T01:36:46.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.524+0000 7f1c76a00640 1 -- 172.21.15.1:0/3119768718 learned_addr learned my addr 172.21.15.1:0/3119768718 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:36:46.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.524+0000 7f1c76000640 1 --2- 172.21.15.1:0/3119768718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c781041f0 0x7f1c7819a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:46.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.524+0000 7f1c77400640 1 --2- 172.21.15.1:0/3119768718 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c78105300 0x7f1c781a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:46.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.524+0000 7f1c76a00640 1 -- 172.21.15.1:0/3119768718 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c78105300 msgr2=0x7f1c781a1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:46.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.524+0000 7f1c76a00640 1 --2- 172.21.15.1:0/3119768718 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c78105300 0x7f1c781a1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:46.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.524+0000 7f1c76a00640 1 -- 172.21.15.1:0/3119768718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c781041f0 msgr2=0x7f1c7819a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:46.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.524+0000 7f1c76a00640 1 --2- 172.21.15.1:0/3119768718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c781041f0 0x7f1c7819a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:46.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.524+0000 7f1c76a00640 1 -- 172.21.15.1:0/3119768718 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c781a1ca0 con 0x7f1c78103830 2024-09-12T01:36:46.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.524+0000 7f1c77400640 1 --2- 172.21.15.1:0/3119768718 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c78105300 0x7f1c781a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:36:46.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.524+0000 7f1c76a00640 1 --2- 172.21.15.1:0/3119768718 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c78103830 0x7f1c7819a080 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1c6800e9c0 tx=0x7f1c6800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:36:46.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.525+0000 7f1c6f400640 1 -- 172.21.15.1:0/3119768718 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6800cd80 con 0x7f1c78103830 2024-09-12T01:36:46.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.525+0000 7f1c6f400640 1 -- 172.21.15.1:0/3119768718 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c680035f0 con 0x7f1c78103830 2024-09-12T01:36:46.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.525+0000 7f1c77e00640 1 -- 172.21.15.1:0/3119768718 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c781a1f90 con 0x7f1c78103830 2024-09-12T01:36:46.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.525+0000 7f1c6f400640 1 -- 172.21.15.1:0/3119768718 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c68010690 con 0x7f1c78103830 2024-09-12T01:36:46.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.525+0000 7f1c77e00640 1 -- 172.21.15.1:0/3119768718 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c781a23f0 con 0x7f1c78103830 2024-09-12T01:36:46.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.526+0000 7f1c77e00640 1 -- 172.21.15.1:0/3119768718 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c78103cb0 con 0x7f1c78103830 2024-09-12T01:36:46.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.528+0000 7f1c6f400640 1 -- 172.21.15.1:0/3119768718 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1c680037a0 con 0x7f1c78103830 2024-09-12T01:36:46.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.529+0000 7f1c6f400640 1 --2- 172.21.15.1:0/3119768718 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1c5c0a2c00 0x7f1c5c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:36:46.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.529+0000 7f1c6f400640 1 -- 172.21.15.1:0/3119768718 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f1c680cf7c0 con 0x7f1c78103830 2024-09-12T01:36:46.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.529+0000 7f1c76000640 1 --2- 172.21.15.1:0/3119768718 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1c5c0a2c00 0x7f1c5c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:36:46.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.530+0000 7f1c76000640 1 --2- 172.21.15.1:0/3119768718 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1c5c0a2c00 0x7f1c5c0a50c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1c64031440 tx=0x7f1c640315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:36:46.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.533+0000 7f1c6f400640 1 -- 172.21.15.1:0/3119768718 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1c6809c330 con 0x7f1c78103830 2024-09-12T01:36:46.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:46 smithi001 ceph-mon[30387]: pgmap v613: 129 pgs: 129 active+clean; 110 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 417 op/s 2024-09-12T01:36:46.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:46 smithi001 ceph-mon[30387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T01:36:46.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:46 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/3615942135' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T01:36:46.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:46 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/4115488039' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T01:36:46.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:46 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/3779144598' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-12T01:36:46.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:46 smithi053 ceph-mon[35158]: pgmap v613: 129 pgs: 129 active+clean; 110 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 417 op/s 2024-09-12T01:36:46.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:46 smithi053 ceph-mon[35158]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T01:36:46.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:46 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/3615942135' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T01:36:46.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:46 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/4115488039' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T01:36:46.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:46 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/3779144598' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-12T01:36:46.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.769+0000 7f1c77e00640 1 -- 172.21.15.1:0/3119768718 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f1c7810afb0 con 0x7f1c78103830 2024-09-12T01:36:46.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.780+0000 7f1c6f400640 1 -- 172.21.15.1:0/3119768718 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+740362 (secure 0 0 0) 0x7f1c6809bc30 con 0x7f1c78103830 2024-09-12T01:36:46.791 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:36:46.791 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-12T01:36:46.791 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 20, 2024-09-12T01:36:46.791 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.791 INFO:teuthology.orchestra.run.smithi001.stdout: "active_gid": 14150, 2024-09-12T01:36:46.791 INFO:teuthology.orchestra.run.smithi001.stdout: "active_name": "x", 2024-09-12T01:36:46.791 INFO:teuthology.orchestra.run.smithi001.stdout: "active_addrs": { 2024-09-12T01:36:46.791 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-12T01:36:46.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:46.791 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-12T01:36:46.791 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6800", 2024-09-12T01:36:46.791 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1928662456 2024-09-12T01:36:46.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:46.792 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-12T01:36:46.792 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6801", 2024-09-12T01:36:46.792 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1928662456 2024-09-12T01:36:46.792 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:46.792 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-12T01:36:46.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.792 INFO:teuthology.orchestra.run.smithi001.stdout: "active_addr": "172.21.15.1:6801/1928662456", 2024-09-12T01:36:46.792 INFO:teuthology.orchestra.run.smithi001.stdout: "active_change": "2024-09-12T01:19:14.580745+0000", 2024-09-12T01:36:46.792 INFO:teuthology.orchestra.run.smithi001.stdout: "active_mgr_features": 4540701547738038271, 2024-09-12T01:36:46.792 INFO:teuthology.orchestra.run.smithi001.stdout: "available": true, 2024-09-12T01:36:46.792 INFO:teuthology.orchestra.run.smithi001.stdout: "standbys": [ 2024-09-12T01:36:46.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:46.793 INFO:teuthology.orchestra.run.smithi001.stdout: "gid": 24109, 2024-09-12T01:36:46.793 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "z", 2024-09-12T01:36:46.793 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_features": 4540701547738038271, 2024-09-12T01:36:46.793 INFO:teuthology.orchestra.run.smithi001.stdout: "available_modules": [ 2024-09-12T01:36:46.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:46.793 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "alerts", 2024-09-12T01:36:46.793 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:46.793 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:46.793 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:46.793 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-12T01:36:46.793 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-12T01:36:46.793 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:46.793 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.793 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.794 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-12T01:36:46.794 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.794 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.794 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.794 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T01:36:46.794 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.794 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.794 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.794 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:46.794 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:46.794 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.794 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.795 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.795 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:46.795 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.795 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.795 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:46.795 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:46.795 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:46.795 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:46.795 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:46.795 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:46.795 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:46.795 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:46.795 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.795 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.796 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.796 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.796 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:46.796 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:46.796 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.796 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.796 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.796 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:46.796 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.796 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.796 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.796 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.797 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.797 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.797 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.797 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:46.797 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:46.797 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.797 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.797 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.797 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:46.797 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.797 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.797 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:46.797 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:46.798 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:46.798 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:46.798 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:46.798 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:46.798 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:46.798 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:46.798 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.798 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.798 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.798 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.798 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:46.798 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:46.798 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.799 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.799 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.799 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:46.799 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.799 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.799 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.799 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.799 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.799 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.799 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.799 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_destination": { 2024-09-12T01:36:46.799 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_destination", 2024-09-12T01:36:46.799 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.800 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.800 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.800 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:46.800 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.800 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.800 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.800 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Email address to send alerts to", 2024-09-12T01:36:46.800 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.800 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.800 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.800 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_from_name": { 2024-09-12T01:36:46.800 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_from_name", 2024-09-12T01:36:46.800 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.801 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.801 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.801 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "Ceph", 2024-09-12T01:36:46.801 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.801 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.801 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.801 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Email From: name", 2024-09-12T01:36:46.801 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.801 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.801 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.801 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_host": { 2024-09-12T01:36:46.801 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_host", 2024-09-12T01:36:46.802 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.802 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.802 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.802 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:46.802 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.802 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.802 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.802 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP server", 2024-09-12T01:36:46.802 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.802 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.802 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.802 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_password": { 2024-09-12T01:36:46.803 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_password", 2024-09-12T01:36:46.803 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.803 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.803 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.803 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:46.803 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.803 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.803 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.803 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Password to authenticate with", 2024-09-12T01:36:46.803 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.803 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.803 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.803 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_port": { 2024-09-12T01:36:46.804 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_port", 2024-09-12T01:36:46.804 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:46.804 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.804 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.804 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "465", 2024-09-12T01:36:46.804 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.804 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.804 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.804 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP port", 2024-09-12T01:36:46.804 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.804 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.804 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.804 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_sender": { 2024-09-12T01:36:46.805 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_sender", 2024-09-12T01:36:46.805 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.805 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.805 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.805 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:46.805 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.805 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.805 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.805 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP envelope sender", 2024-09-12T01:36:46.805 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.805 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.805 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.806 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_ssl": { 2024-09-12T01:36:46.806 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_ssl", 2024-09-12T01:36:46.806 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.806 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.806 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.806 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:46.806 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.806 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.806 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.806 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T01:36:46.806 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.806 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.806 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.807 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_user": { 2024-09-12T01:36:46.807 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_user", 2024-09-12T01:36:46.807 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.807 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.807 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.807 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:46.807 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.807 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.807 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.807 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "User to authenticate as", 2024-09-12T01:36:46.807 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.807 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.807 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.808 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:46.808 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:46.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:46.808 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "balancer", 2024-09-12T01:36:46.808 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:46.808 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:46.808 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:46.808 INFO:teuthology.orchestra.run.smithi001.stdout: "active": { 2024-09-12T01:36:46.808 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "active", 2024-09-12T01:36:46.808 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.808 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.808 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.808 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:46.809 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.809 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.809 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.809 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T01:36:46.809 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.809 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.809 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.809 INFO:teuthology.orchestra.run.smithi001.stdout: "begin_time": { 2024-09-12T01:36:46.809 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "begin_time", 2024-09-12T01:36:46.809 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.809 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.809 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.809 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0000", 2024-09-12T01:36:46.810 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.810 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.810 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.810 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T01:36:46.810 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T01:36:46.810 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.810 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.810 INFO:teuthology.orchestra.run.smithi001.stdout: "begin_weekday": { 2024-09-12T01:36:46.810 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "begin_weekday", 2024-09-12T01:36:46.810 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-12T01:36:46.810 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.810 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.810 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-12T01:36:46.811 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0", 2024-09-12T01:36:46.811 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "6", 2024-09-12T01:36:46.811 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.811 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T01:36:46.811 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T01:36:46.811 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.811 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.811 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_max_iterations": { 2024-09-12T01:36:46.811 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_max_iterations", 2024-09-12T01:36:46.811 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-12T01:36:46.811 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.811 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.811 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "25", 2024-09-12T01:36:46.812 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-09-12T01:36:46.812 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "250", 2024-09-12T01:36:46.812 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.812 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T01:36:46.812 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.812 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.812 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.812 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_metrics": { 2024-09-12T01:36:46.812 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_metrics", 2024-09-12T01:36:46.812 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.812 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.812 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.812 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T01:36:46.813 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.813 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.813 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.813 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T01:36:46.813 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T01:36:46.813 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.813 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.813 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_step": { 2024-09-12T01:36:46.813 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_step", 2024-09-12T01:36:46.813 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-12T01:36:46.813 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.813 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.813 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0.5", 2024-09-12T01:36:46.814 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0.001", 2024-09-12T01:36:46.814 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "0.999", 2024-09-12T01:36:46.814 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.814 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "aggressiveness of optimization", 2024-09-12T01:36:46.814 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T01:36:46.814 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.814 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.814 INFO:teuthology.orchestra.run.smithi001.stdout: "end_time": { 2024-09-12T01:36:46.814 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "end_time", 2024-09-12T01:36:46.814 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.814 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.814 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.814 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "2359", 2024-09-12T01:36:46.815 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.815 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.815 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.815 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T01:36:46.815 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T01:36:46.815 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.815 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.815 INFO:teuthology.orchestra.run.smithi001.stdout: "end_weekday": { 2024-09-12T01:36:46.815 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "end_weekday", 2024-09-12T01:36:46.815 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-12T01:36:46.815 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.815 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.815 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-12T01:36:46.815 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0", 2024-09-12T01:36:46.816 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "6", 2024-09-12T01:36:46.816 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.816 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T01:36:46.816 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T01:36:46.816 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.816 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.816 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:46.816 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:46.816 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.816 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.816 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.816 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:46.816 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.817 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.817 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:46.817 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:46.817 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:46.817 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:46.817 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:46.817 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:46.817 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:46.817 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:46.817 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.817 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.817 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.817 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.818 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:46.818 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:46.818 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.818 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.818 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.818 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:46.818 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.818 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.818 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.818 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.818 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.818 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.818 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.819 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:46.819 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:46.819 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.819 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.819 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.819 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:46.819 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.819 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.819 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:46.819 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:46.819 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:46.819 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:46.819 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:46.820 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:46.820 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:46.820 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:46.820 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.820 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.820 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.820 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.820 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:46.820 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:46.820 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.820 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.820 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.820 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:46.821 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.821 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.821 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.821 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.821 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.821 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.821 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.821 INFO:teuthology.orchestra.run.smithi001.stdout: "min_score": { 2024-09-12T01:36:46.821 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "min_score", 2024-09-12T01:36:46.821 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-12T01:36:46.821 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.821 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.821 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-12T01:36:46.821 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.822 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.822 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.822 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T01:36:46.822 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.822 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.822 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.822 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": { 2024-09-12T01:36:46.822 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mode", 2024-09-12T01:36:46.822 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.822 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.822 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.822 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "upmap", 2024-09-12T01:36:46.822 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.823 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.823 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:46.823 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-compat", 2024-09-12T01:36:46.823 INFO:teuthology.orchestra.run.smithi001.stdout: "none", 2024-09-12T01:36:46.823 INFO:teuthology.orchestra.run.smithi001.stdout: "read", 2024-09-12T01:36:46.823 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap", 2024-09-12T01:36:46.823 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap-read" 2024-09-12T01:36:46.823 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:46.823 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Balancer mode", 2024-09-12T01:36:46.823 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.823 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.823 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.824 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_ids": { 2024-09-12T01:36:46.824 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pool_ids", 2024-09-12T01:36:46.824 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.824 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.824 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.824 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:46.824 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.824 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.824 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.824 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T01:36:46.824 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.824 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.825 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.825 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-12T01:36:46.825 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-12T01:36:46.825 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:46.825 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.825 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.825 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-12T01:36:46.825 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.825 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.825 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.825 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T01:36:46.825 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.825 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.826 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.826 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap_max_deviation": { 2024-09-12T01:36:46.826 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "upmap_max_deviation", 2024-09-12T01:36:46.826 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:46.826 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.826 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.826 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-09-12T01:36:46.826 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-09-12T01:36:46.826 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.826 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.826 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T01:36:46.826 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T01:36:46.826 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.827 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.827 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap_max_optimizations": { 2024-09-12T01:36:46.827 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "upmap_max_optimizations", 2024-09-12T01:36:46.827 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-12T01:36:46.827 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.827 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.827 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-12T01:36:46.827 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.827 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.827 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.827 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T01:36:46.827 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.827 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.828 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.828 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:46.828 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:46.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:46.828 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cephadm", 2024-09-12T01:36:46.828 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:46.828 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:46.828 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:46.828 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_down_multiplier": { 2024-09-12T01:36:46.828 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_down_multiplier", 2024-09-12T01:36:46.828 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-12T01:36:46.828 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.829 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.829 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3.0", 2024-09-12T01:36:46.829 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.829 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.829 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.829 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T01:36:46.829 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.829 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.829 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.829 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_refresh_rate": { 2024-09-12T01:36:46.829 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_refresh_rate", 2024-09-12T01:36:46.829 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:46.829 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.830 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.830 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "20", 2024-09-12T01:36:46.830 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.830 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.830 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.830 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T01:36:46.830 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.830 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.830 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.830 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_starting_port": { 2024-09-12T01:36:46.830 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_starting_port", 2024-09-12T01:36:46.830 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:46.830 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.831 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.831 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "4721", 2024-09-12T01:36:46.831 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.831 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.831 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.831 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T01:36:46.831 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.831 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.831 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.831 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_ptrace": { 2024-09-12T01:36:46.831 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_ptrace", 2024-09-12T01:36:46.831 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.831 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.832 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.832 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:46.832 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.832 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.832 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.832 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T01:36:46.832 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T01:36:46.832 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.832 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.832 INFO:teuthology.orchestra.run.smithi001.stdout: "autotune_interval": { 2024-09-12T01:36:46.832 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "autotune_interval", 2024-09-12T01:36:46.832 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:46.832 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.833 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.833 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-12T01:36:46.833 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.833 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.833 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.833 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T01:36:46.833 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.833 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.833 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.833 INFO:teuthology.orchestra.run.smithi001.stdout: "autotune_memory_target_ratio": { 2024-09-12T01:36:46.833 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T01:36:46.833 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-12T01:36:46.833 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.834 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.834 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0.7", 2024-09-12T01:36:46.834 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.834 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.834 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.834 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T01:36:46.834 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.834 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.834 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.834 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm_log_destination": { 2024-09-12T01:36:46.834 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cephadm_log_destination", 2024-09-12T01:36:46.834 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.834 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.835 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.835 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:46.835 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.835 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.835 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:46.835 INFO:teuthology.orchestra.run.smithi001.stdout: "file", 2024-09-12T01:36:46.835 INFO:teuthology.orchestra.run.smithi001.stdout: "file,syslog", 2024-09-12T01:36:46.835 INFO:teuthology.orchestra.run.smithi001.stdout: "syslog" 2024-09-12T01:36:46.835 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:46.835 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T01:36:46.835 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.835 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.835 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.835 INFO:teuthology.orchestra.run.smithi001.stdout: "cgroups_split": { 2024-09-12T01:36:46.836 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cgroups_split", 2024-09-12T01:36:46.836 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.836 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.836 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.836 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:46.836 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.836 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.836 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.836 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T01:36:46.836 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.836 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.836 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.836 INFO:teuthology.orchestra.run.smithi001.stdout: "config_checks_enabled": { 2024-09-12T01:36:46.837 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_checks_enabled", 2024-09-12T01:36:46.837 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.837 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.837 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.837 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:46.837 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.837 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.837 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.837 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T01:36:46.837 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.837 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.837 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.838 INFO:teuthology.orchestra.run.smithi001.stdout: "config_dashboard": { 2024-09-12T01:36:46.838 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_dashboard", 2024-09-12T01:36:46.838 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.838 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.838 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.838 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:46.838 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.838 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.838 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.838 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T01:36:46.838 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.838 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.838 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.839 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_alertmanager": { 2024-09-12T01:36:46.839 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_alertmanager", 2024-09-12T01:36:46.839 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.839 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.839 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.839 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T01:36:46.839 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.839 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.839 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.839 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-09-12T01:36:46.839 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.839 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.839 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.839 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_base": { 2024-09-12T01:36:46.840 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_base", 2024-09-12T01:36:46.840 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.840 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.840 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.840 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T01:36:46.840 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.840 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.840 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.840 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Container image name, without the tag", 2024-09-12T01:36:46.840 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.840 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.840 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.840 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_elasticsearch": { 2024-09-12T01:36:46.841 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_elasticsearch", 2024-09-12T01:36:46.841 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.841 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.841 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.841 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T01:36:46.841 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.841 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.841 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.841 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "elasticsearch container image", 2024-09-12T01:36:46.841 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.841 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.841 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.841 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_grafana": { 2024-09-12T01:36:46.842 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_grafana", 2024-09-12T01:36:46.842 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.842 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.845 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.845 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T01:36:46.845 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.845 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.845 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.845 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-09-12T01:36:46.845 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.846 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.846 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.846 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_haproxy": { 2024-09-12T01:36:46.846 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_haproxy", 2024-09-12T01:36:46.846 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.846 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.846 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.846 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T01:36:46.846 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.846 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.846 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.846 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "HAproxy container image", 2024-09-12T01:36:46.847 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.847 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.847 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.847 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_agent": { 2024-09-12T01:36:46.847 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_agent", 2024-09-12T01:36:46.847 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.847 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.847 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.847 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T01:36:46.847 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.847 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.847 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.847 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger agent container image", 2024-09-12T01:36:46.848 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.848 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.848 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.848 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_collector": { 2024-09-12T01:36:46.848 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_collector", 2024-09-12T01:36:46.848 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.848 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.848 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.848 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T01:36:46.848 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.848 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.848 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.849 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger collector container image", 2024-09-12T01:36:46.849 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.849 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.849 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.849 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_query": { 2024-09-12T01:36:46.849 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_query", 2024-09-12T01:36:46.849 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.849 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.849 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.849 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T01:36:46.849 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.849 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.849 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.850 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger query container image", 2024-09-12T01:36:46.850 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.850 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.850 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.850 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_keepalived": { 2024-09-12T01:36:46.850 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_keepalived", 2024-09-12T01:36:46.850 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.850 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.850 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.850 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T01:36:46.850 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.850 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.850 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.851 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Keepalived container image", 2024-09-12T01:36:46.851 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.851 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.851 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.851 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_loki": { 2024-09-12T01:36:46.851 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_loki", 2024-09-12T01:36:46.851 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.851 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.851 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.851 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T01:36:46.851 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.851 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.851 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.852 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Loki container image", 2024-09-12T01:36:46.852 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.852 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.852 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.852 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_node_exporter": { 2024-09-12T01:36:46.852 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_node_exporter", 2024-09-12T01:36:46.852 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.852 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.852 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.852 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T01:36:46.852 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.852 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.852 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.853 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-09-12T01:36:46.853 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.853 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.853 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.853 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_nvmeof": { 2024-09-12T01:36:46.853 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_nvmeof", 2024-09-12T01:36:46.853 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.853 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.853 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.853 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T01:36:46.853 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.853 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.853 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.854 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Nvme-of container image", 2024-09-12T01:36:46.854 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.854 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.854 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.854 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_prometheus": { 2024-09-12T01:36:46.854 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_prometheus", 2024-09-12T01:36:46.854 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.854 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.854 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.854 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T01:36:46.854 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.854 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.854 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.855 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-09-12T01:36:46.855 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.855 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.855 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.855 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_promtail": { 2024-09-12T01:36:46.855 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_promtail", 2024-09-12T01:36:46.855 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.855 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.855 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.855 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T01:36:46.855 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.855 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.855 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.855 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Promtail container image", 2024-09-12T01:36:46.856 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.856 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.856 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.856 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_samba": { 2024-09-12T01:36:46.856 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_samba", 2024-09-12T01:36:46.856 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.856 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.856 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.856 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T01:36:46.856 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.856 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.856 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.857 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Samba/SMB container image", 2024-09-12T01:36:46.857 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.857 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.857 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.857 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_snmp_gateway": { 2024-09-12T01:36:46.857 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_snmp_gateway", 2024-09-12T01:36:46.857 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.857 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.857 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.857 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T01:36:46.857 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.857 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.857 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.857 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SNMP Gateway container image", 2024-09-12T01:36:46.858 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.858 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.858 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.858 INFO:teuthology.orchestra.run.smithi001.stdout: "container_init": { 2024-09-12T01:36:46.858 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_init", 2024-09-12T01:36:46.858 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.858 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.858 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.858 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:46.858 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.858 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.858 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.858 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T01:36:46.859 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.859 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.859 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.859 INFO:teuthology.orchestra.run.smithi001.stdout: "daemon_cache_timeout": { 2024-09-12T01:36:46.859 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "daemon_cache_timeout", 2024-09-12T01:36:46.859 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:46.859 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.859 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.859 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-12T01:36:46.859 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.859 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.859 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.859 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T01:36:46.859 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.860 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.860 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.860 INFO:teuthology.orchestra.run.smithi001.stdout: "default_cephadm_command_timeout": { 2024-09-12T01:36:46.860 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T01:36:46.860 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:46.860 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.860 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.860 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "900", 2024-09-12T01:36:46.860 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.860 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.860 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.860 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T01:36:46.861 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.861 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.861 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.861 INFO:teuthology.orchestra.run.smithi001.stdout: "default_registry": { 2024-09-12T01:36:46.861 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "default_registry", 2024-09-12T01:36:46.861 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.861 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.861 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.861 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io", 2024-09-12T01:36:46.861 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.861 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.861 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.861 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T01:36:46.862 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.862 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.862 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.862 INFO:teuthology.orchestra.run.smithi001.stdout: "device_cache_timeout": { 2024-09-12T01:36:46.862 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_cache_timeout", 2024-09-12T01:36:46.862 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:46.862 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.862 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.862 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1800", 2024-09-12T01:36:46.862 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.862 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.862 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.863 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache device inventory", 2024-09-12T01:36:46.863 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.863 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.863 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.863 INFO:teuthology.orchestra.run.smithi001.stdout: "device_enhanced_scan": { 2024-09-12T01:36:46.863 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_enhanced_scan", 2024-09-12T01:36:46.863 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.863 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.863 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.863 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:46.863 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.863 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.863 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.863 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T01:36:46.864 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.864 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.864 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.864 INFO:teuthology.orchestra.run.smithi001.stdout: "facts_cache_timeout": { 2024-09-12T01:36:46.864 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "facts_cache_timeout", 2024-09-12T01:36:46.864 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:46.864 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.864 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.864 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-12T01:36:46.864 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.864 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.864 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.864 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache host facts data", 2024-09-12T01:36:46.865 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.865 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.865 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.865 INFO:teuthology.orchestra.run.smithi001.stdout: "grafana_dashboards_path": { 2024-09-12T01:36:46.865 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "grafana_dashboards_path", 2024-09-12T01:36:46.865 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.865 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.865 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.865 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T01:36:46.865 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.865 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.865 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.865 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T01:36:46.865 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.865 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.866 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.866 INFO:teuthology.orchestra.run.smithi001.stdout: "host_check_interval": { 2024-09-12T01:36:46.866 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "host_check_interval", 2024-09-12T01:36:46.866 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:46.866 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.866 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.866 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-12T01:36:46.866 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.866 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.866 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.866 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to perform a host check", 2024-09-12T01:36:46.866 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.866 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.867 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.867 INFO:teuthology.orchestra.run.smithi001.stdout: "hw_monitoring": { 2024-09-12T01:36:46.867 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "hw_monitoring", 2024-09-12T01:36:46.867 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.867 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.867 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.867 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:46.867 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.867 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.867 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.867 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T01:36:46.867 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.867 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.867 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.868 INFO:teuthology.orchestra.run.smithi001.stdout: "inventory_list_all": { 2024-09-12T01:36:46.868 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "inventory_list_all", 2024-09-12T01:36:46.868 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.868 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.868 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.868 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:46.868 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.868 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.868 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.868 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T01:36:46.868 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.868 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.868 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.869 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:46.869 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:46.869 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.869 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.869 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.869 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:46.869 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.869 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.869 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:46.869 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:46.869 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:46.869 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:46.869 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:46.870 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:46.870 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:46.870 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:46.870 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.870 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.870 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.870 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.870 INFO:teuthology.orchestra.run.smithi001.stdout: "log_refresh_metadata": { 2024-09-12T01:36:46.870 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_refresh_metadata", 2024-09-12T01:36:46.870 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.870 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.870 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.871 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:46.871 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.871 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.871 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.871 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T01:36:46.871 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.871 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.871 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.871 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:46.871 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:46.871 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.871 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.871 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.872 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:46.872 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.872 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.872 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.872 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T01:36:46.872 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.872 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.872 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.872 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:46.872 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:46.872 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.872 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.873 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.873 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:46.873 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.873 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.873 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:46.873 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:46.873 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:46.873 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:46.873 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:46.873 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:46.873 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:46.873 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:46.873 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.874 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.874 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.874 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.874 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:46.874 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:46.874 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.874 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.874 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.874 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:46.874 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.874 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.874 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.875 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.875 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.875 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.875 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.875 INFO:teuthology.orchestra.run.smithi001.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T01:36:46.875 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T01:36:46.875 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.875 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.875 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.875 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:46.875 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.875 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.875 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.876 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T01:36:46.876 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.876 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.876 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.876 INFO:teuthology.orchestra.run.smithi001.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T01:36:46.876 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T01:36:46.876 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.876 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.876 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.876 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "*", 2024-09-12T01:36:46.876 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.876 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.876 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.877 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T01:36:46.877 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.877 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.877 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.877 INFO:teuthology.orchestra.run.smithi001.stdout: "max_count_per_host": { 2024-09-12T01:36:46.877 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_count_per_host", 2024-09-12T01:36:46.877 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:46.877 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.877 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.877 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-12T01:36:46.877 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.877 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.877 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.877 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "max number of daemons per service per host", 2024-09-12T01:36:46.878 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.878 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.878 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.878 INFO:teuthology.orchestra.run.smithi001.stdout: "max_osd_draining_count": { 2024-09-12T01:36:46.878 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_osd_draining_count", 2024-09-12T01:36:46.878 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:46.878 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.878 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.878 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-12T01:36:46.878 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.878 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.878 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.878 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T01:36:46.878 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.879 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.879 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.879 INFO:teuthology.orchestra.run.smithi001.stdout: "migration_current": { 2024-09-12T01:36:46.879 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "migration_current", 2024-09-12T01:36:46.879 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:46.879 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.879 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.879 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:46.879 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.879 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.879 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.879 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "internal - do not modify", 2024-09-12T01:36:46.879 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.879 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.880 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.880 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": { 2024-09-12T01:36:46.880 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mode", 2024-09-12T01:36:46.880 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.880 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.880 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.880 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "root", 2024-09-12T01:36:46.880 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.880 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.880 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:46.880 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm-package", 2024-09-12T01:36:46.880 INFO:teuthology.orchestra.run.smithi001.stdout: "root" 2024-09-12T01:36:46.880 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:46.880 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T01:36:46.881 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.881 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.881 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.881 INFO:teuthology.orchestra.run.smithi001.stdout: "oob_default_addr": { 2024-09-12T01:36:46.881 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "oob_default_addr", 2024-09-12T01:36:46.881 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.881 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.881 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.881 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "169.254.1.1", 2024-09-12T01:36:46.881 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.881 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.881 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.881 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T01:36:46.882 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.882 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.882 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.882 INFO:teuthology.orchestra.run.smithi001.stdout: "prometheus_alerts_path": { 2024-09-12T01:36:46.882 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus_alerts_path", 2024-09-12T01:36:46.882 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.882 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.882 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.882 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T01:36:46.882 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.882 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.882 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.882 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T01:36:46.882 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.883 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.883 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.883 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_insecure": { 2024-09-12T01:36:46.883 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_insecure", 2024-09-12T01:36:46.883 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.883 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.883 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.883 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:46.883 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.883 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.883 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.883 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T01:36:46.884 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.884 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.884 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.884 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_password": { 2024-09-12T01:36:46.884 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_password", 2024-09-12T01:36:46.884 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.884 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.884 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.884 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:46.884 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.884 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.884 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.884 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T01:36:46.885 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.885 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.885 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.885 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_url": { 2024-09-12T01:36:46.885 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_url", 2024-09-12T01:36:46.885 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.885 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.885 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.885 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:46.885 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.885 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.885 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.885 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T01:36:46.886 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.886 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.886 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.886 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_username": { 2024-09-12T01:36:46.886 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_username", 2024-09-12T01:36:46.886 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.886 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.886 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.886 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:46.886 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.886 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.886 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.886 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T01:36:46.887 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.887 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.887 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.887 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.887 INFO:teuthology.orchestra.run.smithi001.stdout: "secure_monitoring_stack": { 2024-09-12T01:36:46.887 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "secure_monitoring_stack", 2024-09-12T01:36:46.887 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.887 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.887 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.887 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:46.887 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.887 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.887 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.887 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T01:36:46.888 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.888 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.888 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.888 INFO:teuthology.orchestra.run.smithi001.stdout: "service_discovery_port": { 2024-09-12T01:36:46.888 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "service_discovery_port", 2024-09-12T01:36:46.888 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:46.888 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.888 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.888 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8765", 2024-09-12T01:36:46.888 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.888 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.888 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.888 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "cephadm service discovery port", 2024-09-12T01:36:46.889 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.889 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.889 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.889 INFO:teuthology.orchestra.run.smithi001.stdout: "ssh_config_file": { 2024-09-12T01:36:46.889 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssh_config_file", 2024-09-12T01:36:46.889 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.889 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.889 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.889 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:46.889 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.889 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.889 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.890 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T01:36:46.890 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.890 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.890 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.890 INFO:teuthology.orchestra.run.smithi001.stdout: "use_agent": { 2024-09-12T01:36:46.890 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "use_agent", 2024-09-12T01:36:46.890 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.890 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.890 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.890 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:46.890 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.890 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.890 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.891 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T01:36:46.891 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.891 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.891 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.891 INFO:teuthology.orchestra.run.smithi001.stdout: "use_repo_digest": { 2024-09-12T01:36:46.891 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "use_repo_digest", 2024-09-12T01:36:46.891 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.891 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.891 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.891 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:46.891 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.891 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.891 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.891 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T01:36:46.892 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.892 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.892 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.892 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_failed_host_check": { 2024-09-12T01:36:46.892 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_failed_host_check", 2024-09-12T01:36:46.892 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.892 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.892 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.892 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:46.892 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.892 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.892 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.892 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T01:36:46.893 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.893 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.893 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.893 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_stray_daemons": { 2024-09-12T01:36:46.893 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_stray_daemons", 2024-09-12T01:36:46.893 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.893 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.893 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.893 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:46.893 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.893 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.893 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.893 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T01:36:46.894 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.894 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.894 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.894 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_stray_hosts": { 2024-09-12T01:36:46.894 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_stray_hosts", 2024-09-12T01:36:46.894 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.894 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.894 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.894 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:46.894 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.894 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.894 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.894 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T01:36:46.894 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.895 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.895 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.895 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:46.895 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:46.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:46.895 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crash", 2024-09-12T01:36:46.895 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:46.895 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:46.895 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:46.895 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:46.895 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:46.895 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.896 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.896 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.896 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:46.896 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.896 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.896 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:46.896 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:46.896 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:46.896 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:46.896 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:46.896 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:46.896 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:46.896 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:46.896 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.897 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.897 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.897 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.897 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:46.897 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:46.897 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.897 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.897 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.897 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:46.897 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.897 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.897 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.897 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.897 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.898 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.898 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.898 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:46.898 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:46.898 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.898 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.898 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.898 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:46.898 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.898 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.898 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:46.898 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:46.898 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:46.898 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:46.899 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:46.899 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:46.899 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:46.899 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:46.899 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.899 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.899 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.899 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.899 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:46.899 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:46.899 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.899 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.899 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.899 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:46.899 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.900 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.900 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.900 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.900 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.900 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.900 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.900 INFO:teuthology.orchestra.run.smithi001.stdout: "retain_interval": { 2024-09-12T01:36:46.900 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "retain_interval", 2024-09-12T01:36:46.900 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:46.900 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.900 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.900 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "31536000", 2024-09-12T01:36:46.901 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.901 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.901 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.901 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T01:36:46.901 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.901 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.901 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.901 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_recent_interval": { 2024-09-12T01:36:46.901 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_recent_interval", 2024-09-12T01:36:46.901 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:46.901 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.901 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.901 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1209600", 2024-09-12T01:36:46.901 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.902 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.902 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.902 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T01:36:46.902 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.902 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.902 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.902 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:46.902 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:46.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:46.902 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "dashboard", 2024-09-12T01:36:46.902 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:46.902 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:46.902 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:46.903 INFO:teuthology.orchestra.run.smithi001.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T01:36:46.903 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T01:36:46.903 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:46.903 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.903 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.903 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-12T01:36:46.903 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.903 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.903 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.903 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.903 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.903 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.903 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.904 INFO:teuthology.orchestra.run.smithi001.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T01:36:46.904 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T01:36:46.904 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.904 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.904 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.904 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:46.904 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.904 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.904 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.904 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.904 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.904 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.904 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.905 INFO:teuthology.orchestra.run.smithi001.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T01:36:46.905 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T01:36:46.905 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.905 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.905 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.905 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:46.905 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.905 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.905 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.905 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.905 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.905 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.905 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.906 INFO:teuthology.orchestra.run.smithi001.stdout: "AUDIT_API_ENABLED": { 2024-09-12T01:36:46.906 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T01:36:46.906 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.906 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.906 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.906 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:46.906 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.906 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.906 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.906 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.906 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.906 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.906 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.907 INFO:teuthology.orchestra.run.smithi001.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T01:36:46.907 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T01:36:46.907 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.907 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.907 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.907 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:46.907 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.907 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.907 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.907 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.907 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.907 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.907 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.908 INFO:teuthology.orchestra.run.smithi001.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T01:36:46.908 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T01:36:46.908 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.908 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.908 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.908 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:46.908 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.908 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.908 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.908 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.908 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.909 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.909 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.909 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T01:36:46.909 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T01:36:46.909 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.909 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.909 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.909 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:46.909 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.909 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.909 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.909 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.910 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.910 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.910 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.910 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T01:36:46.910 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T01:36:46.910 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.910 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.910 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.910 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:46.910 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.910 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.911 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.911 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.911 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.911 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.911 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.911 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T01:36:46.911 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T01:36:46.911 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.911 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.911 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.911 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:46.911 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.911 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.912 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.912 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.912 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.912 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.912 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.912 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T01:36:46.912 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T01:36:46.912 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.912 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.912 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.912 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:46.912 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.913 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.913 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.913 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.913 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.913 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.913 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.913 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T01:36:46.913 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T01:36:46.913 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.913 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.913 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.913 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:46.914 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.914 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.914 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.914 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.914 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.914 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.914 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.914 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T01:36:46.914 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T01:36:46.914 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.914 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.914 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.914 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:46.915 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.915 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.915 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.915 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.915 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.915 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.915 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.915 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T01:36:46.915 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T01:36:46.915 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.915 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.915 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.915 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:46.916 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.916 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.916 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.916 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.916 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.916 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.916 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.916 INFO:teuthology.orchestra.run.smithi001.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T01:36:46.916 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T01:36:46.916 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.916 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.916 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.917 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:46.917 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.917 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.917 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.917 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.917 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.917 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.917 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.917 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T01:36:46.917 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T01:36:46.917 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.917 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.918 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.918 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-09-12T01:36:46.918 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.918 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.918 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.918 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.918 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.918 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.918 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.918 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T01:36:46.918 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T01:36:46.918 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.918 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.918 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.919 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:46.919 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.919 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.919 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.919 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.919 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.919 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.919 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.919 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_URL": { 2024-09-12T01:36:46.919 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_URL", 2024-09-12T01:36:46.919 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.919 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.919 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.920 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:46.920 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.920 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.920 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.920 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.920 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.920 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.920 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.920 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T01:36:46.920 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T01:36:46.920 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.920 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.920 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.921 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-09-12T01:36:46.921 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.921 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.921 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.921 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.921 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.921 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.921 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.921 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T01:36:46.921 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T01:36:46.921 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.921 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.921 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.922 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:46.922 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.922 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.922 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.922 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.922 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.922 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.922 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.922 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T01:36:46.922 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T01:36:46.922 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.923 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.923 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.923 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:46.923 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.923 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.923 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.923 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.923 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.923 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.923 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.923 INFO:teuthology.orchestra.run.smithi001.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T01:36:46.923 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T01:36:46.923 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.924 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.924 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.924 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:46.924 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.924 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.924 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.924 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.924 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.924 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.924 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.924 INFO:teuthology.orchestra.run.smithi001.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T01:36:46.924 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T01:36:46.924 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.925 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.925 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.925 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:46.925 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.925 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.925 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.925 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.925 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.925 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.925 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.925 INFO:teuthology.orchestra.run.smithi001.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T01:36:46.925 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T01:36:46.926 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.926 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.926 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.926 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:46.926 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.926 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.926 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.926 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.926 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.926 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.926 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.926 INFO:teuthology.orchestra.run.smithi001.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T01:36:46.926 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T01:36:46.927 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.927 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.927 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.927 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:46.927 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.927 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.927 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.927 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.927 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.927 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.927 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.927 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T01:36:46.928 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T01:36:46.928 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.928 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.928 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.928 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:46.928 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.928 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.928 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.928 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.928 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.928 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.928 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.928 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T01:36:46.929 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T01:36:46.929 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.929 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.929 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.929 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:46.929 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.929 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.929 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.929 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.929 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.929 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.929 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.929 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T01:36:46.929 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T01:36:46.930 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.930 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.930 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.930 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:46.930 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.930 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.930 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.930 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.930 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.930 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.930 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.930 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T01:36:46.930 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T01:36:46.931 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.931 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.931 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.931 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:46.931 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.931 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.931 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.931 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.931 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.931 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.931 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.931 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T01:36:46.931 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T01:36:46.931 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.932 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.932 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.932 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:46.932 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.932 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.932 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.932 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.932 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.932 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.932 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.932 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T01:36:46.932 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T01:36:46.932 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.933 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.933 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.933 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:46.933 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.933 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.933 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.933 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.933 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.933 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.933 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.933 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T01:36:46.933 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T01:36:46.933 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.933 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.934 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.934 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:46.934 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.934 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.934 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.934 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.934 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.934 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.934 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.934 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T01:36:46.934 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T01:36:46.934 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.934 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.934 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.935 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:46.935 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.935 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.935 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.935 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.935 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.935 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.935 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.935 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T01:36:46.935 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T01:36:46.935 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.935 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.936 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.936 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T01:36:46.936 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.936 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.936 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.936 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.936 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.936 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.936 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.936 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T01:36:46.936 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T01:36:46.936 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:46.936 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.937 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.937 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-12T01:36:46.937 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.937 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.937 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.937 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.937 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.937 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.937 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.937 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T01:36:46.937 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T01:36:46.937 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:46.937 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.938 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.938 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8", 2024-09-12T01:36:46.938 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.938 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.938 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.938 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.938 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.938 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.938 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.938 INFO:teuthology.orchestra.run.smithi001.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T01:36:46.938 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T01:36:46.938 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:46.938 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.939 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.939 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "45", 2024-09-12T01:36:46.939 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.939 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.939 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.939 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.939 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.939 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.939 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.939 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T01:36:46.939 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T01:36:46.939 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.940 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.940 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.940 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:46.940 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.940 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.940 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.940 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.940 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.940 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.940 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.940 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T01:36:46.940 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T01:36:46.940 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.941 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.941 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.941 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-09-12T01:36:46.941 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.941 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.941 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.941 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.941 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.941 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.941 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.941 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T01:36:46.941 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T01:36:46.942 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.942 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.942 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.942 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:46.942 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.942 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.942 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.942 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.942 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.942 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.942 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.942 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T01:36:46.943 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T01:36:46.943 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.943 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.943 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.943 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:46.943 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.943 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.943 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.943 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.943 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.943 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.943 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.943 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T01:36:46.944 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T01:36:46.944 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:46.944 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.944 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.944 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-12T01:36:46.944 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.944 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.944 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.944 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.944 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.944 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.944 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.944 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T01:36:46.945 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T01:36:46.945 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:46.945 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.945 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.945 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-12T01:36:46.945 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.945 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.945 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.945 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.945 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.945 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.945 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.945 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T01:36:46.945 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T01:36:46.946 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:46.946 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.946 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.946 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-09-12T01:36:46.946 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.946 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.946 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.946 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.946 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.946 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.946 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.946 INFO:teuthology.orchestra.run.smithi001.stdout: "cross_origin_url": { 2024-09-12T01:36:46.946 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cross_origin_url", 2024-09-12T01:36:46.947 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.947 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.947 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.947 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:46.947 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.947 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.947 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.947 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.947 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.947 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.947 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.947 INFO:teuthology.orchestra.run.smithi001.stdout: "crt_file": { 2024-09-12T01:36:46.947 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crt_file", 2024-09-12T01:36:46.947 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.948 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.948 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.948 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:46.948 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.948 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.948 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.948 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.948 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.948 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.948 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.948 INFO:teuthology.orchestra.run.smithi001.stdout: "debug": { 2024-09-12T01:36:46.948 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "debug", 2024-09-12T01:36:46.949 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.949 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.949 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.949 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:46.949 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.949 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.949 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.949 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable/disable debug options", 2024-09-12T01:36:46.949 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.949 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.949 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.949 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.949 INFO:teuthology.orchestra.run.smithi001.stdout: "jwt_token_ttl": { 2024-09-12T01:36:46.949 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "jwt_token_ttl", 2024-09-12T01:36:46.950 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:46.950 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.950 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.950 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "28800", 2024-09-12T01:36:46.950 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.950 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.950 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.950 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.950 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.950 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.950 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.950 INFO:teuthology.orchestra.run.smithi001.stdout: "key_file": { 2024-09-12T01:36:46.950 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "key_file", 2024-09-12T01:36:46.950 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.951 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.951 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.951 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:46.951 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.951 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.951 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.951 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.951 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.951 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.951 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.951 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:46.951 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:46.951 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.952 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.952 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.952 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:46.952 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.952 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.952 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:46.952 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:46.952 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:46.952 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:46.952 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:46.952 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:46.952 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:46.952 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:46.953 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.953 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.953 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.953 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.953 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:46.953 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:46.953 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.953 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.953 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.953 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:46.953 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.953 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.953 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.953 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.954 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.954 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.954 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.954 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:46.954 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:46.954 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.954 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.954 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.954 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:46.954 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.954 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.954 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:46.954 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:46.955 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:46.955 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:46.955 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:46.955 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:46.955 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:46.955 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:46.955 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.955 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.955 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.955 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.955 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:46.955 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:46.955 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.956 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.956 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.956 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:46.956 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.956 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.956 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.956 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.956 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.956 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.956 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.956 INFO:teuthology.orchestra.run.smithi001.stdout: "motd": { 2024-09-12T01:36:46.956 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "motd", 2024-09-12T01:36:46.957 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.957 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.957 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.957 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:46.957 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.957 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.957 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.957 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "The message of the day", 2024-09-12T01:36:46.957 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.957 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.957 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.957 INFO:teuthology.orchestra.run.smithi001.stdout: "redirect_resolve_ip_addr": { 2024-09-12T01:36:46.957 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T01:36:46.957 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.958 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.958 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.958 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:46.958 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.958 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.958 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.958 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.958 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.958 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.958 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.958 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-09-12T01:36:46.958 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-09-12T01:36:46.958 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.959 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.959 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.959 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "::", 2024-09-12T01:36:46.959 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.959 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.959 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.959 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.959 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.959 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.959 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.959 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-09-12T01:36:46.959 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-09-12T01:36:46.959 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:46.959 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.960 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.960 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8080", 2024-09-12T01:36:46.960 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.960 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.960 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.960 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.960 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.960 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.960 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.960 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl": { 2024-09-12T01:36:46.960 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl", 2024-09-12T01:36:46.960 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.961 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.961 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.961 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:46.961 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.961 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.961 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.961 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.961 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.961 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.961 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.961 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl_server_port": { 2024-09-12T01:36:46.961 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl_server_port", 2024-09-12T01:36:46.961 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:46.962 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.962 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.962 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8443", 2024-09-12T01:36:46.962 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.962 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.962 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.962 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.962 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.962 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.962 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.962 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_behaviour": { 2024-09-12T01:36:46.962 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_behaviour", 2024-09-12T01:36:46.962 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.963 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.963 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.963 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "redirect", 2024-09-12T01:36:46.963 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.963 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.963 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:46.963 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:46.963 INFO:teuthology.orchestra.run.smithi001.stdout: "redirect" 2024-09-12T01:36:46.963 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:46.963 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.963 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.963 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.963 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.964 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_error_status_code": { 2024-09-12T01:36:46.964 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_error_status_code", 2024-09-12T01:36:46.964 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:46.964 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.964 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.964 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "500", 2024-09-12T01:36:46.964 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "400", 2024-09-12T01:36:46.964 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "599", 2024-09-12T01:36:46.964 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.964 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.964 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.964 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.965 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.965 INFO:teuthology.orchestra.run.smithi001.stdout: "url_prefix": { 2024-09-12T01:36:46.965 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "url_prefix", 2024-09-12T01:36:46.965 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.965 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.965 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.965 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:46.965 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.965 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.965 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.965 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.965 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.965 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.966 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.966 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:46.966 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:46.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:46.966 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "devicehealth", 2024-09-12T01:36:46.966 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:46.966 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:46.966 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:46.966 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_monitoring": { 2024-09-12T01:36:46.966 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enable_monitoring", 2024-09-12T01:36:46.966 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.966 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.966 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.967 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:46.967 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.967 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.967 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.967 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "monitor device health metrics", 2024-09-12T01:36:46.967 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.967 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.967 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.967 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:46.967 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:46.967 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.967 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.967 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.967 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:46.968 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.968 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.968 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:46.968 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:46.968 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:46.968 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:46.968 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:46.968 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:46.968 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:46.968 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:46.968 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.968 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.968 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.968 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.969 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:46.969 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:46.969 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.969 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.969 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.969 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:46.969 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.969 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.969 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.969 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.969 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.969 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.970 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.970 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:46.970 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:46.970 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.970 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.970 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.970 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:46.970 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.970 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.970 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:46.970 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:46.970 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:46.970 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:46.971 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:46.971 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:46.971 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:46.971 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:46.971 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.971 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.971 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.971 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.971 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:46.971 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:46.971 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.971 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.971 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.971 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:46.972 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.972 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.972 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.972 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.972 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.972 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.972 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.972 INFO:teuthology.orchestra.run.smithi001.stdout: "mark_out_threshold": { 2024-09-12T01:36:46.972 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mark_out_threshold", 2024-09-12T01:36:46.972 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:46.972 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.972 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.972 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "2419200", 2024-09-12T01:36:46.973 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.973 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.973 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.973 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T01:36:46.973 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.973 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.973 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.973 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": { 2024-09-12T01:36:46.973 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pool_name", 2024-09-12T01:36:46.973 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.973 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.973 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.973 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "device_health_metrics", 2024-09-12T01:36:46.974 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.974 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.974 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.974 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T01:36:46.974 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.974 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.974 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.974 INFO:teuthology.orchestra.run.smithi001.stdout: "retention_period": { 2024-09-12T01:36:46.974 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "retention_period", 2024-09-12T01:36:46.974 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:46.974 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.974 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.974 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15552000", 2024-09-12T01:36:46.974 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.975 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.975 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.975 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long to retain device health metrics", 2024-09-12T01:36:46.975 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.975 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.975 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.975 INFO:teuthology.orchestra.run.smithi001.stdout: "scrape_frequency": { 2024-09-12T01:36:46.975 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "scrape_frequency", 2024-09-12T01:36:46.975 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:46.975 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.975 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.975 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "86400", 2024-09-12T01:36:46.975 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.976 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.976 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.976 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T01:36:46.976 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.976 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.976 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.976 INFO:teuthology.orchestra.run.smithi001.stdout: "self_heal": { 2024-09-12T01:36:46.976 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "self_heal", 2024-09-12T01:36:46.976 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.976 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.976 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.976 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:46.976 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.977 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.977 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.977 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T01:36:46.977 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.980 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.981 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.981 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-12T01:36:46.981 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-12T01:36:46.981 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:46.981 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.981 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.981 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-12T01:36:46.981 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.981 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.981 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.981 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T01:36:46.981 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.982 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.982 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.982 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_threshold": { 2024-09-12T01:36:46.982 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_threshold", 2024-09-12T01:36:46.982 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:46.982 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.982 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.982 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "7257600", 2024-09-12T01:36:46.982 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.982 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.982 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.982 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T01:36:46.982 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.982 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.983 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.983 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:46.983 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:46.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:46.983 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "diskprediction_local", 2024-09-12T01:36:46.983 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:46.983 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:46.983 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:46.983 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:46.983 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:46.983 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.984 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.984 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.984 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:46.984 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.984 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.984 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:46.984 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:46.984 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:46.984 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:46.984 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:46.984 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:46.984 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:46.984 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:46.984 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.985 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.985 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.985 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.985 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:46.985 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:46.985 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.985 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.985 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.985 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:46.985 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.985 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.985 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.986 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.986 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.986 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.986 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.986 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:46.986 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:46.986 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.986 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.986 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.986 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:46.986 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.986 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.987 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:46.987 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:46.987 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:46.987 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:46.987 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:46.987 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:46.987 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:46.987 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:46.987 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.987 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.987 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.987 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.988 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:46.988 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:46.988 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.988 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.988 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.988 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:46.988 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.988 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.988 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.988 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.988 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.988 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.988 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.989 INFO:teuthology.orchestra.run.smithi001.stdout: "predict_interval": { 2024-09-12T01:36:46.989 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "predict_interval", 2024-09-12T01:36:46.989 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.989 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.989 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.989 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "86400", 2024-09-12T01:36:46.989 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.989 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.989 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.989 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.989 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.989 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.989 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.990 INFO:teuthology.orchestra.run.smithi001.stdout: "predictor_model": { 2024-09-12T01:36:46.990 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "predictor_model", 2024-09-12T01:36:46.990 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.990 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.990 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.990 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "prophetstor", 2024-09-12T01:36:46.990 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.990 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.990 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.990 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.990 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.990 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.990 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.991 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-12T01:36:46.991 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-12T01:36:46.991 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.991 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.991 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.991 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-12T01:36:46.991 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.991 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.991 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.991 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.991 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.991 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.991 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.991 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:46.992 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:46.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:46.992 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "influx", 2024-09-12T01:36:46.992 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": false, 2024-09-12T01:36:46.992 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "influxdb python module not found", 2024-09-12T01:36:46.992 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:46.992 INFO:teuthology.orchestra.run.smithi001.stdout: "batch_size": { 2024-09-12T01:36:46.992 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "batch_size", 2024-09-12T01:36:46.992 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:46.992 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.992 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.992 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5000", 2024-09-12T01:36:46.992 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.992 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.993 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.993 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T01:36:46.993 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.993 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.993 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.993 INFO:teuthology.orchestra.run.smithi001.stdout: "database": { 2024-09-12T01:36:46.993 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "database", 2024-09-12T01:36:46.993 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.993 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.993 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.993 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "ceph", 2024-09-12T01:36:46.993 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.994 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.994 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.994 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T01:36:46.994 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.994 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.994 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.994 INFO:teuthology.orchestra.run.smithi001.stdout: "hostname": { 2024-09-12T01:36:46.994 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "hostname", 2024-09-12T01:36:46.994 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.994 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.994 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.994 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:46.995 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.995 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.995 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.995 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB server hostname", 2024-09-12T01:36:46.995 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.995 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.995 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.995 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-12T01:36:46.995 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-12T01:36:46.995 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:46.995 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.995 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:46.995 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "30", 2024-09-12T01:36:46.995 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "5", 2024-09-12T01:36:46.996 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.996 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.996 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T01:36:46.996 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.996 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.996 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.996 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:46.996 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:46.996 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.996 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.996 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.996 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:46.997 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.997 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.997 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:46.997 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:46.997 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:46.997 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:46.997 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:46.997 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:46.997 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:46.997 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:46.997 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.997 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.997 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.997 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.998 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:46.998 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:46.998 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:46.998 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.998 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.998 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:46.998 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.998 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.998 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:46.998 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:46.998 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:46.998 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:46.998 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:46.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:46.999 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:46.999 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:46.999 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:46.999 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:46.999 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:46.999 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:46.999 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:46.999 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:46.999 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:46.999 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:46.999 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:46.999 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:46.999 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:46.999 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.000 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.000 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.000 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.000 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.000 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.000 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.000 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.000 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.000 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.000 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.000 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.000 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.000 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.001 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.001 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.001 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.001 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.001 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.001 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.001 INFO:teuthology.orchestra.run.smithi001.stdout: "password": { 2024-09-12T01:36:47.001 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "password", 2024-09-12T01:36:47.001 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.001 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.001 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.001 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.001 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.002 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.002 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.002 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "password of InfluxDB server user", 2024-09-12T01:36:47.002 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.002 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.002 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.002 INFO:teuthology.orchestra.run.smithi001.stdout: "port": { 2024-09-12T01:36:47.002 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "port", 2024-09-12T01:36:47.002 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.002 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.002 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.002 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8086", 2024-09-12T01:36:47.002 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.003 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.003 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.003 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB server port", 2024-09-12T01:36:47.003 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.003 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.003 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.003 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl": { 2024-09-12T01:36:47.003 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl", 2024-09-12T01:36:47.003 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.003 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.003 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.003 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "false", 2024-09-12T01:36:47.003 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.004 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.004 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.004 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T01:36:47.004 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.004 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.004 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.004 INFO:teuthology.orchestra.run.smithi001.stdout: "threads": { 2024-09-12T01:36:47.004 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "threads", 2024-09-12T01:36:47.004 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.004 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.004 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.004 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-09-12T01:36:47.005 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-09-12T01:36:47.005 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "32", 2024-09-12T01:36:47.005 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.005 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T01:36:47.005 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.005 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.005 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.005 INFO:teuthology.orchestra.run.smithi001.stdout: "username": { 2024-09-12T01:36:47.005 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "username", 2024-09-12T01:36:47.005 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.005 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.005 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.005 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.006 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.006 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.006 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.006 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "username of InfluxDB server user", 2024-09-12T01:36:47.006 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.006 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.006 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.006 INFO:teuthology.orchestra.run.smithi001.stdout: "verify_ssl": { 2024-09-12T01:36:47.006 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "verify_ssl", 2024-09-12T01:36:47.006 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.006 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.006 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.007 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "true", 2024-09-12T01:36:47.007 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.007 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.007 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.007 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T01:36:47.007 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.007 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.007 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.007 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.007 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.007 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "insights", 2024-09-12T01:36:47.007 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.008 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.008 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.008 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.008 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.008 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.008 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.008 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.008 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.008 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.008 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.008 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.008 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.008 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.008 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.008 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.009 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.009 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.009 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.009 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.009 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.009 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.009 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.009 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.009 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.009 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.009 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.009 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.009 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.010 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.010 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.010 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.010 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.010 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.010 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.010 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.010 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.010 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.010 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.010 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.010 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.010 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.011 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.011 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.011 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.011 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.011 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.011 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.011 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.011 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.011 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.011 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.011 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.011 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.011 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.011 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.012 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.012 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.012 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.012 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.012 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.012 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.012 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.012 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.012 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.012 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.012 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.012 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.012 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.013 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.013 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.013 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "iostat", 2024-09-12T01:36:47.013 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.013 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.013 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.013 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.013 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.013 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.013 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.013 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.013 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.014 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.014 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.014 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.014 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.014 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.014 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.014 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.014 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.014 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.014 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.014 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.014 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.014 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.014 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.015 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.015 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.015 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.015 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.015 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.015 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.015 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.015 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.015 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.015 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.015 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.015 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.015 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.016 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.016 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.016 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.016 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.016 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.016 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.016 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.016 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.016 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.016 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.016 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.016 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.016 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.016 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.017 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.017 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.017 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.017 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.017 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.017 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.017 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.017 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.017 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.017 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.017 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.017 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.018 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.018 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.018 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.018 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.018 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.018 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.018 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.018 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.018 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.018 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.018 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "k8sevents", 2024-09-12T01:36:47.018 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.018 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.019 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.019 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_event_retention_days": { 2024-09-12T01:36:47.019 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ceph_event_retention_days", 2024-09-12T01:36:47.019 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.019 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.019 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.019 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "7", 2024-09-12T01:36:47.019 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.019 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.019 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.019 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T01:36:47.019 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.019 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.019 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.020 INFO:teuthology.orchestra.run.smithi001.stdout: "config_check_secs": { 2024-09-12T01:36:47.020 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_check_secs", 2024-09-12T01:36:47.020 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.020 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.020 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.020 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-12T01:36:47.020 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "10", 2024-09-12T01:36:47.020 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.020 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.020 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T01:36:47.020 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.020 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.020 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.021 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.021 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.021 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.021 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.021 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.021 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.021 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.021 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.021 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.021 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.021 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.021 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.021 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.021 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.021 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.022 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.022 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.022 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.022 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.022 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.022 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.022 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.022 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.022 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.022 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.022 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.022 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.022 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.023 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.023 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.023 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.023 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.023 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.023 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.023 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.023 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.023 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.023 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.023 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.023 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.023 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.023 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.024 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.024 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.024 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.024 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.024 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.024 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.024 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.024 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.024 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.024 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.024 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.024 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.024 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.025 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.025 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.025 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.025 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.025 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.025 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.025 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.025 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.025 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.025 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.025 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.025 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.025 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.026 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.026 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "localpool", 2024-09-12T01:36:47.026 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.026 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.026 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.026 INFO:teuthology.orchestra.run.smithi001.stdout: "failure_domain": { 2024-09-12T01:36:47.026 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "failure_domain", 2024-09-12T01:36:47.026 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.026 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.026 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.026 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "host", 2024-09-12T01:36:47.026 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.027 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.027 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.027 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "failure domain for any created local pool", 2024-09-12T01:36:47.027 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T01:36:47.027 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.027 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.027 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.027 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.027 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.027 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.027 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.027 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.028 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.028 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.028 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.028 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.028 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.028 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.028 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.028 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.028 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.028 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.028 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.028 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.028 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.028 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.029 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.029 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.029 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.029 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.029 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.029 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.029 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.029 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.029 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.029 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.029 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.029 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.029 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.029 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.030 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.030 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.030 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.030 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.030 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.030 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.030 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.030 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.030 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.030 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.030 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.030 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.030 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.030 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.031 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.031 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.031 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.031 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.031 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.031 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.031 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.031 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.031 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.031 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.031 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.031 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.032 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.032 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.032 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.032 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.032 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.032 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.032 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": { 2024-09-12T01:36:47.032 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "min_size", 2024-09-12T01:36:47.032 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.032 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.032 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.032 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.032 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.033 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.033 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.033 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default min_size for any created local pool", 2024-09-12T01:36:47.033 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T01:36:47.033 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.033 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.033 INFO:teuthology.orchestra.run.smithi001.stdout: "num_rep": { 2024-09-12T01:36:47.033 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "num_rep", 2024-09-12T01:36:47.033 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.033 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.033 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.033 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3", 2024-09-12T01:36:47.033 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.034 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.034 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.034 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default replica count for any created local pool", 2024-09-12T01:36:47.034 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.034 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.034 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.034 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": { 2024-09-12T01:36:47.034 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pg_num", 2024-09-12T01:36:47.034 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.034 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.034 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.034 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "128", 2024-09-12T01:36:47.034 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.035 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.035 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.035 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T01:36:47.035 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.035 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.035 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.035 INFO:teuthology.orchestra.run.smithi001.stdout: "prefix": { 2024-09-12T01:36:47.035 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prefix", 2024-09-12T01:36:47.035 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.035 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.035 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.035 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.035 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.036 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.036 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.036 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "name prefix for any created local pool", 2024-09-12T01:36:47.036 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.036 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.036 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.036 INFO:teuthology.orchestra.run.smithi001.stdout: "subtree": { 2024-09-12T01:36:47.036 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "subtree", 2024-09-12T01:36:47.036 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.036 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.036 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.037 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "rack", 2024-09-12T01:36:47.037 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.037 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.037 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.037 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T01:36:47.037 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T01:36:47.037 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.037 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.037 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.037 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.037 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.037 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mds_autoscaler", 2024-09-12T01:36:47.037 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.038 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.038 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.038 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.038 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.038 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.038 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.038 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.038 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.038 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.038 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.038 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.038 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.038 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.039 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.039 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.039 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.039 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.039 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.039 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.039 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.039 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.039 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.039 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.039 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.039 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.039 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.040 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.040 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.040 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.040 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.040 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.040 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.040 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.040 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.040 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.040 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.040 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.040 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.041 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.041 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.041 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.041 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.041 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.041 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.041 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.041 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.041 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.041 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.041 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.041 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.041 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.041 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.041 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.042 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.042 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.042 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.042 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.042 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.042 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.042 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.042 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.042 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.042 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.042 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.042 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.042 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.042 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.042 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.043 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.043 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.043 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.043 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mirroring", 2024-09-12T01:36:47.043 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.043 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.043 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.043 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.043 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.043 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.043 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.043 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.043 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.044 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.044 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.044 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.044 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.044 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.044 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.044 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.044 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.044 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.044 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.044 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.044 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.044 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.044 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.045 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.045 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.045 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.045 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.045 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.045 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.045 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.045 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.045 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.045 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.045 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.045 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.045 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.046 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.046 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.046 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.046 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.046 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.046 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.046 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.046 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.046 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.046 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.046 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.046 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.046 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.046 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.047 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.047 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.047 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.047 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.047 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.047 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.047 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.047 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.047 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.047 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.047 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.047 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.047 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.048 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.048 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.048 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.048 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.048 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.048 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.048 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.048 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.048 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.048 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "nfs", 2024-09-12T01:36:47.048 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.048 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.049 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.049 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.049 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.049 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.049 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.049 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.049 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.049 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.049 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.049 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.049 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.049 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.049 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.049 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.050 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.050 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.050 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.050 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.050 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.050 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.050 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.050 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.050 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.050 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.050 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.050 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.050 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.050 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.051 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.051 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.051 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.051 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.051 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.051 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.051 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.051 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.051 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.051 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.051 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.051 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.051 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.052 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.052 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.052 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.052 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.052 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.052 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.052 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.052 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.052 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.052 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.052 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.052 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.052 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.052 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.053 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.053 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.053 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.053 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.053 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.053 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.053 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.053 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.053 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.053 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.053 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.053 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.054 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.054 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.054 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.054 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.054 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.054 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.054 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "orchestrator", 2024-09-12T01:36:47.054 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.054 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.054 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.054 INFO:teuthology.orchestra.run.smithi001.stdout: "fail_fs": { 2024-09-12T01:36:47.054 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "fail_fs", 2024-09-12T01:36:47.054 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.054 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.055 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.055 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.055 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.055 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.055 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.055 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T01:36:47.055 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.055 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.055 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.055 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.055 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.055 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.055 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.055 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.056 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.056 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.056 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.056 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.056 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.056 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.056 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.056 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.056 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.056 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.056 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.056 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.056 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.056 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.057 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.057 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.057 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.057 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.057 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.057 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.057 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.057 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.057 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.057 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.057 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.057 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.057 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.058 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.058 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.058 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.058 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.058 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.058 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.058 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.058 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.058 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.058 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.058 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.058 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.058 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.058 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.059 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.059 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.059 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.059 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.059 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.059 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.059 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.059 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.059 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.059 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.059 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.059 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.059 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.059 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.060 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.060 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.060 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.060 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.060 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.060 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.060 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.060 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator": { 2024-09-12T01:36:47.060 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "orchestrator", 2024-09-12T01:36:47.060 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.060 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.060 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.061 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.061 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.061 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.061 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.061 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm", 2024-09-12T01:36:47.061 INFO:teuthology.orchestra.run.smithi001.stdout: "rook", 2024-09-12T01:36:47.061 INFO:teuthology.orchestra.run.smithi001.stdout: "test_orchestrator" 2024-09-12T01:36:47.061 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.061 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Orchestrator backend", 2024-09-12T01:36:47.061 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.061 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.061 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.061 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.061 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.062 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.062 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.062 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "osd_perf_query", 2024-09-12T01:36:47.062 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.062 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.062 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.062 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.062 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.062 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.062 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.062 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.062 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.062 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.063 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.063 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.063 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.063 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.063 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.063 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.063 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.063 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.063 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.063 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.063 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.063 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.063 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.064 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.064 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.064 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.064 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.064 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.064 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.064 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.064 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.064 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.064 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.064 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.064 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.065 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.065 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.065 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.065 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.065 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.065 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.065 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.065 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.065 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.065 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.065 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.065 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.066 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.066 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.066 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.066 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.066 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.066 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.066 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.066 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.066 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.066 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.066 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.066 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.066 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.066 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.067 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.067 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.067 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.067 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.067 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.067 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.067 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.067 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.067 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.067 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.067 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.067 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.067 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.067 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "osd_support", 2024-09-12T01:36:47.068 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.068 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.068 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.068 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.068 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.068 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.068 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.068 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.068 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.068 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.068 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.068 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.068 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.068 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.068 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.069 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.069 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.069 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.069 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.069 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.069 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.069 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.069 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.069 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.069 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.069 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.069 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.069 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.069 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.070 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.070 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.070 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.070 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.070 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.070 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.070 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.070 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.070 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.070 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.070 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.070 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.070 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.071 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.071 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.071 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.071 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.071 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.071 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.071 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.071 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.071 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.071 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.071 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.071 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.071 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.071 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.071 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.072 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.072 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.072 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.072 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.072 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.072 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.072 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.072 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.072 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.072 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.072 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.072 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.072 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.072 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.073 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.073 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.073 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.073 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pg_autoscaler", 2024-09-12T01:36:47.073 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.073 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.073 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.073 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.073 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.073 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.073 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.073 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.073 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.073 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.074 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.074 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.074 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.074 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.074 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.074 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.074 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.074 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.074 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.074 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.074 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.074 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.074 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.074 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.075 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.075 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.075 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.075 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.075 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.075 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.075 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.075 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.075 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.075 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.075 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.075 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.075 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.076 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.076 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.076 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.076 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.076 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.076 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.076 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.076 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.076 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.076 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.076 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.076 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.077 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.077 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.077 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.077 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.077 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.077 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.077 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.077 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.077 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.077 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.077 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.077 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.077 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.077 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.078 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.078 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.078 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.078 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.078 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.078 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.078 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.078 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.078 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-12T01:36:47.078 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-12T01:36:47.078 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:47.078 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.078 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.078 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-12T01:36:47.078 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.079 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.079 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.079 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.079 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.079 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.079 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.079 INFO:teuthology.orchestra.run.smithi001.stdout: "threshold": { 2024-09-12T01:36:47.079 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "threshold", 2024-09-12T01:36:47.079 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-12T01:36:47.079 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.079 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.079 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3.0", 2024-09-12T01:36:47.079 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1.0", 2024-09-12T01:36:47.080 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.080 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.080 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "scaling threshold", 2024-09-12T01:36:47.080 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T01:36:47.080 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.080 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.080 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.080 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.080 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.080 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "progress", 2024-09-12T01:36:47.080 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.080 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.080 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.081 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_pg_recovery_event": { 2024-09-12T01:36:47.081 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_pg_recovery_event", 2024-09-12T01:36:47.081 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.081 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.081 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.081 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.081 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.081 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.081 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.081 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T01:36:47.081 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.081 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.081 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.081 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.082 INFO:teuthology.orchestra.run.smithi001.stdout: "enabled": { 2024-09-12T01:36:47.082 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enabled", 2024-09-12T01:36:47.082 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.082 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.082 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.082 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.082 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.082 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.082 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.082 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.082 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.082 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.082 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.082 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.082 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.083 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.083 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.083 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.083 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.083 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.083 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.083 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.083 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.083 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.083 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.083 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.083 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.083 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.083 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.083 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.084 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.084 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.084 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.084 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.084 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.084 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.084 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.084 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.084 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.084 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.084 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.084 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.084 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.085 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.085 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.085 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.085 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.085 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.085 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.085 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.085 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.085 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.085 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.085 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.085 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.085 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.086 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.086 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.086 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.086 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.086 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.086 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.086 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.086 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.086 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.086 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.086 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.086 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.086 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.086 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.087 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.087 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.087 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.087 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.087 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.087 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.087 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.087 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.087 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.087 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.087 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.087 INFO:teuthology.orchestra.run.smithi001.stdout: "max_completed_events": { 2024-09-12T01:36:47.087 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_completed_events", 2024-09-12T01:36:47.087 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.087 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.088 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.088 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "50", 2024-09-12T01:36:47.088 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.088 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.088 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.088 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "number of past completed events to remember", 2024-09-12T01:36:47.088 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.088 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.088 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.088 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-12T01:36:47.088 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-12T01:36:47.088 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:47.088 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.088 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.089 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-09-12T01:36:47.089 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.089 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.089 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.089 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long the module is going to sleep", 2024-09-12T01:36:47.089 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.089 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.089 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.089 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.089 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.089 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.089 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus", 2024-09-12T01:36:47.089 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.089 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.090 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.090 INFO:teuthology.orchestra.run.smithi001.stdout: "cache": { 2024-09-12T01:36:47.090 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cache", 2024-09-12T01:36:47.090 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.090 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.090 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.090 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.090 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.090 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.090 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.090 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.090 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.090 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.090 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.091 INFO:teuthology.orchestra.run.smithi001.stdout: "exclude_perf_counters": { 2024-09-12T01:36:47.091 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "exclude_perf_counters", 2024-09-12T01:36:47.091 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.091 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.091 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.091 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.091 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.091 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.091 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.091 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T01:36:47.091 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T01:36:47.091 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.091 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.091 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.091 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.092 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.092 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.092 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.092 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.092 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.092 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.092 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.092 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.092 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.092 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.092 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.092 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.092 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.092 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.092 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.093 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.093 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.093 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.093 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.093 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.093 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.093 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.093 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.093 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.093 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.093 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.093 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.094 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.094 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.094 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.094 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.094 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.094 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.094 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.094 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.094 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.094 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.094 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.094 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.095 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.095 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.095 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.095 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.095 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.095 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.095 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.095 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.095 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.095 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.095 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.095 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.095 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.095 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.096 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.096 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.096 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.096 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.096 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.096 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.096 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.096 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.096 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.096 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.096 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.096 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_stats_pools": { 2024-09-12T01:36:47.097 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_stats_pools", 2024-09-12T01:36:47.097 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.097 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.097 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.097 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.097 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.097 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.097 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.097 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.097 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.097 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.097 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.097 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T01:36:47.098 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T01:36:47.098 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.098 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.098 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.098 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "300", 2024-09-12T01:36:47.098 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.098 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.098 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.098 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.098 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.098 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.098 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.099 INFO:teuthology.orchestra.run.smithi001.stdout: "scrape_interval": { 2024-09-12T01:36:47.099 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "scrape_interval", 2024-09-12T01:36:47.099 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-12T01:36:47.099 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.099 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.099 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15.0", 2024-09-12T01:36:47.099 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.099 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.099 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.099 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.099 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.099 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.099 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.100 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-09-12T01:36:47.100 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-09-12T01:36:47.100 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.100 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.100 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.100 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "::", 2024-09-12T01:36:47.100 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.100 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.100 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.100 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T01:36:47.100 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.100 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.100 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.101 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.101 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-09-12T01:36:47.101 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-09-12T01:36:47.101 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.101 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.101 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.101 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "9283", 2024-09-12T01:36:47.101 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.101 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.101 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.101 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T01:36:47.101 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.101 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.101 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.102 INFO:teuthology.orchestra.run.smithi001.stdout: "stale_cache_strategy": { 2024-09-12T01:36:47.102 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "stale_cache_strategy", 2024-09-12T01:36:47.102 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.102 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.102 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.102 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "log", 2024-09-12T01:36:47.102 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.102 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.102 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.102 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.102 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.102 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.102 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.103 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_behaviour": { 2024-09-12T01:36:47.103 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_behaviour", 2024-09-12T01:36:47.103 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.103 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.103 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.103 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "default", 2024-09-12T01:36:47.103 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.103 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.103 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.103 INFO:teuthology.orchestra.run.smithi001.stdout: "default", 2024-09-12T01:36:47.103 INFO:teuthology.orchestra.run.smithi001.stdout: "error" 2024-09-12T01:36:47.103 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.103 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.103 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.104 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.104 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.104 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_error_status_code": { 2024-09-12T01:36:47.104 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_error_status_code", 2024-09-12T01:36:47.104 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.104 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.104 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.104 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "500", 2024-09-12T01:36:47.104 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "400", 2024-09-12T01:36:47.104 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "599", 2024-09-12T01:36:47.104 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.104 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.105 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.105 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.105 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.105 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.105 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.105 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.105 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_support", 2024-09-12T01:36:47.105 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.105 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.105 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.105 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.105 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.105 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.105 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.106 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.106 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.106 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.106 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.106 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.106 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.106 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.106 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.106 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.106 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.106 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.106 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.106 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.106 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.107 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.107 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.107 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.107 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.107 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.107 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.107 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.107 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.107 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.107 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.107 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.107 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.107 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.107 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.108 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.108 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.108 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.108 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.108 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.108 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.108 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.108 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.108 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.108 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.108 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.108 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.109 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.109 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.109 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.109 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.109 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.109 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.109 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.109 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.109 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.109 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.109 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.109 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.109 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.110 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.110 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.110 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.110 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.110 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.110 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.110 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.110 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.110 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.110 INFO:teuthology.orchestra.run.smithi001.stdout: "max_concurrent_snap_create": { 2024-09-12T01:36:47.110 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_concurrent_snap_create", 2024-09-12T01:36:47.110 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.110 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.110 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.111 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-12T01:36:47.111 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.111 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.111 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.111 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.111 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.111 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.111 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.111 INFO:teuthology.orchestra.run.smithi001.stdout: "mirror_snapshot_schedule": { 2024-09-12T01:36:47.111 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T01:36:47.111 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.111 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.111 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.112 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.112 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.112 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.112 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.112 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.112 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.112 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.112 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.112 INFO:teuthology.orchestra.run.smithi001.stdout: "trash_purge_schedule": { 2024-09-12T01:36:47.112 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "trash_purge_schedule", 2024-09-12T01:36:47.112 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.112 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.112 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.113 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.113 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.113 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.113 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.113 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.113 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.113 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.113 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.113 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.113 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.113 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "restful", 2024-09-12T01:36:47.113 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.113 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.114 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.114 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_auth": { 2024-09-12T01:36:47.114 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enable_auth", 2024-09-12T01:36:47.114 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.114 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.114 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.114 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.114 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.114 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.114 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.114 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.114 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.114 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.114 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.115 INFO:teuthology.orchestra.run.smithi001.stdout: "key_file": { 2024-09-12T01:36:47.115 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "key_file", 2024-09-12T01:36:47.115 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.115 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.115 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.115 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.115 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.115 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.115 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.115 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.115 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.116 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.116 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.116 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.116 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.116 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.116 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.116 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.116 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.116 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.116 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.116 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.117 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.117 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.117 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.117 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.117 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.117 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.117 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.117 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.117 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.117 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.117 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.117 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.117 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.117 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.118 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.118 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.118 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.118 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.118 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.118 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.118 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.118 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.118 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.118 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.118 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.118 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.118 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.119 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.119 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.119 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.119 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.119 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.119 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.119 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.119 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.119 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.119 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.119 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.119 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.119 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.119 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.119 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.120 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.120 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.120 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.120 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.120 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.120 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.120 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.120 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.120 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.120 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.120 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.120 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.120 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.121 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.121 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.121 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-09-12T01:36:47.121 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-09-12T01:36:47.121 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.121 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.121 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.121 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.121 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.121 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.121 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.122 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.122 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.122 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.122 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.122 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-09-12T01:36:47.122 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-09-12T01:36:47.122 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.122 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.122 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.122 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.122 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.122 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.122 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.123 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.123 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.123 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.123 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.123 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.123 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.123 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rgw", 2024-09-12T01:36:47.123 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.123 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.123 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.123 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.123 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.124 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.124 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.124 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.124 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.124 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.124 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.124 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.124 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.124 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.124 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.124 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.124 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.124 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.124 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.125 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.125 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.125 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.125 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.125 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.125 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.125 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.125 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.125 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.125 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.125 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.125 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.125 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.126 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.126 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.126 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.126 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.126 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.126 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.126 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.126 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.126 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.126 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.126 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.126 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.126 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.127 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.127 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.127 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.127 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.127 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.127 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.127 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.127 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.127 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.127 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.127 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.127 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.127 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.128 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.128 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.128 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.128 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.128 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.128 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.128 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.128 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.128 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.128 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.128 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.128 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.128 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.129 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rook", 2024-09-12T01:36:47.129 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.129 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.129 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.129 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.129 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.129 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.129 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.129 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.129 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.129 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.129 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.129 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.129 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.130 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.130 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.130 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.130 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.130 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.130 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.130 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.130 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.130 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.130 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.130 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.130 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.131 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.131 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.131 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.131 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.131 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.131 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.131 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.131 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.131 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.131 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.131 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.131 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.131 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.132 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.132 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.132 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.132 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.132 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.132 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.132 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.132 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.132 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.132 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.132 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.132 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.132 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.133 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.133 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.133 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.133 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.133 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.133 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.133 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.133 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.133 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.133 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.133 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.133 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.134 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.134 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.134 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.134 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.134 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.134 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.134 INFO:teuthology.orchestra.run.smithi001.stdout: "storage_class": { 2024-09-12T01:36:47.134 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "storage_class", 2024-09-12T01:36:47.134 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.134 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.134 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.134 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "local", 2024-09-12T01:36:47.134 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.135 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.135 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.135 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T01:36:47.135 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.135 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.135 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.135 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.135 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.135 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "selftest", 2024-09-12T01:36:47.135 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.135 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.135 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.136 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.136 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.136 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.136 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.136 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.136 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.136 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.136 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.136 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.136 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.136 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.136 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.136 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.136 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.137 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.137 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.137 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.137 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.137 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.137 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.137 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.137 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.137 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.137 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.137 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.137 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.137 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.138 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.138 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.138 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.138 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.138 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.138 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.138 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.138 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.138 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.138 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.138 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.138 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.138 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.139 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.139 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.139 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.139 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.139 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.139 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.139 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.139 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.139 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.139 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.139 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.139 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.139 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.139 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.140 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.140 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.140 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.140 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.140 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.140 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.140 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.140 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.140 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.140 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.140 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.140 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.140 INFO:teuthology.orchestra.run.smithi001.stdout: "roption1": { 2024-09-12T01:36:47.141 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "roption1", 2024-09-12T01:36:47.141 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.141 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.141 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.141 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.141 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.141 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.141 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.141 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.141 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.141 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.141 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.141 INFO:teuthology.orchestra.run.smithi001.stdout: "roption2": { 2024-09-12T01:36:47.142 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "roption2", 2024-09-12T01:36:47.142 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.142 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.142 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.142 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "xyz", 2024-09-12T01:36:47.142 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.142 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.142 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.142 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.142 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.142 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.142 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.142 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption1": { 2024-09-12T01:36:47.142 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption1", 2024-09-12T01:36:47.143 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.143 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.143 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.143 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.143 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.143 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.143 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.143 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.143 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.143 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.143 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.143 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption2": { 2024-09-12T01:36:47.143 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption2", 2024-09-12T01:36:47.143 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.144 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.144 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.144 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.144 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.144 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.144 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.144 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.144 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.144 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.144 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.144 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption3": { 2024-09-12T01:36:47.144 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption3", 2024-09-12T01:36:47.145 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-12T01:36:47.145 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.145 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.145 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.145 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.145 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.145 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.145 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.145 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.145 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.145 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.145 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption4": { 2024-09-12T01:36:47.145 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption4", 2024-09-12T01:36:47.146 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.146 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.146 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.146 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.146 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.146 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.146 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.146 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.146 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.146 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.146 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.147 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption5": { 2024-09-12T01:36:47.147 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption5", 2024-09-12T01:36:47.147 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.147 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.147 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.147 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.147 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.147 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.147 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.147 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.147 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.147 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.147 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.148 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption6": { 2024-09-12T01:36:47.148 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption6", 2024-09-12T01:36:47.148 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.148 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.148 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.148 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.148 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.148 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.148 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.148 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.148 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.148 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.148 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.149 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption7": { 2024-09-12T01:36:47.149 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption7", 2024-09-12T01:36:47.149 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.149 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.149 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.149 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.149 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-09-12T01:36:47.149 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "42", 2024-09-12T01:36:47.149 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.149 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.149 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.149 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.149 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.150 INFO:teuthology.orchestra.run.smithi001.stdout: "testkey": { 2024-09-12T01:36:47.150 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testkey", 2024-09-12T01:36:47.150 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.150 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.150 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.150 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.150 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.150 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.150 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.150 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.150 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.150 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.150 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.151 INFO:teuthology.orchestra.run.smithi001.stdout: "testlkey": { 2024-09-12T01:36:47.151 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testlkey", 2024-09-12T01:36:47.151 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.151 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.151 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.151 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.151 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.151 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.151 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.151 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.151 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.151 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.152 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.152 INFO:teuthology.orchestra.run.smithi001.stdout: "testnewline": { 2024-09-12T01:36:47.152 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testnewline", 2024-09-12T01:36:47.152 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.152 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.152 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.152 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.152 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.152 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.152 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.152 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.152 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.152 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.153 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.153 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.153 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.153 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snap_schedule", 2024-09-12T01:36:47.153 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.153 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.153 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.153 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_m_granularity": { 2024-09-12T01:36:47.153 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_m_granularity", 2024-09-12T01:36:47.153 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.153 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.153 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.154 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.154 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.154 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.154 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.154 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T01:36:47.154 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.154 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.154 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.154 INFO:teuthology.orchestra.run.smithi001.stdout: "dump_on_update": { 2024-09-12T01:36:47.154 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "dump_on_update", 2024-09-12T01:36:47.154 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.154 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.154 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.154 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.154 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.155 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.155 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.155 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "dump database to debug log on update", 2024-09-12T01:36:47.155 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.155 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.155 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.155 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.155 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.155 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.155 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.155 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.155 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.155 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.155 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.156 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.156 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.156 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.156 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.156 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.156 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.156 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.156 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.156 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.156 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.156 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.156 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.156 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.156 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.157 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.157 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.157 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.157 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.157 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.157 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.157 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.157 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.157 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.157 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.157 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.157 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.157 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.158 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.158 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.158 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.158 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.158 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.158 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.158 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.158 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.158 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.158 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.158 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.158 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.158 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.158 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.158 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.158 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.159 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.159 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.159 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.159 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.159 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.159 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.159 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.159 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.159 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.159 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.159 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.159 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.159 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.160 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.160 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.160 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.160 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.160 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "stats", 2024-09-12T01:36:47.160 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.160 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.160 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.160 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.160 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.160 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.160 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.160 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.161 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.161 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.161 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.161 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.161 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.161 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.161 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.161 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.161 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.161 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.161 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.161 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.161 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.161 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.161 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.162 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.162 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.162 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.162 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.162 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.162 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.162 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.162 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.162 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.162 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.162 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.162 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.162 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.163 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.163 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.163 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.163 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.163 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.163 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.163 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.163 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.163 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.163 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.163 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.163 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.164 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.164 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.164 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.164 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.164 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.164 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.164 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.164 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.164 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.164 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.164 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.164 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.165 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.165 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.165 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.165 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.165 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.165 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.165 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.165 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.165 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.165 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.165 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.165 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.165 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "status", 2024-09-12T01:36:47.166 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.166 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.166 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.166 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.166 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.166 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.166 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.166 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.166 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.166 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.166 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.166 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.166 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.166 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.166 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.167 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.167 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.167 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.167 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.167 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.167 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.167 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.167 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.167 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.167 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.167 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.167 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.167 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.168 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.168 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.168 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.168 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.168 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.168 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.168 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.168 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.168 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.168 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.168 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.168 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.168 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.169 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.169 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.169 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.169 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.169 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.169 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.169 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.169 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.169 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.169 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.169 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.169 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.169 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.169 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.169 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.170 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.170 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.170 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.170 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.170 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.170 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.170 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.170 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.170 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.170 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.170 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.170 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.170 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.170 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.170 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.171 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.171 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.171 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "telegraf", 2024-09-12T01:36:47.171 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.171 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.171 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.171 INFO:teuthology.orchestra.run.smithi001.stdout: "address": { 2024-09-12T01:36:47.171 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "address", 2024-09-12T01:36:47.171 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.171 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.171 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.171 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T01:36:47.171 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.171 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.172 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.172 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.172 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.172 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.172 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.172 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-12T01:36:47.172 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-12T01:36:47.172 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:47.172 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.172 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.172 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15", 2024-09-12T01:36:47.172 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.172 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.173 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.173 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.173 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.173 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.173 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.173 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.173 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.173 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.173 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.173 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.173 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.173 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.173 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.174 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.174 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.174 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.174 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.174 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.174 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.174 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.174 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.174 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.174 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.174 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.174 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.174 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.175 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.175 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.175 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.175 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.175 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.175 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.175 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.175 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.175 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.175 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.175 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.175 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.175 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.175 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.176 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.176 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.176 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.176 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.176 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.176 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.176 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.176 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.176 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.176 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.176 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.176 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.176 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.176 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.177 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.177 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.177 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.177 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.177 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.177 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.177 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.177 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.177 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.177 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.177 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.177 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.177 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.178 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.178 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.178 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.178 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.178 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.178 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.178 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.178 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.178 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "telemetry", 2024-09-12T01:36:47.178 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.178 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.178 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.178 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_basic": { 2024-09-12T01:36:47.179 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_basic", 2024-09-12T01:36:47.179 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.179 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.179 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.179 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.179 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.179 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.179 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.179 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T01:36:47.179 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.179 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.179 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.179 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_crash": { 2024-09-12T01:36:47.180 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_crash", 2024-09-12T01:36:47.180 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.180 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.180 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.180 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.180 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.180 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.180 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.180 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T01:36:47.180 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.180 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.180 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.180 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_device": { 2024-09-12T01:36:47.181 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_device", 2024-09-12T01:36:47.181 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.181 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.181 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.181 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.181 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.181 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.181 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.181 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T01:36:47.181 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.181 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.181 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.181 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_ident": { 2024-09-12T01:36:47.182 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_ident", 2024-09-12T01:36:47.182 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.182 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.182 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.182 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.182 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.182 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.182 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.182 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T01:36:47.182 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.182 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.182 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.182 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_perf": { 2024-09-12T01:36:47.183 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_perf", 2024-09-12T01:36:47.183 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.183 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.183 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.183 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.183 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.183 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.183 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.183 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T01:36:47.183 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.183 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.183 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.183 INFO:teuthology.orchestra.run.smithi001.stdout: "contact": { 2024-09-12T01:36:47.183 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "contact", 2024-09-12T01:36:47.184 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.184 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.184 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.184 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.184 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.184 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.184 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.184 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.184 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.184 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.184 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.184 INFO:teuthology.orchestra.run.smithi001.stdout: "description": { 2024-09-12T01:36:47.184 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "description", 2024-09-12T01:36:47.185 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.185 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.185 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.185 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.185 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.185 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.185 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.185 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.185 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.185 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.185 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.185 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.185 INFO:teuthology.orchestra.run.smithi001.stdout: "device_url": { 2024-09-12T01:36:47.185 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_url", 2024-09-12T01:36:47.186 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.186 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.186 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.186 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T01:36:47.186 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.186 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.186 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.186 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.186 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.186 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.186 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.186 INFO:teuthology.orchestra.run.smithi001.stdout: "enabled": { 2024-09-12T01:36:47.186 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enabled", 2024-09-12T01:36:47.186 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.187 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.187 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.187 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.187 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.187 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.187 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.187 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.187 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.187 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.187 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.187 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-12T01:36:47.187 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-12T01:36:47.187 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.188 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.188 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.188 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "24", 2024-09-12T01:36:47.188 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "8", 2024-09-12T01:36:47.188 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.188 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.188 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.188 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.188 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.188 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.188 INFO:teuthology.orchestra.run.smithi001.stdout: "last_opt_revision": { 2024-09-12T01:36:47.188 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "last_opt_revision", 2024-09-12T01:36:47.188 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.189 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.189 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.189 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1", 2024-09-12T01:36:47.189 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.189 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.189 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.189 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.189 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.189 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.189 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.189 INFO:teuthology.orchestra.run.smithi001.stdout: "leaderboard": { 2024-09-12T01:36:47.189 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "leaderboard", 2024-09-12T01:36:47.189 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.189 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.190 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.190 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.190 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.190 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.190 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.190 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.190 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.190 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.190 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.190 INFO:teuthology.orchestra.run.smithi001.stdout: "leaderboard_description": { 2024-09-12T01:36:47.190 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "leaderboard_description", 2024-09-12T01:36:47.190 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.190 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.190 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.191 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.191 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.191 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.191 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.191 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.191 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.191 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.191 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.191 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.191 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.191 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.191 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.191 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.191 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.192 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.192 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.192 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.192 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.192 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.192 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.192 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.192 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.192 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.192 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.192 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.192 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.192 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.192 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.193 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.193 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.193 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.193 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.193 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.193 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.193 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.193 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.193 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.193 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.193 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.193 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.193 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.193 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.194 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.194 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.194 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.194 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.194 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.194 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.194 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.194 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.194 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.194 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.194 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.194 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.194 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.195 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.195 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.195 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.195 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.195 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.195 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.195 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.195 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.195 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.195 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.195 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.195 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.195 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.195 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.195 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.196 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.196 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.196 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.196 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.196 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.196 INFO:teuthology.orchestra.run.smithi001.stdout: "organization": { 2024-09-12T01:36:47.196 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "organization", 2024-09-12T01:36:47.196 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.196 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.196 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.196 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.196 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.196 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.196 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.196 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.197 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.197 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.197 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.197 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.197 INFO:teuthology.orchestra.run.smithi001.stdout: "proxy": { 2024-09-12T01:36:47.197 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "proxy", 2024-09-12T01:36:47.197 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.197 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.197 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.197 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.197 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.197 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.197 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.197 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.198 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.198 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.198 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.198 INFO:teuthology.orchestra.run.smithi001.stdout: "url": { 2024-09-12T01:36:47.198 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "url", 2024-09-12T01:36:47.198 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.198 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.198 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.198 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T01:36:47.198 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.198 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.198 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.198 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.199 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.199 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.199 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.199 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.199 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.199 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "test_orchestrator", 2024-09-12T01:36:47.199 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.199 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.199 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.199 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.199 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.199 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.200 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.200 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.200 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.200 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.200 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.200 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.200 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.200 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.200 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.200 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.200 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.200 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.200 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.201 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.201 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.201 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.201 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.201 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.201 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.201 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.201 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.201 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.201 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.201 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.201 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.201 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.202 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.202 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.202 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.202 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.202 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.202 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.202 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.202 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.202 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.202 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.202 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.202 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.202 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.202 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.203 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.203 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.203 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.203 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.203 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.203 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.203 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.203 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.203 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.203 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.203 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.203 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.203 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.203 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.203 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.204 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.204 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.204 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.204 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.204 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.204 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.204 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.204 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.204 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.204 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.204 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "volumes", 2024-09-12T01:36:47.205 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.205 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.205 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.205 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.205 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.205 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.205 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.205 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.205 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.205 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.205 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.205 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.205 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.205 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.206 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.206 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.206 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.206 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.206 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.206 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.206 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.206 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.206 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.206 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.206 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.206 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.206 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.206 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.206 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.207 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.207 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.207 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.207 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.207 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.207 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.207 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.207 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.207 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.207 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.207 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.207 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.207 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.208 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.208 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.208 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.208 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.208 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.208 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.208 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.208 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.208 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.208 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.208 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.208 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.208 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.208 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.208 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.209 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.209 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.209 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.209 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.209 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.209 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.209 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.209 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.209 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.209 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.209 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.209 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.210 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.210 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.210 INFO:teuthology.orchestra.run.smithi001.stdout: "max_concurrent_clones": { 2024-09-12T01:36:47.210 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_concurrent_clones", 2024-09-12T01:36:47.210 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.210 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.210 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.210 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "4", 2024-09-12T01:36:47.210 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.210 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.210 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.210 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T01:36:47.210 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.210 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.210 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.211 INFO:teuthology.orchestra.run.smithi001.stdout: "periodic_async_work": { 2024-09-12T01:36:47.211 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "periodic_async_work", 2024-09-12T01:36:47.211 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.211 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.211 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.211 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.211 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.211 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.211 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.211 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Periodically check for async work", 2024-09-12T01:36:47.211 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.211 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.212 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.212 INFO:teuthology.orchestra.run.smithi001.stdout: "snapshot_clone_delay": { 2024-09-12T01:36:47.212 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snapshot_clone_delay", 2024-09-12T01:36:47.212 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.212 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.212 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.212 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-12T01:36:47.212 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.212 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.212 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.212 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T01:36:47.212 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.213 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.213 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.213 INFO:teuthology.orchestra.run.smithi001.stdout: "snapshot_clone_no_wait": { 2024-09-12T01:36:47.213 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T01:36:47.213 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.213 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.213 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.213 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.213 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.213 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.213 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.213 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T01:36:47.213 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.214 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.214 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.214 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.214 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.214 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix", 2024-09-12T01:36:47.214 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.214 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.214 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.214 INFO:teuthology.orchestra.run.smithi001.stdout: "discovery_interval": { 2024-09-12T01:36:47.214 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "discovery_interval", 2024-09-12T01:36:47.214 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-12T01:36:47.214 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.215 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.215 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "100", 2024-09-12T01:36:47.215 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.215 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.215 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.215 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.215 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.215 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.215 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.215 INFO:teuthology.orchestra.run.smithi001.stdout: "identifier": { 2024-09-12T01:36:47.215 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "identifier", 2024-09-12T01:36:47.215 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.215 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.216 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.216 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.216 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.216 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.216 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.216 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.216 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.216 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.216 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.216 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-12T01:36:47.216 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-12T01:36:47.216 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:47.217 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.217 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.217 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-12T01:36:47.217 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.217 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.217 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.217 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.217 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.217 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.217 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.217 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.217 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.217 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.218 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.218 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.218 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.218 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.218 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.218 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.218 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.218 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.218 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.218 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.218 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.218 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.218 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.218 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.219 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.219 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.219 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.219 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.219 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.219 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.219 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.219 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.219 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.219 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.219 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.219 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.219 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.220 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.220 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.220 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.220 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.220 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.220 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.220 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.220 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.220 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.220 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.220 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.220 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.220 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.221 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.221 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.221 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.221 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.221 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.221 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.221 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.221 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.221 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.221 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.221 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.221 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.221 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.222 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.222 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.222 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.222 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.222 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.222 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.222 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.222 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.222 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.222 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.222 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_host": { 2024-09-12T01:36:47.222 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_host", 2024-09-12T01:36:47.222 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.223 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.223 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.223 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.223 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.223 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.223 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.223 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.223 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.223 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.223 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.223 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_port": { 2024-09-12T01:36:47.223 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_port", 2024-09-12T01:36:47.223 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.224 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.224 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.224 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10051", 2024-09-12T01:36:47.224 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.224 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.224 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.224 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.224 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.224 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.224 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.224 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_sender": { 2024-09-12T01:36:47.224 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_sender", 2024-09-12T01:36:47.224 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.225 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.225 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.225 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T01:36:47.225 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.225 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.225 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.225 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.225 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.225 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.225 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.225 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.225 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.225 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.225 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-12T01:36:47.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.226 INFO:teuthology.orchestra.run.smithi001.stdout: "gid": 24113, 2024-09-12T01:36:47.226 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "y", 2024-09-12T01:36:47.226 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_features": 4540701547738038271, 2024-09-12T01:36:47.226 INFO:teuthology.orchestra.run.smithi001.stdout: "available_modules": [ 2024-09-12T01:36:47.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.226 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "alerts", 2024-09-12T01:36:47.226 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.226 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.226 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.226 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-12T01:36:47.226 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-12T01:36:47.226 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:47.227 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.227 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.227 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-12T01:36:47.227 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.227 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.227 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.227 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T01:36:47.227 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.227 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.227 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.227 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.227 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.227 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.227 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.227 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.228 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.228 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.228 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.228 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.228 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.228 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.228 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.228 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.228 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.228 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.228 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.228 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.228 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.229 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.229 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.229 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.229 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.229 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.229 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.229 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.229 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.229 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.229 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.229 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.229 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.229 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.229 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.229 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.230 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.230 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.230 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.230 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.230 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.230 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.230 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.230 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.230 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.230 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.230 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.230 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.230 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.231 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.231 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.231 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.231 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.231 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.231 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.231 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.231 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.231 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.231 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.231 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.231 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.231 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.232 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.232 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.232 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.232 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.232 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.232 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.232 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.232 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.232 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.232 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_destination": { 2024-09-12T01:36:47.232 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_destination", 2024-09-12T01:36:47.232 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.233 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.233 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.233 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.233 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.233 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.233 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.233 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Email address to send alerts to", 2024-09-12T01:36:47.233 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.233 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.233 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.233 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_from_name": { 2024-09-12T01:36:47.233 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_from_name", 2024-09-12T01:36:47.233 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.234 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.234 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.234 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "Ceph", 2024-09-12T01:36:47.234 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.234 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.234 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.234 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Email From: name", 2024-09-12T01:36:47.234 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.234 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.234 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.234 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_host": { 2024-09-12T01:36:47.234 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_host", 2024-09-12T01:36:47.234 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.235 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.235 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.235 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.235 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.235 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.235 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.235 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP server", 2024-09-12T01:36:47.235 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.235 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.235 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.235 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_password": { 2024-09-12T01:36:47.235 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_password", 2024-09-12T01:36:47.235 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.235 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.236 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.236 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.236 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.236 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.236 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.236 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Password to authenticate with", 2024-09-12T01:36:47.236 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.236 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.236 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.236 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_port": { 2024-09-12T01:36:47.236 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_port", 2024-09-12T01:36:47.236 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.236 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.236 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.237 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "465", 2024-09-12T01:36:47.237 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.237 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.237 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.237 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP port", 2024-09-12T01:36:47.237 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.237 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.237 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.237 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_sender": { 2024-09-12T01:36:47.237 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_sender", 2024-09-12T01:36:47.237 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.237 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.237 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.237 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.238 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.238 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.238 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.238 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP envelope sender", 2024-09-12T01:36:47.238 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.238 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.238 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.238 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_ssl": { 2024-09-12T01:36:47.238 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_ssl", 2024-09-12T01:36:47.238 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.238 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.238 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.238 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.238 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.239 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.239 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.239 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T01:36:47.239 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.239 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.239 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.239 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_user": { 2024-09-12T01:36:47.239 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_user", 2024-09-12T01:36:47.239 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.239 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.239 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.239 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.239 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.239 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.239 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.240 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "User to authenticate as", 2024-09-12T01:36:47.240 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.240 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.240 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.240 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.240 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.240 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "balancer", 2024-09-12T01:36:47.240 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.240 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.240 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.240 INFO:teuthology.orchestra.run.smithi001.stdout: "active": { 2024-09-12T01:36:47.240 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "active", 2024-09-12T01:36:47.240 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.241 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.241 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.241 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.241 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.241 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.241 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.241 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T01:36:47.241 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.241 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.241 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.241 INFO:teuthology.orchestra.run.smithi001.stdout: "begin_time": { 2024-09-12T01:36:47.241 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "begin_time", 2024-09-12T01:36:47.241 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.241 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.242 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.242 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0000", 2024-09-12T01:36:47.242 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.242 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.242 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.242 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T01:36:47.242 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T01:36:47.242 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.242 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.242 INFO:teuthology.orchestra.run.smithi001.stdout: "begin_weekday": { 2024-09-12T01:36:47.242 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "begin_weekday", 2024-09-12T01:36:47.242 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-12T01:36:47.242 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.243 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.243 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-12T01:36:47.243 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0", 2024-09-12T01:36:47.243 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "6", 2024-09-12T01:36:47.243 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.243 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T01:36:47.243 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T01:36:47.243 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.243 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.243 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.243 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_max_iterations": { 2024-09-12T01:36:47.243 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_max_iterations", 2024-09-12T01:36:47.243 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-12T01:36:47.244 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.244 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.244 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "25", 2024-09-12T01:36:47.244 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-09-12T01:36:47.244 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "250", 2024-09-12T01:36:47.244 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.244 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T01:36:47.244 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.244 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.244 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.244 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_metrics": { 2024-09-12T01:36:47.244 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_metrics", 2024-09-12T01:36:47.245 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.245 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.245 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.245 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T01:36:47.245 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.245 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.245 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.245 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T01:36:47.245 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T01:36:47.245 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.245 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.245 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_step": { 2024-09-12T01:36:47.245 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_step", 2024-09-12T01:36:47.246 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-12T01:36:47.246 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.246 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.246 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0.5", 2024-09-12T01:36:47.246 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0.001", 2024-09-12T01:36:47.246 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "0.999", 2024-09-12T01:36:47.246 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.246 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "aggressiveness of optimization", 2024-09-12T01:36:47.246 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T01:36:47.246 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.246 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.246 INFO:teuthology.orchestra.run.smithi001.stdout: "end_time": { 2024-09-12T01:36:47.246 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "end_time", 2024-09-12T01:36:47.246 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.246 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.246 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.247 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "2359", 2024-09-12T01:36:47.247 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.247 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.247 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.247 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T01:36:47.247 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T01:36:47.247 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.247 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.247 INFO:teuthology.orchestra.run.smithi001.stdout: "end_weekday": { 2024-09-12T01:36:47.247 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "end_weekday", 2024-09-12T01:36:47.247 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-12T01:36:47.247 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.248 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.248 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-12T01:36:47.248 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0", 2024-09-12T01:36:47.248 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "6", 2024-09-12T01:36:47.248 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.248 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T01:36:47.248 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T01:36:47.248 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.248 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.248 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.248 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.248 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.248 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.248 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.249 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.249 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.249 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.249 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.249 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.249 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.249 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.249 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.249 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.249 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.249 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.249 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.249 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.249 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.250 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.250 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.250 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.250 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.250 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.250 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.250 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.250 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.250 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.250 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.250 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.250 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.250 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.250 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.251 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.251 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.251 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.251 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.251 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.251 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.251 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.251 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.251 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.251 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.251 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.251 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.251 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.252 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.252 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.252 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.252 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.252 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.252 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.252 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.252 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.252 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.252 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.252 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.252 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.252 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.253 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.253 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.253 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.253 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.253 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.253 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.253 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.253 INFO:teuthology.orchestra.run.smithi001.stdout: "min_score": { 2024-09-12T01:36:47.253 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "min_score", 2024-09-12T01:36:47.253 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-12T01:36:47.253 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.253 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.253 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-12T01:36:47.254 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.254 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.254 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.254 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T01:36:47.254 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.254 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.254 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.254 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.254 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": { 2024-09-12T01:36:47.254 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mode", 2024-09-12T01:36:47.254 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.254 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.254 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.254 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "upmap", 2024-09-12T01:36:47.254 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.255 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.255 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.255 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-compat", 2024-09-12T01:36:47.255 INFO:teuthology.orchestra.run.smithi001.stdout: "none", 2024-09-12T01:36:47.255 INFO:teuthology.orchestra.run.smithi001.stdout: "read", 2024-09-12T01:36:47.255 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap", 2024-09-12T01:36:47.255 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap-read" 2024-09-12T01:36:47.255 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.255 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Balancer mode", 2024-09-12T01:36:47.255 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.255 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.255 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.255 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_ids": { 2024-09-12T01:36:47.256 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pool_ids", 2024-09-12T01:36:47.256 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.256 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.256 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.256 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.256 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.256 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.256 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.256 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T01:36:47.256 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.256 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.256 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.256 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-12T01:36:47.257 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-12T01:36:47.257 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:47.257 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.257 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.257 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-12T01:36:47.257 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.257 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.257 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.257 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T01:36:47.257 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.257 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.257 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.257 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap_max_deviation": { 2024-09-12T01:36:47.258 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "upmap_max_deviation", 2024-09-12T01:36:47.258 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.258 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.258 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.258 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-09-12T01:36:47.258 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-09-12T01:36:47.258 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.258 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.258 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T01:36:47.258 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T01:36:47.258 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.258 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.258 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap_max_optimizations": { 2024-09-12T01:36:47.259 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "upmap_max_optimizations", 2024-09-12T01:36:47.259 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-12T01:36:47.259 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.259 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.259 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-12T01:36:47.259 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.259 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.259 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.259 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T01:36:47.259 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.259 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.259 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.259 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.259 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.260 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.260 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cephadm", 2024-09-12T01:36:47.260 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.260 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.260 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.260 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_down_multiplier": { 2024-09-12T01:36:47.260 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_down_multiplier", 2024-09-12T01:36:47.260 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-12T01:36:47.260 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.260 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.260 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3.0", 2024-09-12T01:36:47.260 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.260 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.261 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.261 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T01:36:47.261 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.261 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.261 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.261 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_refresh_rate": { 2024-09-12T01:36:47.261 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_refresh_rate", 2024-09-12T01:36:47.261 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:47.261 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.261 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.261 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "20", 2024-09-12T01:36:47.261 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.261 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.262 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.262 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T01:36:47.262 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.262 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.262 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.262 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_starting_port": { 2024-09-12T01:36:47.262 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_starting_port", 2024-09-12T01:36:47.262 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.262 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.262 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.262 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "4721", 2024-09-12T01:36:47.262 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.262 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.263 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.263 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T01:36:47.263 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.263 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.263 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.263 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_ptrace": { 2024-09-12T01:36:47.263 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_ptrace", 2024-09-12T01:36:47.263 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.263 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.263 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.263 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.263 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.263 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.263 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.264 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T01:36:47.264 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T01:36:47.264 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.264 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.264 INFO:teuthology.orchestra.run.smithi001.stdout: "autotune_interval": { 2024-09-12T01:36:47.264 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "autotune_interval", 2024-09-12T01:36:47.264 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:47.264 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.264 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.264 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-12T01:36:47.264 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.264 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.264 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.265 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T01:36:47.265 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.265 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.265 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.265 INFO:teuthology.orchestra.run.smithi001.stdout: "autotune_memory_target_ratio": { 2024-09-12T01:36:47.265 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T01:36:47.265 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-12T01:36:47.265 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.265 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.265 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0.7", 2024-09-12T01:36:47.265 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.265 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.265 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.266 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T01:36:47.266 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.266 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.266 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.266 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm_log_destination": { 2024-09-12T01:36:47.266 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cephadm_log_destination", 2024-09-12T01:36:47.266 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.266 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.266 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.266 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.266 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.266 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.266 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.266 INFO:teuthology.orchestra.run.smithi001.stdout: "file", 2024-09-12T01:36:47.267 INFO:teuthology.orchestra.run.smithi001.stdout: "file,syslog", 2024-09-12T01:36:47.267 INFO:teuthology.orchestra.run.smithi001.stdout: "syslog" 2024-09-12T01:36:47.267 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.267 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T01:36:47.267 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.267 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.267 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.267 INFO:teuthology.orchestra.run.smithi001.stdout: "cgroups_split": { 2024-09-12T01:36:47.267 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cgroups_split", 2024-09-12T01:36:47.267 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.267 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.267 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.267 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.268 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.268 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.268 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.268 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T01:36:47.268 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.268 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.268 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.268 INFO:teuthology.orchestra.run.smithi001.stdout: "config_checks_enabled": { 2024-09-12T01:36:47.268 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_checks_enabled", 2024-09-12T01:36:47.268 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.268 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.268 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.268 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.269 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.269 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.269 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.269 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T01:36:47.269 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.269 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.269 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.269 INFO:teuthology.orchestra.run.smithi001.stdout: "config_dashboard": { 2024-09-12T01:36:47.269 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_dashboard", 2024-09-12T01:36:47.269 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.269 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.269 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.269 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.270 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.270 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.270 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.270 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T01:36:47.270 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.270 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.270 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.270 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_alertmanager": { 2024-09-12T01:36:47.270 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_alertmanager", 2024-09-12T01:36:47.270 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.270 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.270 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.270 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T01:36:47.271 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.271 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.271 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.271 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-09-12T01:36:47.271 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.271 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.271 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.271 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_base": { 2024-09-12T01:36:47.271 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_base", 2024-09-12T01:36:47.271 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.271 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.271 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.271 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T01:36:47.272 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.272 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.272 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.272 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Container image name, without the tag", 2024-09-12T01:36:47.272 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.272 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.272 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.272 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_elasticsearch": { 2024-09-12T01:36:47.272 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_elasticsearch", 2024-09-12T01:36:47.272 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.272 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.272 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.272 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T01:36:47.273 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.273 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.273 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.273 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "elasticsearch container image", 2024-09-12T01:36:47.273 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.273 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.273 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.273 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_grafana": { 2024-09-12T01:36:47.273 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_grafana", 2024-09-12T01:36:47.273 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.273 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.273 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.274 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T01:36:47.274 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.274 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.274 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.274 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-09-12T01:36:47.274 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.274 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.274 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.274 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_haproxy": { 2024-09-12T01:36:47.274 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_haproxy", 2024-09-12T01:36:47.274 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.274 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.274 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.275 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T01:36:47.275 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.275 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.275 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.275 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "HAproxy container image", 2024-09-12T01:36:47.275 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.275 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.275 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.275 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_agent": { 2024-09-12T01:36:47.275 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_agent", 2024-09-12T01:36:47.275 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.275 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.275 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.275 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T01:36:47.276 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.276 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.276 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.276 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger agent container image", 2024-09-12T01:36:47.276 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.276 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.276 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.276 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_collector": { 2024-09-12T01:36:47.276 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_collector", 2024-09-12T01:36:47.276 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.276 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.276 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.276 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T01:36:47.277 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.277 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.277 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.277 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger collector container image", 2024-09-12T01:36:47.277 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.277 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.277 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.277 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_query": { 2024-09-12T01:36:47.277 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_query", 2024-09-12T01:36:47.277 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.277 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.277 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.277 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T01:36:47.277 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.277 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.278 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.278 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger query container image", 2024-09-12T01:36:47.278 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.278 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.278 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.278 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.278 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_keepalived": { 2024-09-12T01:36:47.278 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_keepalived", 2024-09-12T01:36:47.278 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.278 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.278 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.278 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T01:36:47.278 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.279 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.279 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.279 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Keepalived container image", 2024-09-12T01:36:47.279 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.279 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.279 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.279 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_loki": { 2024-09-12T01:36:47.279 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_loki", 2024-09-12T01:36:47.279 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.279 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.279 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.279 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T01:36:47.279 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.279 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.280 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.280 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Loki container image", 2024-09-12T01:36:47.280 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.280 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.280 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.280 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_node_exporter": { 2024-09-12T01:36:47.280 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_node_exporter", 2024-09-12T01:36:47.280 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.280 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.280 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.280 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T01:36:47.280 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.280 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.281 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.281 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-09-12T01:36:47.281 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.281 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.281 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.281 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_nvmeof": { 2024-09-12T01:36:47.281 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_nvmeof", 2024-09-12T01:36:47.281 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.281 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.281 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.281 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T01:36:47.281 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.281 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.282 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.282 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Nvme-of container image", 2024-09-12T01:36:47.282 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.282 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.282 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.282 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_prometheus": { 2024-09-12T01:36:47.282 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_prometheus", 2024-09-12T01:36:47.282 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.282 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.282 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.282 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T01:36:47.282 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.282 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.283 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.283 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-09-12T01:36:47.283 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.283 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.283 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.283 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_promtail": { 2024-09-12T01:36:47.283 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_promtail", 2024-09-12T01:36:47.283 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.283 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.283 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.283 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T01:36:47.283 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.283 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.284 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.284 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Promtail container image", 2024-09-12T01:36:47.284 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.284 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.284 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.284 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_samba": { 2024-09-12T01:36:47.284 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_samba", 2024-09-12T01:36:47.284 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.284 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.284 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.284 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T01:36:47.284 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.284 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.285 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.285 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Samba/SMB container image", 2024-09-12T01:36:47.285 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.285 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.285 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.285 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_snmp_gateway": { 2024-09-12T01:36:47.285 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_snmp_gateway", 2024-09-12T01:36:47.285 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.285 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.285 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.285 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T01:36:47.285 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.286 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.286 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.286 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SNMP Gateway container image", 2024-09-12T01:36:47.286 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.286 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.286 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.286 INFO:teuthology.orchestra.run.smithi001.stdout: "container_init": { 2024-09-12T01:36:47.286 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_init", 2024-09-12T01:36:47.286 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.286 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.286 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.286 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.286 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.286 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.287 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.287 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T01:36:47.287 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.287 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.287 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.287 INFO:teuthology.orchestra.run.smithi001.stdout: "daemon_cache_timeout": { 2024-09-12T01:36:47.287 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "daemon_cache_timeout", 2024-09-12T01:36:47.287 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:47.287 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.287 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.287 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-12T01:36:47.287 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.288 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.288 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.288 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T01:36:47.288 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.288 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.288 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.288 INFO:teuthology.orchestra.run.smithi001.stdout: "default_cephadm_command_timeout": { 2024-09-12T01:36:47.288 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T01:36:47.288 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.288 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.288 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.288 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "900", 2024-09-12T01:36:47.289 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.289 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.289 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.289 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T01:36:47.289 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.289 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.289 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.289 INFO:teuthology.orchestra.run.smithi001.stdout: "default_registry": { 2024-09-12T01:36:47.289 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "default_registry", 2024-09-12T01:36:47.289 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.289 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.289 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.289 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io", 2024-09-12T01:36:47.290 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.290 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.290 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.290 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T01:36:47.290 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.290 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.290 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.290 INFO:teuthology.orchestra.run.smithi001.stdout: "device_cache_timeout": { 2024-09-12T01:36:47.290 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_cache_timeout", 2024-09-12T01:36:47.290 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:47.290 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.290 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.290 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1800", 2024-09-12T01:36:47.291 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.291 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.291 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.291 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache device inventory", 2024-09-12T01:36:47.291 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.291 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.291 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.291 INFO:teuthology.orchestra.run.smithi001.stdout: "device_enhanced_scan": { 2024-09-12T01:36:47.291 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_enhanced_scan", 2024-09-12T01:36:47.291 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.291 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.291 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.291 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.292 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.292 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.292 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.292 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T01:36:47.292 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.292 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.292 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.292 INFO:teuthology.orchestra.run.smithi001.stdout: "facts_cache_timeout": { 2024-09-12T01:36:47.292 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "facts_cache_timeout", 2024-09-12T01:36:47.292 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:47.292 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.292 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.292 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-12T01:36:47.292 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.293 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.293 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.293 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache host facts data", 2024-09-12T01:36:47.293 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.293 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.293 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.293 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.293 INFO:teuthology.orchestra.run.smithi001.stdout: "grafana_dashboards_path": { 2024-09-12T01:36:47.293 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "grafana_dashboards_path", 2024-09-12T01:36:47.293 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.293 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.293 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.293 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T01:36:47.294 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.294 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.294 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.294 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T01:36:47.294 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.294 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.294 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.294 INFO:teuthology.orchestra.run.smithi001.stdout: "host_check_interval": { 2024-09-12T01:36:47.294 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "host_check_interval", 2024-09-12T01:36:47.294 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:47.294 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.294 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.295 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-12T01:36:47.295 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.295 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.295 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.295 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to perform a host check", 2024-09-12T01:36:47.295 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.295 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.295 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.295 INFO:teuthology.orchestra.run.smithi001.stdout: "hw_monitoring": { 2024-09-12T01:36:47.295 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "hw_monitoring", 2024-09-12T01:36:47.295 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.295 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.295 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.296 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.296 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.296 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.296 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.296 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T01:36:47.296 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.296 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.296 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.296 INFO:teuthology.orchestra.run.smithi001.stdout: "inventory_list_all": { 2024-09-12T01:36:47.296 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "inventory_list_all", 2024-09-12T01:36:47.296 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.296 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.296 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.296 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.297 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.297 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.297 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.297 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T01:36:47.297 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.297 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.297 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.297 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.297 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.297 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.297 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.297 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.298 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.298 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.298 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.298 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.298 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.298 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.298 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.298 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.298 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.298 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.298 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.298 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.298 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.298 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.299 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.299 INFO:teuthology.orchestra.run.smithi001.stdout: "log_refresh_metadata": { 2024-09-12T01:36:47.299 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_refresh_metadata", 2024-09-12T01:36:47.299 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.299 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.299 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.299 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.299 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.299 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.299 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.299 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T01:36:47.299 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.299 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.300 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.300 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.300 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.300 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.300 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.300 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.300 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.300 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.300 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.300 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.300 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T01:36:47.300 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.301 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.301 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.301 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.301 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.301 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.301 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.301 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.301 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.301 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.301 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.301 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.301 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.301 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.302 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.302 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.302 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.302 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.302 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.302 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.302 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.302 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.302 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.302 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.302 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.302 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.302 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.302 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.303 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.303 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.303 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.303 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.303 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.303 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.303 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.303 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.303 INFO:teuthology.orchestra.run.smithi001.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T01:36:47.303 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T01:36:47.303 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.303 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.304 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.304 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.304 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.304 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.304 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.304 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T01:36:47.304 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.304 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.304 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.304 INFO:teuthology.orchestra.run.smithi001.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T01:36:47.304 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T01:36:47.304 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.304 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.305 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.305 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "*", 2024-09-12T01:36:47.305 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.305 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.305 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.305 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T01:36:47.305 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.305 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.305 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.305 INFO:teuthology.orchestra.run.smithi001.stdout: "max_count_per_host": { 2024-09-12T01:36:47.305 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_count_per_host", 2024-09-12T01:36:47.305 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.305 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.306 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.306 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-12T01:36:47.306 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.306 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.306 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.306 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "max number of daemons per service per host", 2024-09-12T01:36:47.306 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.306 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.306 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.306 INFO:teuthology.orchestra.run.smithi001.stdout: "max_osd_draining_count": { 2024-09-12T01:36:47.306 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_osd_draining_count", 2024-09-12T01:36:47.306 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.306 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.307 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.307 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-12T01:36:47.307 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.307 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.307 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.307 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T01:36:47.307 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.307 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.307 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.307 INFO:teuthology.orchestra.run.smithi001.stdout: "migration_current": { 2024-09-12T01:36:47.307 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "migration_current", 2024-09-12T01:36:47.307 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.307 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.308 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.308 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.308 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.308 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.308 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.308 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "internal - do not modify", 2024-09-12T01:36:47.308 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.308 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.308 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.308 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": { 2024-09-12T01:36:47.308 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mode", 2024-09-12T01:36:47.308 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.308 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.309 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.309 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "root", 2024-09-12T01:36:47.309 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.309 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.309 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.309 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm-package", 2024-09-12T01:36:47.309 INFO:teuthology.orchestra.run.smithi001.stdout: "root" 2024-09-12T01:36:47.309 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.309 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T01:36:47.309 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.309 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.309 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.309 INFO:teuthology.orchestra.run.smithi001.stdout: "oob_default_addr": { 2024-09-12T01:36:47.309 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "oob_default_addr", 2024-09-12T01:36:47.310 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.310 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.310 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.310 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "169.254.1.1", 2024-09-12T01:36:47.310 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.310 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.310 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.310 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T01:36:47.310 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.310 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.310 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.310 INFO:teuthology.orchestra.run.smithi001.stdout: "prometheus_alerts_path": { 2024-09-12T01:36:47.310 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus_alerts_path", 2024-09-12T01:36:47.310 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.311 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.311 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.311 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T01:36:47.311 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.311 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.311 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.311 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T01:36:47.311 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.311 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.311 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.311 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_insecure": { 2024-09-12T01:36:47.311 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_insecure", 2024-09-12T01:36:47.311 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.312 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.312 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.312 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.312 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.312 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.312 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.312 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T01:36:47.312 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.312 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.312 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.312 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.312 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_password": { 2024-09-12T01:36:47.312 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_password", 2024-09-12T01:36:47.312 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.313 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.313 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.313 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.313 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.313 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.313 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.313 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T01:36:47.313 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.313 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.313 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.313 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_url": { 2024-09-12T01:36:47.313 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_url", 2024-09-12T01:36:47.313 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.314 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.314 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.314 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.314 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.314 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.314 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.314 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T01:36:47.314 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.314 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.314 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.314 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_username": { 2024-09-12T01:36:47.314 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_username", 2024-09-12T01:36:47.314 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.315 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.315 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.315 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.315 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.315 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.315 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.315 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T01:36:47.315 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.315 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.315 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.315 INFO:teuthology.orchestra.run.smithi001.stdout: "secure_monitoring_stack": { 2024-09-12T01:36:47.315 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "secure_monitoring_stack", 2024-09-12T01:36:47.315 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.316 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.316 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.316 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.316 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.316 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.316 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.316 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T01:36:47.316 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.316 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.316 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.316 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.316 INFO:teuthology.orchestra.run.smithi001.stdout: "service_discovery_port": { 2024-09-12T01:36:47.316 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "service_discovery_port", 2024-09-12T01:36:47.316 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.317 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.317 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.317 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8765", 2024-09-12T01:36:47.317 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.317 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.317 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.317 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "cephadm service discovery port", 2024-09-12T01:36:47.317 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.317 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.317 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.317 INFO:teuthology.orchestra.run.smithi001.stdout: "ssh_config_file": { 2024-09-12T01:36:47.317 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssh_config_file", 2024-09-12T01:36:47.317 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.318 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.318 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.318 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.318 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.318 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.318 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.318 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T01:36:47.318 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.318 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.318 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.318 INFO:teuthology.orchestra.run.smithi001.stdout: "use_agent": { 2024-09-12T01:36:47.318 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "use_agent", 2024-09-12T01:36:47.319 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.319 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.319 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.319 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.319 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.319 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.319 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.319 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T01:36:47.319 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.319 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.319 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.319 INFO:teuthology.orchestra.run.smithi001.stdout: "use_repo_digest": { 2024-09-12T01:36:47.319 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "use_repo_digest", 2024-09-12T01:36:47.320 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.320 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.320 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.320 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.320 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.320 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.320 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.320 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T01:36:47.320 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.320 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.320 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.320 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.320 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_failed_host_check": { 2024-09-12T01:36:47.320 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_failed_host_check", 2024-09-12T01:36:47.321 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.321 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.321 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.321 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.321 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.321 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.321 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.321 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T01:36:47.321 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.321 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.321 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.321 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_stray_daemons": { 2024-09-12T01:36:47.321 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_stray_daemons", 2024-09-12T01:36:47.321 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.322 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.322 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.322 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.322 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.322 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.322 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.322 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T01:36:47.322 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.322 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.322 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.322 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_stray_hosts": { 2024-09-12T01:36:47.322 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_stray_hosts", 2024-09-12T01:36:47.322 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.323 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.323 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.323 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.323 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.323 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.323 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.323 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T01:36:47.323 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.323 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.323 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.323 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.323 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.324 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crash", 2024-09-12T01:36:47.324 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.324 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.324 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.324 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.324 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.324 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.324 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.324 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.324 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.324 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.324 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.324 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.324 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.325 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.325 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.325 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.325 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.325 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.325 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.325 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.325 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.325 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.325 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.325 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.325 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.326 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.326 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.326 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.326 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.326 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.326 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.326 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.326 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.326 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.326 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.326 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.326 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.326 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.327 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.327 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.327 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.327 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.327 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.327 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.327 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.327 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.327 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.327 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.327 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.327 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.327 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.327 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.328 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.328 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.328 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.328 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.328 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.328 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.328 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.328 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.328 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.328 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.328 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.328 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.328 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.329 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.329 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.329 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.329 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.329 INFO:teuthology.orchestra.run.smithi001.stdout: "retain_interval": { 2024-09-12T01:36:47.329 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "retain_interval", 2024-09-12T01:36:47.329 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:47.329 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.329 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.329 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "31536000", 2024-09-12T01:36:47.329 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.329 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.330 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.330 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T01:36:47.330 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.330 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.330 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.330 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_recent_interval": { 2024-09-12T01:36:47.330 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_recent_interval", 2024-09-12T01:36:47.330 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:47.330 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.330 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.330 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1209600", 2024-09-12T01:36:47.330 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.331 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.331 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.331 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T01:36:47.331 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.331 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.331 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.331 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.331 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.331 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "dashboard", 2024-09-12T01:36:47.331 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.331 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.331 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.332 INFO:teuthology.orchestra.run.smithi001.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T01:36:47.332 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T01:36:47.332 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.332 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.332 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.332 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-12T01:36:47.332 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.332 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.332 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.332 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.332 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.332 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.332 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.333 INFO:teuthology.orchestra.run.smithi001.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T01:36:47.333 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T01:36:47.333 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.333 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.333 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.333 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.333 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.333 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.333 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.333 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.333 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.333 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.333 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.333 INFO:teuthology.orchestra.run.smithi001.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T01:36:47.334 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T01:36:47.334 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.334 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.334 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.334 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.334 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.334 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.334 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.334 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.334 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.334 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.334 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.334 INFO:teuthology.orchestra.run.smithi001.stdout: "AUDIT_API_ENABLED": { 2024-09-12T01:36:47.334 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T01:36:47.335 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.335 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.335 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.335 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.335 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.335 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.335 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.335 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.335 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.335 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.335 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.335 INFO:teuthology.orchestra.run.smithi001.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T01:36:47.335 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T01:36:47.336 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.336 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.336 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.336 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.336 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.336 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.336 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.336 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.336 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.336 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.336 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.336 INFO:teuthology.orchestra.run.smithi001.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T01:36:47.336 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T01:36:47.337 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.337 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.337 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.337 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.337 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.337 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.337 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.337 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.337 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.337 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.337 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.337 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T01:36:47.337 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T01:36:47.337 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.338 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.338 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.338 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.338 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.338 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.338 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.338 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.338 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.338 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.338 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.338 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T01:36:47.338 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T01:36:47.339 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.339 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.339 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.339 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.339 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.339 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.339 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.339 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.339 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.339 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.339 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.339 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T01:36:47.339 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T01:36:47.340 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.340 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.340 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.340 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.340 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.340 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.340 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.340 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.340 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.340 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.340 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.340 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T01:36:47.340 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T01:36:47.340 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.341 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.341 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.341 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.341 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.341 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.341 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.341 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.341 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.341 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.341 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.341 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T01:36:47.341 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T01:36:47.342 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.342 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.342 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.342 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.342 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.342 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.342 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.342 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.342 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.342 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.342 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.342 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T01:36:47.342 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T01:36:47.342 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.343 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.343 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.343 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.343 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.343 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.343 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.343 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.343 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.343 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.343 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.343 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T01:36:47.343 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T01:36:47.344 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.344 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.344 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.344 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.344 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.344 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.344 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.344 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.344 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.344 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.344 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.344 INFO:teuthology.orchestra.run.smithi001.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T01:36:47.345 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T01:36:47.345 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.345 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.345 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.345 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.345 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.345 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.345 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.345 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.345 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.345 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.345 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.346 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T01:36:47.346 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T01:36:47.346 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.346 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.346 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.346 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-09-12T01:36:47.346 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.346 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.346 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.346 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.346 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.346 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.346 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.347 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T01:36:47.347 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T01:36:47.347 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.347 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.347 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.347 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.347 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.347 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.347 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.347 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.347 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.347 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.347 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.348 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_URL": { 2024-09-12T01:36:47.348 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_URL", 2024-09-12T01:36:47.348 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.348 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.348 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.348 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.348 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.348 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.348 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.348 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.348 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.348 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.348 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.349 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T01:36:47.349 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T01:36:47.349 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.349 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.349 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.349 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-09-12T01:36:47.349 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.349 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.349 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.349 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.349 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.349 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.349 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.350 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T01:36:47.350 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T01:36:47.350 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.350 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.350 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.350 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.350 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.350 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.350 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.350 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.350 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.350 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.350 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.350 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T01:36:47.350 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T01:36:47.351 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.351 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.351 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.351 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.351 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.351 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.351 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.351 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.351 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.351 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.351 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.351 INFO:teuthology.orchestra.run.smithi001.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T01:36:47.352 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T01:36:47.352 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.352 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.352 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.352 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.352 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.352 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.352 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.352 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.352 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.352 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.352 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.352 INFO:teuthology.orchestra.run.smithi001.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T01:36:47.353 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T01:36:47.353 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.353 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.353 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.353 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.353 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.353 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.353 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.353 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.353 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.353 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.353 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.353 INFO:teuthology.orchestra.run.smithi001.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T01:36:47.354 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T01:36:47.354 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.354 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.354 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.354 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.354 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.354 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.354 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.354 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.354 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.354 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.354 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.355 INFO:teuthology.orchestra.run.smithi001.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T01:36:47.355 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T01:36:47.355 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.355 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.355 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.355 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.355 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.355 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.355 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.355 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.355 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.355 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.355 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.355 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T01:36:47.356 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T01:36:47.356 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.356 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.356 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.356 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.356 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.356 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.356 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.356 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.356 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.356 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.356 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.356 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T01:36:47.357 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T01:36:47.357 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.357 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.357 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.357 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.357 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.357 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.357 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.357 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.357 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.357 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.357 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.357 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T01:36:47.358 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T01:36:47.358 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.358 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.358 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.358 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.358 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.358 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.358 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.358 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.358 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.358 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.358 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.359 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T01:36:47.359 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T01:36:47.359 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.359 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.359 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.359 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.359 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.359 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.359 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.359 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.359 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.359 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.359 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.359 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T01:36:47.360 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T01:36:47.360 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.360 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.360 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.360 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.360 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.360 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.360 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.360 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.360 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.360 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.360 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.360 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T01:36:47.361 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T01:36:47.361 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.361 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.361 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.361 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.361 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.361 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.361 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.361 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.361 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.361 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.361 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.361 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T01:36:47.362 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T01:36:47.362 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.362 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.362 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.362 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.362 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.362 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.362 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.362 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.362 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.362 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.362 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.363 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T01:36:47.363 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T01:36:47.363 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.363 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.363 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.363 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.363 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.363 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.363 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.363 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.363 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.363 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.363 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.363 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T01:36:47.364 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T01:36:47.364 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.364 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.364 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.364 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T01:36:47.364 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.364 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.364 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.364 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.364 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.364 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.364 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.365 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T01:36:47.365 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T01:36:47.365 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.365 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.365 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.365 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-12T01:36:47.365 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.365 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.365 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.365 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.365 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.365 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.365 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.366 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T01:36:47.366 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T01:36:47.366 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.366 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.366 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.366 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8", 2024-09-12T01:36:47.366 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.366 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.366 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.366 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.366 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.366 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.366 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.367 INFO:teuthology.orchestra.run.smithi001.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T01:36:47.367 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T01:36:47.367 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.367 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.367 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.367 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "45", 2024-09-12T01:36:47.367 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.367 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.367 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.367 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.367 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.367 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.367 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.368 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T01:36:47.368 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T01:36:47.368 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.368 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.368 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.368 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.368 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.368 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.368 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.368 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.368 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.368 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.368 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.369 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T01:36:47.369 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T01:36:47.369 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.369 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.369 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.369 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-09-12T01:36:47.369 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.369 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.369 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.369 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.369 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.369 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.369 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.370 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T01:36:47.370 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T01:36:47.370 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.370 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.370 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.370 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.370 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.370 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.370 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.370 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.370 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.370 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.370 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.371 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T01:36:47.371 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T01:36:47.371 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.371 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.371 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.371 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.371 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.371 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.371 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.371 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.371 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.371 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.371 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.372 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T01:36:47.372 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T01:36:47.372 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.372 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.372 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.372 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-12T01:36:47.372 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.372 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.372 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.372 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.372 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.372 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.372 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.373 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T01:36:47.373 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T01:36:47.373 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.373 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.373 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.373 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-12T01:36:47.373 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.373 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.373 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.373 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.373 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.373 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.373 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.374 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T01:36:47.374 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T01:36:47.374 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.374 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.374 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.374 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-09-12T01:36:47.374 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.374 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.374 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.374 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.374 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.374 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.374 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.374 INFO:teuthology.orchestra.run.smithi001.stdout: "cross_origin_url": { 2024-09-12T01:36:47.375 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cross_origin_url", 2024-09-12T01:36:47.375 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.375 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.375 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.375 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.375 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.375 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.375 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.375 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.375 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.375 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.375 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.375 INFO:teuthology.orchestra.run.smithi001.stdout: "crt_file": { 2024-09-12T01:36:47.375 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crt_file", 2024-09-12T01:36:47.376 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.376 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.376 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.376 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.376 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.376 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.376 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.376 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.376 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.376 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.376 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.376 INFO:teuthology.orchestra.run.smithi001.stdout: "debug": { 2024-09-12T01:36:47.376 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "debug", 2024-09-12T01:36:47.377 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.377 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.377 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.377 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.377 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.377 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.377 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.377 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable/disable debug options", 2024-09-12T01:36:47.377 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.377 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.377 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.377 INFO:teuthology.orchestra.run.smithi001.stdout: "jwt_token_ttl": { 2024-09-12T01:36:47.377 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "jwt_token_ttl", 2024-09-12T01:36:47.378 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.378 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.378 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.378 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "28800", 2024-09-12T01:36:47.378 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.378 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.378 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.378 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.378 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.378 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.378 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.378 INFO:teuthology.orchestra.run.smithi001.stdout: "key_file": { 2024-09-12T01:36:47.378 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "key_file", 2024-09-12T01:36:47.379 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.379 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.379 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.379 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.379 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.379 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.379 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.379 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.379 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.379 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.379 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.379 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.379 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.379 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.380 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.380 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.380 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.380 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.380 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.380 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.380 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.380 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.380 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.380 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.380 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.380 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.380 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.380 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.381 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.381 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.381 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.381 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.381 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.381 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.381 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.381 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.381 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.381 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.381 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.381 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.381 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.382 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.382 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.382 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.382 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.382 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.382 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.382 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.382 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.382 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.382 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.382 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.382 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.382 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.382 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.383 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.383 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.383 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.383 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.383 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.383 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.383 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.383 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.383 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.383 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.383 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.383 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.384 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.384 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.384 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.384 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.384 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.384 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.384 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.384 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.384 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.384 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.384 INFO:teuthology.orchestra.run.smithi001.stdout: "motd": { 2024-09-12T01:36:47.384 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "motd", 2024-09-12T01:36:47.384 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.385 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.385 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.385 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.385 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.385 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.385 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.385 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "The message of the day", 2024-09-12T01:36:47.385 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.385 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.385 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.385 INFO:teuthology.orchestra.run.smithi001.stdout: "redirect_resolve_ip_addr": { 2024-09-12T01:36:47.385 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T01:36:47.385 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.385 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.385 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.386 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.386 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.386 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.386 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.386 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.386 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.386 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.386 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.386 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-09-12T01:36:47.386 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-09-12T01:36:47.386 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.386 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.386 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.386 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "::", 2024-09-12T01:36:47.387 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.387 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.387 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.387 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.387 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.387 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.387 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.387 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-09-12T01:36:47.387 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-09-12T01:36:47.387 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.387 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.387 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.387 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8080", 2024-09-12T01:36:47.388 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.388 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.388 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.388 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.388 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.388 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.388 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.388 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl": { 2024-09-12T01:36:47.388 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl", 2024-09-12T01:36:47.388 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.388 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.388 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.388 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.389 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.389 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.389 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.389 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.389 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.389 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.389 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.389 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl_server_port": { 2024-09-12T01:36:47.389 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl_server_port", 2024-09-12T01:36:47.389 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.389 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.389 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.390 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8443", 2024-09-12T01:36:47.390 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.390 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.390 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.390 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.390 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.390 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.390 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.390 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_behaviour": { 2024-09-12T01:36:47.390 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_behaviour", 2024-09-12T01:36:47.390 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.390 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.390 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.390 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "redirect", 2024-09-12T01:36:47.391 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.391 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.391 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.391 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.391 INFO:teuthology.orchestra.run.smithi001.stdout: "redirect" 2024-09-12T01:36:47.391 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.391 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.391 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.391 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.391 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.391 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_error_status_code": { 2024-09-12T01:36:47.391 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_error_status_code", 2024-09-12T01:36:47.391 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.391 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.392 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.392 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "500", 2024-09-12T01:36:47.392 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "400", 2024-09-12T01:36:47.392 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "599", 2024-09-12T01:36:47.392 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.392 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.392 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.392 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.392 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.392 INFO:teuthology.orchestra.run.smithi001.stdout: "url_prefix": { 2024-09-12T01:36:47.392 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "url_prefix", 2024-09-12T01:36:47.392 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.392 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.393 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.393 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.393 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.393 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.393 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.393 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.393 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.393 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.393 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.393 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.393 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.394 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "devicehealth", 2024-09-12T01:36:47.394 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.394 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.394 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.394 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_monitoring": { 2024-09-12T01:36:47.394 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enable_monitoring", 2024-09-12T01:36:47.394 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.394 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.394 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.394 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.394 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.394 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.394 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.394 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "monitor device health metrics", 2024-09-12T01:36:47.394 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.394 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.395 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.395 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.395 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.395 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.395 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.395 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.395 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.395 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.395 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.395 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.395 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.395 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.395 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.395 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.395 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.396 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.396 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.396 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.396 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.396 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.396 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.396 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.396 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.396 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.396 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.396 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.396 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.396 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.397 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.397 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.397 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.397 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.397 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.397 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.397 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.397 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.397 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.397 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.397 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.397 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.397 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.397 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.397 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.398 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.398 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.398 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.398 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.398 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.398 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.398 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.398 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.398 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.398 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.398 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.398 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.398 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.398 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.399 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.399 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.399 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.399 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.399 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.399 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.399 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.399 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.399 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.399 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.399 INFO:teuthology.orchestra.run.smithi001.stdout: "mark_out_threshold": { 2024-09-12T01:36:47.399 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mark_out_threshold", 2024-09-12T01:36:47.399 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:47.399 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.400 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.400 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "2419200", 2024-09-12T01:36:47.400 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.400 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.400 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.400 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T01:36:47.400 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.400 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.400 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.400 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": { 2024-09-12T01:36:47.400 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pool_name", 2024-09-12T01:36:47.400 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.401 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.401 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.401 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "device_health_metrics", 2024-09-12T01:36:47.401 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.401 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.401 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.401 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T01:36:47.401 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.401 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.401 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.401 INFO:teuthology.orchestra.run.smithi001.stdout: "retention_period": { 2024-09-12T01:36:47.401 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "retention_period", 2024-09-12T01:36:47.402 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:47.402 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.402 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.402 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15552000", 2024-09-12T01:36:47.402 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.402 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.402 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.402 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long to retain device health metrics", 2024-09-12T01:36:47.402 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.402 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.402 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.402 INFO:teuthology.orchestra.run.smithi001.stdout: "scrape_frequency": { 2024-09-12T01:36:47.402 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "scrape_frequency", 2024-09-12T01:36:47.402 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:47.403 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.403 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.403 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "86400", 2024-09-12T01:36:47.403 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.403 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.403 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.403 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T01:36:47.403 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.403 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.403 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.403 INFO:teuthology.orchestra.run.smithi001.stdout: "self_heal": { 2024-09-12T01:36:47.403 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "self_heal", 2024-09-12T01:36:47.404 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.404 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.404 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.404 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.404 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.404 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.404 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.404 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T01:36:47.404 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.404 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.404 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.404 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-12T01:36:47.405 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-12T01:36:47.405 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:47.405 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.405 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.405 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-12T01:36:47.405 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.405 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.405 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.405 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T01:36:47.405 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.405 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.405 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.405 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_threshold": { 2024-09-12T01:36:47.406 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_threshold", 2024-09-12T01:36:47.406 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:47.406 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.406 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.406 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "7257600", 2024-09-12T01:36:47.406 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.406 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.406 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.406 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T01:36:47.406 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.406 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.406 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.406 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.406 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.407 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "diskprediction_local", 2024-09-12T01:36:47.407 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.407 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.407 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.407 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.407 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.407 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.407 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.407 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.407 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.407 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.408 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.408 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.408 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.408 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.408 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.408 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.408 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.408 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.408 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.408 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.408 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.408 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.408 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.409 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.409 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.409 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.409 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.409 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.409 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.409 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.409 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.409 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.409 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.409 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.409 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.409 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.410 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.410 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.410 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.410 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.410 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.410 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.410 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.410 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.410 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.410 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.410 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.410 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.410 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.411 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.411 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.411 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.411 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.411 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.411 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.411 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.411 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.411 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.411 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.411 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.411 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.411 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.411 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.412 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.412 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.412 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.412 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.412 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.412 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.412 INFO:teuthology.orchestra.run.smithi001.stdout: "predict_interval": { 2024-09-12T01:36:47.412 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "predict_interval", 2024-09-12T01:36:47.412 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.412 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.412 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.412 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "86400", 2024-09-12T01:36:47.412 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.413 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.413 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.413 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.413 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.413 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.413 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.413 INFO:teuthology.orchestra.run.smithi001.stdout: "predictor_model": { 2024-09-12T01:36:47.413 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "predictor_model", 2024-09-12T01:36:47.413 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.413 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.413 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.413 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "prophetstor", 2024-09-12T01:36:47.413 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.413 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.414 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.414 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.414 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.414 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.414 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.414 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-12T01:36:47.414 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-12T01:36:47.414 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.414 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.414 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.414 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-12T01:36:47.414 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.414 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.415 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.415 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.415 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.415 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.415 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.415 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.415 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.415 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "influx", 2024-09-12T01:36:47.415 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": false, 2024-09-12T01:36:47.415 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "influxdb python module not found", 2024-09-12T01:36:47.415 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.415 INFO:teuthology.orchestra.run.smithi001.stdout: "batch_size": { 2024-09-12T01:36:47.415 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "batch_size", 2024-09-12T01:36:47.416 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.416 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.416 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.416 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5000", 2024-09-12T01:36:47.416 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.416 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.416 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.416 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T01:36:47.416 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.416 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.416 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.416 INFO:teuthology.orchestra.run.smithi001.stdout: "database": { 2024-09-12T01:36:47.416 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "database", 2024-09-12T01:36:47.417 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.417 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.417 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.417 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "ceph", 2024-09-12T01:36:47.417 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.417 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.417 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.417 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T01:36:47.417 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.417 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.417 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.417 INFO:teuthology.orchestra.run.smithi001.stdout: "hostname": { 2024-09-12T01:36:47.417 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "hostname", 2024-09-12T01:36:47.418 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.418 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.418 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.418 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.418 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.418 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.418 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.418 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB server hostname", 2024-09-12T01:36:47.418 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.418 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.418 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.418 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-12T01:36:47.418 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-12T01:36:47.419 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:47.419 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.419 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.419 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "30", 2024-09-12T01:36:47.419 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "5", 2024-09-12T01:36:47.419 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.419 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.419 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T01:36:47.419 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.419 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.419 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.419 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.419 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.420 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.420 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.420 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.420 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.420 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.420 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.420 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.420 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.420 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.420 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.420 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.420 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.420 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.421 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.421 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.421 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.421 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.421 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.421 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.421 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.421 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.421 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.421 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.421 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.421 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.421 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.422 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.422 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.422 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.422 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.422 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.422 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.422 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.422 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.422 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.422 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.422 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.422 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.423 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.423 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.423 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.423 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.423 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.423 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.423 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.423 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.423 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.423 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.423 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.423 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.423 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.423 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.424 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.424 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.424 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.424 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.424 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.424 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.424 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.424 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.424 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.424 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.424 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.424 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.425 INFO:teuthology.orchestra.run.smithi001.stdout: "password": { 2024-09-12T01:36:47.425 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "password", 2024-09-12T01:36:47.425 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.425 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.425 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.425 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.425 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.425 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.425 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.425 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "password of InfluxDB server user", 2024-09-12T01:36:47.425 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.425 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.426 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.426 INFO:teuthology.orchestra.run.smithi001.stdout: "port": { 2024-09-12T01:36:47.426 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "port", 2024-09-12T01:36:47.426 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.426 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.426 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.426 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8086", 2024-09-12T01:36:47.426 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.426 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.426 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.426 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB server port", 2024-09-12T01:36:47.426 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.426 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.426 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.427 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl": { 2024-09-12T01:36:47.427 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl", 2024-09-12T01:36:47.427 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.427 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.427 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.427 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "false", 2024-09-12T01:36:47.427 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.427 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.427 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.427 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T01:36:47.427 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.427 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.427 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.428 INFO:teuthology.orchestra.run.smithi001.stdout: "threads": { 2024-09-12T01:36:47.428 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "threads", 2024-09-12T01:36:47.428 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.428 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.428 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.428 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-09-12T01:36:47.428 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-09-12T01:36:47.428 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "32", 2024-09-12T01:36:47.428 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.428 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T01:36:47.428 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.428 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.428 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.429 INFO:teuthology.orchestra.run.smithi001.stdout: "username": { 2024-09-12T01:36:47.429 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "username", 2024-09-12T01:36:47.429 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.429 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.429 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.429 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.429 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.429 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.429 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.429 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "username of InfluxDB server user", 2024-09-12T01:36:47.429 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.429 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.429 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.430 INFO:teuthology.orchestra.run.smithi001.stdout: "verify_ssl": { 2024-09-12T01:36:47.430 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "verify_ssl", 2024-09-12T01:36:47.430 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.430 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.430 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.430 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "true", 2024-09-12T01:36:47.430 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.430 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.430 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.430 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T01:36:47.430 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.430 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.430 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.430 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.431 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.431 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "insights", 2024-09-12T01:36:47.431 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.431 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.431 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.431 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.431 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.431 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.431 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.431 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.431 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.431 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.432 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.432 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.432 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.432 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.432 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.432 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.432 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.432 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.432 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.432 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.432 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.432 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.432 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.433 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.433 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.433 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.433 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.433 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.433 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.433 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.433 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.433 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.433 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.433 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.433 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.433 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.434 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.434 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.434 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.434 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.434 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.434 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.434 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.434 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.434 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.434 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.434 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.434 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.435 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.435 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.435 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.435 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.435 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.435 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.435 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.435 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.435 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.435 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.435 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.435 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.435 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.436 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.436 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.436 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.436 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.436 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.436 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.436 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.436 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.436 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.436 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.436 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.436 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "iostat", 2024-09-12T01:36:47.436 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.437 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.437 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.437 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.437 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.437 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.437 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.437 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.437 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.437 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.437 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.437 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.437 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.437 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.438 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.438 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.438 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.438 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.438 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.438 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.438 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.438 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.438 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.438 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.438 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.438 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.438 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.439 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.439 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.439 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.439 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.439 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.439 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.439 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.439 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.439 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.439 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.439 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.439 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.439 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.440 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.440 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.440 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.440 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.440 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.440 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.440 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.440 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.440 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.440 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.440 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.440 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.440 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.440 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.441 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.441 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.441 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.441 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.441 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.441 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.441 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.441 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.441 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.441 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.441 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.441 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.441 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.442 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.442 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.442 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.442 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.442 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.442 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "k8sevents", 2024-09-12T01:36:47.442 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.442 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.442 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.442 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_event_retention_days": { 2024-09-12T01:36:47.442 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ceph_event_retention_days", 2024-09-12T01:36:47.442 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.442 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.443 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.443 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "7", 2024-09-12T01:36:47.443 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.443 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.443 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.443 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T01:36:47.443 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.443 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.443 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.443 INFO:teuthology.orchestra.run.smithi001.stdout: "config_check_secs": { 2024-09-12T01:36:47.443 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_check_secs", 2024-09-12T01:36:47.443 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.443 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.443 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.444 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-12T01:36:47.444 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "10", 2024-09-12T01:36:47.444 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.444 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.444 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T01:36:47.444 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.444 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.444 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.444 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.444 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.444 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.444 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.444 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.445 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.445 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.445 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.445 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.445 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.445 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.445 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.445 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.445 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.445 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.445 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.445 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.445 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.446 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.446 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.446 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.446 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.446 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.446 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.446 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.446 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.446 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.446 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.446 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.446 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.446 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.447 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.447 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.447 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.447 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.447 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.447 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.447 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.447 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.447 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.447 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.447 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.447 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.447 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.448 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.448 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.448 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.448 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.448 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.448 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.448 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.448 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.448 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.448 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.448 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.448 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.448 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.449 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.449 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.449 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.449 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.449 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.449 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.449 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.449 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.449 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.449 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.449 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.449 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "localpool", 2024-09-12T01:36:47.450 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.450 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.450 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.450 INFO:teuthology.orchestra.run.smithi001.stdout: "failure_domain": { 2024-09-12T01:36:47.450 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "failure_domain", 2024-09-12T01:36:47.450 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.450 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.450 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.450 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "host", 2024-09-12T01:36:47.450 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.450 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.450 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.450 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "failure domain for any created local pool", 2024-09-12T01:36:47.450 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T01:36:47.451 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.451 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.451 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.451 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.451 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.451 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.451 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.451 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.451 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.451 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.451 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.451 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.451 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.452 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.452 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.452 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.452 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.452 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.452 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.452 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.452 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.452 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.452 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.452 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.452 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.452 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.453 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.453 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.453 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.453 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.453 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.453 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.453 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.453 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.453 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.453 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.453 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.453 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.453 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.454 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.454 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.454 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.454 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.454 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.454 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.454 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.454 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.454 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.454 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.454 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.454 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.454 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.455 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.455 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.455 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.455 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.455 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.455 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.455 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.455 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.455 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.455 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.455 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.455 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.455 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.456 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.456 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.456 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.456 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": { 2024-09-12T01:36:47.456 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "min_size", 2024-09-12T01:36:47.456 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.456 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.456 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.456 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.456 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.456 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.456 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.456 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default min_size for any created local pool", 2024-09-12T01:36:47.456 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T01:36:47.457 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.457 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.457 INFO:teuthology.orchestra.run.smithi001.stdout: "num_rep": { 2024-09-12T01:36:47.457 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "num_rep", 2024-09-12T01:36:47.457 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.457 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.457 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.457 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3", 2024-09-12T01:36:47.457 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.457 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.457 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.457 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default replica count for any created local pool", 2024-09-12T01:36:47.457 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.457 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.458 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.458 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": { 2024-09-12T01:36:47.458 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pg_num", 2024-09-12T01:36:47.458 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.458 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.458 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.458 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "128", 2024-09-12T01:36:47.458 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.458 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.458 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.458 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T01:36:47.459 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.459 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.459 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.459 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.459 INFO:teuthology.orchestra.run.smithi001.stdout: "prefix": { 2024-09-12T01:36:47.459 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prefix", 2024-09-12T01:36:47.459 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.459 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.459 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.459 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.459 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.459 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.459 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.459 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "name prefix for any created local pool", 2024-09-12T01:36:47.459 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.460 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.460 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.460 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.460 INFO:teuthology.orchestra.run.smithi001.stdout: "subtree": { 2024-09-12T01:36:47.460 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "subtree", 2024-09-12T01:36:47.460 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.460 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.460 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.460 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "rack", 2024-09-12T01:36:47.460 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.460 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.460 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.460 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T01:36:47.460 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T01:36:47.461 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.461 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.461 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.461 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.461 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mds_autoscaler", 2024-09-12T01:36:47.461 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.461 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.461 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.461 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.461 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.461 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.461 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.461 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.462 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.462 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.462 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.462 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.462 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.462 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.462 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.462 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.462 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.462 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.462 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.462 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.462 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.462 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.462 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.463 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.463 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.463 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.463 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.463 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.463 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.463 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.463 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.463 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.463 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.463 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.463 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.464 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.464 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.464 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.464 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.464 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.464 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.464 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.464 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.464 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.464 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.464 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.464 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.464 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.464 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.465 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.465 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.465 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.465 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.465 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.465 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.465 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.465 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.465 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.465 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.465 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.465 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.466 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.466 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.466 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.466 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.466 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.466 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.466 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.466 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.466 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.466 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.466 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.466 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mirroring", 2024-09-12T01:36:47.466 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.467 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.467 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.467 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.467 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.467 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.467 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.467 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.467 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.467 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.467 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.467 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.467 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.467 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.467 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.468 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.468 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.468 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.468 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.468 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.468 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.468 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.468 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.468 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.468 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.468 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.468 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.468 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.469 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.469 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.469 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.469 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.469 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.469 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.469 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.469 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.469 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.469 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.469 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.469 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.470 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.470 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.470 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.470 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.470 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.470 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.470 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.470 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.470 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.470 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.470 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.470 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.470 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.470 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.471 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.471 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.471 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.471 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.471 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.471 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.471 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.471 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.471 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.471 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.471 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.471 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.471 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.472 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.472 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.472 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.472 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.472 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "nfs", 2024-09-12T01:36:47.472 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.472 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.472 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.472 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.472 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.472 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.472 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.473 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.473 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.473 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.473 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.473 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.473 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.473 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.473 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.473 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.473 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.473 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.473 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.473 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.474 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.474 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.474 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.474 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.474 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.474 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.474 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.474 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.474 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.474 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.474 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.474 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.475 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.475 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.475 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.475 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.475 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.475 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.475 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.475 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.475 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.475 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.475 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.475 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.475 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.475 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.476 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.476 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.476 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.476 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.476 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.476 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.476 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.476 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.476 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.476 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.476 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.476 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.477 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.477 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.477 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.477 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.477 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.477 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.477 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.477 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.477 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.477 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.477 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.477 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.477 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.478 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.478 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "orchestrator", 2024-09-12T01:36:47.478 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.478 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.478 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.478 INFO:teuthology.orchestra.run.smithi001.stdout: "fail_fs": { 2024-09-12T01:36:47.478 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "fail_fs", 2024-09-12T01:36:47.478 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.478 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.478 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.478 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.478 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.478 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.478 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.479 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T01:36:47.479 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.479 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.479 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.479 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.479 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.479 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.479 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.479 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.479 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.479 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.479 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.479 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.480 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.480 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.480 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.480 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.480 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.480 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.480 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.480 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.480 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.480 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.480 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.480 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.480 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.481 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.481 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.481 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.481 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.481 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.481 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.481 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.481 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.481 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.481 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.481 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.481 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.481 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.482 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.482 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.482 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.482 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.482 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.482 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.482 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.482 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.482 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.482 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.482 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.482 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.482 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.482 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.483 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.483 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.483 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.483 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.483 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.483 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.483 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.483 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.483 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.483 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.483 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.483 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.483 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.484 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.484 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.484 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.484 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.484 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator": { 2024-09-12T01:36:47.484 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "orchestrator", 2024-09-12T01:36:47.484 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.484 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.484 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.484 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.484 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.484 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.484 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.485 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm", 2024-09-12T01:36:47.485 INFO:teuthology.orchestra.run.smithi001.stdout: "rook", 2024-09-12T01:36:47.485 INFO:teuthology.orchestra.run.smithi001.stdout: "test_orchestrator" 2024-09-12T01:36:47.485 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.485 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Orchestrator backend", 2024-09-12T01:36:47.485 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.485 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.485 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.485 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.485 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.485 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "osd_perf_query", 2024-09-12T01:36:47.485 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.486 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.486 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.486 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.486 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.486 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.486 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.486 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.486 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.486 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.486 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.486 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.486 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.486 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.486 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.487 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.487 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.487 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.487 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.487 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.487 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.487 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.487 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.487 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.487 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.487 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.487 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.487 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.488 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.488 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.488 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.488 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.488 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.488 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.488 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.488 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.488 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.488 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.488 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.488 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.488 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.489 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.489 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.489 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.489 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.489 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.489 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.489 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.489 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.489 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.489 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.489 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.489 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.489 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.489 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.490 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.490 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.490 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.490 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.490 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.490 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.490 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.490 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.490 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.490 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.490 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.490 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.490 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.490 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.490 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.491 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.491 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.491 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "osd_support", 2024-09-12T01:36:47.491 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.491 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.491 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.491 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.491 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.491 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.491 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.491 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.491 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.491 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.491 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.492 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.492 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.492 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.492 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.492 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.492 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.492 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.492 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.492 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.492 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.492 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.492 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.492 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.492 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.493 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.493 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.493 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.493 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.493 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.493 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.493 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.493 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.493 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.493 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.493 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.493 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.493 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.494 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.494 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.494 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.494 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.494 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.494 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.494 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.494 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.494 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.494 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.494 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.494 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.494 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.494 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.494 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.495 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.495 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.495 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.495 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.495 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.495 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.495 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.495 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.495 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.495 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.495 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.495 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.495 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.495 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.496 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.496 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.496 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.496 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.496 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.496 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.496 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.496 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.496 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pg_autoscaler", 2024-09-12T01:36:47.496 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.496 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.496 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.497 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.497 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.497 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.497 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.497 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.497 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.497 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.497 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.497 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.497 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.497 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.497 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.497 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.497 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.497 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.498 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.498 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.498 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.498 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.498 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.498 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.498 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.498 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.498 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.498 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.498 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.498 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.498 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.499 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.499 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.499 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.499 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.499 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.499 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.499 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.499 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.499 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.499 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.499 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.499 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.499 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.500 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.500 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.500 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.500 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.500 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.500 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.500 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.500 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.500 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.500 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.500 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.500 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.501 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.501 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.501 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.501 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.501 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.501 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.501 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.501 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.501 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.501 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.501 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.501 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.502 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.502 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.502 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.502 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-12T01:36:47.502 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-12T01:36:47.502 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:47.502 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.502 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.502 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-12T01:36:47.502 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.502 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.502 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.502 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.503 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.503 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.503 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.503 INFO:teuthology.orchestra.run.smithi001.stdout: "threshold": { 2024-09-12T01:36:47.503 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "threshold", 2024-09-12T01:36:47.503 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-12T01:36:47.503 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.503 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.503 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3.0", 2024-09-12T01:36:47.503 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1.0", 2024-09-12T01:36:47.503 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.503 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.504 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "scaling threshold", 2024-09-12T01:36:47.504 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T01:36:47.504 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.504 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.504 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.504 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.504 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.504 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "progress", 2024-09-12T01:36:47.504 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.504 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.504 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.504 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_pg_recovery_event": { 2024-09-12T01:36:47.505 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_pg_recovery_event", 2024-09-12T01:36:47.505 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.505 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.505 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.505 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.505 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.505 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.505 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.505 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T01:36:47.505 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.505 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.505 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.505 INFO:teuthology.orchestra.run.smithi001.stdout: "enabled": { 2024-09-12T01:36:47.506 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enabled", 2024-09-12T01:36:47.506 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.506 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.506 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.506 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.506 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.506 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.506 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.506 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.506 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.506 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.506 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.506 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.507 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.507 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.507 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.507 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.507 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.507 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.507 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.507 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.507 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.507 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.507 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.507 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.507 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.508 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.508 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.508 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.508 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.508 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.508 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.508 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.508 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.508 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.508 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.508 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.508 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.508 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.509 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.509 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.509 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.509 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.509 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.509 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.509 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.509 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.509 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.509 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.509 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.509 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.509 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.510 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.510 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.510 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.510 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.510 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.510 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.510 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.510 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.510 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.510 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.510 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.510 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.510 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.510 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.511 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.511 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.511 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.511 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.511 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.511 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.511 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.511 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.511 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.511 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.511 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.511 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.511 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.512 INFO:teuthology.orchestra.run.smithi001.stdout: "max_completed_events": { 2024-09-12T01:36:47.512 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_completed_events", 2024-09-12T01:36:47.512 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.512 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.512 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.512 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "50", 2024-09-12T01:36:47.512 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.512 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.512 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.512 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "number of past completed events to remember", 2024-09-12T01:36:47.512 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.512 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.512 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.513 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-12T01:36:47.513 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-12T01:36:47.513 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:47.513 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.513 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.513 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-09-12T01:36:47.513 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.513 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.513 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.513 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long the module is going to sleep", 2024-09-12T01:36:47.513 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.513 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.514 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.514 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.514 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.514 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.514 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.514 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus", 2024-09-12T01:36:47.514 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.514 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.514 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.514 INFO:teuthology.orchestra.run.smithi001.stdout: "cache": { 2024-09-12T01:36:47.514 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cache", 2024-09-12T01:36:47.514 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.514 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.514 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.515 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.515 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.515 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.515 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.515 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.515 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.515 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.515 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.515 INFO:teuthology.orchestra.run.smithi001.stdout: "exclude_perf_counters": { 2024-09-12T01:36:47.515 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "exclude_perf_counters", 2024-09-12T01:36:47.515 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.515 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.516 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.516 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.516 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.516 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.516 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.516 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T01:36:47.516 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T01:36:47.516 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.516 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.516 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.516 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.516 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.516 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.517 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.517 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.517 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.517 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.517 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.517 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.517 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.517 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.517 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.517 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.517 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.517 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.517 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.517 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.518 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.518 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.518 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.518 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.518 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.518 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.518 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.518 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.518 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.518 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.518 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.518 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.518 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.519 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.519 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.519 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.519 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.519 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.519 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.519 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.519 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.519 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.519 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.519 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.519 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.519 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.520 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.520 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.520 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.520 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.520 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.520 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.520 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.520 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.520 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.520 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.520 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.520 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.520 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.521 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.521 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.521 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.521 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.521 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.521 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.521 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.521 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.521 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.521 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_stats_pools": { 2024-09-12T01:36:47.521 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_stats_pools", 2024-09-12T01:36:47.521 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.522 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.522 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.522 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.522 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.522 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.522 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.522 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.522 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.522 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.522 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.522 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T01:36:47.522 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T01:36:47.522 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.522 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.523 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.523 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "300", 2024-09-12T01:36:47.523 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.523 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.523 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.523 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.523 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.523 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.523 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.523 INFO:teuthology.orchestra.run.smithi001.stdout: "scrape_interval": { 2024-09-12T01:36:47.523 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "scrape_interval", 2024-09-12T01:36:47.523 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-12T01:36:47.524 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.524 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.524 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15.0", 2024-09-12T01:36:47.524 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.524 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.524 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.524 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.524 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.524 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.524 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.524 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-09-12T01:36:47.524 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-09-12T01:36:47.524 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.525 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.525 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.525 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "::", 2024-09-12T01:36:47.525 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.525 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.525 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.525 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T01:36:47.525 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.525 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.525 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.525 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-09-12T01:36:47.525 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-09-12T01:36:47.525 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.526 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.526 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.526 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "9283", 2024-09-12T01:36:47.526 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.526 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.526 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.526 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T01:36:47.526 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.526 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.526 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.526 INFO:teuthology.orchestra.run.smithi001.stdout: "stale_cache_strategy": { 2024-09-12T01:36:47.526 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "stale_cache_strategy", 2024-09-12T01:36:47.526 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.526 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.527 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.527 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "log", 2024-09-12T01:36:47.527 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.527 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.527 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.527 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.527 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.527 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.527 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.527 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.527 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_behaviour": { 2024-09-12T01:36:47.527 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_behaviour", 2024-09-12T01:36:47.527 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.527 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.528 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.528 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "default", 2024-09-12T01:36:47.528 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.528 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.528 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.528 INFO:teuthology.orchestra.run.smithi001.stdout: "default", 2024-09-12T01:36:47.528 INFO:teuthology.orchestra.run.smithi001.stdout: "error" 2024-09-12T01:36:47.528 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.528 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.528 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.528 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.528 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.528 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.529 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_error_status_code": { 2024-09-12T01:36:47.529 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_error_status_code", 2024-09-12T01:36:47.529 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.529 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.529 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.529 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "500", 2024-09-12T01:36:47.529 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "400", 2024-09-12T01:36:47.529 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "599", 2024-09-12T01:36:47.529 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.529 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.529 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.529 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.529 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.529 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.530 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.530 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_support", 2024-09-12T01:36:47.530 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.530 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.530 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.530 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.530 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.530 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.530 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.530 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.530 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.531 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.531 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.531 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.531 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.531 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.531 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.531 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.531 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.531 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.531 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.531 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.531 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.531 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.531 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.532 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.532 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.532 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.532 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.532 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.532 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.532 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.532 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.532 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.532 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.532 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.532 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.532 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.533 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.533 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.533 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.533 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.533 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.533 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.533 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.533 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.533 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.533 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.533 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.533 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.533 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.533 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.534 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.534 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.534 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.534 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.534 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.534 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.534 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.534 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.534 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.534 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.534 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.534 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.534 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.535 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.535 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.535 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.535 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.535 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.535 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.535 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.535 INFO:teuthology.orchestra.run.smithi001.stdout: "max_concurrent_snap_create": { 2024-09-12T01:36:47.535 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_concurrent_snap_create", 2024-09-12T01:36:47.535 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.535 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.535 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.535 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-12T01:36:47.536 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.536 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.536 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.536 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.536 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.536 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.536 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.536 INFO:teuthology.orchestra.run.smithi001.stdout: "mirror_snapshot_schedule": { 2024-09-12T01:36:47.536 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T01:36:47.536 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.536 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.536 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.537 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.537 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.537 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.537 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.537 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.537 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.537 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.537 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.537 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.537 INFO:teuthology.orchestra.run.smithi001.stdout: "trash_purge_schedule": { 2024-09-12T01:36:47.537 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "trash_purge_schedule", 2024-09-12T01:36:47.537 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.537 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.537 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.538 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.538 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.538 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.538 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.538 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.538 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.538 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.538 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.538 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.538 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.538 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.538 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "restful", 2024-09-12T01:36:47.538 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.539 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.539 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.539 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_auth": { 2024-09-12T01:36:47.539 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enable_auth", 2024-09-12T01:36:47.539 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.539 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.539 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.539 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.539 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.539 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.539 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.539 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.539 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.539 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.539 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.540 INFO:teuthology.orchestra.run.smithi001.stdout: "key_file": { 2024-09-12T01:36:47.540 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "key_file", 2024-09-12T01:36:47.540 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.540 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.540 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.540 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.540 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.540 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.540 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.540 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.540 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.540 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.540 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.541 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.541 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.541 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.541 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.541 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.541 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.541 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.541 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.541 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.541 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.541 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.541 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.541 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.541 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.542 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.542 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.542 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.542 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.542 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.542 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.542 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.542 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.542 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.542 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.542 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.542 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.542 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.542 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.543 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.543 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.543 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.543 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.543 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.543 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.543 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.543 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.543 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.543 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.543 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.543 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.543 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.544 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.544 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.544 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.544 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.544 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.544 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.544 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.544 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.544 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.544 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.544 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.544 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.544 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.544 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.545 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.545 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.545 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.545 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.545 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.545 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.545 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.545 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.545 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.545 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.545 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.545 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.545 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.545 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.546 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.546 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-09-12T01:36:47.546 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-09-12T01:36:47.546 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.546 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.546 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.546 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.546 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.546 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.546 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.546 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.546 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.546 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.547 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.547 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-09-12T01:36:47.547 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-09-12T01:36:47.547 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.547 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.547 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.547 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.547 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.547 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.547 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.547 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.547 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.547 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.547 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.548 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.548 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.548 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rgw", 2024-09-12T01:36:47.548 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.548 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.548 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.548 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.548 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.548 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.548 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.548 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.548 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.549 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.549 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.549 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.549 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.549 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.549 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.549 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.549 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.549 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.549 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.549 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.549 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.549 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.550 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.550 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.550 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.550 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.550 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.550 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.550 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.550 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.550 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.550 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.550 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.550 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.550 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.550 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.550 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.551 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.551 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.551 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.551 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.551 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.551 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.551 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.551 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.551 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.551 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.551 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.551 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.551 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.551 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.552 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.552 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.552 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.552 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.552 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.552 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.552 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.552 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.552 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.552 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.552 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.552 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.553 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.553 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.553 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.553 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.553 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.553 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.553 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.553 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.553 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.553 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.553 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rook", 2024-09-12T01:36:47.553 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.554 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.554 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.554 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.554 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.554 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.554 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.554 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.554 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.554 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.554 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.554 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.554 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.554 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.554 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.555 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.555 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.555 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.555 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.555 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.555 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.555 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.555 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.555 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.555 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.555 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.555 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.555 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.555 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.556 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.556 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.556 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.556 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.556 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.556 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.556 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.556 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.556 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.556 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.556 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.556 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.557 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.557 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.557 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.557 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.557 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.557 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.557 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.557 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.557 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.557 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.557 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.557 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.557 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.557 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.557 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.558 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.558 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.558 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.558 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.558 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.558 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.558 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.558 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.558 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.558 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.558 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.558 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.558 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.559 INFO:teuthology.orchestra.run.smithi001.stdout: "storage_class": { 2024-09-12T01:36:47.559 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "storage_class", 2024-09-12T01:36:47.559 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.559 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.559 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.559 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "local", 2024-09-12T01:36:47.559 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.559 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.559 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.559 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T01:36:47.559 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.559 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.559 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.560 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.560 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.560 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "selftest", 2024-09-12T01:36:47.560 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.560 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.560 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.560 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.560 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.560 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.560 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.560 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.560 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.561 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.561 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.561 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.561 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.561 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.561 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.561 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.561 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.561 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.561 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.561 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.561 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.561 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.561 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.562 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.562 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.562 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.562 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.562 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.562 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.562 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.562 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.562 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.562 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.562 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.562 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.562 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.563 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.563 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.563 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.563 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.563 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.563 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.563 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.563 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.563 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.563 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.563 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.563 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.564 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.564 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.564 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.564 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.564 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.564 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.564 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.564 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.564 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.564 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.564 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.564 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.564 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.565 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.565 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.565 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.565 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.565 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.565 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.565 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.565 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.565 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.565 INFO:teuthology.orchestra.run.smithi001.stdout: "roption1": { 2024-09-12T01:36:47.565 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "roption1", 2024-09-12T01:36:47.565 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.565 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.565 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.566 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.566 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.566 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.566 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.566 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.566 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.566 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.566 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.566 INFO:teuthology.orchestra.run.smithi001.stdout: "roption2": { 2024-09-12T01:36:47.566 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "roption2", 2024-09-12T01:36:47.566 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.566 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.566 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.567 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "xyz", 2024-09-12T01:36:47.567 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.567 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.567 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.567 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.567 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.567 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.567 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.567 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption1": { 2024-09-12T01:36:47.567 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption1", 2024-09-12T01:36:47.567 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.567 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.567 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.568 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.568 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.568 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.568 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.568 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.568 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.568 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.568 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.568 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption2": { 2024-09-12T01:36:47.568 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption2", 2024-09-12T01:36:47.568 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.568 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.568 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.569 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.569 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.569 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.569 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.569 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.569 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.569 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.569 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.569 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption3": { 2024-09-12T01:36:47.569 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption3", 2024-09-12T01:36:47.569 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-12T01:36:47.569 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.570 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.570 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.570 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.570 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.570 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.570 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.570 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.570 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.570 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.570 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption4": { 2024-09-12T01:36:47.570 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption4", 2024-09-12T01:36:47.570 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.570 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.571 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.571 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.571 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.571 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.571 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.571 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.571 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.571 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.571 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.571 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption5": { 2024-09-12T01:36:47.571 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption5", 2024-09-12T01:36:47.571 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.572 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.572 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.572 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.572 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.572 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.572 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.572 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.572 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.572 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.572 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.572 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption6": { 2024-09-12T01:36:47.572 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption6", 2024-09-12T01:36:47.572 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.573 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.573 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.573 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.573 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.573 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.573 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.573 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.573 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.573 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.573 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.573 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption7": { 2024-09-12T01:36:47.573 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption7", 2024-09-12T01:36:47.573 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.574 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.574 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.574 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.574 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-09-12T01:36:47.574 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "42", 2024-09-12T01:36:47.574 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.574 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.574 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.574 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.574 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.574 INFO:teuthology.orchestra.run.smithi001.stdout: "testkey": { 2024-09-12T01:36:47.574 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testkey", 2024-09-12T01:36:47.574 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.574 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.575 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.575 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.575 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.575 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.575 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.575 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.575 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.575 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.575 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.575 INFO:teuthology.orchestra.run.smithi001.stdout: "testlkey": { 2024-09-12T01:36:47.575 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testlkey", 2024-09-12T01:36:47.575 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.575 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.576 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.576 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.576 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.576 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.576 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.576 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.576 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.576 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.576 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.576 INFO:teuthology.orchestra.run.smithi001.stdout: "testnewline": { 2024-09-12T01:36:47.576 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testnewline", 2024-09-12T01:36:47.576 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.576 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.577 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.577 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.577 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.577 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.577 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.577 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.577 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.577 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.577 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.577 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.577 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.577 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snap_schedule", 2024-09-12T01:36:47.578 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.578 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.578 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.578 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_m_granularity": { 2024-09-12T01:36:47.578 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_m_granularity", 2024-09-12T01:36:47.578 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.578 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.578 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.578 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.578 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.578 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.578 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.578 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T01:36:47.578 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.579 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.579 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.579 INFO:teuthology.orchestra.run.smithi001.stdout: "dump_on_update": { 2024-09-12T01:36:47.579 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "dump_on_update", 2024-09-12T01:36:47.579 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.579 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.579 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.579 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.579 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.579 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.579 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.579 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "dump database to debug log on update", 2024-09-12T01:36:47.579 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.580 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.580 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.580 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.580 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.580 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.580 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.580 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.580 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.580 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.580 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.580 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.580 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.580 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.581 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.581 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.581 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.581 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.581 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.581 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.581 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.581 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.581 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.581 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.581 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.581 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.581 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.581 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.581 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.582 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.582 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.582 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.582 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.582 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.582 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.582 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.582 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.582 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.582 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.582 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.582 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.583 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.583 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.583 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.583 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.583 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.583 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.583 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.583 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.583 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.583 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.583 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.583 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.583 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.583 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.584 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.584 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.584 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.584 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.584 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.584 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.584 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.584 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.584 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.584 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.584 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.584 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.585 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.585 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.585 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.585 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.585 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "stats", 2024-09-12T01:36:47.585 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.585 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.585 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.585 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.585 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.585 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.585 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.586 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.586 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.586 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.586 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.586 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.586 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.586 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.586 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.586 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.586 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.586 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.586 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.586 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.586 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.587 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.587 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.587 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.587 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.587 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.587 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.587 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.587 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.587 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.587 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.587 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.587 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.587 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.587 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.588 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.588 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.588 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.588 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.588 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.588 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.588 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.588 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.588 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.588 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.588 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.588 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.588 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.589 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.589 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.589 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.589 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.589 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.589 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.589 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.589 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.589 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.589 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.589 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.590 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.590 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.590 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.590 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.590 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.590 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.590 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.590 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.590 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.590 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.590 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.590 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.591 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "status", 2024-09-12T01:36:47.591 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.591 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.591 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.591 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.591 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.591 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.591 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.591 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.591 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.591 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.591 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.591 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.591 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.592 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.592 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.592 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.592 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.592 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.592 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.592 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.592 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.592 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.592 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.592 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.592 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.592 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.593 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.593 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.593 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.593 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.593 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.593 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.593 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.593 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.593 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.593 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.593 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.593 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.593 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.594 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.594 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.594 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.594 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.594 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.594 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.594 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.594 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.594 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.594 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.594 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.594 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.594 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.594 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.595 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.595 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.595 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.595 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.595 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.595 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.595 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.595 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.595 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.595 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.595 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.595 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.595 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.596 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.596 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.596 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.596 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.596 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.596 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "telegraf", 2024-09-12T01:36:47.596 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.596 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.596 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.596 INFO:teuthology.orchestra.run.smithi001.stdout: "address": { 2024-09-12T01:36:47.596 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "address", 2024-09-12T01:36:47.596 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.597 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.597 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.597 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T01:36:47.597 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.597 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.597 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.597 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.597 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.597 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.597 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.597 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-12T01:36:47.597 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-12T01:36:47.597 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:47.598 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.598 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.598 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15", 2024-09-12T01:36:47.598 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.598 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.598 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.598 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.598 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.598 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.598 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.598 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.598 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.599 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.599 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.599 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.599 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.599 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.599 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.599 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.599 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.599 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.599 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.599 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.599 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.599 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.599 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.599 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.600 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.600 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.600 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.600 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.600 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.600 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.600 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.600 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.600 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.600 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.600 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.600 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.601 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.601 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.601 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.601 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.601 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.601 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.601 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.601 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.601 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.601 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.601 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.602 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.602 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.602 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.602 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.602 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.602 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.602 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.602 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.602 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.602 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.602 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.602 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.602 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.603 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.603 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.603 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.603 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.603 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.603 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.603 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.603 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.603 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.603 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.603 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.603 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.603 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.604 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.604 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.604 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "telemetry", 2024-09-12T01:36:47.604 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.604 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.604 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.604 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_basic": { 2024-09-12T01:36:47.604 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_basic", 2024-09-12T01:36:47.604 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.604 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.604 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.605 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.605 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.605 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.605 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.605 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T01:36:47.605 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.605 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.605 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.605 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_crash": { 2024-09-12T01:36:47.605 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_crash", 2024-09-12T01:36:47.605 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.605 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.606 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.606 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.606 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.606 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.606 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.606 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T01:36:47.606 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.606 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.606 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.606 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_device": { 2024-09-12T01:36:47.606 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_device", 2024-09-12T01:36:47.606 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.606 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.607 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.607 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.607 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.607 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.607 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.607 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T01:36:47.607 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.607 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.607 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.607 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_ident": { 2024-09-12T01:36:47.607 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_ident", 2024-09-12T01:36:47.607 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.607 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.608 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.608 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.608 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.608 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.608 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.608 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T01:36:47.608 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.608 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.608 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.608 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_perf": { 2024-09-12T01:36:47.608 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_perf", 2024-09-12T01:36:47.608 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.608 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.609 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.609 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.609 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.609 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.609 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.609 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T01:36:47.609 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.609 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.609 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.609 INFO:teuthology.orchestra.run.smithi001.stdout: "contact": { 2024-09-12T01:36:47.609 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "contact", 2024-09-12T01:36:47.609 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.609 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.610 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.610 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.610 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.610 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.610 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.610 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.610 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.610 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.610 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.610 INFO:teuthology.orchestra.run.smithi001.stdout: "description": { 2024-09-12T01:36:47.610 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "description", 2024-09-12T01:36:47.610 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.610 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.610 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.611 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.611 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.611 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.611 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.611 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.611 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.611 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.611 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.611 INFO:teuthology.orchestra.run.smithi001.stdout: "device_url": { 2024-09-12T01:36:47.611 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_url", 2024-09-12T01:36:47.611 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.611 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.612 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.612 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T01:36:47.612 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.612 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.612 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.612 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.612 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.612 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.612 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.612 INFO:teuthology.orchestra.run.smithi001.stdout: "enabled": { 2024-09-12T01:36:47.612 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enabled", 2024-09-12T01:36:47.612 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.612 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.613 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.613 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.613 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.613 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.613 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.613 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.613 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.613 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.613 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.613 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-12T01:36:47.613 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-12T01:36:47.614 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.614 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.614 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.614 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "24", 2024-09-12T01:36:47.614 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "8", 2024-09-12T01:36:47.614 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.614 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.614 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.614 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.614 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.614 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.614 INFO:teuthology.orchestra.run.smithi001.stdout: "last_opt_revision": { 2024-09-12T01:36:47.615 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "last_opt_revision", 2024-09-12T01:36:47.615 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.615 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.615 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.615 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1", 2024-09-12T01:36:47.615 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.615 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.615 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.615 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.615 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.615 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.615 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.615 INFO:teuthology.orchestra.run.smithi001.stdout: "leaderboard": { 2024-09-12T01:36:47.616 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "leaderboard", 2024-09-12T01:36:47.616 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.616 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.616 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.616 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.616 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.616 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.616 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.616 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.616 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.616 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.616 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.616 INFO:teuthology.orchestra.run.smithi001.stdout: "leaderboard_description": { 2024-09-12T01:36:47.617 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "leaderboard_description", 2024-09-12T01:36:47.617 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.617 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.617 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.617 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.617 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.617 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.617 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.617 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.617 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.617 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.617 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.618 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.618 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.618 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.618 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.618 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.618 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.618 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.618 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.618 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.618 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.618 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.618 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.618 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.618 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.619 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.619 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.619 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.619 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.619 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.619 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.619 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.619 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.619 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.619 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.619 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.619 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.619 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.620 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.620 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.620 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.620 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.620 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.620 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.620 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.620 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.620 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.620 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.620 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.620 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.620 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.620 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.621 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.621 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.621 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.621 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.621 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.621 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.621 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.621 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.621 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.621 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.621 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.621 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.621 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.621 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.622 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.622 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.622 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.622 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.622 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.622 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.622 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.622 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.622 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.622 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.622 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.622 INFO:teuthology.orchestra.run.smithi001.stdout: "organization": { 2024-09-12T01:36:47.622 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "organization", 2024-09-12T01:36:47.623 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.623 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.623 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.623 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.623 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.623 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.623 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.623 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.623 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.623 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.623 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.623 INFO:teuthology.orchestra.run.smithi001.stdout: "proxy": { 2024-09-12T01:36:47.623 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "proxy", 2024-09-12T01:36:47.624 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.624 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.624 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.624 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.624 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.624 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.624 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.624 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.624 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.624 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.624 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.624 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.624 INFO:teuthology.orchestra.run.smithi001.stdout: "url": { 2024-09-12T01:36:47.624 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "url", 2024-09-12T01:36:47.625 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.625 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.625 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.625 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T01:36:47.625 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.625 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.625 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.625 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.625 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.625 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.625 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.625 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.625 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.626 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.626 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "test_orchestrator", 2024-09-12T01:36:47.626 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.626 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.626 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.626 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.626 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.626 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.626 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.626 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.626 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.626 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.626 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.626 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.627 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.627 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.627 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.627 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.627 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.627 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.627 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.627 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.627 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.627 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.627 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.627 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.627 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.628 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.628 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.628 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.628 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.628 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.628 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.628 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.628 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.628 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.628 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.628 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.628 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.629 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.629 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.629 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.629 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.629 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.629 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.629 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.629 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.629 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.629 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.629 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.629 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.629 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.630 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.630 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.630 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.630 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.630 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.630 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.630 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.630 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.630 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.630 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.630 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.630 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.630 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.631 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.631 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.631 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.631 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.631 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.631 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.631 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.631 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.631 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "volumes", 2024-09-12T01:36:47.631 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.631 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.631 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.632 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.632 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.632 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.632 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.632 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.632 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.632 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.632 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.632 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.632 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.632 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.632 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.632 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.632 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.633 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.633 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.633 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.633 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.633 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.633 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.633 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.633 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.633 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.633 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.633 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.633 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.633 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.634 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.634 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.634 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.634 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.634 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.634 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.634 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.634 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.634 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.634 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.634 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.634 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.634 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.635 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.635 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.635 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.635 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.635 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.635 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.635 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.635 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.635 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.635 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.635 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.635 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.635 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.635 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.636 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.636 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.636 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.636 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.636 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.636 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.636 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.636 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.636 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.636 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.636 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.636 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.636 INFO:teuthology.orchestra.run.smithi001.stdout: "max_concurrent_clones": { 2024-09-12T01:36:47.637 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_concurrent_clones", 2024-09-12T01:36:47.637 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.637 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.637 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.637 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "4", 2024-09-12T01:36:47.637 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.637 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.637 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.637 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T01:36:47.637 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.637 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.637 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.638 INFO:teuthology.orchestra.run.smithi001.stdout: "periodic_async_work": { 2024-09-12T01:36:47.638 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "periodic_async_work", 2024-09-12T01:36:47.638 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.638 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.638 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.638 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.638 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.638 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.638 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.638 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Periodically check for async work", 2024-09-12T01:36:47.638 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.638 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.638 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.638 INFO:teuthology.orchestra.run.smithi001.stdout: "snapshot_clone_delay": { 2024-09-12T01:36:47.639 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snapshot_clone_delay", 2024-09-12T01:36:47.639 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.639 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.639 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.639 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-12T01:36:47.639 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.639 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.639 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.639 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T01:36:47.639 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.639 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.639 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.640 INFO:teuthology.orchestra.run.smithi001.stdout: "snapshot_clone_no_wait": { 2024-09-12T01:36:47.640 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T01:36:47.640 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.640 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.640 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.640 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.640 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.640 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.640 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.640 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T01:36:47.640 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.640 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.640 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.641 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.641 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.641 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.641 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix", 2024-09-12T01:36:47.641 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.641 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.641 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.641 INFO:teuthology.orchestra.run.smithi001.stdout: "discovery_interval": { 2024-09-12T01:36:47.641 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "discovery_interval", 2024-09-12T01:36:47.641 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-12T01:36:47.641 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.641 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.642 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "100", 2024-09-12T01:36:47.642 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.642 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.642 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.642 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.642 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.642 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.642 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.642 INFO:teuthology.orchestra.run.smithi001.stdout: "identifier": { 2024-09-12T01:36:47.642 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "identifier", 2024-09-12T01:36:47.642 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.642 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.642 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.642 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.643 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.643 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.643 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.643 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.643 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.643 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.643 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.643 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-12T01:36:47.643 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-12T01:36:47.643 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:47.643 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.643 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.644 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-12T01:36:47.644 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.644 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.644 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.644 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.644 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.644 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.644 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.644 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.644 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.644 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.644 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.645 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.645 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.645 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.645 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.645 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.645 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.645 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.645 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.645 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.645 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.645 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.645 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.646 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.646 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.646 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.646 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.646 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.646 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.646 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.646 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.646 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.646 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.646 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.646 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.647 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.647 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.647 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.647 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.647 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.647 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.647 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.647 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.647 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.647 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.647 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.647 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.648 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.648 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.648 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.648 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.648 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.648 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.648 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.648 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.648 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.648 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.648 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.648 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.648 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.649 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.649 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.649 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.649 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.649 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.649 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.649 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.649 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.649 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.649 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.649 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.649 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.649 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.650 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_host": { 2024-09-12T01:36:47.650 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_host", 2024-09-12T01:36:47.650 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.650 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.650 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.650 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.650 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.650 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.650 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.650 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.650 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.650 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.650 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.651 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_port": { 2024-09-12T01:36:47.651 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_port", 2024-09-12T01:36:47.651 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.651 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.651 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.651 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10051", 2024-09-12T01:36:47.651 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.651 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.651 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.651 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.651 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.651 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.651 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.652 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_sender": { 2024-09-12T01:36:47.652 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_sender", 2024-09-12T01:36:47.652 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.652 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.652 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.652 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T01:36:47.652 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.652 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.652 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.652 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.652 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.652 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.652 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.653 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.653 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.653 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.653 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-12T01:36:47.653 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.653 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.653 INFO:teuthology.orchestra.run.smithi001.stdout: "modules": [ 2024-09-12T01:36:47.653 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm", 2024-09-12T01:36:47.653 INFO:teuthology.orchestra.run.smithi001.stdout: "dashboard", 2024-09-12T01:36:47.653 INFO:teuthology.orchestra.run.smithi001.stdout: "iostat", 2024-09-12T01:36:47.653 INFO:teuthology.orchestra.run.smithi001.stdout: "nfs", 2024-09-12T01:36:47.653 INFO:teuthology.orchestra.run.smithi001.stdout: "restful" 2024-09-12T01:36:47.653 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.653 INFO:teuthology.orchestra.run.smithi001.stdout: "available_modules": [ 2024-09-12T01:36:47.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.654 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "alerts", 2024-09-12T01:36:47.654 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.654 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.654 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.654 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-12T01:36:47.654 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-12T01:36:47.654 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:47.654 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.654 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.654 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-12T01:36:47.654 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.654 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.655 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.655 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T01:36:47.655 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.655 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.655 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.655 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.655 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.655 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.655 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.655 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.655 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.655 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.656 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.656 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.656 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.656 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.656 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.656 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.656 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.656 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.656 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.656 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.656 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.656 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.656 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.657 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.657 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.657 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.657 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.657 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.657 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.657 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.657 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.657 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.657 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.657 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.657 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.657 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.657 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.658 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.658 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.658 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.658 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.658 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.658 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.658 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.658 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.658 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.658 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.658 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.658 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.658 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.658 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.659 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.659 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.659 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.659 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.659 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.659 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.659 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.659 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.659 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.659 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.659 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.659 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.660 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.660 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.660 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.660 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.660 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.660 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.660 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_destination": { 2024-09-12T01:36:47.660 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_destination", 2024-09-12T01:36:47.660 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.660 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.660 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.660 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.661 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.661 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.661 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.661 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Email address to send alerts to", 2024-09-12T01:36:47.661 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.661 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.661 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.661 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_from_name": { 2024-09-12T01:36:47.661 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_from_name", 2024-09-12T01:36:47.661 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.661 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.661 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.661 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "Ceph", 2024-09-12T01:36:47.661 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.662 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.662 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.662 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Email From: name", 2024-09-12T01:36:47.662 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.662 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.662 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.662 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_host": { 2024-09-12T01:36:47.662 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_host", 2024-09-12T01:36:47.662 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.662 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.662 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.662 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.662 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.663 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.663 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.663 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP server", 2024-09-12T01:36:47.663 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.663 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.663 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.663 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_password": { 2024-09-12T01:36:47.663 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_password", 2024-09-12T01:36:47.663 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.663 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.663 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.663 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.663 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.664 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.664 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.664 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Password to authenticate with", 2024-09-12T01:36:47.664 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.664 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.664 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.664 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_port": { 2024-09-12T01:36:47.664 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_port", 2024-09-12T01:36:47.664 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.664 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.664 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.664 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "465", 2024-09-12T01:36:47.664 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.665 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.665 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.665 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP port", 2024-09-12T01:36:47.665 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.665 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.665 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.665 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.668 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_sender": { 2024-09-12T01:36:47.668 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_sender", 2024-09-12T01:36:47.669 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.669 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.669 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.669 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.669 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.669 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.669 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.669 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP envelope sender", 2024-09-12T01:36:47.669 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.669 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.669 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.669 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_ssl": { 2024-09-12T01:36:47.670 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_ssl", 2024-09-12T01:36:47.670 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.670 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.670 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.670 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.670 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.670 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.670 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.670 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T01:36:47.670 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.670 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.670 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.670 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_user": { 2024-09-12T01:36:47.671 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_user", 2024-09-12T01:36:47.671 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.671 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.671 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.671 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.671 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.671 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.671 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.671 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "User to authenticate as", 2024-09-12T01:36:47.671 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.671 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.671 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.671 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.671 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.672 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "balancer", 2024-09-12T01:36:47.672 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.672 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.672 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.672 INFO:teuthology.orchestra.run.smithi001.stdout: "active": { 2024-09-12T01:36:47.672 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "active", 2024-09-12T01:36:47.672 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.672 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.672 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.672 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.672 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.672 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.672 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.673 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T01:36:47.673 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.673 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.673 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.673 INFO:teuthology.orchestra.run.smithi001.stdout: "begin_time": { 2024-09-12T01:36:47.673 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "begin_time", 2024-09-12T01:36:47.673 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.673 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.673 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.673 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0000", 2024-09-12T01:36:47.673 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.673 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.673 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.673 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T01:36:47.674 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T01:36:47.674 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.674 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.674 INFO:teuthology.orchestra.run.smithi001.stdout: "begin_weekday": { 2024-09-12T01:36:47.674 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "begin_weekday", 2024-09-12T01:36:47.674 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-12T01:36:47.674 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.674 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.674 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-12T01:36:47.674 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0", 2024-09-12T01:36:47.674 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "6", 2024-09-12T01:36:47.674 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.674 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T01:36:47.675 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T01:36:47.675 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.675 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.675 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_max_iterations": { 2024-09-12T01:36:47.675 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_max_iterations", 2024-09-12T01:36:47.675 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-12T01:36:47.675 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.675 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.675 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "25", 2024-09-12T01:36:47.675 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-09-12T01:36:47.675 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "250", 2024-09-12T01:36:47.675 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.675 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T01:36:47.676 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.676 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.676 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.676 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_metrics": { 2024-09-12T01:36:47.676 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_metrics", 2024-09-12T01:36:47.676 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.676 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.676 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.676 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T01:36:47.676 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.676 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.676 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.677 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T01:36:47.677 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T01:36:47.677 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.677 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.677 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_step": { 2024-09-12T01:36:47.677 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_step", 2024-09-12T01:36:47.677 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-12T01:36:47.677 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.677 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.677 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0.5", 2024-09-12T01:36:47.677 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0.001", 2024-09-12T01:36:47.677 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "0.999", 2024-09-12T01:36:47.677 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.677 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "aggressiveness of optimization", 2024-09-12T01:36:47.678 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T01:36:47.678 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.678 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.678 INFO:teuthology.orchestra.run.smithi001.stdout: "end_time": { 2024-09-12T01:36:47.678 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "end_time", 2024-09-12T01:36:47.678 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.678 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.678 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.678 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "2359", 2024-09-12T01:36:47.678 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.678 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.678 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.678 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T01:36:47.679 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T01:36:47.679 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.679 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.679 INFO:teuthology.orchestra.run.smithi001.stdout: "end_weekday": { 2024-09-12T01:36:47.679 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "end_weekday", 2024-09-12T01:36:47.679 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-12T01:36:47.679 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.679 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.679 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-12T01:36:47.679 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0", 2024-09-12T01:36:47.679 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "6", 2024-09-12T01:36:47.679 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.679 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T01:36:47.680 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T01:36:47.680 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.680 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.680 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.680 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.680 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.680 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.680 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.680 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.680 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.680 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.680 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.681 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.681 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.681 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.681 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.681 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.681 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.681 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.681 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.681 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.681 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.681 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.681 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.681 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.681 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.682 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.682 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.682 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.682 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.682 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.682 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.682 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.682 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.682 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.682 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.682 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.682 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.682 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.682 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.682 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.683 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.683 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.683 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.683 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.683 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.683 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.683 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.683 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.683 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.683 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.683 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.683 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.683 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.684 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.684 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.684 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.684 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.684 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.684 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.684 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.684 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.684 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.684 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.684 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.684 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.685 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.685 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.685 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.685 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.685 INFO:teuthology.orchestra.run.smithi001.stdout: "min_score": { 2024-09-12T01:36:47.685 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "min_score", 2024-09-12T01:36:47.685 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-12T01:36:47.685 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.685 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.685 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-12T01:36:47.685 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.685 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.685 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.685 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T01:36:47.685 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.686 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.686 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.686 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": { 2024-09-12T01:36:47.686 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mode", 2024-09-12T01:36:47.686 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.686 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.686 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.686 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "upmap", 2024-09-12T01:36:47.686 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.686 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.686 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.686 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-compat", 2024-09-12T01:36:47.686 INFO:teuthology.orchestra.run.smithi001.stdout: "none", 2024-09-12T01:36:47.686 INFO:teuthology.orchestra.run.smithi001.stdout: "read", 2024-09-12T01:36:47.686 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap", 2024-09-12T01:36:47.687 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap-read" 2024-09-12T01:36:47.687 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.687 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Balancer mode", 2024-09-12T01:36:47.687 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.687 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.687 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.687 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.687 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_ids": { 2024-09-12T01:36:47.687 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pool_ids", 2024-09-12T01:36:47.687 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.687 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.687 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.687 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.687 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.687 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.687 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.688 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T01:36:47.688 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.688 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.688 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.688 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-12T01:36:47.688 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-12T01:36:47.688 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:47.688 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.688 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.688 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-12T01:36:47.688 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.688 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.688 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.688 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T01:36:47.688 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.689 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.689 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.689 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap_max_deviation": { 2024-09-12T01:36:47.689 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "upmap_max_deviation", 2024-09-12T01:36:47.689 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.689 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.689 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.689 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-09-12T01:36:47.689 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-09-12T01:36:47.689 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.689 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.689 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T01:36:47.689 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T01:36:47.690 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.690 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.690 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap_max_optimizations": { 2024-09-12T01:36:47.690 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "upmap_max_optimizations", 2024-09-12T01:36:47.690 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-12T01:36:47.690 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.690 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.690 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-12T01:36:47.690 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.690 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.690 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.690 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T01:36:47.690 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.690 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.691 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.691 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.691 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.691 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cephadm", 2024-09-12T01:36:47.691 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.691 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.691 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.691 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_down_multiplier": { 2024-09-12T01:36:47.691 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_down_multiplier", 2024-09-12T01:36:47.691 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-12T01:36:47.691 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.692 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.692 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3.0", 2024-09-12T01:36:47.692 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.692 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.692 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.692 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T01:36:47.692 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.692 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.692 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.692 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_refresh_rate": { 2024-09-12T01:36:47.692 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_refresh_rate", 2024-09-12T01:36:47.692 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:47.693 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.693 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.693 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "20", 2024-09-12T01:36:47.693 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.693 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.693 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.693 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T01:36:47.693 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.693 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.693 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.693 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_starting_port": { 2024-09-12T01:36:47.693 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_starting_port", 2024-09-12T01:36:47.693 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.694 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.694 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.694 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "4721", 2024-09-12T01:36:47.694 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.694 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.694 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.694 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T01:36:47.694 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.694 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.694 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.694 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_ptrace": { 2024-09-12T01:36:47.694 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_ptrace", 2024-09-12T01:36:47.694 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.695 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.695 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.695 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.695 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.695 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.695 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.695 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T01:36:47.695 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T01:36:47.695 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.695 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.695 INFO:teuthology.orchestra.run.smithi001.stdout: "autotune_interval": { 2024-09-12T01:36:47.695 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "autotune_interval", 2024-09-12T01:36:47.695 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:47.695 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.696 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.696 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-12T01:36:47.696 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.696 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.696 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.696 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T01:36:47.696 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.696 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.696 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.696 INFO:teuthology.orchestra.run.smithi001.stdout: "autotune_memory_target_ratio": { 2024-09-12T01:36:47.696 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T01:36:47.696 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-12T01:36:47.696 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.697 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.697 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0.7", 2024-09-12T01:36:47.697 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.697 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.697 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.697 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T01:36:47.697 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.697 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.697 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.697 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm_log_destination": { 2024-09-12T01:36:47.697 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cephadm_log_destination", 2024-09-12T01:36:47.697 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.697 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.698 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.698 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.698 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.698 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.698 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.698 INFO:teuthology.orchestra.run.smithi001.stdout: "file", 2024-09-12T01:36:47.698 INFO:teuthology.orchestra.run.smithi001.stdout: "file,syslog", 2024-09-12T01:36:47.698 INFO:teuthology.orchestra.run.smithi001.stdout: "syslog" 2024-09-12T01:36:47.698 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.698 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T01:36:47.698 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.698 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.698 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.698 INFO:teuthology.orchestra.run.smithi001.stdout: "cgroups_split": { 2024-09-12T01:36:47.699 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cgroups_split", 2024-09-12T01:36:47.699 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.699 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.699 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.699 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.699 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.699 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.699 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.699 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T01:36:47.699 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.699 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.699 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.699 INFO:teuthology.orchestra.run.smithi001.stdout: "config_checks_enabled": { 2024-09-12T01:36:47.699 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_checks_enabled", 2024-09-12T01:36:47.700 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.700 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.700 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.700 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.700 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.700 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.700 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.700 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T01:36:47.700 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.700 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.700 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.700 INFO:teuthology.orchestra.run.smithi001.stdout: "config_dashboard": { 2024-09-12T01:36:47.700 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_dashboard", 2024-09-12T01:36:47.700 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.701 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.701 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.701 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.701 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.701 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.701 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.701 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T01:36:47.701 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.701 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.701 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.701 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_alertmanager": { 2024-09-12T01:36:47.701 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_alertmanager", 2024-09-12T01:36:47.701 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.702 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.702 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.702 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T01:36:47.702 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.702 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.702 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.702 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-09-12T01:36:47.702 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.702 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.702 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.702 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_base": { 2024-09-12T01:36:47.702 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_base", 2024-09-12T01:36:47.702 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.702 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.703 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.703 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T01:36:47.703 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.703 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.703 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.703 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Container image name, without the tag", 2024-09-12T01:36:47.703 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.703 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.703 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.703 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_elasticsearch": { 2024-09-12T01:36:47.703 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_elasticsearch", 2024-09-12T01:36:47.703 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.703 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.704 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.704 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T01:36:47.704 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.704 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.704 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.704 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "elasticsearch container image", 2024-09-12T01:36:47.704 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.704 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.704 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.704 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_grafana": { 2024-09-12T01:36:47.704 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_grafana", 2024-09-12T01:36:47.704 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.705 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.705 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.705 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T01:36:47.705 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.705 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.705 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.705 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-09-12T01:36:47.705 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.705 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.705 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.705 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_haproxy": { 2024-09-12T01:36:47.705 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_haproxy", 2024-09-12T01:36:47.705 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.706 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.706 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.706 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T01:36:47.706 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.706 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.706 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.706 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "HAproxy container image", 2024-09-12T01:36:47.706 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.706 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.706 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.706 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_agent": { 2024-09-12T01:36:47.706 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_agent", 2024-09-12T01:36:47.706 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.707 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.707 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.707 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T01:36:47.707 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.707 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.707 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.707 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger agent container image", 2024-09-12T01:36:47.707 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.707 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.707 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.707 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_collector": { 2024-09-12T01:36:47.707 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_collector", 2024-09-12T01:36:47.707 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.708 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.708 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.708 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T01:36:47.708 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.708 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.708 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.708 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger collector container image", 2024-09-12T01:36:47.708 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.708 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.708 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.708 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_query": { 2024-09-12T01:36:47.708 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_query", 2024-09-12T01:36:47.709 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.709 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.709 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.709 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T01:36:47.709 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.709 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.709 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.709 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger query container image", 2024-09-12T01:36:47.709 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.709 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.709 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.709 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_keepalived": { 2024-09-12T01:36:47.709 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_keepalived", 2024-09-12T01:36:47.710 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.710 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.710 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.710 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T01:36:47.710 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.710 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.710 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.710 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Keepalived container image", 2024-09-12T01:36:47.710 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.710 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.710 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.710 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_loki": { 2024-09-12T01:36:47.710 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_loki", 2024-09-12T01:36:47.711 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.711 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.711 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.711 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T01:36:47.711 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.711 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.711 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.711 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Loki container image", 2024-09-12T01:36:47.711 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.711 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.711 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.711 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_node_exporter": { 2024-09-12T01:36:47.711 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_node_exporter", 2024-09-12T01:36:47.712 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.712 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.712 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.712 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T01:36:47.712 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.712 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.712 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.712 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-09-12T01:36:47.712 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.712 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.712 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.712 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_nvmeof": { 2024-09-12T01:36:47.713 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_nvmeof", 2024-09-12T01:36:47.713 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.713 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.713 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.713 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T01:36:47.713 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.713 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.713 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.713 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Nvme-of container image", 2024-09-12T01:36:47.713 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.713 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.713 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.713 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_prometheus": { 2024-09-12T01:36:47.714 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_prometheus", 2024-09-12T01:36:47.714 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.714 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.714 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.714 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T01:36:47.714 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.714 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.714 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.714 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-09-12T01:36:47.714 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.714 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.714 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.714 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_promtail": { 2024-09-12T01:36:47.715 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_promtail", 2024-09-12T01:36:47.715 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.715 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.715 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.715 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T01:36:47.715 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.715 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.715 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.715 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Promtail container image", 2024-09-12T01:36:47.715 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.715 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.715 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.715 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_samba": { 2024-09-12T01:36:47.716 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_samba", 2024-09-12T01:36:47.716 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.716 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.716 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.716 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T01:36:47.716 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.716 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.716 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.716 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Samba/SMB container image", 2024-09-12T01:36:47.716 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.716 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.716 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.716 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_snmp_gateway": { 2024-09-12T01:36:47.717 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_snmp_gateway", 2024-09-12T01:36:47.717 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.717 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.717 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.717 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T01:36:47.717 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.717 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.717 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.717 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SNMP Gateway container image", 2024-09-12T01:36:47.717 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.717 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.717 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.717 INFO:teuthology.orchestra.run.smithi001.stdout: "container_init": { 2024-09-12T01:36:47.718 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_init", 2024-09-12T01:36:47.718 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.718 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.718 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.718 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.718 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.718 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.718 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.718 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T01:36:47.718 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.718 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.718 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.718 INFO:teuthology.orchestra.run.smithi001.stdout: "daemon_cache_timeout": { 2024-09-12T01:36:47.719 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "daemon_cache_timeout", 2024-09-12T01:36:47.719 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:47.719 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.719 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.719 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-12T01:36:47.719 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.719 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.719 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.719 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T01:36:47.719 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.719 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.719 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.719 INFO:teuthology.orchestra.run.smithi001.stdout: "default_cephadm_command_timeout": { 2024-09-12T01:36:47.720 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T01:36:47.720 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.720 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.720 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.720 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "900", 2024-09-12T01:36:47.720 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.720 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.720 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.720 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T01:36:47.720 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.720 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.720 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.721 INFO:teuthology.orchestra.run.smithi001.stdout: "default_registry": { 2024-09-12T01:36:47.721 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "default_registry", 2024-09-12T01:36:47.721 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.721 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.721 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.721 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io", 2024-09-12T01:36:47.721 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.721 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.721 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.721 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T01:36:47.721 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.721 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.721 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.722 INFO:teuthology.orchestra.run.smithi001.stdout: "device_cache_timeout": { 2024-09-12T01:36:47.722 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_cache_timeout", 2024-09-12T01:36:47.722 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:47.722 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.722 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.722 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1800", 2024-09-12T01:36:47.722 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.722 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.722 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.722 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache device inventory", 2024-09-12T01:36:47.722 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.722 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.722 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.723 INFO:teuthology.orchestra.run.smithi001.stdout: "device_enhanced_scan": { 2024-09-12T01:36:47.723 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_enhanced_scan", 2024-09-12T01:36:47.723 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.723 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.723 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.723 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.723 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.723 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.723 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.723 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T01:36:47.723 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.723 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.723 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.724 INFO:teuthology.orchestra.run.smithi001.stdout: "facts_cache_timeout": { 2024-09-12T01:36:47.724 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "facts_cache_timeout", 2024-09-12T01:36:47.724 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:47.724 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.724 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.724 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-12T01:36:47.724 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.724 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.724 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.724 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache host facts data", 2024-09-12T01:36:47.724 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.724 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.724 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.725 INFO:teuthology.orchestra.run.smithi001.stdout: "grafana_dashboards_path": { 2024-09-12T01:36:47.725 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "grafana_dashboards_path", 2024-09-12T01:36:47.725 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.725 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.725 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.725 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T01:36:47.725 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.725 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.725 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.725 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T01:36:47.725 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.725 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.725 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.726 INFO:teuthology.orchestra.run.smithi001.stdout: "host_check_interval": { 2024-09-12T01:36:47.726 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "host_check_interval", 2024-09-12T01:36:47.726 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:47.726 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.726 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.726 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-12T01:36:47.726 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.726 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.726 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.726 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to perform a host check", 2024-09-12T01:36:47.726 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.726 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.726 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.727 INFO:teuthology.orchestra.run.smithi001.stdout: "hw_monitoring": { 2024-09-12T01:36:47.727 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "hw_monitoring", 2024-09-12T01:36:47.727 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.727 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.727 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.727 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.727 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.727 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.727 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.727 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T01:36:47.727 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.727 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.727 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.728 INFO:teuthology.orchestra.run.smithi001.stdout: "inventory_list_all": { 2024-09-12T01:36:47.728 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "inventory_list_all", 2024-09-12T01:36:47.728 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.728 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.728 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.728 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.728 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.728 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.728 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.728 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T01:36:47.728 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.728 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.728 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.729 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.729 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.729 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.729 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.729 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.729 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.729 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.729 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.729 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.729 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.729 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.729 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.729 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.729 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.730 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.730 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.730 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.730 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.730 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.730 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.730 INFO:teuthology.orchestra.run.smithi001.stdout: "log_refresh_metadata": { 2024-09-12T01:36:47.730 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_refresh_metadata", 2024-09-12T01:36:47.730 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.730 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.730 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.730 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.730 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.730 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.731 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.731 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T01:36:47.731 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.731 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.731 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.731 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.731 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.731 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.731 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.731 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.731 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.731 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.732 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.732 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.732 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T01:36:47.732 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.732 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.732 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.732 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.732 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.732 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.732 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.732 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.732 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.732 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.733 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.733 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.733 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.733 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.733 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.733 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.733 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.733 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.733 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.733 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.733 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.733 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.733 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.734 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.734 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.734 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.734 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.734 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.734 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.734 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.734 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.734 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.734 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.734 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.734 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.734 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.735 INFO:teuthology.orchestra.run.smithi001.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T01:36:47.735 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T01:36:47.735 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.735 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.735 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.735 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.735 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.735 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.735 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.735 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T01:36:47.735 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.735 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.735 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.736 INFO:teuthology.orchestra.run.smithi001.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T01:36:47.736 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T01:36:47.736 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.736 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.736 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.736 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "*", 2024-09-12T01:36:47.736 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.736 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.736 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.736 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T01:36:47.736 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.736 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.737 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.737 INFO:teuthology.orchestra.run.smithi001.stdout: "max_count_per_host": { 2024-09-12T01:36:47.737 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_count_per_host", 2024-09-12T01:36:47.737 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.737 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.737 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.737 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-12T01:36:47.737 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.737 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.737 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.737 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "max number of daemons per service per host", 2024-09-12T01:36:47.737 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.737 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.738 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.738 INFO:teuthology.orchestra.run.smithi001.stdout: "max_osd_draining_count": { 2024-09-12T01:36:47.738 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_osd_draining_count", 2024-09-12T01:36:47.738 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.738 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.738 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.738 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-12T01:36:47.738 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.738 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.738 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.738 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T01:36:47.738 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.738 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.738 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.739 INFO:teuthology.orchestra.run.smithi001.stdout: "migration_current": { 2024-09-12T01:36:47.739 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "migration_current", 2024-09-12T01:36:47.739 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.739 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.739 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.739 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.739 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.739 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.739 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.739 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "internal - do not modify", 2024-09-12T01:36:47.739 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.739 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.739 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.740 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": { 2024-09-12T01:36:47.740 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mode", 2024-09-12T01:36:47.740 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.740 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.740 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.740 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "root", 2024-09-12T01:36:47.740 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.740 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.740 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.740 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm-package", 2024-09-12T01:36:47.740 INFO:teuthology.orchestra.run.smithi001.stdout: "root" 2024-09-12T01:36:47.740 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.740 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T01:36:47.741 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.741 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.741 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.741 INFO:teuthology.orchestra.run.smithi001.stdout: "oob_default_addr": { 2024-09-12T01:36:47.741 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "oob_default_addr", 2024-09-12T01:36:47.741 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.741 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.741 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.741 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "169.254.1.1", 2024-09-12T01:36:47.741 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.741 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.741 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.741 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T01:36:47.742 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.742 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.742 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.742 INFO:teuthology.orchestra.run.smithi001.stdout: "prometheus_alerts_path": { 2024-09-12T01:36:47.742 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus_alerts_path", 2024-09-12T01:36:47.742 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.742 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.742 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.742 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T01:36:47.742 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.742 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.742 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.742 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T01:36:47.742 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.743 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.743 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.743 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_insecure": { 2024-09-12T01:36:47.743 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_insecure", 2024-09-12T01:36:47.743 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.743 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.743 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.743 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.743 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.743 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.743 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.743 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T01:36:47.743 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.744 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.744 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.744 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_password": { 2024-09-12T01:36:47.744 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_password", 2024-09-12T01:36:47.744 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.744 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.744 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.744 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.744 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.744 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.744 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.744 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T01:36:47.744 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.745 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.745 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.745 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_url": { 2024-09-12T01:36:47.745 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_url", 2024-09-12T01:36:47.745 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.745 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.745 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.745 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.745 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.745 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.745 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.745 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T01:36:47.745 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.745 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.746 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.746 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_username": { 2024-09-12T01:36:47.746 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_username", 2024-09-12T01:36:47.746 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.746 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.746 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.746 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.746 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.746 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.746 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.746 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T01:36:47.746 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.746 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.747 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.747 INFO:teuthology.orchestra.run.smithi001.stdout: "secure_monitoring_stack": { 2024-09-12T01:36:47.747 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "secure_monitoring_stack", 2024-09-12T01:36:47.747 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.747 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.747 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.747 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.747 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.747 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.747 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.747 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T01:36:47.747 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.747 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.748 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.748 INFO:teuthology.orchestra.run.smithi001.stdout: "service_discovery_port": { 2024-09-12T01:36:47.748 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "service_discovery_port", 2024-09-12T01:36:47.748 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.748 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.748 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.748 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8765", 2024-09-12T01:36:47.748 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.748 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.748 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.748 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "cephadm service discovery port", 2024-09-12T01:36:47.748 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.748 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.749 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.749 INFO:teuthology.orchestra.run.smithi001.stdout: "ssh_config_file": { 2024-09-12T01:36:47.749 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssh_config_file", 2024-09-12T01:36:47.749 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.749 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.749 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.749 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.749 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.749 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.749 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.749 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T01:36:47.749 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.749 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.750 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.750 INFO:teuthology.orchestra.run.smithi001.stdout: "use_agent": { 2024-09-12T01:36:47.750 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "use_agent", 2024-09-12T01:36:47.750 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.750 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.750 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.750 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.750 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.750 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.750 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.750 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T01:36:47.750 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.750 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.750 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.751 INFO:teuthology.orchestra.run.smithi001.stdout: "use_repo_digest": { 2024-09-12T01:36:47.751 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "use_repo_digest", 2024-09-12T01:36:47.751 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.751 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.751 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.751 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.751 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.751 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.751 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.751 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T01:36:47.751 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.751 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.751 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.752 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_failed_host_check": { 2024-09-12T01:36:47.752 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_failed_host_check", 2024-09-12T01:36:47.752 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.752 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.752 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.752 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.752 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.752 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.752 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.752 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T01:36:47.752 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.752 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.752 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.753 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_stray_daemons": { 2024-09-12T01:36:47.753 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_stray_daemons", 2024-09-12T01:36:47.753 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.753 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.753 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.753 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.753 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.753 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.753 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.753 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T01:36:47.753 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.753 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.753 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.754 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_stray_hosts": { 2024-09-12T01:36:47.754 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_stray_hosts", 2024-09-12T01:36:47.754 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.754 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.754 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.754 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.754 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.754 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.754 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.754 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T01:36:47.754 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.754 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.754 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.755 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.755 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.755 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crash", 2024-09-12T01:36:47.755 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.755 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.755 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.755 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.755 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.755 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.755 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.755 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.755 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.755 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.756 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.756 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.756 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.756 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.756 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.756 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.756 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.756 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.756 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.756 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.756 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.756 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.756 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.757 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.757 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.757 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.757 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.757 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.757 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.757 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.757 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.757 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.757 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.757 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.757 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.757 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.758 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.758 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.758 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.758 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.758 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.758 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.758 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.758 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.758 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.758 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.758 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.758 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.758 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.758 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.758 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.759 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.759 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.759 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.759 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.759 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.759 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.759 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.759 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.759 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.759 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.759 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.759 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.759 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.760 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.760 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.760 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.760 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.760 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.760 INFO:teuthology.orchestra.run.smithi001.stdout: "retain_interval": { 2024-09-12T01:36:47.760 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "retain_interval", 2024-09-12T01:36:47.760 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:47.760 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.760 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.760 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "31536000", 2024-09-12T01:36:47.760 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.760 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.761 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.761 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T01:36:47.761 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.761 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.761 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.761 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_recent_interval": { 2024-09-12T01:36:47.761 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_recent_interval", 2024-09-12T01:36:47.761 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:47.761 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.761 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.761 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1209600", 2024-09-12T01:36:47.761 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.762 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.762 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.762 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T01:36:47.762 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.762 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.762 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.762 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.762 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.762 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "dashboard", 2024-09-12T01:36:47.762 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.762 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.762 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.763 INFO:teuthology.orchestra.run.smithi001.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T01:36:47.763 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T01:36:47.763 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.763 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.763 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.763 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-12T01:36:47.763 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.763 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.763 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.763 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.763 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.763 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.764 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.764 INFO:teuthology.orchestra.run.smithi001.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T01:36:47.764 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T01:36:47.764 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.764 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.764 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.764 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.764 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.764 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.764 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.764 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.764 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.765 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.765 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.765 INFO:teuthology.orchestra.run.smithi001.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T01:36:47.765 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T01:36:47.765 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.765 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.765 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.765 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.765 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.765 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.765 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.765 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.766 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.766 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.766 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.766 INFO:teuthology.orchestra.run.smithi001.stdout: "AUDIT_API_ENABLED": { 2024-09-12T01:36:47.766 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T01:36:47.766 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.766 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.766 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.766 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.766 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.766 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.766 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.766 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.767 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.767 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.767 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.767 INFO:teuthology.orchestra.run.smithi001.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T01:36:47.767 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T01:36:47.767 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.767 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.767 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.767 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.767 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.767 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.767 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.768 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.768 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.768 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.768 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.768 INFO:teuthology.orchestra.run.smithi001.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T01:36:47.768 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T01:36:47.768 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.768 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.768 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.768 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.768 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.768 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.768 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.769 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.769 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.769 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.769 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.769 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T01:36:47.769 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T01:36:47.769 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.769 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.769 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.769 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.769 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.769 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.770 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.770 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.770 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.770 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.770 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.770 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T01:36:47.770 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T01:36:47.770 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.770 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.770 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.770 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.770 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.770 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.771 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.771 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.771 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.771 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.771 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.771 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T01:36:47.771 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T01:36:47.771 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.771 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.771 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.771 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.771 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.771 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.772 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.772 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.772 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.772 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.772 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.772 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T01:36:47.772 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T01:36:47.772 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.772 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.772 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.772 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.772 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.773 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.773 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.773 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.773 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.773 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.773 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.773 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T01:36:47.773 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T01:36:47.773 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.773 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.773 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.773 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.773 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.774 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.774 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.774 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.774 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.774 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.774 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.774 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T01:36:47.774 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T01:36:47.774 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.774 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.774 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.774 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.774 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.775 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.775 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.775 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.775 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.775 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.775 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.775 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T01:36:47.775 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T01:36:47.775 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.775 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.775 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.775 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.775 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.776 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.776 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.776 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.776 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.776 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.776 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.776 INFO:teuthology.orchestra.run.smithi001.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T01:36:47.776 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T01:36:47.776 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.776 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.776 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.776 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.776 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.777 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.777 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.777 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.777 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.777 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.777 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.777 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T01:36:47.777 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T01:36:47.777 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.777 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.777 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.777 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-09-12T01:36:47.777 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.777 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.778 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.778 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.778 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.778 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.778 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.778 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T01:36:47.778 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T01:36:47.778 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.778 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.778 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.778 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.778 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.778 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.779 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.779 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.779 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.779 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.779 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.779 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_URL": { 2024-09-12T01:36:47.779 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_URL", 2024-09-12T01:36:47.779 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.779 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.779 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.779 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.779 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.779 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.779 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.780 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.780 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.780 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.780 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.780 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T01:36:47.780 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T01:36:47.780 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.780 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.780 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.780 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-09-12T01:36:47.780 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.780 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.781 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.781 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.781 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.781 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.781 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.781 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T01:36:47.781 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T01:36:47.781 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.781 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.781 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.781 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.781 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.781 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.781 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.782 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.782 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.782 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.782 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.782 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T01:36:47.782 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T01:36:47.782 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.782 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.782 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.782 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.782 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.782 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.782 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.783 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.783 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.783 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.783 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.783 INFO:teuthology.orchestra.run.smithi001.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T01:36:47.783 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T01:36:47.783 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.783 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.783 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.783 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.783 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.783 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.783 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.783 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.784 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.784 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.784 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.784 INFO:teuthology.orchestra.run.smithi001.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T01:36:47.784 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T01:36:47.784 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.784 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.784 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.784 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.784 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.784 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.784 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.785 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.785 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.785 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.785 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.785 INFO:teuthology.orchestra.run.smithi001.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T01:36:47.785 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T01:36:47.785 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.785 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.785 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.785 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.785 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.785 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.785 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.786 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.786 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.786 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.786 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.786 INFO:teuthology.orchestra.run.smithi001.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T01:36:47.786 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T01:36:47.786 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.786 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.786 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.786 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.786 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.786 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.786 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.787 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.787 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.787 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.787 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.787 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T01:36:47.787 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T01:36:47.787 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.787 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.787 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.787 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.787 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.787 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.787 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.787 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.788 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.788 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.788 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.788 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T01:36:47.788 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T01:36:47.788 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.788 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.788 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.788 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.788 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.788 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.788 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.788 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.789 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.789 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.789 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.789 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T01:36:47.789 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T01:36:47.789 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.789 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.789 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.789 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.789 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.789 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.789 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.789 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.790 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.790 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.790 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.790 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T01:36:47.790 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T01:36:47.790 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.790 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.790 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.790 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.790 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.790 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.790 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.790 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.791 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.791 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.791 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.791 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T01:36:47.791 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T01:36:47.791 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.791 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.791 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.791 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.791 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.791 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.791 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.791 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.791 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.792 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.792 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.792 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T01:36:47.792 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T01:36:47.792 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.792 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.792 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.792 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.792 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.792 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.792 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.792 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.792 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.793 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.793 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.793 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T01:36:47.793 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T01:36:47.793 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.793 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.793 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.793 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.793 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.793 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.793 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.793 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.793 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.793 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.794 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.794 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T01:36:47.794 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T01:36:47.794 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.794 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.794 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.794 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.794 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.794 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.794 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.794 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.794 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.794 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.794 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.795 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T01:36:47.795 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T01:36:47.795 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.795 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.795 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.795 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T01:36:47.795 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.795 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.795 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.795 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.795 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.795 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.796 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.796 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T01:36:47.796 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T01:36:47.796 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.796 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.796 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.796 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-12T01:36:47.796 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.796 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.796 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.796 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.796 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.796 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.797 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.797 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T01:36:47.797 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T01:36:47.797 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.797 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.797 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.797 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8", 2024-09-12T01:36:47.797 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.797 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.797 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.797 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.797 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.797 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.798 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.798 INFO:teuthology.orchestra.run.smithi001.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T01:36:47.798 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T01:36:47.798 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.798 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.798 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.798 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "45", 2024-09-12T01:36:47.798 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.798 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.798 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.798 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.798 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.798 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.799 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.799 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T01:36:47.799 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T01:36:47.799 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.799 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.799 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.799 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.799 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.799 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.799 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.799 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.799 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.799 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.799 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.800 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T01:36:47.800 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T01:36:47.800 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.800 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.800 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.800 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-09-12T01:36:47.800 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.800 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.800 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.800 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.800 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.800 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.800 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.801 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T01:36:47.801 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T01:36:47.801 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.801 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.801 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.801 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.801 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.801 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.801 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.801 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.801 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.801 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.801 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.802 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T01:36:47.802 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T01:36:47.802 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.802 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.802 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.802 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.802 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.802 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.802 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.802 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.802 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.802 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.802 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.803 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T01:36:47.803 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T01:36:47.803 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.803 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.803 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.803 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-12T01:36:47.803 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.803 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.803 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.803 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.803 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.803 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.803 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.804 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T01:36:47.804 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T01:36:47.804 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.804 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.804 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.804 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-12T01:36:47.804 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.804 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.804 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.804 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.804 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.804 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.804 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.805 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T01:36:47.805 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T01:36:47.805 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.805 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.805 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.805 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-09-12T01:36:47.805 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.805 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.805 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.805 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.805 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.805 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.805 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.806 INFO:teuthology.orchestra.run.smithi001.stdout: "cross_origin_url": { 2024-09-12T01:36:47.806 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cross_origin_url", 2024-09-12T01:36:47.806 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.806 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.806 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.806 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.806 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.806 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.806 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.806 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.806 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.806 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.806 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.806 INFO:teuthology.orchestra.run.smithi001.stdout: "crt_file": { 2024-09-12T01:36:47.807 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crt_file", 2024-09-12T01:36:47.807 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.807 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.807 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.807 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.807 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.807 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.807 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.807 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.807 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.807 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.807 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.807 INFO:teuthology.orchestra.run.smithi001.stdout: "debug": { 2024-09-12T01:36:47.808 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "debug", 2024-09-12T01:36:47.808 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.808 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.808 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.808 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.808 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.808 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.808 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.808 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable/disable debug options", 2024-09-12T01:36:47.808 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.808 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.808 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.808 INFO:teuthology.orchestra.run.smithi001.stdout: "jwt_token_ttl": { 2024-09-12T01:36:47.809 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "jwt_token_ttl", 2024-09-12T01:36:47.809 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.809 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.809 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.809 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "28800", 2024-09-12T01:36:47.809 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.809 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.809 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.809 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.809 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.809 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.809 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.810 INFO:teuthology.orchestra.run.smithi001.stdout: "key_file": { 2024-09-12T01:36:47.810 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "key_file", 2024-09-12T01:36:47.810 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.810 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.810 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.810 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.810 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.810 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.810 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.810 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.810 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.810 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.810 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.811 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.811 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.811 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.811 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.811 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.811 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.811 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.811 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.811 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.811 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.811 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.811 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.811 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.812 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.812 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.812 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.812 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.812 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.812 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.812 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.812 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.812 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.812 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.812 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.812 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.813 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.813 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.813 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.813 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.813 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.813 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.813 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.813 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.813 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.814 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.814 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.814 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.814 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.814 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.814 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.814 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.814 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.814 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.814 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.814 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.814 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.815 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.815 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.815 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.815 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.815 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.815 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.815 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.815 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.815 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.815 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.816 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.816 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.816 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.816 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.816 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.816 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.816 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.816 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.816 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.816 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.817 INFO:teuthology.orchestra.run.smithi001.stdout: "motd": { 2024-09-12T01:36:47.817 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "motd", 2024-09-12T01:36:47.817 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.817 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.817 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.817 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.817 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.817 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.817 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.818 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "The message of the day", 2024-09-12T01:36:47.818 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.818 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.818 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.818 INFO:teuthology.orchestra.run.smithi001.stdout: "redirect_resolve_ip_addr": { 2024-09-12T01:36:47.818 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T01:36:47.818 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.818 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.818 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.819 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.819 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.819 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.819 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.819 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.819 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.819 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.819 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.819 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-09-12T01:36:47.819 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-09-12T01:36:47.819 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.820 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.820 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.820 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "::", 2024-09-12T01:36:47.820 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.820 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.820 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.820 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.820 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.820 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.820 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.820 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-09-12T01:36:47.820 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-09-12T01:36:47.821 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.821 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.821 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.821 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8080", 2024-09-12T01:36:47.821 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.821 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.821 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.821 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.821 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.821 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.821 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.821 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl": { 2024-09-12T01:36:47.821 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl", 2024-09-12T01:36:47.822 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.822 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.822 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.822 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.822 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.822 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.822 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.822 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.822 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.822 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.822 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.822 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl_server_port": { 2024-09-12T01:36:47.822 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl_server_port", 2024-09-12T01:36:47.823 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.823 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.823 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.823 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8443", 2024-09-12T01:36:47.823 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.823 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.823 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.823 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.823 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.823 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.823 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.823 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_behaviour": { 2024-09-12T01:36:47.823 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_behaviour", 2024-09-12T01:36:47.824 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.824 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.824 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.824 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "redirect", 2024-09-12T01:36:47.824 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.824 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.824 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.824 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.824 INFO:teuthology.orchestra.run.smithi001.stdout: "redirect" 2024-09-12T01:36:47.824 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.824 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.824 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.824 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.824 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.825 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_error_status_code": { 2024-09-12T01:36:47.825 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_error_status_code", 2024-09-12T01:36:47.825 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.825 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.825 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.825 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "500", 2024-09-12T01:36:47.825 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "400", 2024-09-12T01:36:47.825 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "599", 2024-09-12T01:36:47.825 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.825 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.825 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.825 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.825 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.826 INFO:teuthology.orchestra.run.smithi001.stdout: "url_prefix": { 2024-09-12T01:36:47.826 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "url_prefix", 2024-09-12T01:36:47.826 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.826 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.826 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.826 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.826 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.826 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.826 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.826 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.826 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.826 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.826 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.827 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.827 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.827 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "devicehealth", 2024-09-12T01:36:47.827 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.827 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.827 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.827 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_monitoring": { 2024-09-12T01:36:47.827 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enable_monitoring", 2024-09-12T01:36:47.827 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.827 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.827 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.827 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.827 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.828 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.828 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.828 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "monitor device health metrics", 2024-09-12T01:36:47.828 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.828 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.828 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.828 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.828 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.828 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.828 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.828 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.828 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.829 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.829 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.829 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.829 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.829 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.829 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.829 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.829 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.829 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.829 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.829 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.829 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.829 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.829 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.830 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.830 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.830 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.830 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.830 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.830 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.830 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.830 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.830 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.830 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.830 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.830 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.831 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.831 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.831 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.831 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.831 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.831 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.831 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.831 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.831 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.831 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.831 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.831 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.831 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.832 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.832 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.832 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.832 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.832 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.832 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.832 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.832 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.832 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.832 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.832 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.832 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.832 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.833 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.833 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.833 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.833 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.833 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.833 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.833 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.833 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.833 INFO:teuthology.orchestra.run.smithi001.stdout: "mark_out_threshold": { 2024-09-12T01:36:47.833 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mark_out_threshold", 2024-09-12T01:36:47.833 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:47.833 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.833 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.834 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "2419200", 2024-09-12T01:36:47.834 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.834 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.834 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.834 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T01:36:47.834 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.834 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.834 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.834 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": { 2024-09-12T01:36:47.834 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pool_name", 2024-09-12T01:36:47.834 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.834 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.834 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.835 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "device_health_metrics", 2024-09-12T01:36:47.835 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.835 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.835 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.835 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T01:36:47.835 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.835 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.835 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.835 INFO:teuthology.orchestra.run.smithi001.stdout: "retention_period": { 2024-09-12T01:36:47.835 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "retention_period", 2024-09-12T01:36:47.835 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:47.835 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.835 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.836 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15552000", 2024-09-12T01:36:47.836 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.836 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.836 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.836 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long to retain device health metrics", 2024-09-12T01:36:47.836 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.836 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.836 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.836 INFO:teuthology.orchestra.run.smithi001.stdout: "scrape_frequency": { 2024-09-12T01:36:47.836 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "scrape_frequency", 2024-09-12T01:36:47.836 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:47.836 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.836 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.837 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "86400", 2024-09-12T01:36:47.837 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.837 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.837 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.837 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T01:36:47.837 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.837 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.837 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.837 INFO:teuthology.orchestra.run.smithi001.stdout: "self_heal": { 2024-09-12T01:36:47.837 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "self_heal", 2024-09-12T01:36:47.837 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.837 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.837 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.838 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.838 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.838 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.838 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.838 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T01:36:47.838 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.838 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.838 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.838 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-12T01:36:47.838 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-12T01:36:47.838 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:47.838 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.839 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.839 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-12T01:36:47.839 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.839 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.839 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.839 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T01:36:47.839 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.839 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.839 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.839 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_threshold": { 2024-09-12T01:36:47.839 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_threshold", 2024-09-12T01:36:47.839 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:47.839 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.840 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.840 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "7257600", 2024-09-12T01:36:47.840 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.840 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.840 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.840 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T01:36:47.840 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.840 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.840 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.840 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.840 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.840 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.840 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "diskprediction_local", 2024-09-12T01:36:47.841 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.841 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.841 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.841 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.841 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.841 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.841 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.841 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.841 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.841 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.841 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.841 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.841 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.841 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.842 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.842 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.842 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.842 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.842 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.842 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.842 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.842 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.842 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.842 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.842 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.842 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.842 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.843 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.843 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.843 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.843 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.843 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.843 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.843 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.843 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.843 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.843 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.843 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.843 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.843 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.844 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.844 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.844 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.844 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.844 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.844 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.844 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.844 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.844 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.844 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.844 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.844 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.844 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.844 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.845 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.845 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.845 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.845 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.845 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.845 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.845 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.845 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.845 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.845 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.845 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.845 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.845 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.846 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.846 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.846 INFO:teuthology.orchestra.run.smithi001.stdout: "predict_interval": { 2024-09-12T01:36:47.846 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "predict_interval", 2024-09-12T01:36:47.846 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.846 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.846 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.846 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "86400", 2024-09-12T01:36:47.846 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.846 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.847 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.847 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.847 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.847 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.847 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.847 INFO:teuthology.orchestra.run.smithi001.stdout: "predictor_model": { 2024-09-12T01:36:47.847 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "predictor_model", 2024-09-12T01:36:47.847 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.847 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.847 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.847 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "prophetstor", 2024-09-12T01:36:47.847 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.847 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.848 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.848 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.848 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.848 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.848 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.848 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-12T01:36:47.848 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-12T01:36:47.848 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.848 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.848 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.848 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-12T01:36:47.848 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.848 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.849 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.849 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.849 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.849 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.849 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.849 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.849 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.849 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "influx", 2024-09-12T01:36:47.849 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": false, 2024-09-12T01:36:47.849 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "influxdb python module not found", 2024-09-12T01:36:47.849 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.849 INFO:teuthology.orchestra.run.smithi001.stdout: "batch_size": { 2024-09-12T01:36:47.850 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "batch_size", 2024-09-12T01:36:47.850 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.850 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.850 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.850 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5000", 2024-09-12T01:36:47.850 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.850 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.850 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.850 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T01:36:47.850 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.850 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.850 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.850 INFO:teuthology.orchestra.run.smithi001.stdout: "database": { 2024-09-12T01:36:47.851 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "database", 2024-09-12T01:36:47.851 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.851 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.851 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.851 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "ceph", 2024-09-12T01:36:47.851 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.851 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.851 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.851 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T01:36:47.851 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.851 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.851 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.851 INFO:teuthology.orchestra.run.smithi001.stdout: "hostname": { 2024-09-12T01:36:47.851 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "hostname", 2024-09-12T01:36:47.852 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.852 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.852 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.852 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.852 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.852 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.852 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.852 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB server hostname", 2024-09-12T01:36:47.852 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.852 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.852 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.852 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-12T01:36:47.853 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-12T01:36:47.853 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:47.853 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.853 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.853 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "30", 2024-09-12T01:36:47.853 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "5", 2024-09-12T01:36:47.853 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.853 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.853 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T01:36:47.853 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.853 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.853 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.854 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.854 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.854 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.854 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.854 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.854 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.854 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.854 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.854 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.854 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.854 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.854 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.854 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.854 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.855 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.855 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.855 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.855 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.855 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.855 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.855 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.855 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.855 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.855 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.855 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.855 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.856 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.856 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.856 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.856 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.856 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.856 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.856 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.856 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.856 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.856 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.856 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.856 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.856 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.857 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.857 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.857 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.857 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.857 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.857 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.857 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.857 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.857 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.857 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.857 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.857 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.857 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.857 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.858 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.858 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.858 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.858 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.858 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.858 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.858 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.858 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.858 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.858 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.858 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.858 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.859 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.859 INFO:teuthology.orchestra.run.smithi001.stdout: "password": { 2024-09-12T01:36:47.859 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "password", 2024-09-12T01:36:47.859 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.859 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.859 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.859 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.859 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.859 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.859 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.859 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "password of InfluxDB server user", 2024-09-12T01:36:47.859 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.859 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.859 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.860 INFO:teuthology.orchestra.run.smithi001.stdout: "port": { 2024-09-12T01:36:47.860 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "port", 2024-09-12T01:36:47.860 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.860 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.860 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.860 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8086", 2024-09-12T01:36:47.860 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.860 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.860 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.860 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB server port", 2024-09-12T01:36:47.860 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.860 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.860 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.861 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl": { 2024-09-12T01:36:47.861 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl", 2024-09-12T01:36:47.861 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.861 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.861 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.861 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "false", 2024-09-12T01:36:47.861 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.861 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.861 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.861 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T01:36:47.861 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.861 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.861 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.862 INFO:teuthology.orchestra.run.smithi001.stdout: "threads": { 2024-09-12T01:36:47.862 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "threads", 2024-09-12T01:36:47.862 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.862 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.862 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.862 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-09-12T01:36:47.862 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-09-12T01:36:47.862 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "32", 2024-09-12T01:36:47.862 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.862 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T01:36:47.862 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.862 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.862 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.863 INFO:teuthology.orchestra.run.smithi001.stdout: "username": { 2024-09-12T01:36:47.863 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "username", 2024-09-12T01:36:47.863 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.863 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.863 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.863 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.863 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.863 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.863 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.863 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "username of InfluxDB server user", 2024-09-12T01:36:47.863 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.863 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.863 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.864 INFO:teuthology.orchestra.run.smithi001.stdout: "verify_ssl": { 2024-09-12T01:36:47.864 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "verify_ssl", 2024-09-12T01:36:47.864 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.864 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.864 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.864 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "true", 2024-09-12T01:36:47.864 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.864 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.864 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.864 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T01:36:47.864 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.864 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.864 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.864 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.865 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.865 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "insights", 2024-09-12T01:36:47.865 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.865 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.865 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.865 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.865 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.865 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.865 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.865 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.865 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.865 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.865 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.866 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.866 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.866 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.866 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.866 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.866 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.866 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.866 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.866 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.866 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.866 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.866 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.866 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.867 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.867 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.867 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.867 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.867 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.867 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.867 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.867 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.867 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.867 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.867 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.867 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.867 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.867 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.868 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.868 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.868 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.868 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.868 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.868 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.868 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.868 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.868 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.868 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.868 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.868 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.868 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.868 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.869 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.869 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.869 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.869 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.869 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.869 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.869 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.869 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.869 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.869 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.869 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.869 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.869 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.870 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.870 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.870 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.870 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.870 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.870 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.870 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "iostat", 2024-09-12T01:36:47.870 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.870 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.870 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.870 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.870 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.870 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.871 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.871 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.871 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.871 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.871 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.871 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.871 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.871 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.871 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.871 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.871 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.871 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.871 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.871 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.872 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.872 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.872 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.872 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.872 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.872 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.872 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.872 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.872 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.872 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.872 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.872 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.872 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.873 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.873 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.873 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.873 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.873 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.873 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.873 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.873 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.873 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.873 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.873 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.873 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.873 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.874 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.874 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.874 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.874 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.874 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.874 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.874 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.874 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.874 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.874 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.874 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.874 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.874 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.874 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.875 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.875 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.875 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.875 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.875 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.875 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.875 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.875 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.875 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.875 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.875 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.875 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "k8sevents", 2024-09-12T01:36:47.876 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.876 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.876 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.876 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_event_retention_days": { 2024-09-12T01:36:47.876 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ceph_event_retention_days", 2024-09-12T01:36:47.876 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.876 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.876 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.876 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "7", 2024-09-12T01:36:47.876 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.876 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.876 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.876 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T01:36:47.876 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.877 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.877 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.877 INFO:teuthology.orchestra.run.smithi001.stdout: "config_check_secs": { 2024-09-12T01:36:47.877 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_check_secs", 2024-09-12T01:36:47.877 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.877 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.877 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.877 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-12T01:36:47.877 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "10", 2024-09-12T01:36:47.877 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.877 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.877 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T01:36:47.877 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.878 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.878 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.878 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.878 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.878 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.878 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.878 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.878 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.878 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.878 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.878 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.878 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.878 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.878 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.879 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.879 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.879 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.879 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.879 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.879 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.879 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.879 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.879 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.879 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.879 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.879 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.879 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.880 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.880 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.880 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.880 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.880 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.880 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.880 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.880 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.880 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.880 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.880 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.880 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.880 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.881 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.881 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.881 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.881 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.881 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.881 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.881 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.881 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.881 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.881 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.881 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.881 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.881 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.881 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.882 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.882 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.882 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.882 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.882 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.882 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.882 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.882 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.882 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.882 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.882 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.882 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.882 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.882 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.883 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.883 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.883 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "localpool", 2024-09-12T01:36:47.883 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.883 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.883 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.883 INFO:teuthology.orchestra.run.smithi001.stdout: "failure_domain": { 2024-09-12T01:36:47.883 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "failure_domain", 2024-09-12T01:36:47.883 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.883 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.883 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.883 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "host", 2024-09-12T01:36:47.884 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.884 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.884 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.884 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "failure domain for any created local pool", 2024-09-12T01:36:47.884 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T01:36:47.884 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.884 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.884 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.884 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.884 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.884 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.884 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.884 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.885 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.885 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.885 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.885 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.885 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.885 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.885 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.885 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.885 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.885 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.885 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.885 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.885 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.885 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.886 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.886 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.886 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.886 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.886 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.886 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.886 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.886 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.886 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.886 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.886 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.886 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.886 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.887 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.887 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.887 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.887 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.887 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.887 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.887 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.887 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.887 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.887 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.887 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.887 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.887 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.887 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.888 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.888 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.888 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.888 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.888 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.888 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.888 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.888 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.888 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.888 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.888 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.888 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.888 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.889 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.889 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.889 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.889 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.889 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.889 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.889 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": { 2024-09-12T01:36:47.889 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "min_size", 2024-09-12T01:36:47.889 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.889 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.889 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.889 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.889 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.889 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.890 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.890 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default min_size for any created local pool", 2024-09-12T01:36:47.890 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T01:36:47.890 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.890 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.890 INFO:teuthology.orchestra.run.smithi001.stdout: "num_rep": { 2024-09-12T01:36:47.890 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "num_rep", 2024-09-12T01:36:47.890 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.890 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.890 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.890 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3", 2024-09-12T01:36:47.890 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.890 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.891 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.891 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default replica count for any created local pool", 2024-09-12T01:36:47.891 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.891 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.891 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.891 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": { 2024-09-12T01:36:47.891 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pg_num", 2024-09-12T01:36:47.891 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.891 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.891 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.891 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "128", 2024-09-12T01:36:47.891 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.891 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.891 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.892 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T01:36:47.892 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.892 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.892 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.892 INFO:teuthology.orchestra.run.smithi001.stdout: "prefix": { 2024-09-12T01:36:47.892 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prefix", 2024-09-12T01:36:47.892 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.892 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.892 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.892 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.892 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.892 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.892 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.893 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "name prefix for any created local pool", 2024-09-12T01:36:47.893 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.893 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.893 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.893 INFO:teuthology.orchestra.run.smithi001.stdout: "subtree": { 2024-09-12T01:36:47.893 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "subtree", 2024-09-12T01:36:47.893 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.893 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.893 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.893 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "rack", 2024-09-12T01:36:47.893 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.893 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.893 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.894 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T01:36:47.894 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T01:36:47.894 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.894 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.894 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.894 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.894 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mds_autoscaler", 2024-09-12T01:36:47.894 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.894 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.894 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.894 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.894 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.895 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.895 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.895 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.895 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.895 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.895 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.895 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.895 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.895 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.895 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.895 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.895 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.895 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.895 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.895 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.896 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.896 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.896 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.896 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.896 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.896 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.896 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.896 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.896 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.896 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.896 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.896 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.896 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.897 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.897 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.897 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.897 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.897 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.897 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.897 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.897 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.897 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.897 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.897 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.897 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.897 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.898 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.898 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.898 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.898 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.898 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.898 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.898 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.898 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.898 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.898 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.898 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.898 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.898 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.899 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.899 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.899 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.899 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.899 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.899 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.899 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.899 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.899 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.899 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.899 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.899 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.899 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mirroring", 2024-09-12T01:36:47.900 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.900 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.900 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.900 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.900 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.900 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.900 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.900 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.900 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.900 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.900 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.900 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.900 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.900 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.901 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.901 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.901 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.901 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.901 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.901 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.901 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.901 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.901 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.901 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.901 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.901 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.901 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.902 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.902 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.902 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.902 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.902 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.902 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.902 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.902 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.902 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.902 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.902 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.902 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.902 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.903 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.903 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.903 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.903 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.903 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.903 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.903 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.903 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.903 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.903 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.903 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.903 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.903 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.904 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.904 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.904 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.904 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.904 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.904 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.904 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.904 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.904 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.904 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.904 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.904 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.904 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.905 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.905 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.905 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.905 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.905 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.905 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.905 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "nfs", 2024-09-12T01:36:47.905 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.905 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.905 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.905 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.905 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.905 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.905 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.906 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.906 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.906 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.906 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.906 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.906 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.906 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.906 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.906 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.906 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.906 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.906 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.906 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.906 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.907 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.907 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.907 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.907 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.907 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.907 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.907 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.907 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.907 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.907 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.907 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.907 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.907 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.907 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.908 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.908 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.908 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.908 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.908 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.908 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.908 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.908 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.908 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.908 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.908 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.908 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.908 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.909 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.909 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.909 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.909 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.909 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.909 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.909 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.909 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.909 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.909 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.909 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.909 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.909 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.910 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.910 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.910 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.910 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.910 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.910 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.910 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.910 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.910 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.910 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.910 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "orchestrator", 2024-09-12T01:36:47.910 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.911 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.911 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.911 INFO:teuthology.orchestra.run.smithi001.stdout: "fail_fs": { 2024-09-12T01:36:47.911 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "fail_fs", 2024-09-12T01:36:47.911 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.911 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.911 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.911 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.911 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.911 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.911 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.911 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T01:36:47.911 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.911 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.911 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.912 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.912 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.912 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.912 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.912 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.912 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.912 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.912 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.912 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.912 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.912 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.912 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.912 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.912 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.913 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.913 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.913 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.913 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.913 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.913 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.913 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.913 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.913 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.913 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.913 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.913 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.913 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.914 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.914 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.914 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.914 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.914 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.914 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.914 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.914 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.914 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.914 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.914 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.914 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.914 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.915 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.915 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.915 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.915 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.915 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.915 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.915 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.915 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.915 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.915 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.915 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.915 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.915 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.916 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.916 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.916 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.916 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.916 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.916 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.916 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.916 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.916 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.916 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.916 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.916 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.916 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.917 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator": { 2024-09-12T01:36:47.917 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "orchestrator", 2024-09-12T01:36:47.917 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.917 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.917 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.917 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.917 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.917 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.917 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.917 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm", 2024-09-12T01:36:47.917 INFO:teuthology.orchestra.run.smithi001.stdout: "rook", 2024-09-12T01:36:47.917 INFO:teuthology.orchestra.run.smithi001.stdout: "test_orchestrator" 2024-09-12T01:36:47.917 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.917 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Orchestrator backend", 2024-09-12T01:36:47.918 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.918 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.918 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.918 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.918 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.918 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "osd_perf_query", 2024-09-12T01:36:47.918 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.918 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.918 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.918 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.918 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.918 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.919 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.919 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.919 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.919 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.919 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.919 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.919 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.919 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.919 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.919 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.919 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.919 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.919 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.919 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.920 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.920 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.920 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.920 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.920 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.920 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.920 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.920 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.920 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.920 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.920 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.920 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.920 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.921 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.921 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.921 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.921 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.921 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.921 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.921 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.921 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.921 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.921 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.921 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.921 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.921 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.921 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.922 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.922 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.922 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.922 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.922 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.922 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.922 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.922 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.922 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.922 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.922 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.922 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.922 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.923 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.923 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.923 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.923 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.923 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.923 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.923 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.923 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.923 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.923 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.923 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.923 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "osd_support", 2024-09-12T01:36:47.923 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.924 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.924 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.924 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.924 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.924 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.924 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.924 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.924 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.924 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.924 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.924 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.924 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.924 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.924 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.925 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.925 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.925 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.925 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.925 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.925 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.925 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.925 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.925 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.925 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.925 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.925 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.925 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.926 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.926 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.926 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.926 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.926 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.926 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.926 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.926 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.926 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.926 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.926 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.926 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.926 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.926 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.927 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.927 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.927 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.927 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.927 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.927 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.927 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.927 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.927 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.927 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.927 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.927 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.927 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.927 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.928 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.928 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.928 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.928 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.928 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.928 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.928 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.928 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.928 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.928 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.928 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.928 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.928 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.928 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.929 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.929 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pg_autoscaler", 2024-09-12T01:36:47.929 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.929 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.929 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.929 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.929 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.929 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.929 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.929 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.929 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.929 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.930 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.930 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.930 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.930 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.930 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.930 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.930 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.930 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.930 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.930 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.930 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.930 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.930 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.931 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.931 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.931 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.931 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.931 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.931 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.931 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.931 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.931 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.931 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.931 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.931 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.931 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.932 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.932 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.932 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.932 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.932 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.932 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.932 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.932 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.932 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.932 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.932 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.932 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.932 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.932 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.933 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.933 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.933 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.933 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.933 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.933 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.933 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.933 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.933 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.933 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.933 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.933 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.933 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.934 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.934 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.934 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.934 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.934 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.934 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.934 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-12T01:36:47.934 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-12T01:36:47.934 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:47.934 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.934 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.934 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-12T01:36:47.934 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.935 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.935 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.935 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.935 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.935 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.935 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.935 INFO:teuthology.orchestra.run.smithi001.stdout: "threshold": { 2024-09-12T01:36:47.935 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "threshold", 2024-09-12T01:36:47.935 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-12T01:36:47.935 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.935 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.935 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3.0", 2024-09-12T01:36:47.936 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1.0", 2024-09-12T01:36:47.936 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.936 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.936 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "scaling threshold", 2024-09-12T01:36:47.936 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T01:36:47.936 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.936 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.936 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.936 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.936 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "progress", 2024-09-12T01:36:47.936 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.936 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.937 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.937 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_pg_recovery_event": { 2024-09-12T01:36:47.937 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_pg_recovery_event", 2024-09-12T01:36:47.937 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.937 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.937 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.937 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.937 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.937 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.937 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.937 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T01:36:47.937 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.937 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.937 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.938 INFO:teuthology.orchestra.run.smithi001.stdout: "enabled": { 2024-09-12T01:36:47.938 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enabled", 2024-09-12T01:36:47.938 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.938 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.938 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.938 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.938 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.938 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.938 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.938 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.938 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.939 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.939 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.939 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.939 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.939 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.939 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.939 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.939 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.939 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.939 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.939 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.939 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.939 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.940 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.940 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.940 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.940 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.940 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.940 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.940 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.940 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.940 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.940 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.940 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.940 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.940 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.941 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.941 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.941 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.941 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.941 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.941 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.941 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.941 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.941 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.941 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.941 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.941 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.941 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.942 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.942 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.942 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.942 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.942 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.942 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.942 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.942 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.942 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.942 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.942 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.942 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.942 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.942 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.943 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.943 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.943 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.943 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.943 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.943 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.943 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.943 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.943 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.943 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.943 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.943 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.943 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.943 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.944 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.944 INFO:teuthology.orchestra.run.smithi001.stdout: "max_completed_events": { 2024-09-12T01:36:47.944 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_completed_events", 2024-09-12T01:36:47.944 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.944 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.944 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.944 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "50", 2024-09-12T01:36:47.944 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.944 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.944 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.944 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "number of past completed events to remember", 2024-09-12T01:36:47.944 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.944 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.945 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.945 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-12T01:36:47.945 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-12T01:36:47.945 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:47.945 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.945 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.945 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-09-12T01:36:47.945 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.945 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.945 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.945 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long the module is going to sleep", 2024-09-12T01:36:47.945 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.945 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.946 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.946 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.946 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.946 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.946 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus", 2024-09-12T01:36:47.946 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.946 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.946 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.946 INFO:teuthology.orchestra.run.smithi001.stdout: "cache": { 2024-09-12T01:36:47.946 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cache", 2024-09-12T01:36:47.946 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.946 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.946 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.946 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.947 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.947 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.947 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.947 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.947 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.947 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.947 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.947 INFO:teuthology.orchestra.run.smithi001.stdout: "exclude_perf_counters": { 2024-09-12T01:36:47.947 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "exclude_perf_counters", 2024-09-12T01:36:47.947 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.947 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.947 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.948 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.948 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.948 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.948 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.948 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T01:36:47.948 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T01:36:47.948 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.948 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.948 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.948 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.948 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.948 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.948 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.949 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.949 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.949 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.949 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.949 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.949 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.949 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.949 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.949 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.949 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.949 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.949 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.949 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.949 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.950 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.950 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.950 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.950 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.950 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.950 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.950 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.950 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.950 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.950 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.950 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.950 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.950 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.951 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.951 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.951 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.951 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.951 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.951 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.951 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.951 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.951 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.951 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.951 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.951 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.951 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.952 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.952 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.952 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.952 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.952 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.952 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.952 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.952 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.952 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.952 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.952 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.952 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.953 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.953 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.953 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.953 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.953 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.953 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.953 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.953 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.953 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.953 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_stats_pools": { 2024-09-12T01:36:47.953 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_stats_pools", 2024-09-12T01:36:47.953 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.953 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.954 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.954 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.954 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.954 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.954 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.954 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.954 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.954 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.954 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.954 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T01:36:47.954 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T01:36:47.954 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.954 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.955 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.955 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "300", 2024-09-12T01:36:47.955 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.955 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.955 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.955 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.955 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.955 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.955 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.955 INFO:teuthology.orchestra.run.smithi001.stdout: "scrape_interval": { 2024-09-12T01:36:47.955 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "scrape_interval", 2024-09-12T01:36:47.955 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-12T01:36:47.955 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.956 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.956 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15.0", 2024-09-12T01:36:47.956 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.956 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.956 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.956 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.956 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.956 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.956 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.956 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-09-12T01:36:47.956 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-09-12T01:36:47.956 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.956 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.957 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.957 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "::", 2024-09-12T01:36:47.957 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.957 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.957 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.957 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T01:36:47.957 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.957 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.957 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.957 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-09-12T01:36:47.957 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-09-12T01:36:47.957 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.957 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.958 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.958 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "9283", 2024-09-12T01:36:47.958 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.958 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.958 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.958 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T01:36:47.958 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.958 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.958 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.958 INFO:teuthology.orchestra.run.smithi001.stdout: "stale_cache_strategy": { 2024-09-12T01:36:47.958 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "stale_cache_strategy", 2024-09-12T01:36:47.958 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.958 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.958 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.959 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "log", 2024-09-12T01:36:47.959 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.959 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.959 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.959 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.959 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.959 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.959 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.959 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_behaviour": { 2024-09-12T01:36:47.959 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_behaviour", 2024-09-12T01:36:47.959 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.959 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.959 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.960 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "default", 2024-09-12T01:36:47.960 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.960 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.960 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.960 INFO:teuthology.orchestra.run.smithi001.stdout: "default", 2024-09-12T01:36:47.960 INFO:teuthology.orchestra.run.smithi001.stdout: "error" 2024-09-12T01:36:47.960 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.960 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.960 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.960 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.960 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.960 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_error_status_code": { 2024-09-12T01:36:47.960 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_error_status_code", 2024-09-12T01:36:47.960 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.961 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.961 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.961 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "500", 2024-09-12T01:36:47.961 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "400", 2024-09-12T01:36:47.961 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "599", 2024-09-12T01:36:47.961 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.961 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.961 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.961 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.961 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.961 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.961 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.962 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_support", 2024-09-12T01:36:47.962 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.962 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.962 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.962 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.962 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.962 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.962 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.962 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.962 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.962 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.962 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.962 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.962 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.963 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.963 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.963 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.963 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.963 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.963 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.963 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.963 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.963 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.963 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.963 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.963 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.963 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.963 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.964 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.964 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.964 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.964 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.964 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.964 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.964 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.964 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.964 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.964 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.964 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.964 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.964 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.965 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.965 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.965 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.965 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.965 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.965 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.965 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.965 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.965 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.965 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.965 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.965 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.965 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.966 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.966 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.966 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.966 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.966 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.966 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.966 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.966 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.966 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.966 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.966 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.966 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.966 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.966 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.967 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.967 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.967 INFO:teuthology.orchestra.run.smithi001.stdout: "max_concurrent_snap_create": { 2024-09-12T01:36:47.967 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_concurrent_snap_create", 2024-09-12T01:36:47.967 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:47.967 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.967 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.967 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-12T01:36:47.967 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.967 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.967 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.967 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.967 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.968 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.968 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.968 INFO:teuthology.orchestra.run.smithi001.stdout: "mirror_snapshot_schedule": { 2024-09-12T01:36:47.968 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T01:36:47.968 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.968 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.968 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.968 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.968 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.968 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.968 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.968 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.968 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.969 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.969 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.969 INFO:teuthology.orchestra.run.smithi001.stdout: "trash_purge_schedule": { 2024-09-12T01:36:47.969 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "trash_purge_schedule", 2024-09-12T01:36:47.969 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.969 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.969 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.969 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.969 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.969 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.969 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.969 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.970 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.970 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.970 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.970 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.970 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.970 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "restful", 2024-09-12T01:36:47.970 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.970 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.970 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.970 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_auth": { 2024-09-12T01:36:47.970 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enable_auth", 2024-09-12T01:36:47.970 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.971 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.971 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.971 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:47.971 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.971 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.971 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.971 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.971 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.971 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.971 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.971 INFO:teuthology.orchestra.run.smithi001.stdout: "key_file": { 2024-09-12T01:36:47.971 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "key_file", 2024-09-12T01:36:47.971 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.971 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.972 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.972 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.972 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.972 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.972 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.972 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.972 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.972 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.972 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.972 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.972 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.972 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.972 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.973 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.973 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.973 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.973 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.973 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.973 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.973 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.973 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.973 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.973 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.973 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.973 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.973 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.973 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.974 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.974 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.974 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.974 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.974 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.974 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.974 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.974 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.974 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.974 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.974 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.974 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.975 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.975 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.975 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.975 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.975 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.975 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.975 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.975 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.975 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.975 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.975 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.975 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.975 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.976 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.976 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.976 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.976 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.976 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.976 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.976 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.976 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.976 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.976 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.976 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.976 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.976 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.976 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.977 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.977 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.977 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.977 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.977 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.977 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.977 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.977 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.977 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.977 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-09-12T01:36:47.977 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-09-12T01:36:47.977 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.978 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.978 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.978 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.978 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.978 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.978 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.978 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.978 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.978 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.978 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.978 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-09-12T01:36:47.978 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-09-12T01:36:47.978 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.979 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.979 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.979 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.979 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.979 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.979 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.979 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.979 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.979 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.979 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.979 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.979 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.979 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.980 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rgw", 2024-09-12T01:36:47.980 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.980 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.980 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.980 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.980 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.980 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.980 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.980 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.980 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.980 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.980 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.980 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.980 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.981 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.981 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.981 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.981 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.981 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.981 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.981 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.981 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.981 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.981 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.981 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.981 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.981 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.982 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.982 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.982 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.982 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.982 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.982 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.982 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.982 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.982 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.982 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.982 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.982 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.982 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.983 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.983 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.983 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.983 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.983 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.983 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.983 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.983 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.983 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.983 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.983 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.983 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.983 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.983 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.983 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.984 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.984 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.984 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.984 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.984 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.984 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.984 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.984 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.984 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.984 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.984 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.984 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.984 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.985 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.985 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.985 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.985 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.985 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.985 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rook", 2024-09-12T01:36:47.985 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.985 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.985 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.985 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.985 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.985 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.985 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.986 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.986 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.986 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.986 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.986 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.986 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.986 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.986 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.986 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.986 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.986 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.986 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.986 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.986 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.987 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.987 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.987 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.987 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.987 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.987 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.987 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.987 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.987 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.987 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.987 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.987 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.987 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.988 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.988 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.988 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.988 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.988 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.988 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.988 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.988 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.988 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.988 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.988 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.988 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.988 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.989 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.989 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.989 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.989 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.989 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.989 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.989 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.989 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.989 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.989 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.989 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.989 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.989 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.989 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.990 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.990 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.990 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.990 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.990 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.990 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.990 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.990 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.990 INFO:teuthology.orchestra.run.smithi001.stdout: "storage_class": { 2024-09-12T01:36:47.990 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "storage_class", 2024-09-12T01:36:47.990 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.990 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.990 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.991 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "local", 2024-09-12T01:36:47.991 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.991 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.991 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.991 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T01:36:47.991 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.991 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.991 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.991 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.991 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:47.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:47.991 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "selftest", 2024-09-12T01:36:47.991 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:47.992 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:47.992 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:47.992 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:47.992 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:47.992 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.992 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.992 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.992 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.992 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.992 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.992 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.992 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.992 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.992 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.993 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.993 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.993 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.993 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.993 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.993 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.993 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.993 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.993 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:47.993 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:47.993 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.993 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.993 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.994 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.994 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.994 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.994 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.994 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.994 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.994 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.994 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.994 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:47.994 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:47.994 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.994 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.994 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.995 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:47.995 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.995 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.995 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:47.995 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:47.995 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:47.995 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:47.995 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:47.995 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:47.995 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:47.995 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:47.995 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.995 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.995 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.996 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.996 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:47.996 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:47.996 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:47.996 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.996 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:47.996 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:47.996 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.996 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.996 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.996 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.996 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.997 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.997 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.997 INFO:teuthology.orchestra.run.smithi001.stdout: "roption1": { 2024-09-12T01:36:47.997 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "roption1", 2024-09-12T01:36:47.997 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.997 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.997 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.997 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.997 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.997 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.997 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.997 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.998 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.998 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.998 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.998 INFO:teuthology.orchestra.run.smithi001.stdout: "roption2": { 2024-09-12T01:36:47.998 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "roption2", 2024-09-12T01:36:47.998 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.998 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.998 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.998 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "xyz", 2024-09-12T01:36:47.998 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.998 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.998 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.998 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:47.999 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:47.999 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:47.999 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:47.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:47.999 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption1": { 2024-09-12T01:36:47.999 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption1", 2024-09-12T01:36:47.999 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:47.999 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:47.999 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:47.999 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:47.999 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:47.999 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:47.999 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:47.999 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.000 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.000 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.000 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.000 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption2": { 2024-09-12T01:36:48.000 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption2", 2024-09-12T01:36:48.000 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:48.000 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.000 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:48.000 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:48.000 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.000 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.000 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.000 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.001 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.001 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.001 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.001 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption3": { 2024-09-12T01:36:48.001 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption3", 2024-09-12T01:36:48.001 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-12T01:36:48.001 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.001 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:48.001 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:48.001 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.001 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.001 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.001 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.002 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.002 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.002 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.002 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption4": { 2024-09-12T01:36:48.002 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption4", 2024-09-12T01:36:48.002 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:48.002 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.002 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:48.002 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:48.002 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.002 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.002 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.002 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.002 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.003 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.003 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.003 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption5": { 2024-09-12T01:36:48.003 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption5", 2024-09-12T01:36:48.003 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:48.003 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.003 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:48.003 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:48.003 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.003 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.003 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.003 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.003 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.004 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.004 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.004 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption6": { 2024-09-12T01:36:48.004 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption6", 2024-09-12T01:36:48.004 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:48.004 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.004 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:48.004 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:48.004 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.004 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.004 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.004 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.004 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.005 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.005 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.005 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption7": { 2024-09-12T01:36:48.005 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption7", 2024-09-12T01:36:48.005 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:48.005 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.005 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:48.005 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:48.005 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-09-12T01:36:48.005 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "42", 2024-09-12T01:36:48.005 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.005 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.005 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.006 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.006 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.006 INFO:teuthology.orchestra.run.smithi001.stdout: "testkey": { 2024-09-12T01:36:48.006 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testkey", 2024-09-12T01:36:48.006 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:48.006 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.006 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:48.006 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:48.006 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.006 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.006 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.006 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.006 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.007 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.007 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.007 INFO:teuthology.orchestra.run.smithi001.stdout: "testlkey": { 2024-09-12T01:36:48.007 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testlkey", 2024-09-12T01:36:48.007 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:48.007 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.007 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:48.007 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:48.007 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.007 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.007 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.007 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.007 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.008 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.008 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.008 INFO:teuthology.orchestra.run.smithi001.stdout: "testnewline": { 2024-09-12T01:36:48.008 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testnewline", 2024-09-12T01:36:48.008 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:48.008 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.008 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:48.008 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:48.008 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.008 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.008 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.008 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.008 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.009 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.009 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.009 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:48.009 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:48.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:48.009 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snap_schedule", 2024-09-12T01:36:48.009 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:48.009 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:48.009 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:48.009 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_m_granularity": { 2024-09-12T01:36:48.009 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_m_granularity", 2024-09-12T01:36:48.009 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:48.010 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.010 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:48.010 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:48.010 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.010 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.010 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.010 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T01:36:48.010 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.010 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.010 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.010 INFO:teuthology.orchestra.run.smithi001.stdout: "dump_on_update": { 2024-09-12T01:36:48.010 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "dump_on_update", 2024-09-12T01:36:48.010 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:48.010 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.011 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:48.011 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:48.011 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.011 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.011 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.011 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "dump database to debug log on update", 2024-09-12T01:36:48.011 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.011 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.011 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.011 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:48.011 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:48.011 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:48.011 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.012 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:48.012 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:48.012 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.012 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.012 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:48.012 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:48.012 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:48.012 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:48.012 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:48.012 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:48.012 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:48.012 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:48.012 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.013 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.013 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.013 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.013 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:48.013 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:48.013 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:48.013 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.013 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:48.013 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:48.013 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.013 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.013 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.014 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.014 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.014 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.014 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.014 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:48.014 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:48.014 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:48.014 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.014 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:48.014 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:48.015 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.015 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.015 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:48.015 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:48.015 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:48.015 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:48.015 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:48.015 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:48.015 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:48.015 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:48.015 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.015 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.015 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.016 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.016 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:48.016 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:48.016 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:48.016 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.016 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:48.016 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:48.016 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.016 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.016 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.016 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.017 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.017 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.017 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.017 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:48.017 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:48.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:48.017 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "stats", 2024-09-12T01:36:48.017 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:48.017 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:48.017 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:48.017 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:48.017 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:48.018 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:48.018 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.018 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:48.018 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:48.018 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.018 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.018 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:48.018 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:48.018 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:48.018 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:48.018 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:48.018 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:48.018 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:48.018 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:48.019 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.019 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.019 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.019 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.019 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:48.019 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:48.019 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:48.019 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.019 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:48.019 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:48.019 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.019 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.019 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.020 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.020 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.020 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.020 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.020 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:48.020 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:48.020 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:48.020 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.020 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:48.020 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:48.020 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.020 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.021 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:48.021 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:48.021 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:48.021 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:48.021 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:48.021 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:48.021 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:48.021 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:48.021 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.021 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.021 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.021 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.021 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:48.022 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:48.022 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:48.022 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.022 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:48.022 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:48.022 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.022 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.022 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.022 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.022 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.022 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.022 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.022 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:48.022 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:48.023 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.023 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:48.023 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "status", 2024-09-12T01:36:48.023 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:48.023 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:48.023 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:48.023 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:48.023 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:48.023 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:48.023 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.023 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:48.023 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:48.023 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.023 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.024 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:48.024 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:48.024 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:48.024 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:48.024 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:48.024 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:48.024 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:48.024 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:48.024 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.024 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.024 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.024 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.025 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:48.025 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:48.025 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:48.025 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.025 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:48.025 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:48.025 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.025 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.025 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.025 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.025 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.025 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.025 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.026 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:48.026 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:48.026 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:48.026 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.026 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:48.026 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:48.026 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.026 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.026 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:48.026 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:48.026 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:48.026 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:48.026 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:48.026 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:48.026 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:48.027 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:48.027 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.027 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.027 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.027 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.027 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:48.027 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:48.027 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:48.027 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.027 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:48.027 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:48.027 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.027 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.028 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.028 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.028 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.028 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.028 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.028 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:48.028 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:48.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.028 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:48.028 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "telegraf", 2024-09-12T01:36:48.028 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:48.028 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:48.028 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:48.028 INFO:teuthology.orchestra.run.smithi001.stdout: "address": { 2024-09-12T01:36:48.028 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "address", 2024-09-12T01:36:48.029 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:48.029 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.029 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:48.029 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T01:36:48.029 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.029 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.029 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.029 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.029 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.029 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.029 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.029 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-12T01:36:48.029 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-12T01:36:48.030 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:48.030 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.030 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:48.030 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15", 2024-09-12T01:36:48.030 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.030 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.030 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.030 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.030 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.030 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.030 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.030 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:48.030 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:48.031 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:48.031 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.031 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:48.031 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:48.031 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.031 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.031 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:48.031 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:48.031 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:48.031 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:48.031 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:48.031 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:48.031 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:48.031 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:48.031 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.032 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.032 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.032 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.032 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:48.032 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:48.032 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:48.032 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.032 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:48.032 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:48.032 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.032 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.032 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.032 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.033 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.033 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.033 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.033 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:48.033 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:48.033 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:48.033 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.033 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:48.033 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:48.033 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.033 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.033 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:48.033 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:48.034 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:48.034 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:48.034 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:48.034 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:48.034 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:48.034 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:48.034 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.034 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.034 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.034 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.034 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:48.034 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:48.034 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:48.035 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.035 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:48.035 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:48.035 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.035 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.035 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.035 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.035 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.035 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.035 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.035 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:48.035 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:48.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.035 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:48.036 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "telemetry", 2024-09-12T01:36:48.036 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:48.036 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:48.036 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:48.036 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_basic": { 2024-09-12T01:36:48.036 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_basic", 2024-09-12T01:36:48.036 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:48.036 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.036 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:48.036 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:48.036 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.036 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.036 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.036 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T01:36:48.037 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.037 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.037 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.037 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_crash": { 2024-09-12T01:36:48.037 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_crash", 2024-09-12T01:36:48.037 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:48.037 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.037 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:48.037 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:48.037 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.037 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.037 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.037 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T01:36:48.038 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.038 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.038 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.038 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_device": { 2024-09-12T01:36:48.038 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_device", 2024-09-12T01:36:48.038 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:48.038 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.038 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:48.038 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:48.038 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.038 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.038 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.038 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T01:36:48.039 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.039 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.039 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.039 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_ident": { 2024-09-12T01:36:48.039 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_ident", 2024-09-12T01:36:48.039 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:48.039 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.039 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:48.039 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:48.039 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.039 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.039 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.039 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T01:36:48.040 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.040 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.040 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.040 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_perf": { 2024-09-12T01:36:48.040 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_perf", 2024-09-12T01:36:48.040 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:48.040 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.040 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:48.040 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:48.040 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.040 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.040 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.040 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T01:36:48.041 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.041 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.041 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.041 INFO:teuthology.orchestra.run.smithi001.stdout: "contact": { 2024-09-12T01:36:48.041 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "contact", 2024-09-12T01:36:48.041 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:48.041 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.041 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:48.041 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:48.041 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.041 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.041 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.041 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.041 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.042 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.042 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.042 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.042 INFO:teuthology.orchestra.run.smithi001.stdout: "description": { 2024-09-12T01:36:48.042 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "description", 2024-09-12T01:36:48.042 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:48.042 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.042 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:48.042 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:48.042 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.042 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.042 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.042 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.042 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.043 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.043 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.043 INFO:teuthology.orchestra.run.smithi001.stdout: "device_url": { 2024-09-12T01:36:48.043 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_url", 2024-09-12T01:36:48.043 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:48.043 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.043 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:48.043 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T01:36:48.043 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.043 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.043 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.043 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.043 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.044 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.044 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.044 INFO:teuthology.orchestra.run.smithi001.stdout: "enabled": { 2024-09-12T01:36:48.044 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enabled", 2024-09-12T01:36:48.044 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:48.044 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.044 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:48.044 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:48.044 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.044 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.044 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.044 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.044 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.044 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.045 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.045 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-12T01:36:48.045 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-12T01:36:48.045 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:48.045 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.045 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:48.045 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "24", 2024-09-12T01:36:48.045 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "8", 2024-09-12T01:36:48.045 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.045 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.045 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.045 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.045 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.046 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.046 INFO:teuthology.orchestra.run.smithi001.stdout: "last_opt_revision": { 2024-09-12T01:36:48.046 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "last_opt_revision", 2024-09-12T01:36:48.046 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:48.046 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.046 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:48.046 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1", 2024-09-12T01:36:48.046 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.046 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.046 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.046 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.046 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.047 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.047 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.047 INFO:teuthology.orchestra.run.smithi001.stdout: "leaderboard": { 2024-09-12T01:36:48.047 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "leaderboard", 2024-09-12T01:36:48.047 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:48.047 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.047 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:48.047 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:48.047 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.047 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.047 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.047 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.047 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.047 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.048 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.048 INFO:teuthology.orchestra.run.smithi001.stdout: "leaderboard_description": { 2024-09-12T01:36:48.048 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "leaderboard_description", 2024-09-12T01:36:48.048 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:48.048 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.048 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:48.048 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:48.048 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.048 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.048 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.048 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.048 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.048 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.049 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.049 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:48.049 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:48.049 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:48.049 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.049 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:48.049 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:48.049 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.049 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.049 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:48.049 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:48.049 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:48.050 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:48.050 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:48.050 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:48.050 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:48.050 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:48.050 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.050 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.050 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.050 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.050 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.050 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:48.050 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:48.050 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:48.050 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.051 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:48.051 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:48.051 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.051 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.051 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.051 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.051 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.051 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.051 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.051 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.051 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:48.051 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:48.051 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:48.051 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.052 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:48.052 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:48.052 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.052 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.052 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:48.052 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:48.052 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:48.052 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:48.052 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:48.052 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:48.052 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:48.052 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:48.052 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.052 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.053 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.053 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.053 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:48.053 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:48.053 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:48.053 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.053 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:48.053 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:48.053 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.053 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.053 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.053 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.053 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.054 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.054 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.054 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.054 INFO:teuthology.orchestra.run.smithi001.stdout: "organization": { 2024-09-12T01:36:48.054 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "organization", 2024-09-12T01:36:48.054 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:48.054 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.054 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:48.054 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:48.054 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.054 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.054 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.054 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.054 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.055 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.055 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.055 INFO:teuthology.orchestra.run.smithi001.stdout: "proxy": { 2024-09-12T01:36:48.055 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "proxy", 2024-09-12T01:36:48.055 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:48.055 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.055 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:48.055 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:48.055 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.055 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.055 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.055 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.055 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.056 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.056 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.056 INFO:teuthology.orchestra.run.smithi001.stdout: "url": { 2024-09-12T01:36:48.056 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "url", 2024-09-12T01:36:48.056 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:48.056 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.056 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:48.056 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T01:36:48.056 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.056 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.056 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.056 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.056 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.057 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.057 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.057 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:48.057 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:48.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:48.057 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "test_orchestrator", 2024-09-12T01:36:48.057 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:48.057 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:48.057 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:48.057 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:48.057 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:48.057 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:48.057 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.058 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:48.058 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:48.058 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.058 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.058 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:48.058 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:48.058 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:48.058 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:48.058 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:48.058 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:48.058 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:48.058 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:48.058 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.058 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.059 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.059 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.059 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:48.059 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:48.059 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:48.059 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.059 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:48.059 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:48.059 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.059 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.059 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.059 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.059 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.060 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.060 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.060 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.060 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:48.060 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:48.060 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:48.060 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.060 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:48.060 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:48.060 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.060 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.060 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:48.060 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:48.060 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:48.060 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:48.061 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:48.061 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:48.061 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:48.061 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:48.061 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.061 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.061 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.061 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.061 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.061 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:48.061 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:48.061 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:48.061 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.061 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:48.062 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:48.062 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.062 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.062 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.062 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.062 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.062 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.062 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.062 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:48.062 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:48.062 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.062 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:48.062 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "volumes", 2024-09-12T01:36:48.062 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:48.062 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:48.063 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:48.063 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:48.063 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:48.063 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:48.063 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.063 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:48.063 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:48.063 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.063 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.063 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:48.063 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:48.063 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:48.063 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:48.063 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:48.064 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:48.064 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:48.064 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:48.064 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.064 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.064 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.064 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.064 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:48.064 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:48.064 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:48.064 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.064 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:48.064 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:48.064 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.065 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.065 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.065 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.065 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.065 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.065 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.065 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:48.065 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:48.065 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:48.065 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.065 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:48.065 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:48.065 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.066 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.066 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:48.066 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:48.066 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:48.066 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:48.066 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:48.066 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:48.066 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:48.066 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:48.066 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.066 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.066 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.066 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.066 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.066 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:48.067 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:48.067 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:48.067 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.067 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:48.067 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:48.067 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.067 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.067 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.067 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.067 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.067 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.067 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.067 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.067 INFO:teuthology.orchestra.run.smithi001.stdout: "max_concurrent_clones": { 2024-09-12T01:36:48.068 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_concurrent_clones", 2024-09-12T01:36:48.068 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:48.068 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.068 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:48.068 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "4", 2024-09-12T01:36:48.068 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.068 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.068 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.068 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T01:36:48.068 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.068 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.068 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.069 INFO:teuthology.orchestra.run.smithi001.stdout: "periodic_async_work": { 2024-09-12T01:36:48.069 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "periodic_async_work", 2024-09-12T01:36:48.069 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:48.069 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.069 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:48.069 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:48.069 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.069 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.069 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.069 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Periodically check for async work", 2024-09-12T01:36:48.069 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.069 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.069 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.069 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.069 INFO:teuthology.orchestra.run.smithi001.stdout: "snapshot_clone_delay": { 2024-09-12T01:36:48.070 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snapshot_clone_delay", 2024-09-12T01:36:48.070 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:48.070 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.070 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:48.070 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-12T01:36:48.070 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.070 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.070 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.070 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T01:36:48.070 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.070 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.070 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.070 INFO:teuthology.orchestra.run.smithi001.stdout: "snapshot_clone_no_wait": { 2024-09-12T01:36:48.071 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T01:36:48.071 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:48.071 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.071 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:48.071 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-12T01:36:48.071 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.071 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.071 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.071 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T01:36:48.071 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.071 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.071 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.071 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:48.071 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:48.072 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.072 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:48.072 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix", 2024-09-12T01:36:48.072 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-12T01:36:48.072 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-12T01:36:48.072 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-12T01:36:48.072 INFO:teuthology.orchestra.run.smithi001.stdout: "discovery_interval": { 2024-09-12T01:36:48.072 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "discovery_interval", 2024-09-12T01:36:48.072 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-12T01:36:48.072 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.072 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:48.072 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "100", 2024-09-12T01:36:48.072 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.073 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.073 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.073 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.073 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.073 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.073 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.073 INFO:teuthology.orchestra.run.smithi001.stdout: "identifier": { 2024-09-12T01:36:48.073 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "identifier", 2024-09-12T01:36:48.073 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:48.073 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.073 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:48.073 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:48.073 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.074 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.074 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.074 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.074 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.074 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.074 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.074 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-12T01:36:48.074 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-12T01:36:48.074 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-12T01:36:48.074 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.074 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:48.074 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-12T01:36:48.074 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.074 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.075 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.075 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.075 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.075 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.075 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.075 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-12T01:36:48.075 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-12T01:36:48.075 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:48.075 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.075 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:48.075 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:48.075 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.075 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.076 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:48.076 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:48.076 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:48.076 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:48.076 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:48.076 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:48.076 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:48.076 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:48.076 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.076 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.076 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.076 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.076 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-12T01:36:48.077 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-12T01:36:48.077 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:48.077 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.077 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:48.077 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:48.077 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.077 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.077 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.077 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.077 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.077 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.077 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.077 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.078 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-12T01:36:48.078 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-12T01:36:48.078 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:48.078 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.078 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:48.078 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-12T01:36:48.078 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.078 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.078 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-12T01:36:48.078 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-12T01:36:48.082 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-12T01:36:48.082 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-12T01:36:48.082 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-12T01:36:48.082 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-12T01:36:48.082 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-12T01:36:48.082 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:48.082 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.082 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.083 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.083 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.083 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.083 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-12T01:36:48.083 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-12T01:36:48.083 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-12T01:36:48.083 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.083 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-12T01:36:48.083 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-12T01:36:48.083 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.083 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.083 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.083 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.083 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.084 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.084 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.084 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_host": { 2024-09-12T01:36:48.084 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_host", 2024-09-12T01:36:48.084 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:48.084 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.084 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:48.084 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-12T01:36:48.084 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.084 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.084 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.084 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.085 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.085 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.085 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.085 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_port": { 2024-09-12T01:36:48.085 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_port", 2024-09-12T01:36:48.085 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-12T01:36:48.085 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.085 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:48.085 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10051", 2024-09-12T01:36:48.085 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.085 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.085 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.085 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.086 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.086 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.086 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.086 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_sender": { 2024-09-12T01:36:48.086 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_sender", 2024-09-12T01:36:48.086 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-12T01:36:48.086 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-12T01:36:48.086 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-12T01:36:48.086 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T01:36:48.086 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-12T01:36:48.086 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-12T01:36:48.086 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-12T01:36:48.087 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-12T01:36:48.087 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-12T01:36:48.087 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-12T01:36:48.087 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-12T01:36:48.087 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:48.087 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:48.087 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:48.087 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:48.087 INFO:teuthology.orchestra.run.smithi001.stdout: "services": { 2024-09-12T01:36:48.087 INFO:teuthology.orchestra.run.smithi001.stdout: "dashboard": "https://172.21.15.1:8443/" 2024-09-12T01:36:48.087 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.087 INFO:teuthology.orchestra.run.smithi001.stdout: "always_on_modules": { 2024-09-12T01:36:48.087 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus": [ 2024-09-12T01:36:48.087 INFO:teuthology.orchestra.run.smithi001.stdout: "balancer", 2024-09-12T01:36:48.087 INFO:teuthology.orchestra.run.smithi001.stdout: "crash", 2024-09-12T01:36:48.088 INFO:teuthology.orchestra.run.smithi001.stdout: "devicehealth", 2024-09-12T01:36:48.088 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator", 2024-09-12T01:36:48.088 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscaler", 2024-09-12T01:36:48.088 INFO:teuthology.orchestra.run.smithi001.stdout: "progress", 2024-09-12T01:36:48.088 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_support", 2024-09-12T01:36:48.088 INFO:teuthology.orchestra.run.smithi001.stdout: "status", 2024-09-12T01:36:48.088 INFO:teuthology.orchestra.run.smithi001.stdout: "telemetry", 2024-09-12T01:36:48.088 INFO:teuthology.orchestra.run.smithi001.stdout: "volumes" 2024-09-12T01:36:48.088 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:48.088 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific": [ 2024-09-12T01:36:48.088 INFO:teuthology.orchestra.run.smithi001.stdout: "balancer", 2024-09-12T01:36:48.088 INFO:teuthology.orchestra.run.smithi001.stdout: "crash", 2024-09-12T01:36:48.088 INFO:teuthology.orchestra.run.smithi001.stdout: "devicehealth", 2024-09-12T01:36:48.088 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator", 2024-09-12T01:36:48.089 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscaler", 2024-09-12T01:36:48.089 INFO:teuthology.orchestra.run.smithi001.stdout: "progress", 2024-09-12T01:36:48.089 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_support", 2024-09-12T01:36:48.089 INFO:teuthology.orchestra.run.smithi001.stdout: "status", 2024-09-12T01:36:48.089 INFO:teuthology.orchestra.run.smithi001.stdout: "telemetry", 2024-09-12T01:36:48.089 INFO:teuthology.orchestra.run.smithi001.stdout: "volumes" 2024-09-12T01:36:48.089 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:48.089 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy": [ 2024-09-12T01:36:48.089 INFO:teuthology.orchestra.run.smithi001.stdout: "balancer", 2024-09-12T01:36:48.089 INFO:teuthology.orchestra.run.smithi001.stdout: "crash", 2024-09-12T01:36:48.089 INFO:teuthology.orchestra.run.smithi001.stdout: "devicehealth", 2024-09-12T01:36:48.089 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator", 2024-09-12T01:36:48.089 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscaler", 2024-09-12T01:36:48.089 INFO:teuthology.orchestra.run.smithi001.stdout: "progress", 2024-09-12T01:36:48.090 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_support", 2024-09-12T01:36:48.090 INFO:teuthology.orchestra.run.smithi001.stdout: "status", 2024-09-12T01:36:48.090 INFO:teuthology.orchestra.run.smithi001.stdout: "telemetry", 2024-09-12T01:36:48.090 INFO:teuthology.orchestra.run.smithi001.stdout: "volumes" 2024-09-12T01:36:48.090 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:48.090 INFO:teuthology.orchestra.run.smithi001.stdout: "reef": [ 2024-09-12T01:36:48.090 INFO:teuthology.orchestra.run.smithi001.stdout: "balancer", 2024-09-12T01:36:48.090 INFO:teuthology.orchestra.run.smithi001.stdout: "crash", 2024-09-12T01:36:48.090 INFO:teuthology.orchestra.run.smithi001.stdout: "devicehealth", 2024-09-12T01:36:48.090 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator", 2024-09-12T01:36:48.090 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscaler", 2024-09-12T01:36:48.090 INFO:teuthology.orchestra.run.smithi001.stdout: "progress", 2024-09-12T01:36:48.090 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_support", 2024-09-12T01:36:48.090 INFO:teuthology.orchestra.run.smithi001.stdout: "status", 2024-09-12T01:36:48.090 INFO:teuthology.orchestra.run.smithi001.stdout: "telemetry", 2024-09-12T01:36:48.091 INFO:teuthology.orchestra.run.smithi001.stdout: "volumes" 2024-09-12T01:36:48.091 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-12T01:36:48.091 INFO:teuthology.orchestra.run.smithi001.stdout: "squid": [ 2024-09-12T01:36:48.091 INFO:teuthology.orchestra.run.smithi001.stdout: "balancer", 2024-09-12T01:36:48.091 INFO:teuthology.orchestra.run.smithi001.stdout: "crash", 2024-09-12T01:36:48.091 INFO:teuthology.orchestra.run.smithi001.stdout: "devicehealth", 2024-09-12T01:36:48.091 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator", 2024-09-12T01:36:48.091 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscaler", 2024-09-12T01:36:48.091 INFO:teuthology.orchestra.run.smithi001.stdout: "progress", 2024-09-12T01:36:48.091 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_support", 2024-09-12T01:36:48.091 INFO:teuthology.orchestra.run.smithi001.stdout: "status", 2024-09-12T01:36:48.091 INFO:teuthology.orchestra.run.smithi001.stdout: "telemetry", 2024-09-12T01:36:48.091 INFO:teuthology.orchestra.run.smithi001.stdout: "volumes" 2024-09-12T01:36:48.091 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-12T01:36:48.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.092 INFO:teuthology.orchestra.run.smithi001.stdout: "last_failure_osd_epoch": 3, 2024-09-12T01:36:48.092 INFO:teuthology.orchestra.run.smithi001.stdout: "active_clients": [ 2024-09-12T01:36:48.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:48.092 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "devicehealth", 2024-09-12T01:36:48.092 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-12T01:36:48.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:48.092 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-12T01:36:48.092 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:0", 2024-09-12T01:36:48.092 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 743312396 2024-09-12T01:36:48.092 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:48.092 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-12T01:36:48.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:48.093 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "libcephsqlite", 2024-09-12T01:36:48.093 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-12T01:36:48.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:48.093 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-12T01:36:48.093 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:0", 2024-09-12T01:36:48.093 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4032339702 2024-09-12T01:36:48.093 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:48.093 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-12T01:36:48.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:48.093 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_support", 2024-09-12T01:36:48.093 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-12T01:36:48.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:48.093 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-12T01:36:48.094 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:0", 2024-09-12T01:36:48.094 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1385185651 2024-09-12T01:36:48.094 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:48.094 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-12T01:36:48.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:48.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:48.094 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "volumes", 2024-09-12T01:36:48.094 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-12T01:36:48.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-12T01:36:48.094 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-12T01:36:48.094 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:0", 2024-09-12T01:36:48.094 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2721395899 2024-09-12T01:36:48.094 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:48.094 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-12T01:36:48.095 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:48.095 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-12T01:36:48.095 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-12T01:36:48.096 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:47 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/3119768718' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T01:36:48.096 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:46 smithi104 ceph-mon[35219]: pgmap v613: 129 pgs: 129 active+clean; 110 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 417 op/s 2024-09-12T01:36:48.097 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:46 smithi104 ceph-mon[35219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T01:36:48.097 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:46 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/3615942135' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T01:36:48.097 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:46 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/4115488039' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T01:36:48.097 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:46 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/3779144598' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-12T01:36:48.097 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:47 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/3119768718' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T01:36:48.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.787+0000 7f1c77e00640 1 -- 172.21.15.1:0/3119768718 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1c5c0a2c00 msgr2=0x7f1c5c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:48.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.787+0000 7f1c77e00640 1 --2- 172.21.15.1:0/3119768718 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1c5c0a2c00 0x7f1c5c0a50c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1c64031440 tx=0x7f1c640315b0 comp rx=0 tx=0).stop 2024-09-12T01:36:48.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.787+0000 7f1c77e00640 1 -- 172.21.15.1:0/3119768718 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c78103830 msgr2=0x7f1c7819a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:36:48.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.787+0000 7f1c77e00640 1 --2- 172.21.15.1:0/3119768718 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c78103830 0x7f1c7819a080 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1c6800e9c0 tx=0x7f1c6800ee80 comp rx=0 tx=0).stop 2024-09-12T01:36:48.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.787+0000 7f1c77e00640 1 -- 172.21.15.1:0/3119768718 shutdown_connections 2024-09-12T01:36:48.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.787+0000 7f1c77e00640 1 --2- 172.21.15.1:0/3119768718 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f1c5c0a2c00 0x7f1c5c0a50c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:48.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.787+0000 7f1c77e00640 1 --2- 172.21.15.1:0/3119768718 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c78105300 0x7f1c781a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:48.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.787+0000 7f1c77e00640 1 --2- 172.21.15.1:0/3119768718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c781041f0 0x7f1c7819a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:48.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.787+0000 7f1c77e00640 1 --2- 172.21.15.1:0/3119768718 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c78103830 0x7f1c7819a080 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:36:48.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.787+0000 7f1c77e00640 1 -- 172.21.15.1:0/3119768718 >> 172.21.15.1:0/3119768718 conn(0x7f1c780ff380 msgr2=0x7f1c78108180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:36:48.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.787+0000 7f1c77e00640 1 -- 172.21.15.1:0/3119768718 shutdown_connections 2024-09-12T01:36:48.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:36:46.787+0000 7f1c77e00640 1 -- 172.21.15.1:0/3119768718 wait complete. 2024-09-12T01:36:48.104 DEBUG:tasks.check_counter:Getting stats from a 2024-09-12T01:36:48.104 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-12T01:36:48.105 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:47 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/3119768718' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T01:36:48.494 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:36:48.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:48 smithi053 ceph-mon[35158]: pgmap v614: 129 pgs: 129 active+clean; 101 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 460 op/s 2024-09-12T01:36:48.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:48 smithi053 ceph-mon[35158]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T01:36:48.815 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:48 smithi001 ceph-mon[30387]: pgmap v614: 129 pgs: 129 active+clean; 101 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 460 op/s 2024-09-12T01:36:48.815 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:48 smithi001 ceph-mon[30387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T01:36:48.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:48 smithi104 ceph-mon[35219]: pgmap v614: 129 pgs: 129 active+clean; 101 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 460 op/s 2024-09-12T01:36:48.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:48 smithi104 ceph-mon[35219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T01:36:49.102 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-12T01:36:49.102 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T01:36:49.102 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 737, 2024-09-12T01:36:49.102 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 719, 2024-09-12T01:36:49.102 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 1017232, 2024-09-12T01:36:49.102 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 276754, 2024-09-12T01:36:49.103 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 5, 2024-09-12T01:36:49.103 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 1, 2024-09-12T01:36:49.103 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.104942146, 2024-09-12T01:36:49.103 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.041224298, 2024-09-12T01:36:49.103 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.044461307, 2024-09-12T01:36:49.103 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.017421874, 2024-09-12T01:36:49.103 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:36:49.103 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 719, 2024-09-12T01:36:49.103 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.053510967, 2024-09-12T01:36:49.103 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000074424 2024-09-12T01:36:49.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.104 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:36:49.104 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:49.104 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:49.104 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:49.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.104 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 834784, 2024-09-12T01:36:49.104 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 104080 2024-09-12T01:36:49.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.104 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T01:36:49.104 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 457, 2024-09-12T01:36:49.104 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 583, 2024-09-12T01:36:49.104 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 112988, 2024-09-12T01:36:49.104 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 431380, 2024-09-12T01:36:49.105 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 5, 2024-09-12T01:36:49.105 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 4, 2024-09-12T01:36:49.105 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.060737037, 2024-09-12T01:36:49.105 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.024764168, 2024-09-12T01:36:49.105 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.013916810, 2024-09-12T01:36:49.105 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.016365284, 2024-09-12T01:36:49.105 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:36:49.105 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 581, 2024-09-12T01:36:49.105 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.030568446, 2024-09-12T01:36:49.105 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000052613 2024-09-12T01:36:49.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.105 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:36:49.105 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:49.106 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:49.106 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:49.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.106 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-12T01:36:49.106 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 286304 2024-09-12T01:36:49.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.106 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T01:36:49.106 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 564, 2024-09-12T01:36:49.106 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 566, 2024-09-12T01:36:49.106 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 195182, 2024-09-12T01:36:49.106 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 179353, 2024-09-12T01:36:49.106 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 4, 2024-09-12T01:36:49.106 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 3, 2024-09-12T01:36:49.107 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.053392042, 2024-09-12T01:36:49.107 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.018531962, 2024-09-12T01:36:49.107 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.015325996, 2024-09-12T01:36:49.107 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.014036804, 2024-09-12T01:36:49.107 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:36:49.107 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 564, 2024-09-12T01:36:49.107 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.027170916, 2024-09-12T01:36:49.107 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000048175 2024-09-12T01:36:49.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.107 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:36:49.107 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:49.107 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:49.107 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:49.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.108 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T01:36:49.108 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T01:36:49.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.108 INFO:teuthology.orchestra.run.smithi001.stdout: "cct": { 2024-09-12T01:36:49.108 INFO:teuthology.orchestra.run.smithi001.stdout: "total_workers": 1, 2024-09-12T01:36:49.108 INFO:teuthology.orchestra.run.smithi001.stdout: "unhealthy_workers": 0 2024-09-12T01:36:49.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.108 INFO:teuthology.orchestra.run.smithi001.stdout: "finisher-MDSRank": { 2024-09-12T01:36:49.108 INFO:teuthology.orchestra.run.smithi001.stdout: "queue_len": 0, 2024-09-12T01:36:49.108 INFO:teuthology.orchestra.run.smithi001.stdout: "complete_latency": { 2024-09-12T01:36:49.108 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 2518, 2024-09-12T01:36:49.108 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.073933659, 2024-09-12T01:36:49.109 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000029362 2024-09-12T01:36:49.109 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:49.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.109 INFO:teuthology.orchestra.run.smithi001.stdout: "finisher-PurgeQueue": { 2024-09-12T01:36:49.109 INFO:teuthology.orchestra.run.smithi001.stdout: "queue_len": 0, 2024-09-12T01:36:49.109 INFO:teuthology.orchestra.run.smithi001.stdout: "complete_latency": { 2024-09-12T01:36:49.109 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:49.109 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:49.109 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:49.109 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:49.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.109 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": { 2024-09-12T01:36:49.109 INFO:teuthology.orchestra.run.smithi001.stdout: "request": 0, 2024-09-12T01:36:49.110 INFO:teuthology.orchestra.run.smithi001.stdout: "reply": 0, 2024-09-12T01:36:49.110 INFO:teuthology.orchestra.run.smithi001.stdout: "reply_latency": { 2024-09-12T01:36:49.110 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:49.110 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:49.110 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:49.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.110 INFO:teuthology.orchestra.run.smithi001.stdout: "slow_reply": 0, 2024-09-12T01:36:49.110 INFO:teuthology.orchestra.run.smithi001.stdout: "forward": 0, 2024-09-12T01:36:49.110 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_fetch_complete": 0, 2024-09-12T01:36:49.110 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_fetch_keys": 0, 2024-09-12T01:36:49.110 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_commit": 0, 2024-09-12T01:36:49.110 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_split": 0, 2024-09-12T01:36:49.110 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_merge": 0, 2024-09-12T01:36:49.110 INFO:teuthology.orchestra.run.smithi001.stdout: "inodes": 6, 2024-09-12T01:36:49.110 INFO:teuthology.orchestra.run.smithi001.stdout: "inodes_top": 0, 2024-09-12T01:36:49.111 INFO:teuthology.orchestra.run.smithi001.stdout: "inodes_bottom": 6, 2024-09-12T01:36:49.111 INFO:teuthology.orchestra.run.smithi001.stdout: "inodes_pin_tail": 0, 2024-09-12T01:36:49.111 INFO:teuthology.orchestra.run.smithi001.stdout: "inodes_pinned": 6, 2024-09-12T01:36:49.111 INFO:teuthology.orchestra.run.smithi001.stdout: "inodes_expired": 0, 2024-09-12T01:36:49.111 INFO:teuthology.orchestra.run.smithi001.stdout: "inodes_with_caps": 0, 2024-09-12T01:36:49.111 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": 0, 2024-09-12T01:36:49.111 INFO:teuthology.orchestra.run.smithi001.stdout: "subtrees": 1, 2024-09-12T01:36:49.111 INFO:teuthology.orchestra.run.smithi001.stdout: "traverse": 0, 2024-09-12T01:36:49.111 INFO:teuthology.orchestra.run.smithi001.stdout: "traverse_hit": 0, 2024-09-12T01:36:49.111 INFO:teuthology.orchestra.run.smithi001.stdout: "traverse_forward": 0, 2024-09-12T01:36:49.111 INFO:teuthology.orchestra.run.smithi001.stdout: "traverse_discover": 0, 2024-09-12T01:36:49.111 INFO:teuthology.orchestra.run.smithi001.stdout: "traverse_dir_fetch": 0, 2024-09-12T01:36:49.111 INFO:teuthology.orchestra.run.smithi001.stdout: "traverse_remote_ino": 0, 2024-09-12T01:36:49.111 INFO:teuthology.orchestra.run.smithi001.stdout: "traverse_lock": 0, 2024-09-12T01:36:49.111 INFO:teuthology.orchestra.run.smithi001.stdout: "load_cent": 0, 2024-09-12T01:36:49.112 INFO:teuthology.orchestra.run.smithi001.stdout: "q": 0, 2024-09-12T01:36:49.112 INFO:teuthology.orchestra.run.smithi001.stdout: "exported": 0, 2024-09-12T01:36:49.112 INFO:teuthology.orchestra.run.smithi001.stdout: "exported_inodes": 0, 2024-09-12T01:36:49.112 INFO:teuthology.orchestra.run.smithi001.stdout: "imported": 0, 2024-09-12T01:36:49.112 INFO:teuthology.orchestra.run.smithi001.stdout: "imported_inodes": 0, 2024-09-12T01:36:49.112 INFO:teuthology.orchestra.run.smithi001.stdout: "openino_dir_fetch": 0, 2024-09-12T01:36:49.112 INFO:teuthology.orchestra.run.smithi001.stdout: "openino_backtrace_fetch": 0, 2024-09-12T01:36:49.112 INFO:teuthology.orchestra.run.smithi001.stdout: "openino_peer_discover": 0, 2024-09-12T01:36:49.112 INFO:teuthology.orchestra.run.smithi001.stdout: "root_rfiles": 0, 2024-09-12T01:36:49.112 INFO:teuthology.orchestra.run.smithi001.stdout: "root_rbytes": 0, 2024-09-12T01:36:49.112 INFO:teuthology.orchestra.run.smithi001.stdout: "root_rsnaps": 0, 2024-09-12T01:36:49.112 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T01:36:49.112 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_set_tag": 0, 2024-09-12T01:36:49.112 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T01:36:49.113 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_inotable_repaired": 0, 2024-09-12T01:36:49.113 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_dir_inodes": 0, 2024-09-12T01:36:49.113 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T01:36:49.113 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T01:36:49.113 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_file_inodes": 0, 2024-09-12T01:36:49.113 INFO:teuthology.orchestra.run.smithi001.stdout: "handle_inode_file_caps": 0, 2024-09-12T01:36:49.113 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T01:36:49.113 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_cap_op_grant": 0, 2024-09-12T01:36:49.113 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T01:36:49.113 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T01:36:49.113 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T01:36:49.113 INFO:teuthology.orchestra.run.smithi001.stdout: "handle_client_caps": 0, 2024-09-12T01:36:49.113 INFO:teuthology.orchestra.run.smithi001.stdout: "handle_client_caps_dirty": 0, 2024-09-12T01:36:49.114 INFO:teuthology.orchestra.run.smithi001.stdout: "handle_client_cap_release": 0, 2024-09-12T01:36:49.114 INFO:teuthology.orchestra.run.smithi001.stdout: "process_request_cap_release": 0 2024-09-12T01:36:49.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.114 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_cache": { 2024-09-12T01:36:49.114 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_update": 0, 2024-09-12T01:36:49.114 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_update_receipt": 0, 2024-09-12T01:36:49.114 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_try_discover": 0, 2024-09-12T01:36:49.114 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_send_discover": 0, 2024-09-12T01:36:49.114 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_handle_discover": 0, 2024-09-12T01:36:49.114 INFO:teuthology.orchestra.run.smithi001.stdout: "num_strays": 0, 2024-09-12T01:36:49.114 INFO:teuthology.orchestra.run.smithi001.stdout: "num_strays_delayed": 0, 2024-09-12T01:36:49.114 INFO:teuthology.orchestra.run.smithi001.stdout: "num_strays_enqueuing": 0, 2024-09-12T01:36:49.114 INFO:teuthology.orchestra.run.smithi001.stdout: "strays_created": 0, 2024-09-12T01:36:49.114 INFO:teuthology.orchestra.run.smithi001.stdout: "strays_enqueued": 0, 2024-09-12T01:36:49.115 INFO:teuthology.orchestra.run.smithi001.stdout: "strays_reintegrated": 0, 2024-09-12T01:36:49.115 INFO:teuthology.orchestra.run.smithi001.stdout: "strays_migrated": 0, 2024-09-12T01:36:49.115 INFO:teuthology.orchestra.run.smithi001.stdout: "num_recovering_processing": 0, 2024-09-12T01:36:49.115 INFO:teuthology.orchestra.run.smithi001.stdout: "num_recovering_enqueued": 0, 2024-09-12T01:36:49.115 INFO:teuthology.orchestra.run.smithi001.stdout: "num_recovering_prioritized": 0, 2024-09-12T01:36:49.115 INFO:teuthology.orchestra.run.smithi001.stdout: "recovery_started": 0, 2024-09-12T01:36:49.115 INFO:teuthology.orchestra.run.smithi001.stdout: "recovery_completed": 0, 2024-09-12T01:36:49.115 INFO:teuthology.orchestra.run.smithi001.stdout: "ireq_quiesce_path": 0, 2024-09-12T01:36:49.115 INFO:teuthology.orchestra.run.smithi001.stdout: "ireq_quiesce_inode": 0, 2024-09-12T01:36:49.115 INFO:teuthology.orchestra.run.smithi001.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T01:36:49.115 INFO:teuthology.orchestra.run.smithi001.stdout: "ireq_exportdir": 0, 2024-09-12T01:36:49.115 INFO:teuthology.orchestra.run.smithi001.stdout: "ireq_flush": 0, 2024-09-12T01:36:49.115 INFO:teuthology.orchestra.run.smithi001.stdout: "ireq_fragmentdir": 0, 2024-09-12T01:36:49.115 INFO:teuthology.orchestra.run.smithi001.stdout: "ireq_fragstats": 0, 2024-09-12T01:36:49.115 INFO:teuthology.orchestra.run.smithi001.stdout: "ireq_inodestats": 0 2024-09-12T01:36:49.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.116 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_log": { 2024-09-12T01:36:49.116 INFO:teuthology.orchestra.run.smithi001.stdout: "evlrg": 0, 2024-09-12T01:36:49.116 INFO:teuthology.orchestra.run.smithi001.stdout: "evadd": 0, 2024-09-12T01:36:49.116 INFO:teuthology.orchestra.run.smithi001.stdout: "evex": 0, 2024-09-12T01:36:49.116 INFO:teuthology.orchestra.run.smithi001.stdout: "evtrm": 0, 2024-09-12T01:36:49.116 INFO:teuthology.orchestra.run.smithi001.stdout: "ev": 36, 2024-09-12T01:36:49.116 INFO:teuthology.orchestra.run.smithi001.stdout: "evexg": 0, 2024-09-12T01:36:49.116 INFO:teuthology.orchestra.run.smithi001.stdout: "evexd": 0, 2024-09-12T01:36:49.116 INFO:teuthology.orchestra.run.smithi001.stdout: "segadd": 0, 2024-09-12T01:36:49.116 INFO:teuthology.orchestra.run.smithi001.stdout: "segex": 0, 2024-09-12T01:36:49.116 INFO:teuthology.orchestra.run.smithi001.stdout: "segtrm": 0, 2024-09-12T01:36:49.116 INFO:teuthology.orchestra.run.smithi001.stdout: "seg": 2, 2024-09-12T01:36:49.117 INFO:teuthology.orchestra.run.smithi001.stdout: "segmjr": 2, 2024-09-12T01:36:49.117 INFO:teuthology.orchestra.run.smithi001.stdout: "segexg": 0, 2024-09-12T01:36:49.117 INFO:teuthology.orchestra.run.smithi001.stdout: "segexd": 0, 2024-09-12T01:36:49.117 INFO:teuthology.orchestra.run.smithi001.stdout: "expos": 4194304, 2024-09-12T01:36:49.117 INFO:teuthology.orchestra.run.smithi001.stdout: "wrpos": 4241997, 2024-09-12T01:36:49.117 INFO:teuthology.orchestra.run.smithi001.stdout: "rdpos": 4239713, 2024-09-12T01:36:49.117 INFO:teuthology.orchestra.run.smithi001.stdout: "jlat": { 2024-09-12T01:36:49.117 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:49.117 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:49.117 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:49.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.117 INFO:teuthology.orchestra.run.smithi001.stdout: "replayed": 36 2024-09-12T01:36:49.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.117 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_mem": { 2024-09-12T01:36:49.118 INFO:teuthology.orchestra.run.smithi001.stdout: "ino": 4, 2024-09-12T01:36:49.118 INFO:teuthology.orchestra.run.smithi001.stdout: "ino+": 20, 2024-09-12T01:36:49.118 INFO:teuthology.orchestra.run.smithi001.stdout: "ino-": 16, 2024-09-12T01:36:49.118 INFO:teuthology.orchestra.run.smithi001.stdout: "dir": 2, 2024-09-12T01:36:49.118 INFO:teuthology.orchestra.run.smithi001.stdout: "dir+": 14, 2024-09-12T01:36:49.118 INFO:teuthology.orchestra.run.smithi001.stdout: "dir-": 12, 2024-09-12T01:36:49.118 INFO:teuthology.orchestra.run.smithi001.stdout: "dn": 6, 2024-09-12T01:36:49.118 INFO:teuthology.orchestra.run.smithi001.stdout: "dn+": 22, 2024-09-12T01:36:49.118 INFO:teuthology.orchestra.run.smithi001.stdout: "dn-": 16, 2024-09-12T01:36:49.118 INFO:teuthology.orchestra.run.smithi001.stdout: "cap": 0, 2024-09-12T01:36:49.118 INFO:teuthology.orchestra.run.smithi001.stdout: "cap+": 0, 2024-09-12T01:36:49.118 INFO:teuthology.orchestra.run.smithi001.stdout: "cap-": 0, 2024-09-12T01:36:49.118 INFO:teuthology.orchestra.run.smithi001.stdout: "rss": 46744, 2024-09-12T01:36:49.118 INFO:teuthology.orchestra.run.smithi001.stdout: "heap": 215328 2024-09-12T01:36:49.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.119 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_server": { 2024-09-12T01:36:49.119 INFO:teuthology.orchestra.run.smithi001.stdout: "dispatch_client_request": 0, 2024-09-12T01:36:49.119 INFO:teuthology.orchestra.run.smithi001.stdout: "dispatch_server_request": 0, 2024-09-12T01:36:49.119 INFO:teuthology.orchestra.run.smithi001.stdout: "handle_client_request": 0, 2024-09-12T01:36:49.119 INFO:teuthology.orchestra.run.smithi001.stdout: "handle_client_session": 0, 2024-09-12T01:36:49.119 INFO:teuthology.orchestra.run.smithi001.stdout: "handle_peer_request": 0, 2024-09-12T01:36:49.119 INFO:teuthology.orchestra.run.smithi001.stdout: "req_create_latency": { 2024-09-12T01:36:49.119 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:49.119 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:49.119 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:49.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.119 INFO:teuthology.orchestra.run.smithi001.stdout: "req_getattr_latency": { 2024-09-12T01:36:49.119 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:49.120 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:49.120 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:49.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.120 INFO:teuthology.orchestra.run.smithi001.stdout: "req_getfilelock_latency": { 2024-09-12T01:36:49.120 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:49.120 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:49.120 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:49.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.120 INFO:teuthology.orchestra.run.smithi001.stdout: "req_link_latency": { 2024-09-12T01:36:49.120 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:49.120 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:49.120 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:49.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.120 INFO:teuthology.orchestra.run.smithi001.stdout: "req_lookup_latency": { 2024-09-12T01:36:49.121 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:49.121 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:49.121 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:49.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.121 INFO:teuthology.orchestra.run.smithi001.stdout: "req_lookuphash_latency": { 2024-09-12T01:36:49.121 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:49.121 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:49.121 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:49.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.122 INFO:teuthology.orchestra.run.smithi001.stdout: "req_lookupino_latency": { 2024-09-12T01:36:49.122 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:49.122 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:49.122 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:49.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.122 INFO:teuthology.orchestra.run.smithi001.stdout: "req_lookupname_latency": { 2024-09-12T01:36:49.122 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:49.122 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:49.123 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:49.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.123 INFO:teuthology.orchestra.run.smithi001.stdout: "req_lookupparent_latency": { 2024-09-12T01:36:49.123 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:49.123 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:49.123 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:49.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.123 INFO:teuthology.orchestra.run.smithi001.stdout: "req_lookupsnap_latency": { 2024-09-12T01:36:49.123 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:49.123 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:49.123 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:49.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.123 INFO:teuthology.orchestra.run.smithi001.stdout: "req_lssnap_latency": { 2024-09-12T01:36:49.124 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:49.124 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:49.124 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:49.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.124 INFO:teuthology.orchestra.run.smithi001.stdout: "req_mkdir_latency": { 2024-09-12T01:36:49.124 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:49.124 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:49.125 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:49.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.125 INFO:teuthology.orchestra.run.smithi001.stdout: "req_mknod_latency": { 2024-09-12T01:36:49.125 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:49.125 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:49.125 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:49.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.125 INFO:teuthology.orchestra.run.smithi001.stdout: "req_mksnap_latency": { 2024-09-12T01:36:49.125 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:49.125 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:49.125 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:49.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.126 INFO:teuthology.orchestra.run.smithi001.stdout: "req_open_latency": { 2024-09-12T01:36:49.126 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:49.126 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:49.126 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:49.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.126 INFO:teuthology.orchestra.run.smithi001.stdout: "req_readdir_latency": { 2024-09-12T01:36:49.126 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:49.126 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:49.126 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:49.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.126 INFO:teuthology.orchestra.run.smithi001.stdout: "req_rename_latency": { 2024-09-12T01:36:49.126 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:49.126 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:49.126 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:49.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.127 INFO:teuthology.orchestra.run.smithi001.stdout: "req_renamesnap_latency": { 2024-09-12T01:36:49.127 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:49.127 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:49.127 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:49.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.127 INFO:teuthology.orchestra.run.smithi001.stdout: "req_snapdiff_latency": { 2024-09-12T01:36:49.127 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:49.127 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:49.127 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:49.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.127 INFO:teuthology.orchestra.run.smithi001.stdout: "req_rmdir_latency": { 2024-09-12T01:36:49.127 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:49.127 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:49.128 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:49.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.128 INFO:teuthology.orchestra.run.smithi001.stdout: "req_rmsnap_latency": { 2024-09-12T01:36:49.128 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:49.128 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:49.128 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:49.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.128 INFO:teuthology.orchestra.run.smithi001.stdout: "req_rmxattr_latency": { 2024-09-12T01:36:49.128 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:49.128 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:49.128 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:49.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.128 INFO:teuthology.orchestra.run.smithi001.stdout: "req_setattr_latency": { 2024-09-12T01:36:49.129 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:49.129 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:49.129 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:49.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.129 INFO:teuthology.orchestra.run.smithi001.stdout: "req_setdirlayout_latency": { 2024-09-12T01:36:49.129 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:49.129 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:49.129 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:49.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.129 INFO:teuthology.orchestra.run.smithi001.stdout: "req_setfilelock_latency": { 2024-09-12T01:36:49.129 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:49.129 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:49.129 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:49.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.130 INFO:teuthology.orchestra.run.smithi001.stdout: "req_setlayout_latency": { 2024-09-12T01:36:49.130 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:49.130 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:49.130 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:49.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.130 INFO:teuthology.orchestra.run.smithi001.stdout: "req_setxattr_latency": { 2024-09-12T01:36:49.130 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:49.130 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:49.130 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:49.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.130 INFO:teuthology.orchestra.run.smithi001.stdout: "req_symlink_latency": { 2024-09-12T01:36:49.130 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:49.130 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:49.130 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:49.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.131 INFO:teuthology.orchestra.run.smithi001.stdout: "req_unlink_latency": { 2024-09-12T01:36:49.131 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:49.131 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:49.131 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:49.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.131 INFO:teuthology.orchestra.run.smithi001.stdout: "cap_revoke_eviction": 0, 2024-09-12T01:36:49.131 INFO:teuthology.orchestra.run.smithi001.stdout: "cap_acquisition_throttle": 0, 2024-09-12T01:36:49.131 INFO:teuthology.orchestra.run.smithi001.stdout: "req_getvxattr_latency": { 2024-09-12T01:36:49.131 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:49.131 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:49.131 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:49.131 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:49.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.132 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_sessions": { 2024-09-12T01:36:49.132 INFO:teuthology.orchestra.run.smithi001.stdout: "session_count": 1, 2024-09-12T01:36:49.132 INFO:teuthology.orchestra.run.smithi001.stdout: "session_add": 2, 2024-09-12T01:36:49.132 INFO:teuthology.orchestra.run.smithi001.stdout: "session_remove": 1, 2024-09-12T01:36:49.132 INFO:teuthology.orchestra.run.smithi001.stdout: "sessions_open": 1, 2024-09-12T01:36:49.132 INFO:teuthology.orchestra.run.smithi001.stdout: "sessions_stale": 0, 2024-09-12T01:36:49.132 INFO:teuthology.orchestra.run.smithi001.stdout: "total_load": 0, 2024-09-12T01:36:49.132 INFO:teuthology.orchestra.run.smithi001.stdout: "average_load": 0, 2024-09-12T01:36:49.132 INFO:teuthology.orchestra.run.smithi001.stdout: "avg_session_uptime": 1651, 2024-09-12T01:36:49.132 INFO:teuthology.orchestra.run.smithi001.stdout: "mdthresh_evicted": 0 2024-09-12T01:36:49.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.133 INFO:teuthology.orchestra.run.smithi001.stdout: "mempool": { 2024-09-12T01:36:49.133 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_bytes": 0, 2024-09-12T01:36:49.133 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_items": 0, 2024-09-12T01:36:49.133 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T01:36:49.133 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_items": 0, 2024-09-12T01:36:49.133 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T01:36:49.133 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_items": 0, 2024-09-12T01:36:49.133 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T01:36:49.133 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T01:36:49.133 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T01:36:49.133 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T01:36:49.133 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T01:36:49.134 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_items": 0, 2024-09-12T01:36:49.134 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T01:36:49.134 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T01:36:49.134 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_bytes": 0, 2024-09-12T01:36:49.134 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_items": 0, 2024-09-12T01:36:49.134 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_bytes": 0, 2024-09-12T01:36:49.134 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_items": 0, 2024-09-12T01:36:49.134 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T01:36:49.134 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T01:36:49.134 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T01:36:49.134 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T01:36:49.134 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T01:36:49.134 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_items": 0, 2024-09-12T01:36:49.135 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_bytes": 0, 2024-09-12T01:36:49.135 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_items": 0, 2024-09-12T01:36:49.135 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T01:36:49.135 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T01:36:49.135 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_bytes": 0, 2024-09-12T01:36:49.135 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_items": 0, 2024-09-12T01:36:49.135 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_bytes": 0, 2024-09-12T01:36:49.135 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_items": 0, 2024-09-12T01:36:49.136 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T01:36:49.136 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_items": 0, 2024-09-12T01:36:49.136 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T01:36:49.136 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_items": 0, 2024-09-12T01:36:49.136 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_bytes": 189400, 2024-09-12T01:36:49.136 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_items": 51, 2024-09-12T01:36:49.136 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_bytes": 0, 2024-09-12T01:36:49.136 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_items": 0, 2024-09-12T01:36:49.136 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_bytes": 0, 2024-09-12T01:36:49.136 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_items": 0, 2024-09-12T01:36:49.136 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_bytes": 0, 2024-09-12T01:36:49.136 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_items": 0, 2024-09-12T01:36:49.136 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_bytes": 0, 2024-09-12T01:36:49.137 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_items": 0, 2024-09-12T01:36:49.137 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_bytes": 8856, 2024-09-12T01:36:49.137 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_items": 166, 2024-09-12T01:36:49.137 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T01:36:49.137 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_items": 0, 2024-09-12T01:36:49.137 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_bytes": 0, 2024-09-12T01:36:49.137 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_items": 0, 2024-09-12T01:36:49.137 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_bytes": 16280, 2024-09-12T01:36:49.137 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_items": 53, 2024-09-12T01:36:49.137 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_bytes": 0, 2024-09-12T01:36:49.137 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_items": 0, 2024-09-12T01:36:49.137 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_bytes": 0, 2024-09-12T01:36:49.137 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_items": 0 2024-09-12T01:36:49.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.138 INFO:teuthology.orchestra.run.smithi001.stdout: "objecter": { 2024-09-12T01:36:49.138 INFO:teuthology.orchestra.run.smithi001.stdout: "op_active": 0, 2024-09-12T01:36:49.138 INFO:teuthology.orchestra.run.smithi001.stdout: "op_laggy": 0, 2024-09-12T01:36:49.138 INFO:teuthology.orchestra.run.smithi001.stdout: "op_send": 1563, 2024-09-12T01:36:49.138 INFO:teuthology.orchestra.run.smithi001.stdout: "op_send_bytes": 21, 2024-09-12T01:36:49.138 INFO:teuthology.orchestra.run.smithi001.stdout: "op_resend": 3, 2024-09-12T01:36:49.138 INFO:teuthology.orchestra.run.smithi001.stdout: "op_reply": 1563, 2024-09-12T01:36:49.138 INFO:teuthology.orchestra.run.smithi001.stdout: "op_latency": { 2024-09-12T01:36:49.138 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 1563, 2024-09-12T01:36:49.138 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.947987523, 2024-09-12T01:36:49.138 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000606517 2024-09-12T01:36:49.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.138 INFO:teuthology.orchestra.run.smithi001.stdout: "op_inflight": 0, 2024-09-12T01:36:49.139 INFO:teuthology.orchestra.run.smithi001.stdout: "oplen_avg": { 2024-09-12T01:36:49.139 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 1563, 2024-09-12T01:36:49.139 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 1564 2024-09-12T01:36:49.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.139 INFO:teuthology.orchestra.run.smithi001.stdout: "op": 1563, 2024-09-12T01:36:49.139 INFO:teuthology.orchestra.run.smithi001.stdout: "op_r": 1563, 2024-09-12T01:36:49.139 INFO:teuthology.orchestra.run.smithi001.stdout: "op_w": 0, 2024-09-12T01:36:49.139 INFO:teuthology.orchestra.run.smithi001.stdout: "op_rmw": 0, 2024-09-12T01:36:49.139 INFO:teuthology.orchestra.run.smithi001.stdout: "op_pg": 0, 2024-09-12T01:36:49.139 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_stat": 996, 2024-09-12T01:36:49.139 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_create": 0, 2024-09-12T01:36:49.139 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_read": 565, 2024-09-12T01:36:49.140 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_write": 0, 2024-09-12T01:36:49.140 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_writefull": 0, 2024-09-12T01:36:49.140 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_writesame": 0, 2024-09-12T01:36:49.140 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_append": 0, 2024-09-12T01:36:49.140 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_zero": 0, 2024-09-12T01:36:49.140 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_truncate": 0, 2024-09-12T01:36:49.140 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_delete": 0, 2024-09-12T01:36:49.140 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_mapext": 0, 2024-09-12T01:36:49.140 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_sparse_read": 0, 2024-09-12T01:36:49.140 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_clonerange": 0, 2024-09-12T01:36:49.140 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_getxattr": 1, 2024-09-12T01:36:49.141 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_setxattr": 0, 2024-09-12T01:36:49.141 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_cmpxattr": 0, 2024-09-12T01:36:49.141 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_rmxattr": 0, 2024-09-12T01:36:49.141 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_resetxattrs": 0, 2024-09-12T01:36:49.141 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_call": 0, 2024-09-12T01:36:49.141 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_watch": 0, 2024-09-12T01:36:49.141 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_notify": 0, 2024-09-12T01:36:49.141 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T01:36:49.141 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_pgls": 0, 2024-09-12T01:36:49.141 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_pgls_filter": 0, 2024-09-12T01:36:49.141 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_other": 0, 2024-09-12T01:36:49.141 INFO:teuthology.orchestra.run.smithi001.stdout: "linger_active": 0, 2024-09-12T01:36:49.141 INFO:teuthology.orchestra.run.smithi001.stdout: "linger_send": 0, 2024-09-12T01:36:49.141 INFO:teuthology.orchestra.run.smithi001.stdout: "linger_resend": 0, 2024-09-12T01:36:49.142 INFO:teuthology.orchestra.run.smithi001.stdout: "linger_ping": 0, 2024-09-12T01:36:49.142 INFO:teuthology.orchestra.run.smithi001.stdout: "poolop_active": 0, 2024-09-12T01:36:49.142 INFO:teuthology.orchestra.run.smithi001.stdout: "poolop_send": 0, 2024-09-12T01:36:49.142 INFO:teuthology.orchestra.run.smithi001.stdout: "poolop_resend": 0, 2024-09-12T01:36:49.142 INFO:teuthology.orchestra.run.smithi001.stdout: "poolstat_active": 0, 2024-09-12T01:36:49.142 INFO:teuthology.orchestra.run.smithi001.stdout: "poolstat_send": 0, 2024-09-12T01:36:49.142 INFO:teuthology.orchestra.run.smithi001.stdout: "poolstat_resend": 0, 2024-09-12T01:36:49.142 INFO:teuthology.orchestra.run.smithi001.stdout: "statfs_active": 0, 2024-09-12T01:36:49.142 INFO:teuthology.orchestra.run.smithi001.stdout: "statfs_send": 0, 2024-09-12T01:36:49.142 INFO:teuthology.orchestra.run.smithi001.stdout: "statfs_resend": 0, 2024-09-12T01:36:49.142 INFO:teuthology.orchestra.run.smithi001.stdout: "command_active": 0, 2024-09-12T01:36:49.142 INFO:teuthology.orchestra.run.smithi001.stdout: "command_send": 0, 2024-09-12T01:36:49.142 INFO:teuthology.orchestra.run.smithi001.stdout: "command_resend": 0, 2024-09-12T01:36:49.142 INFO:teuthology.orchestra.run.smithi001.stdout: "map_epoch": 98, 2024-09-12T01:36:49.142 INFO:teuthology.orchestra.run.smithi001.stdout: "map_full": 0, 2024-09-12T01:36:49.142 INFO:teuthology.orchestra.run.smithi001.stdout: "map_inc": 2, 2024-09-12T01:36:49.143 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_sessions": 5, 2024-09-12T01:36:49.143 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_session_open": 5, 2024-09-12T01:36:49.143 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_session_close": 0, 2024-09-12T01:36:49.143 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_laggy": 0, 2024-09-12T01:36:49.143 INFO:teuthology.orchestra.run.smithi001.stdout: "omap_wr": 0, 2024-09-12T01:36:49.143 INFO:teuthology.orchestra.run.smithi001.stdout: "omap_rd": 2, 2024-09-12T01:36:49.143 INFO:teuthology.orchestra.run.smithi001.stdout: "omap_del": 0 2024-09-12T01:36:49.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.143 INFO:teuthology.orchestra.run.smithi001.stdout: "oft": { 2024-09-12T01:36:49.143 INFO:teuthology.orchestra.run.smithi001.stdout: "omap_total_objs": 0, 2024-09-12T01:36:49.143 INFO:teuthology.orchestra.run.smithi001.stdout: "omap_total_kv_pairs": 0, 2024-09-12T01:36:49.143 INFO:teuthology.orchestra.run.smithi001.stdout: "omap_total_updates": 0, 2024-09-12T01:36:49.143 INFO:teuthology.orchestra.run.smithi001.stdout: "omap_total_removes": 0 2024-09-12T01:36:49.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.144 INFO:teuthology.orchestra.run.smithi001.stdout: "osd-slow-ops": { 2024-09-12T01:36:49.144 INFO:teuthology.orchestra.run.smithi001.stdout: "slow_ops_count": 0 2024-09-12T01:36:49.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.144 INFO:teuthology.orchestra.run.smithi001.stdout: "purge_queue": { 2024-09-12T01:36:49.144 INFO:teuthology.orchestra.run.smithi001.stdout: "pq_executing_ops": 0, 2024-09-12T01:36:49.144 INFO:teuthology.orchestra.run.smithi001.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T01:36:49.144 INFO:teuthology.orchestra.run.smithi001.stdout: "pq_executing": 0, 2024-09-12T01:36:49.144 INFO:teuthology.orchestra.run.smithi001.stdout: "pq_executing_high_water": 0, 2024-09-12T01:36:49.144 INFO:teuthology.orchestra.run.smithi001.stdout: "pq_executed_ops": 0, 2024-09-12T01:36:49.144 INFO:teuthology.orchestra.run.smithi001.stdout: "pq_executed": 0, 2024-09-12T01:36:49.144 INFO:teuthology.orchestra.run.smithi001.stdout: "pq_item_in_journal": 0 2024-09-12T01:36:49.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.144 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T01:36:49.145 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-09-12T01:36:49.145 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 104857600, 2024-09-12T01:36:49.145 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-09-12T01:36:49.145 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 1754, 2024-09-12T01:36:49.145 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 1154834, 2024-09-12T01:36:49.145 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-09-12T01:36:49.145 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 1754, 2024-09-12T01:36:49.145 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 0, 2024-09-12T01:36:49.145 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 0, 2024-09-12T01:36:49.145 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 1754, 2024-09-12T01:36:49.145 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 1154834, 2024-09-12T01:36:49.145 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-09-12T01:36:49.145 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:49.145 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:49.145 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:49.146 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:49.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.146 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-objecter_bytes": { 2024-09-12T01:36:49.146 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-09-12T01:36:49.146 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 104857600, 2024-09-12T01:36:49.146 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-09-12T01:36:49.146 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 0, 2024-09-12T01:36:49.146 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 0, 2024-09-12T01:36:49.146 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-09-12T01:36:49.146 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 0, 2024-09-12T01:36:49.146 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 1563, 2024-09-12T01:36:49.146 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 47698, 2024-09-12T01:36:49.146 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 1563, 2024-09-12T01:36:49.146 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 47698, 2024-09-12T01:36:49.146 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-09-12T01:36:49.147 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:49.147 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:49.147 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:49.147 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:49.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.147 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-objecter_ops": { 2024-09-12T01:36:49.147 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-09-12T01:36:49.147 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 1024, 2024-09-12T01:36:49.147 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-09-12T01:36:49.147 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 0, 2024-09-12T01:36:49.147 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 0, 2024-09-12T01:36:49.147 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-09-12T01:36:49.147 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 0, 2024-09-12T01:36:49.147 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 1563, 2024-09-12T01:36:49.147 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 1563, 2024-09-12T01:36:49.148 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 1563, 2024-09-12T01:36:49.148 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 1563, 2024-09-12T01:36:49.148 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-09-12T01:36:49.148 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:49.148 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:49.148 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:49.148 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:49.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.148 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-write_buf_throttle": { 2024-09-12T01:36:49.148 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-09-12T01:36:49.148 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 3758096384, 2024-09-12T01:36:49.148 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-09-12T01:36:49.148 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 0, 2024-09-12T01:36:49.148 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 0, 2024-09-12T01:36:49.149 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-09-12T01:36:49.149 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 0, 2024-09-12T01:36:49.149 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 0, 2024-09-12T01:36:49.149 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 0, 2024-09-12T01:36:49.149 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 0, 2024-09-12T01:36:49.149 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 0, 2024-09-12T01:36:49.149 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-09-12T01:36:49.149 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:49.149 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:49.150 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:49.150 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:49.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:49.150 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-write_buf_throttle-0x555b05522780": { 2024-09-12T01:36:49.150 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-09-12T01:36:49.150 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 3758096384, 2024-09-12T01:36:49.150 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-09-12T01:36:49.150 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 0, 2024-09-12T01:36:49.150 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 0, 2024-09-12T01:36:49.150 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-09-12T01:36:49.151 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 0, 2024-09-12T01:36:49.151 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 0, 2024-09-12T01:36:49.151 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 0, 2024-09-12T01:36:49.151 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 0, 2024-09-12T01:36:49.151 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 0, 2024-09-12T01:36:49.151 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-09-12T01:36:49.151 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:49.151 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:49.151 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:49.151 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:49.151 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:49.151 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-12T01:36:50.182 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-12T01:36:50.182 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-12T01:36:50.183 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2024-09-12T01:36:50.183 DEBUG:tasks.check_counter:Getting stats from d 2024-09-12T01:36:50.183 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-12T01:36:50.564 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:36:50.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:50 smithi001 ceph-mon[30387]: pgmap v615: 129 pgs: 129 active+clean; 101 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 341 op/s 2024-09-12T01:36:50.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:50 smithi053 ceph-mon[35158]: pgmap v615: 129 pgs: 129 active+clean; 101 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 341 op/s 2024-09-12T01:36:50.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:50 smithi104 ceph-mon[35219]: pgmap v615: 129 pgs: 129 active+clean; 101 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 341 op/s 2024-09-12T01:36:51.184 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-12T01:36:51.184 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T01:36:51.184 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 0, 2024-09-12T01:36:51.184 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 2, 2024-09-12T01:36:51.184 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 0, 2024-09-12T01:36:51.184 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 0, 2024-09-12T01:36:51.184 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 4, 2024-09-12T01:36:51.185 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 0, 2024-09-12T01:36:51.185 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.003218374, 2024-09-12T01:36:51.185 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.000000000, 2024-09-12T01:36:51.185 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.000551487, 2024-09-12T01:36:51.185 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T01:36:51.185 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:36:51.185 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:51.185 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:51.185 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:51.185 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:51.186 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:36:51.186 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:51.186 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:51.186 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:51.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:51.186 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T01:36:51.186 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T01:36:51.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:51.186 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T01:36:51.186 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 183, 2024-09-12T01:36:51.186 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 176, 2024-09-12T01:36:51.186 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 745168, 2024-09-12T01:36:51.186 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 106896, 2024-09-12T01:36:51.187 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 2, 2024-09-12T01:36:51.187 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 1, 2024-09-12T01:36:51.187 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.042120513, 2024-09-12T01:36:51.187 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.018808046, 2024-09-12T01:36:51.187 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.031167870, 2024-09-12T01:36:51.187 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T01:36:51.187 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:36:51.187 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 176, 2024-09-12T01:36:51.187 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.018441217, 2024-09-12T01:36:51.187 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000104779 2024-09-12T01:36:51.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:51.188 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:36:51.188 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:51.188 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:51.188 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:51.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:51.188 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 745168, 2024-09-12T01:36:51.188 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 106896 2024-09-12T01:36:51.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:51.188 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T01:36:51.188 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 1, 2024-09-12T01:36:51.188 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 131, 2024-09-12T01:36:51.188 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 160, 2024-09-12T01:36:51.189 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 101584, 2024-09-12T01:36:51.189 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 3, 2024-09-12T01:36:51.189 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 2, 2024-09-12T01:36:51.189 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.015063453, 2024-09-12T01:36:51.189 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.011550722, 2024-09-12T01:36:51.189 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.001702432, 2024-09-12T01:36:51.189 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T01:36:51.189 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:36:51.189 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 129, 2024-09-12T01:36:51.189 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.009584570, 2024-09-12T01:36:51.189 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000074298 2024-09-12T01:36:51.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:51.190 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:36:51.190 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:51.190 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:51.190 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:51.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:51.190 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-12T01:36:51.190 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 101584 2024-09-12T01:36:51.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:51.190 INFO:teuthology.orchestra.run.smithi001.stdout: "cct": { 2024-09-12T01:36:51.191 INFO:teuthology.orchestra.run.smithi001.stdout: "total_workers": 0, 2024-09-12T01:36:51.191 INFO:teuthology.orchestra.run.smithi001.stdout: "unhealthy_workers": 0 2024-09-12T01:36:51.191 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:51.191 INFO:teuthology.orchestra.run.smithi001.stdout: "mempool": { 2024-09-12T01:36:51.191 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_bytes": 0, 2024-09-12T01:36:51.191 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_items": 0, 2024-09-12T01:36:51.191 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T01:36:51.191 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_items": 0, 2024-09-12T01:36:51.191 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T01:36:51.192 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_items": 0, 2024-09-12T01:36:51.192 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T01:36:51.192 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T01:36:51.192 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T01:36:51.192 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T01:36:51.192 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T01:36:51.192 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_items": 0, 2024-09-12T01:36:51.192 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T01:36:51.192 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T01:36:51.192 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_bytes": 0, 2024-09-12T01:36:51.193 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_items": 0, 2024-09-12T01:36:51.193 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_bytes": 0, 2024-09-12T01:36:51.193 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_items": 0, 2024-09-12T01:36:51.193 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T01:36:51.193 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T01:36:51.193 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T01:36:51.193 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T01:36:51.193 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T01:36:51.194 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_items": 0, 2024-09-12T01:36:51.194 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_bytes": 0, 2024-09-12T01:36:51.194 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_items": 0, 2024-09-12T01:36:51.194 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T01:36:51.194 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T01:36:51.195 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_bytes": 0, 2024-09-12T01:36:51.195 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_items": 0, 2024-09-12T01:36:51.195 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_bytes": 0, 2024-09-12T01:36:51.195 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_items": 0, 2024-09-12T01:36:51.195 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T01:36:51.196 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_items": 0, 2024-09-12T01:36:51.196 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T01:36:51.196 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_items": 0, 2024-09-12T01:36:51.197 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_bytes": 148967, 2024-09-12T01:36:51.197 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_items": 29, 2024-09-12T01:36:51.197 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_bytes": 0, 2024-09-12T01:36:51.197 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_items": 0, 2024-09-12T01:36:51.197 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_bytes": 0, 2024-09-12T01:36:51.197 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_items": 0, 2024-09-12T01:36:51.197 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_bytes": 0, 2024-09-12T01:36:51.197 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_items": 0, 2024-09-12T01:36:51.197 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_bytes": 0, 2024-09-12T01:36:51.197 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_items": 0, 2024-09-12T01:36:51.198 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_bytes": 0, 2024-09-12T01:36:51.198 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_items": 0, 2024-09-12T01:36:51.198 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T01:36:51.198 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_items": 0, 2024-09-12T01:36:51.198 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_bytes": 0, 2024-09-12T01:36:51.198 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_items": 0, 2024-09-12T01:36:51.198 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_bytes": 928, 2024-09-12T01:36:51.199 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_items": 2, 2024-09-12T01:36:51.199 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_bytes": 0, 2024-09-12T01:36:51.199 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_items": 0, 2024-09-12T01:36:51.199 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_bytes": 0, 2024-09-12T01:36:51.199 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_items": 0 2024-09-12T01:36:51.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:51.199 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T01:36:51.199 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-09-12T01:36:51.199 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 104857600, 2024-09-12T01:36:51.199 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-09-12T01:36:51.199 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 180, 2024-09-12T01:36:51.199 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 716886, 2024-09-12T01:36:51.199 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-09-12T01:36:51.199 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 180, 2024-09-12T01:36:51.200 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 0, 2024-09-12T01:36:51.200 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 0, 2024-09-12T01:36:51.200 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 180, 2024-09-12T01:36:51.200 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 716886, 2024-09-12T01:36:51.200 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-09-12T01:36:51.200 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:51.200 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:51.200 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:51.200 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:51.200 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:51.200 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-12T01:36:51.989 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-12T01:36:51.990 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-12T01:36:51.990 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-09-12T01:36:51.990 DEBUG:tasks.check_counter:Getting stats from g 2024-09-12T01:36:51.990 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-12T01:36:52.369 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:36:52.718 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:52 smithi001 ceph-mon[30387]: pgmap v616: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 335 op/s 2024-09-12T01:36:52.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:52 smithi053 ceph-mon[35158]: pgmap v616: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 335 op/s 2024-09-12T01:36:52.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:52 smithi104 ceph-mon[35219]: pgmap v616: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 335 op/s 2024-09-12T01:36:52.978 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-12T01:36:52.978 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T01:36:52.978 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 548, 2024-09-12T01:36:52.978 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 550, 2024-09-12T01:36:52.978 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 184073, 2024-09-12T01:36:52.978 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 174265, 2024-09-12T01:36:52.978 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 6, 2024-09-12T01:36:52.978 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 2, 2024-09-12T01:36:52.978 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.078570099, 2024-09-12T01:36:52.978 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.035002136, 2024-09-12T01:36:52.978 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.017178809, 2024-09-12T01:36:52.979 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.017028538, 2024-09-12T01:36:52.979 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:36:52.979 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 548, 2024-09-12T01:36:52.979 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.037260107, 2024-09-12T01:36:52.979 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000067992 2024-09-12T01:36:52.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:52.979 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:36:52.979 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:52.979 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:52.979 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:52.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:52.980 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T01:36:52.980 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T01:36:52.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:52.980 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T01:36:52.980 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 1204, 2024-09-12T01:36:52.980 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 1186, 2024-09-12T01:36:52.980 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 1093234, 2024-09-12T01:36:52.980 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 425961, 2024-09-12T01:36:52.980 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 4, 2024-09-12T01:36:52.980 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 3, 2024-09-12T01:36:52.980 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.121646349, 2024-09-12T01:36:52.980 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.042114351, 2024-09-12T01:36:52.980 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.059025653, 2024-09-12T01:36:52.981 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.026508181, 2024-09-12T01:36:52.981 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:36:52.981 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 1186, 2024-09-12T01:36:52.981 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.061027989, 2024-09-12T01:36:52.981 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000051456 2024-09-12T01:36:52.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:52.981 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:36:52.981 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:52.981 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:52.981 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:52.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:52.982 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 835328, 2024-09-12T01:36:52.982 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 105360 2024-09-12T01:36:52.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:52.982 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T01:36:52.982 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 4, 2024-09-12T01:36:52.982 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 131, 2024-09-12T01:36:52.982 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 1508, 2024-09-12T01:36:52.982 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 287445, 2024-09-12T01:36:52.982 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 5, 2024-09-12T01:36:52.982 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 4, 2024-09-12T01:36:52.982 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.014992431, 2024-09-12T01:36:52.982 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.010455748, 2024-09-12T01:36:52.983 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.002941570, 2024-09-12T01:36:52.983 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000065949, 2024-09-12T01:36:52.983 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:36:52.983 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 129, 2024-09-12T01:36:52.983 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.010325879, 2024-09-12T01:36:52.983 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000080045 2024-09-12T01:36:52.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:52.983 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:36:52.983 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:52.983 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:52.983 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:52.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:52.983 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-12T01:36:52.984 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 286480 2024-09-12T01:36:52.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:52.984 INFO:teuthology.orchestra.run.smithi001.stdout: "cct": { 2024-09-12T01:36:52.984 INFO:teuthology.orchestra.run.smithi001.stdout: "total_workers": 1, 2024-09-12T01:36:52.984 INFO:teuthology.orchestra.run.smithi001.stdout: "unhealthy_workers": 0 2024-09-12T01:36:52.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:52.984 INFO:teuthology.orchestra.run.smithi001.stdout: "finisher-MDSRank": { 2024-09-12T01:36:52.984 INFO:teuthology.orchestra.run.smithi001.stdout: "queue_len": 0, 2024-09-12T01:36:52.984 INFO:teuthology.orchestra.run.smithi001.stdout: "complete_latency": { 2024-09-12T01:36:52.984 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 2625, 2024-09-12T01:36:52.984 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.066374153, 2024-09-12T01:36:52.984 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000025285 2024-09-12T01:36:52.984 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:52.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:52.985 INFO:teuthology.orchestra.run.smithi001.stdout: "finisher-PurgeQueue": { 2024-09-12T01:36:52.985 INFO:teuthology.orchestra.run.smithi001.stdout: "queue_len": 0, 2024-09-12T01:36:52.985 INFO:teuthology.orchestra.run.smithi001.stdout: "complete_latency": { 2024-09-12T01:36:52.985 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:52.985 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:52.985 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:52.985 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:52.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:52.985 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": { 2024-09-12T01:36:52.985 INFO:teuthology.orchestra.run.smithi001.stdout: "request": 0, 2024-09-12T01:36:52.985 INFO:teuthology.orchestra.run.smithi001.stdout: "reply": 0, 2024-09-12T01:36:52.986 INFO:teuthology.orchestra.run.smithi001.stdout: "reply_latency": { 2024-09-12T01:36:52.986 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:52.986 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:52.986 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:52.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:52.986 INFO:teuthology.orchestra.run.smithi001.stdout: "slow_reply": 0, 2024-09-12T01:36:52.986 INFO:teuthology.orchestra.run.smithi001.stdout: "forward": 0, 2024-09-12T01:36:52.986 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_fetch_complete": 0, 2024-09-12T01:36:52.986 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_fetch_keys": 0, 2024-09-12T01:36:52.986 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_commit": 0, 2024-09-12T01:36:52.986 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_split": 0, 2024-09-12T01:36:52.986 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_merge": 0, 2024-09-12T01:36:52.987 INFO:teuthology.orchestra.run.smithi001.stdout: "inodes": 0, 2024-09-12T01:36:52.987 INFO:teuthology.orchestra.run.smithi001.stdout: "inodes_top": 0, 2024-09-12T01:36:52.987 INFO:teuthology.orchestra.run.smithi001.stdout: "inodes_bottom": 0, 2024-09-12T01:36:52.987 INFO:teuthology.orchestra.run.smithi001.stdout: "inodes_pin_tail": 0, 2024-09-12T01:36:52.987 INFO:teuthology.orchestra.run.smithi001.stdout: "inodes_pinned": 0, 2024-09-12T01:36:52.987 INFO:teuthology.orchestra.run.smithi001.stdout: "inodes_expired": 0, 2024-09-12T01:36:52.987 INFO:teuthology.orchestra.run.smithi001.stdout: "inodes_with_caps": 0, 2024-09-12T01:36:52.987 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": 0, 2024-09-12T01:36:52.987 INFO:teuthology.orchestra.run.smithi001.stdout: "subtrees": 1, 2024-09-12T01:36:52.987 INFO:teuthology.orchestra.run.smithi001.stdout: "traverse": 0, 2024-09-12T01:36:52.987 INFO:teuthology.orchestra.run.smithi001.stdout: "traverse_hit": 0, 2024-09-12T01:36:52.987 INFO:teuthology.orchestra.run.smithi001.stdout: "traverse_forward": 0, 2024-09-12T01:36:52.987 INFO:teuthology.orchestra.run.smithi001.stdout: "traverse_discover": 0, 2024-09-12T01:36:52.988 INFO:teuthology.orchestra.run.smithi001.stdout: "traverse_dir_fetch": 0, 2024-09-12T01:36:52.988 INFO:teuthology.orchestra.run.smithi001.stdout: "traverse_remote_ino": 0, 2024-09-12T01:36:52.988 INFO:teuthology.orchestra.run.smithi001.stdout: "traverse_lock": 0, 2024-09-12T01:36:52.988 INFO:teuthology.orchestra.run.smithi001.stdout: "load_cent": 0, 2024-09-12T01:36:52.988 INFO:teuthology.orchestra.run.smithi001.stdout: "q": 0, 2024-09-12T01:36:52.988 INFO:teuthology.orchestra.run.smithi001.stdout: "exported": 0, 2024-09-12T01:36:52.988 INFO:teuthology.orchestra.run.smithi001.stdout: "exported_inodes": 0, 2024-09-12T01:36:52.988 INFO:teuthology.orchestra.run.smithi001.stdout: "imported": 0, 2024-09-12T01:36:52.988 INFO:teuthology.orchestra.run.smithi001.stdout: "imported_inodes": 0, 2024-09-12T01:36:52.988 INFO:teuthology.orchestra.run.smithi001.stdout: "openino_dir_fetch": 0, 2024-09-12T01:36:52.988 INFO:teuthology.orchestra.run.smithi001.stdout: "openino_backtrace_fetch": 0, 2024-09-12T01:36:52.988 INFO:teuthology.orchestra.run.smithi001.stdout: "openino_peer_discover": 0, 2024-09-12T01:36:52.988 INFO:teuthology.orchestra.run.smithi001.stdout: "root_rfiles": 0, 2024-09-12T01:36:52.989 INFO:teuthology.orchestra.run.smithi001.stdout: "root_rbytes": 0, 2024-09-12T01:36:52.989 INFO:teuthology.orchestra.run.smithi001.stdout: "root_rsnaps": 0, 2024-09-12T01:36:52.989 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T01:36:52.989 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_set_tag": 0, 2024-09-12T01:36:52.989 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T01:36:52.989 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_inotable_repaired": 0, 2024-09-12T01:36:52.989 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_dir_inodes": 0, 2024-09-12T01:36:52.989 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T01:36:52.990 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T01:36:52.990 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_file_inodes": 0, 2024-09-12T01:36:52.990 INFO:teuthology.orchestra.run.smithi001.stdout: "handle_inode_file_caps": 0, 2024-09-12T01:36:52.990 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T01:36:52.990 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_cap_op_grant": 0, 2024-09-12T01:36:52.990 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T01:36:52.990 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T01:36:52.990 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T01:36:52.990 INFO:teuthology.orchestra.run.smithi001.stdout: "handle_client_caps": 0, 2024-09-12T01:36:52.990 INFO:teuthology.orchestra.run.smithi001.stdout: "handle_client_caps_dirty": 0, 2024-09-12T01:36:52.991 INFO:teuthology.orchestra.run.smithi001.stdout: "handle_client_cap_release": 0, 2024-09-12T01:36:52.991 INFO:teuthology.orchestra.run.smithi001.stdout: "process_request_cap_release": 0 2024-09-12T01:36:52.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:52.991 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_cache": { 2024-09-12T01:36:52.991 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_update": 0, 2024-09-12T01:36:52.991 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_update_receipt": 0, 2024-09-12T01:36:52.991 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_try_discover": 0, 2024-09-12T01:36:52.991 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_send_discover": 0, 2024-09-12T01:36:52.991 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_handle_discover": 0, 2024-09-12T01:36:52.991 INFO:teuthology.orchestra.run.smithi001.stdout: "num_strays": 0, 2024-09-12T01:36:52.991 INFO:teuthology.orchestra.run.smithi001.stdout: "num_strays_delayed": 0, 2024-09-12T01:36:52.991 INFO:teuthology.orchestra.run.smithi001.stdout: "num_strays_enqueuing": 0, 2024-09-12T01:36:52.992 INFO:teuthology.orchestra.run.smithi001.stdout: "strays_created": 0, 2024-09-12T01:36:52.992 INFO:teuthology.orchestra.run.smithi001.stdout: "strays_enqueued": 0, 2024-09-12T01:36:52.992 INFO:teuthology.orchestra.run.smithi001.stdout: "strays_reintegrated": 0, 2024-09-12T01:36:52.992 INFO:teuthology.orchestra.run.smithi001.stdout: "strays_migrated": 0, 2024-09-12T01:36:52.992 INFO:teuthology.orchestra.run.smithi001.stdout: "num_recovering_processing": 0, 2024-09-12T01:36:52.992 INFO:teuthology.orchestra.run.smithi001.stdout: "num_recovering_enqueued": 0, 2024-09-12T01:36:52.992 INFO:teuthology.orchestra.run.smithi001.stdout: "num_recovering_prioritized": 0, 2024-09-12T01:36:52.992 INFO:teuthology.orchestra.run.smithi001.stdout: "recovery_started": 0, 2024-09-12T01:36:52.992 INFO:teuthology.orchestra.run.smithi001.stdout: "recovery_completed": 0, 2024-09-12T01:36:52.992 INFO:teuthology.orchestra.run.smithi001.stdout: "ireq_quiesce_path": 0, 2024-09-12T01:36:52.992 INFO:teuthology.orchestra.run.smithi001.stdout: "ireq_quiesce_inode": 0, 2024-09-12T01:36:52.992 INFO:teuthology.orchestra.run.smithi001.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T01:36:52.992 INFO:teuthology.orchestra.run.smithi001.stdout: "ireq_exportdir": 0, 2024-09-12T01:36:52.993 INFO:teuthology.orchestra.run.smithi001.stdout: "ireq_flush": 0, 2024-09-12T01:36:52.993 INFO:teuthology.orchestra.run.smithi001.stdout: "ireq_fragmentdir": 0, 2024-09-12T01:36:52.993 INFO:teuthology.orchestra.run.smithi001.stdout: "ireq_fragstats": 0, 2024-09-12T01:36:52.993 INFO:teuthology.orchestra.run.smithi001.stdout: "ireq_inodestats": 0 2024-09-12T01:36:52.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:52.993 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_log": { 2024-09-12T01:36:52.993 INFO:teuthology.orchestra.run.smithi001.stdout: "evlrg": 0, 2024-09-12T01:36:52.993 INFO:teuthology.orchestra.run.smithi001.stdout: "evadd": 0, 2024-09-12T01:36:52.993 INFO:teuthology.orchestra.run.smithi001.stdout: "evex": 0, 2024-09-12T01:36:52.993 INFO:teuthology.orchestra.run.smithi001.stdout: "evtrm": 0, 2024-09-12T01:36:52.993 INFO:teuthology.orchestra.run.smithi001.stdout: "ev": 4, 2024-09-12T01:36:52.993 INFO:teuthology.orchestra.run.smithi001.stdout: "evexg": 0, 2024-09-12T01:36:52.993 INFO:teuthology.orchestra.run.smithi001.stdout: "evexd": 0, 2024-09-12T01:36:52.993 INFO:teuthology.orchestra.run.smithi001.stdout: "segadd": 0, 2024-09-12T01:36:52.994 INFO:teuthology.orchestra.run.smithi001.stdout: "segex": 0, 2024-09-12T01:36:52.994 INFO:teuthology.orchestra.run.smithi001.stdout: "segtrm": 0, 2024-09-12T01:36:52.994 INFO:teuthology.orchestra.run.smithi001.stdout: "seg": 2, 2024-09-12T01:36:52.994 INFO:teuthology.orchestra.run.smithi001.stdout: "segmjr": 2, 2024-09-12T01:36:52.994 INFO:teuthology.orchestra.run.smithi001.stdout: "segexg": 0, 2024-09-12T01:36:52.994 INFO:teuthology.orchestra.run.smithi001.stdout: "segexd": 0, 2024-09-12T01:36:52.994 INFO:teuthology.orchestra.run.smithi001.stdout: "expos": 4194304, 2024-09-12T01:36:52.994 INFO:teuthology.orchestra.run.smithi001.stdout: "wrpos": 4195426, 2024-09-12T01:36:52.994 INFO:teuthology.orchestra.run.smithi001.stdout: "rdpos": 4195285, 2024-09-12T01:36:52.994 INFO:teuthology.orchestra.run.smithi001.stdout: "jlat": { 2024-09-12T01:36:52.994 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:52.994 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:52.994 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:52.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:52.995 INFO:teuthology.orchestra.run.smithi001.stdout: "replayed": 4 2024-09-12T01:36:52.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:52.995 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_mem": { 2024-09-12T01:36:52.995 INFO:teuthology.orchestra.run.smithi001.stdout: "ino": 3, 2024-09-12T01:36:52.995 INFO:teuthology.orchestra.run.smithi001.stdout: "ino+": 3, 2024-09-12T01:36:52.995 INFO:teuthology.orchestra.run.smithi001.stdout: "ino-": 0, 2024-09-12T01:36:52.995 INFO:teuthology.orchestra.run.smithi001.stdout: "dir": 1, 2024-09-12T01:36:52.995 INFO:teuthology.orchestra.run.smithi001.stdout: "dir+": 1, 2024-09-12T01:36:52.995 INFO:teuthology.orchestra.run.smithi001.stdout: "dir-": 0, 2024-09-12T01:36:52.995 INFO:teuthology.orchestra.run.smithi001.stdout: "dn": 0, 2024-09-12T01:36:52.995 INFO:teuthology.orchestra.run.smithi001.stdout: "dn+": 0, 2024-09-12T01:36:52.995 INFO:teuthology.orchestra.run.smithi001.stdout: "dn-": 0, 2024-09-12T01:36:52.995 INFO:teuthology.orchestra.run.smithi001.stdout: "cap": 0, 2024-09-12T01:36:52.995 INFO:teuthology.orchestra.run.smithi001.stdout: "cap+": 0, 2024-09-12T01:36:52.996 INFO:teuthology.orchestra.run.smithi001.stdout: "cap-": 0, 2024-09-12T01:36:52.996 INFO:teuthology.orchestra.run.smithi001.stdout: "rss": 46912, 2024-09-12T01:36:52.996 INFO:teuthology.orchestra.run.smithi001.stdout: "heap": 215328 2024-09-12T01:36:52.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:52.996 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_server": { 2024-09-12T01:36:52.996 INFO:teuthology.orchestra.run.smithi001.stdout: "dispatch_client_request": 0, 2024-09-12T01:36:52.996 INFO:teuthology.orchestra.run.smithi001.stdout: "dispatch_server_request": 0, 2024-09-12T01:36:52.996 INFO:teuthology.orchestra.run.smithi001.stdout: "handle_client_request": 0, 2024-09-12T01:36:52.996 INFO:teuthology.orchestra.run.smithi001.stdout: "handle_client_session": 0, 2024-09-12T01:36:52.996 INFO:teuthology.orchestra.run.smithi001.stdout: "handle_peer_request": 0, 2024-09-12T01:36:52.996 INFO:teuthology.orchestra.run.smithi001.stdout: "req_create_latency": { 2024-09-12T01:36:52.996 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:52.996 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:52.996 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:52.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:52.997 INFO:teuthology.orchestra.run.smithi001.stdout: "req_getattr_latency": { 2024-09-12T01:36:52.997 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:52.997 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:52.997 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:52.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:52.997 INFO:teuthology.orchestra.run.smithi001.stdout: "req_getfilelock_latency": { 2024-09-12T01:36:52.997 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:52.997 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:52.997 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:52.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:52.997 INFO:teuthology.orchestra.run.smithi001.stdout: "req_link_latency": { 2024-09-12T01:36:52.997 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:52.997 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:52.997 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:52.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:52.998 INFO:teuthology.orchestra.run.smithi001.stdout: "req_lookup_latency": { 2024-09-12T01:36:52.998 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:52.998 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:52.998 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:52.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:52.998 INFO:teuthology.orchestra.run.smithi001.stdout: "req_lookuphash_latency": { 2024-09-12T01:36:52.998 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:52.998 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:52.998 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:52.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:52.998 INFO:teuthology.orchestra.run.smithi001.stdout: "req_lookupino_latency": { 2024-09-12T01:36:52.998 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:52.998 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:52.999 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:52.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:52.999 INFO:teuthology.orchestra.run.smithi001.stdout: "req_lookupname_latency": { 2024-09-12T01:36:52.999 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:52.999 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:52.999 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:52.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:52.999 INFO:teuthology.orchestra.run.smithi001.stdout: "req_lookupparent_latency": { 2024-09-12T01:36:52.999 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:52.999 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:52.999 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:52.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:52.999 INFO:teuthology.orchestra.run.smithi001.stdout: "req_lookupsnap_latency": { 2024-09-12T01:36:52.999 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:52.999 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:53.000 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:53.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:53.000 INFO:teuthology.orchestra.run.smithi001.stdout: "req_lssnap_latency": { 2024-09-12T01:36:53.000 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:53.000 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:53.000 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:53.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:53.000 INFO:teuthology.orchestra.run.smithi001.stdout: "req_mkdir_latency": { 2024-09-12T01:36:53.000 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:53.001 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:53.001 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:53.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:53.001 INFO:teuthology.orchestra.run.smithi001.stdout: "req_mknod_latency": { 2024-09-12T01:36:53.001 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:53.001 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:53.001 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:53.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:53.002 INFO:teuthology.orchestra.run.smithi001.stdout: "req_mksnap_latency": { 2024-09-12T01:36:53.002 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:53.002 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:53.002 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:53.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:53.002 INFO:teuthology.orchestra.run.smithi001.stdout: "req_open_latency": { 2024-09-12T01:36:53.002 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:53.002 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:53.002 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:53.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:53.002 INFO:teuthology.orchestra.run.smithi001.stdout: "req_readdir_latency": { 2024-09-12T01:36:53.003 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:53.003 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:53.003 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:53.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:53.003 INFO:teuthology.orchestra.run.smithi001.stdout: "req_rename_latency": { 2024-09-12T01:36:53.003 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:53.003 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:53.003 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:53.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:53.003 INFO:teuthology.orchestra.run.smithi001.stdout: "req_renamesnap_latency": { 2024-09-12T01:36:53.003 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:53.003 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:53.004 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:53.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:53.004 INFO:teuthology.orchestra.run.smithi001.stdout: "req_snapdiff_latency": { 2024-09-12T01:36:53.004 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:53.004 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:53.004 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:53.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:53.004 INFO:teuthology.orchestra.run.smithi001.stdout: "req_rmdir_latency": { 2024-09-12T01:36:53.004 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:53.004 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:53.004 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:53.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:53.004 INFO:teuthology.orchestra.run.smithi001.stdout: "req_rmsnap_latency": { 2024-09-12T01:36:53.005 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:53.005 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:53.005 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:53.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:53.005 INFO:teuthology.orchestra.run.smithi001.stdout: "req_rmxattr_latency": { 2024-09-12T01:36:53.005 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:53.005 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:53.005 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:53.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:53.005 INFO:teuthology.orchestra.run.smithi001.stdout: "req_setattr_latency": { 2024-09-12T01:36:53.005 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:53.005 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:53.005 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:53.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:53.006 INFO:teuthology.orchestra.run.smithi001.stdout: "req_setdirlayout_latency": { 2024-09-12T01:36:53.006 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:53.006 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:53.006 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:53.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:53.006 INFO:teuthology.orchestra.run.smithi001.stdout: "req_setfilelock_latency": { 2024-09-12T01:36:53.006 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:53.006 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:53.006 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:53.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:53.006 INFO:teuthology.orchestra.run.smithi001.stdout: "req_setlayout_latency": { 2024-09-12T01:36:53.006 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:53.006 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:53.006 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:53.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:53.007 INFO:teuthology.orchestra.run.smithi001.stdout: "req_setxattr_latency": { 2024-09-12T01:36:53.007 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:53.007 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:53.007 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:53.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:53.007 INFO:teuthology.orchestra.run.smithi001.stdout: "req_symlink_latency": { 2024-09-12T01:36:53.007 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:53.007 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:53.007 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:53.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:53.007 INFO:teuthology.orchestra.run.smithi001.stdout: "req_unlink_latency": { 2024-09-12T01:36:53.007 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:53.007 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:53.008 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:53.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:53.008 INFO:teuthology.orchestra.run.smithi001.stdout: "cap_revoke_eviction": 0, 2024-09-12T01:36:53.008 INFO:teuthology.orchestra.run.smithi001.stdout: "cap_acquisition_throttle": 0, 2024-09-12T01:36:53.008 INFO:teuthology.orchestra.run.smithi001.stdout: "req_getvxattr_latency": { 2024-09-12T01:36:53.008 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:53.008 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:53.008 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:53.008 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:53.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:53.008 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_sessions": { 2024-09-12T01:36:53.008 INFO:teuthology.orchestra.run.smithi001.stdout: "session_count": 0, 2024-09-12T01:36:53.008 INFO:teuthology.orchestra.run.smithi001.stdout: "session_add": 1, 2024-09-12T01:36:53.009 INFO:teuthology.orchestra.run.smithi001.stdout: "session_remove": 1, 2024-09-12T01:36:53.009 INFO:teuthology.orchestra.run.smithi001.stdout: "sessions_open": 1, 2024-09-12T01:36:53.009 INFO:teuthology.orchestra.run.smithi001.stdout: "sessions_stale": 0, 2024-09-12T01:36:53.009 INFO:teuthology.orchestra.run.smithi001.stdout: "total_load": 0, 2024-09-12T01:36:53.009 INFO:teuthology.orchestra.run.smithi001.stdout: "average_load": 0, 2024-09-12T01:36:53.009 INFO:teuthology.orchestra.run.smithi001.stdout: "avg_session_uptime": 1197, 2024-09-12T01:36:53.009 INFO:teuthology.orchestra.run.smithi001.stdout: "mdthresh_evicted": 0 2024-09-12T01:36:53.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:53.009 INFO:teuthology.orchestra.run.smithi001.stdout: "mempool": { 2024-09-12T01:36:53.009 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_bytes": 0, 2024-09-12T01:36:53.009 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_items": 0, 2024-09-12T01:36:53.009 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T01:36:53.009 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_items": 0, 2024-09-12T01:36:53.009 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T01:36:53.010 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_items": 0, 2024-09-12T01:36:53.010 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T01:36:53.010 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T01:36:53.010 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T01:36:53.010 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T01:36:53.010 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T01:36:53.010 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_items": 0, 2024-09-12T01:36:53.010 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T01:36:53.010 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T01:36:53.010 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_bytes": 0, 2024-09-12T01:36:53.010 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_items": 0, 2024-09-12T01:36:53.010 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_bytes": 0, 2024-09-12T01:36:53.010 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_items": 0, 2024-09-12T01:36:53.010 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T01:36:53.011 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T01:36:53.011 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T01:36:53.011 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T01:36:53.011 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T01:36:53.011 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_items": 0, 2024-09-12T01:36:53.011 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_bytes": 0, 2024-09-12T01:36:53.011 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_items": 0, 2024-09-12T01:36:53.011 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T01:36:53.011 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T01:36:53.011 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_bytes": 0, 2024-09-12T01:36:53.011 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_items": 0, 2024-09-12T01:36:53.011 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_bytes": 0, 2024-09-12T01:36:53.011 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_items": 0, 2024-09-12T01:36:53.011 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T01:36:53.012 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_items": 0, 2024-09-12T01:36:53.012 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T01:36:53.012 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_items": 0, 2024-09-12T01:36:53.012 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_bytes": 197463, 2024-09-12T01:36:53.012 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_items": 54, 2024-09-12T01:36:53.012 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_bytes": 0, 2024-09-12T01:36:53.012 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_items": 0, 2024-09-12T01:36:53.012 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_bytes": 0, 2024-09-12T01:36:53.012 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_items": 0, 2024-09-12T01:36:53.012 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_bytes": 0, 2024-09-12T01:36:53.012 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_items": 0, 2024-09-12T01:36:53.012 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_bytes": 0, 2024-09-12T01:36:53.012 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_items": 0, 2024-09-12T01:36:53.012 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_bytes": 8856, 2024-09-12T01:36:53.013 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_items": 166, 2024-09-12T01:36:53.013 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T01:36:53.013 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_items": 0, 2024-09-12T01:36:53.013 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_bytes": 0, 2024-09-12T01:36:53.013 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_items": 0, 2024-09-12T01:36:53.013 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_bytes": 8512, 2024-09-12T01:36:53.013 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_items": 13, 2024-09-12T01:36:53.013 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_bytes": 0, 2024-09-12T01:36:53.013 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_items": 0, 2024-09-12T01:36:53.013 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_bytes": 0, 2024-09-12T01:36:53.013 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_items": 0 2024-09-12T01:36:53.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:53.013 INFO:teuthology.orchestra.run.smithi001.stdout: "objecter": { 2024-09-12T01:36:53.013 INFO:teuthology.orchestra.run.smithi001.stdout: "op_active": 0, 2024-09-12T01:36:53.014 INFO:teuthology.orchestra.run.smithi001.stdout: "op_laggy": 0, 2024-09-12T01:36:53.014 INFO:teuthology.orchestra.run.smithi001.stdout: "op_send": 1559, 2024-09-12T01:36:53.014 INFO:teuthology.orchestra.run.smithi001.stdout: "op_send_bytes": 21, 2024-09-12T01:36:53.014 INFO:teuthology.orchestra.run.smithi001.stdout: "op_resend": 3, 2024-09-12T01:36:53.014 INFO:teuthology.orchestra.run.smithi001.stdout: "op_reply": 1559, 2024-09-12T01:36:53.014 INFO:teuthology.orchestra.run.smithi001.stdout: "op_latency": { 2024-09-12T01:36:53.014 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 1559, 2024-09-12T01:36:53.014 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 1.321982618, 2024-09-12T01:36:53.014 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000847968 2024-09-12T01:36:53.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:53.014 INFO:teuthology.orchestra.run.smithi001.stdout: "op_inflight": 0, 2024-09-12T01:36:53.014 INFO:teuthology.orchestra.run.smithi001.stdout: "oplen_avg": { 2024-09-12T01:36:53.014 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 1559, 2024-09-12T01:36:53.014 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 1560 2024-09-12T01:36:53.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:53.015 INFO:teuthology.orchestra.run.smithi001.stdout: "op": 1559, 2024-09-12T01:36:53.015 INFO:teuthology.orchestra.run.smithi001.stdout: "op_r": 1559, 2024-09-12T01:36:53.015 INFO:teuthology.orchestra.run.smithi001.stdout: "op_w": 0, 2024-09-12T01:36:53.015 INFO:teuthology.orchestra.run.smithi001.stdout: "op_rmw": 0, 2024-09-12T01:36:53.015 INFO:teuthology.orchestra.run.smithi001.stdout: "op_pg": 0, 2024-09-12T01:36:53.015 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_stat": 1004, 2024-09-12T01:36:53.015 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_create": 0, 2024-09-12T01:36:53.015 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_read": 553, 2024-09-12T01:36:53.015 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_write": 0, 2024-09-12T01:36:53.015 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_writefull": 0, 2024-09-12T01:36:53.015 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_writesame": 0, 2024-09-12T01:36:53.015 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_append": 0, 2024-09-12T01:36:53.015 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_zero": 0, 2024-09-12T01:36:53.015 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_truncate": 0, 2024-09-12T01:36:53.016 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_delete": 0, 2024-09-12T01:36:53.016 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_mapext": 0, 2024-09-12T01:36:53.016 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_sparse_read": 0, 2024-09-12T01:36:53.016 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_clonerange": 0, 2024-09-12T01:36:53.016 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_getxattr": 1, 2024-09-12T01:36:53.016 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_setxattr": 0, 2024-09-12T01:36:53.016 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_cmpxattr": 0, 2024-09-12T01:36:53.016 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_rmxattr": 0, 2024-09-12T01:36:53.016 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_resetxattrs": 0, 2024-09-12T01:36:53.016 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_call": 0, 2024-09-12T01:36:53.016 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_watch": 0, 2024-09-12T01:36:53.016 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_notify": 0, 2024-09-12T01:36:53.016 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T01:36:53.016 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_pgls": 0, 2024-09-12T01:36:53.017 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_pgls_filter": 0, 2024-09-12T01:36:53.017 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_other": 0, 2024-09-12T01:36:53.017 INFO:teuthology.orchestra.run.smithi001.stdout: "linger_active": 0, 2024-09-12T01:36:53.017 INFO:teuthology.orchestra.run.smithi001.stdout: "linger_send": 0, 2024-09-12T01:36:53.017 INFO:teuthology.orchestra.run.smithi001.stdout: "linger_resend": 0, 2024-09-12T01:36:53.017 INFO:teuthology.orchestra.run.smithi001.stdout: "linger_ping": 0, 2024-09-12T01:36:53.017 INFO:teuthology.orchestra.run.smithi001.stdout: "poolop_active": 0, 2024-09-12T01:36:53.017 INFO:teuthology.orchestra.run.smithi001.stdout: "poolop_send": 0, 2024-09-12T01:36:53.017 INFO:teuthology.orchestra.run.smithi001.stdout: "poolop_resend": 0, 2024-09-12T01:36:53.017 INFO:teuthology.orchestra.run.smithi001.stdout: "poolstat_active": 0, 2024-09-12T01:36:53.017 INFO:teuthology.orchestra.run.smithi001.stdout: "poolstat_send": 0, 2024-09-12T01:36:53.017 INFO:teuthology.orchestra.run.smithi001.stdout: "poolstat_resend": 0, 2024-09-12T01:36:53.017 INFO:teuthology.orchestra.run.smithi001.stdout: "statfs_active": 0, 2024-09-12T01:36:53.017 INFO:teuthology.orchestra.run.smithi001.stdout: "statfs_send": 0, 2024-09-12T01:36:53.018 INFO:teuthology.orchestra.run.smithi001.stdout: "statfs_resend": 0, 2024-09-12T01:36:53.018 INFO:teuthology.orchestra.run.smithi001.stdout: "command_active": 0, 2024-09-12T01:36:53.018 INFO:teuthology.orchestra.run.smithi001.stdout: "command_send": 0, 2024-09-12T01:36:53.018 INFO:teuthology.orchestra.run.smithi001.stdout: "command_resend": 0, 2024-09-12T01:36:53.018 INFO:teuthology.orchestra.run.smithi001.stdout: "map_epoch": 98, 2024-09-12T01:36:53.018 INFO:teuthology.orchestra.run.smithi001.stdout: "map_full": 0, 2024-09-12T01:36:53.018 INFO:teuthology.orchestra.run.smithi001.stdout: "map_inc": 2, 2024-09-12T01:36:53.018 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_sessions": 6, 2024-09-12T01:36:53.018 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_session_open": 6, 2024-09-12T01:36:53.018 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_session_close": 0, 2024-09-12T01:36:53.018 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_laggy": 0, 2024-09-12T01:36:53.018 INFO:teuthology.orchestra.run.smithi001.stdout: "omap_wr": 0, 2024-09-12T01:36:53.018 INFO:teuthology.orchestra.run.smithi001.stdout: "omap_rd": 2, 2024-09-12T01:36:53.018 INFO:teuthology.orchestra.run.smithi001.stdout: "omap_del": 0 2024-09-12T01:36:53.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:53.019 INFO:teuthology.orchestra.run.smithi001.stdout: "oft": { 2024-09-12T01:36:53.019 INFO:teuthology.orchestra.run.smithi001.stdout: "omap_total_objs": 0, 2024-09-12T01:36:53.019 INFO:teuthology.orchestra.run.smithi001.stdout: "omap_total_kv_pairs": 0, 2024-09-12T01:36:53.019 INFO:teuthology.orchestra.run.smithi001.stdout: "omap_total_updates": 0, 2024-09-12T01:36:53.019 INFO:teuthology.orchestra.run.smithi001.stdout: "omap_total_removes": 0 2024-09-12T01:36:53.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:53.019 INFO:teuthology.orchestra.run.smithi001.stdout: "osd-slow-ops": { 2024-09-12T01:36:53.019 INFO:teuthology.orchestra.run.smithi001.stdout: "slow_ops_count": 0 2024-09-12T01:36:53.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:53.019 INFO:teuthology.orchestra.run.smithi001.stdout: "purge_queue": { 2024-09-12T01:36:53.019 INFO:teuthology.orchestra.run.smithi001.stdout: "pq_executing_ops": 0, 2024-09-12T01:36:53.019 INFO:teuthology.orchestra.run.smithi001.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T01:36:53.019 INFO:teuthology.orchestra.run.smithi001.stdout: "pq_executing": 0, 2024-09-12T01:36:53.019 INFO:teuthology.orchestra.run.smithi001.stdout: "pq_executing_high_water": 0, 2024-09-12T01:36:53.019 INFO:teuthology.orchestra.run.smithi001.stdout: "pq_executed_ops": 0, 2024-09-12T01:36:53.020 INFO:teuthology.orchestra.run.smithi001.stdout: "pq_executed": 0, 2024-09-12T01:36:53.020 INFO:teuthology.orchestra.run.smithi001.stdout: "pq_item_in_journal": 0 2024-09-12T01:36:53.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:53.020 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T01:36:53.020 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-09-12T01:36:53.020 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 104857600, 2024-09-12T01:36:53.020 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-09-12T01:36:53.020 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 1752, 2024-09-12T01:36:53.020 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 1108323, 2024-09-12T01:36:53.020 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-09-12T01:36:53.020 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 1752, 2024-09-12T01:36:53.020 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 0, 2024-09-12T01:36:53.020 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 0, 2024-09-12T01:36:53.020 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 1752, 2024-09-12T01:36:53.021 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 1108323, 2024-09-12T01:36:53.021 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-09-12T01:36:53.021 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:53.021 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:53.021 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:53.021 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:53.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:53.021 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-objecter_bytes": { 2024-09-12T01:36:53.021 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-09-12T01:36:53.021 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 104857600, 2024-09-12T01:36:53.021 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-09-12T01:36:53.021 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 0, 2024-09-12T01:36:53.021 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 0, 2024-09-12T01:36:53.021 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-09-12T01:36:53.022 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 0, 2024-09-12T01:36:53.022 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 1559, 2024-09-12T01:36:53.022 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 1127, 2024-09-12T01:36:53.022 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 1559, 2024-09-12T01:36:53.022 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 1127, 2024-09-12T01:36:53.022 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-09-12T01:36:53.022 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:53.022 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:53.022 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:53.022 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:53.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:53.022 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-objecter_ops": { 2024-09-12T01:36:53.022 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-09-12T01:36:53.022 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 1024, 2024-09-12T01:36:53.023 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-09-12T01:36:53.023 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 0, 2024-09-12T01:36:53.023 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 0, 2024-09-12T01:36:53.023 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-09-12T01:36:53.023 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 0, 2024-09-12T01:36:53.023 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 1559, 2024-09-12T01:36:53.023 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 1559, 2024-09-12T01:36:53.023 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 1559, 2024-09-12T01:36:53.023 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 1559, 2024-09-12T01:36:53.023 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-09-12T01:36:53.023 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:53.023 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:53.023 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:53.023 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:53.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:53.024 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-write_buf_throttle": { 2024-09-12T01:36:53.024 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-09-12T01:36:53.024 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 3758096384, 2024-09-12T01:36:53.024 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-09-12T01:36:53.024 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 0, 2024-09-12T01:36:53.024 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 0, 2024-09-12T01:36:53.024 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-09-12T01:36:53.024 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 0, 2024-09-12T01:36:53.024 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 0, 2024-09-12T01:36:53.024 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 0, 2024-09-12T01:36:53.024 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 0, 2024-09-12T01:36:53.024 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 0, 2024-09-12T01:36:53.024 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-09-12T01:36:53.025 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:53.025 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:53.025 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:53.025 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:53.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:53.025 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-write_buf_throttle-0x561dd98e2780": { 2024-09-12T01:36:53.025 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-09-12T01:36:53.025 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 3758096384, 2024-09-12T01:36:53.025 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-09-12T01:36:53.026 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 0, 2024-09-12T01:36:53.026 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 0, 2024-09-12T01:36:53.026 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-09-12T01:36:53.026 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 0, 2024-09-12T01:36:53.026 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 0, 2024-09-12T01:36:53.026 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 0, 2024-09-12T01:36:53.026 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 0, 2024-09-12T01:36:53.026 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 0, 2024-09-12T01:36:53.026 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-09-12T01:36:53.026 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:53.026 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:53.026 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:53.026 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:53.026 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:53.027 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-12T01:36:54.152 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-12T01:36:54.153 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-12T01:36:54.153 INFO:tasks.check_counter:Daemon mds.g mds.dir_split=0 2024-09-12T01:36:54.153 DEBUG:tasks.check_counter:Getting stats from j 2024-09-12T01:36:54.153 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-12T01:36:54.541 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:36:54.734 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:54 smithi001 ceph-mon[30387]: pgmap v617: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 793 KiB/s wr, 217 op/s 2024-09-12T01:36:54.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:54 smithi053 ceph-mon[35158]: pgmap v617: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 793 KiB/s wr, 217 op/s 2024-09-12T01:36:54.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:54 smithi104 ceph-mon[35219]: pgmap v617: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 793 KiB/s wr, 217 op/s 2024-09-12T01:36:55.171 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-12T01:36:55.171 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T01:36:55.171 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 0, 2024-09-12T01:36:55.171 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 2, 2024-09-12T01:36:55.171 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 0, 2024-09-12T01:36:55.171 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 0, 2024-09-12T01:36:55.171 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 4, 2024-09-12T01:36:55.171 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 0, 2024-09-12T01:36:55.171 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.003267725, 2024-09-12T01:36:55.172 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.000000000, 2024-09-12T01:36:55.172 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.000538031, 2024-09-12T01:36:55.172 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T01:36:55.172 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:36:55.172 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:55.172 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:55.172 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:55.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:55.172 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:36:55.172 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:55.172 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:55.172 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:55.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:55.173 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T01:36:55.173 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T01:36:55.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:55.173 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T01:36:55.173 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 5, 2024-09-12T01:36:55.173 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 135, 2024-09-12T01:36:55.173 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 3184, 2024-09-12T01:36:55.173 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 102384, 2024-09-12T01:36:55.173 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 3, 2024-09-12T01:36:55.173 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 2, 2024-09-12T01:36:55.173 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.016874548, 2024-09-12T01:36:55.173 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.010730022, 2024-09-12T01:36:55.173 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.004534023, 2024-09-12T01:36:55.173 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T01:36:55.174 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:36:55.174 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 134, 2024-09-12T01:36:55.174 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.016503675, 2024-09-12T01:36:55.174 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000123161 2024-09-12T01:36:55.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:55.174 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:36:55.174 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:55.174 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:55.174 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:55.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:55.174 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-09-12T01:36:55.174 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 102384 2024-09-12T01:36:55.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:55.175 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T01:36:55.175 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 181, 2024-09-12T01:36:55.175 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 176, 2024-09-12T01:36:55.175 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 742688, 2024-09-12T01:36:55.175 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 107728, 2024-09-12T01:36:55.175 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 2, 2024-09-12T01:36:55.175 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 1, 2024-09-12T01:36:55.175 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.041152281, 2024-09-12T01:36:55.175 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.021528379, 2024-09-12T01:36:55.175 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.027853390, 2024-09-12T01:36:55.175 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T01:36:55.175 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:36:55.175 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 175, 2024-09-12T01:36:55.175 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.011450923, 2024-09-12T01:36:55.176 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000065433 2024-09-12T01:36:55.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:55.176 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:36:55.176 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:55.176 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:55.176 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:55.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:55.176 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 742688, 2024-09-12T01:36:55.176 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 107728 2024-09-12T01:36:55.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:55.176 INFO:teuthology.orchestra.run.smithi001.stdout: "cct": { 2024-09-12T01:36:55.176 INFO:teuthology.orchestra.run.smithi001.stdout: "total_workers": 0, 2024-09-12T01:36:55.176 INFO:teuthology.orchestra.run.smithi001.stdout: "unhealthy_workers": 0 2024-09-12T01:36:55.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:55.177 INFO:teuthology.orchestra.run.smithi001.stdout: "mempool": { 2024-09-12T01:36:55.177 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_bytes": 0, 2024-09-12T01:36:55.177 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_items": 0, 2024-09-12T01:36:55.177 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T01:36:55.177 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_items": 0, 2024-09-12T01:36:55.177 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T01:36:55.177 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_items": 0, 2024-09-12T01:36:55.177 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T01:36:55.177 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T01:36:55.177 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T01:36:55.177 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T01:36:55.177 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T01:36:55.177 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_items": 0, 2024-09-12T01:36:55.177 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T01:36:55.178 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T01:36:55.178 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_bytes": 0, 2024-09-12T01:36:55.178 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_items": 0, 2024-09-12T01:36:55.178 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_bytes": 0, 2024-09-12T01:36:55.178 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_items": 0, 2024-09-12T01:36:55.178 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T01:36:55.178 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T01:36:55.178 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T01:36:55.178 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T01:36:55.178 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T01:36:55.178 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_items": 0, 2024-09-12T01:36:55.178 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_bytes": 0, 2024-09-12T01:36:55.178 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_items": 0, 2024-09-12T01:36:55.178 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T01:36:55.178 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T01:36:55.179 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_bytes": 0, 2024-09-12T01:36:55.179 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_items": 0, 2024-09-12T01:36:55.179 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_bytes": 0, 2024-09-12T01:36:55.179 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_items": 0, 2024-09-12T01:36:55.179 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T01:36:55.179 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_items": 0, 2024-09-12T01:36:55.179 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T01:36:55.179 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_items": 0, 2024-09-12T01:36:55.179 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_bytes": 148967, 2024-09-12T01:36:55.179 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_items": 29, 2024-09-12T01:36:55.179 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_bytes": 0, 2024-09-12T01:36:55.179 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_items": 0, 2024-09-12T01:36:55.179 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_bytes": 0, 2024-09-12T01:36:55.179 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_items": 0, 2024-09-12T01:36:55.180 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_bytes": 0, 2024-09-12T01:36:55.180 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_items": 0, 2024-09-12T01:36:55.180 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_bytes": 0, 2024-09-12T01:36:55.180 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_items": 0, 2024-09-12T01:36:55.180 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_bytes": 0, 2024-09-12T01:36:55.180 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_items": 0, 2024-09-12T01:36:55.180 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T01:36:55.180 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_items": 0, 2024-09-12T01:36:55.180 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_bytes": 0, 2024-09-12T01:36:55.180 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_items": 0, 2024-09-12T01:36:55.180 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_bytes": 928, 2024-09-12T01:36:55.180 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_items": 2, 2024-09-12T01:36:55.180 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_bytes": 0, 2024-09-12T01:36:55.180 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_items": 0, 2024-09-12T01:36:55.180 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_bytes": 0, 2024-09-12T01:36:55.181 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_items": 0 2024-09-12T01:36:55.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-12T01:36:55.181 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T01:36:55.181 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-09-12T01:36:55.181 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 104857600, 2024-09-12T01:36:55.181 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-09-12T01:36:55.181 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 182, 2024-09-12T01:36:55.181 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 717146, 2024-09-12T01:36:55.181 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-09-12T01:36:55.181 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 182, 2024-09-12T01:36:55.181 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 0, 2024-09-12T01:36:55.181 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 0, 2024-09-12T01:36:55.181 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 182, 2024-09-12T01:36:55.181 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 717146, 2024-09-12T01:36:55.182 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-09-12T01:36:55.182 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-12T01:36:55.182 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-12T01:36:55.182 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-12T01:36:55.182 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:55.182 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-12T01:36:55.182 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-12T01:36:56.418 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-12T01:36:56.418 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-12T01:36:56.418 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-09-12T01:36:56.418 DEBUG:tasks.check_counter:Getting stats from b 2024-09-12T01:36:56.418 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-12T01:36:56.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:56 smithi001 ceph-mon[30387]: pgmap v618: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 436 KiB/s wr, 182 op/s 2024-09-12T01:36:56.683 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T01:36:56.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:56 smithi053 ceph-mon[35158]: pgmap v618: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 436 KiB/s wr, 182 op/s 2024-09-12T01:36:56.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:56 smithi104 ceph-mon[35219]: pgmap v618: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 436 KiB/s wr, 182 op/s 2024-09-12T01:36:57.323 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-12T01:36:57.324 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T01:36:57.324 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 6128, 2024-09-12T01:36:57.324 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 13505, 2024-09-12T01:36:57.324 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 2073416, 2024-09-12T01:36:57.324 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 12628580, 2024-09-12T01:36:57.324 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 25, 2024-09-12T01:36:57.324 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 4, 2024-09-12T01:36:57.324 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.452876589, 2024-09-12T01:36:57.324 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.192578389, 2024-09-12T01:36:57.324 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.191035452, 2024-09-12T01:36:57.324 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.076083835, 2024-09-12T01:36:57.325 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:36:57.325 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 13506, 2024-09-12T01:36:57.325 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 2.588152997, 2024-09-12T01:36:57.325 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000191629 2024-09-12T01:36:57.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.325 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:36:57.325 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 9589, 2024-09-12T01:36:57.325 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.002025184, 2024-09-12T01:36:57.325 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000211 2024-09-12T01:36:57.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.325 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 56304, 2024-09-12T01:36:57.325 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 29744 2024-09-12T01:36:57.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.325 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T01:36:57.326 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 3825, 2024-09-12T01:36:57.326 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 3212, 2024-09-12T01:36:57.326 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 3590906, 2024-09-12T01:36:57.326 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 5173401, 2024-09-12T01:36:57.326 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 10, 2024-09-12T01:36:57.326 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 6, 2024-09-12T01:36:57.326 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.234689590, 2024-09-12T01:36:57.326 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.100073192, 2024-09-12T01:36:57.326 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.122297081, 2024-09-12T01:36:57.326 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.042155946, 2024-09-12T01:36:57.326 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:36:57.326 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 3215, 2024-09-12T01:36:57.326 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 4.934151688, 2024-09-12T01:36:57.326 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.001534728 2024-09-12T01:36:57.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.327 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:36:57.327 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 2357, 2024-09-12T01:36:57.327 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000398966, 2024-09-12T01:36:57.327 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000169 2024-09-12T01:36:57.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.327 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 874752, 2024-09-12T01:36:57.327 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 111456 2024-09-12T01:36:57.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.327 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T01:36:57.327 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 2390, 2024-09-12T01:36:57.327 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 2338, 2024-09-12T01:36:57.327 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 850186, 2024-09-12T01:36:57.327 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 3174919, 2024-09-12T01:36:57.328 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 65, 2024-09-12T01:36:57.328 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 8, 2024-09-12T01:36:57.328 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.178934779, 2024-09-12T01:36:57.328 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.061602667, 2024-09-12T01:36:57.328 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.080530479, 2024-09-12T01:36:57.328 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.038273276, 2024-09-12T01:36:57.328 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:36:57.328 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 2330, 2024-09-12T01:36:57.328 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 2.219539104, 2024-09-12T01:36:57.328 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000952591 2024-09-12T01:36:57.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.328 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:36:57.328 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1600, 2024-09-12T01:36:57.328 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000280648, 2024-09-12T01:36:57.329 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000175 2024-09-12T01:36:57.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.329 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 40400, 2024-09-12T01:36:57.329 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 310880 2024-09-12T01:36:57.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.329 INFO:teuthology.orchestra.run.smithi053.stdout: "cct": { 2024-09-12T01:36:57.329 INFO:teuthology.orchestra.run.smithi053.stdout: "total_workers": 1, 2024-09-12T01:36:57.329 INFO:teuthology.orchestra.run.smithi053.stdout: "unhealthy_workers": 0 2024-09-12T01:36:57.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.329 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-MDSRank": { 2024-09-12T01:36:57.329 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-09-12T01:36:57.329 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-09-12T01:36:57.329 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 964, 2024-09-12T01:36:57.330 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 1.894677994, 2024-09-12T01:36:57.330 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.001965433 2024-09-12T01:36:57.330 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-12T01:36:57.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.330 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-PurgeQueue": { 2024-09-12T01:36:57.330 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-09-12T01:36:57.330 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-09-12T01:36:57.330 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 91, 2024-09-12T01:36:57.330 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.014581263, 2024-09-12T01:36:57.330 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000160233 2024-09-12T01:36:57.330 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-12T01:36:57.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.330 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": { 2024-09-12T01:36:57.330 INFO:teuthology.orchestra.run.smithi053.stdout: "request": 3169, 2024-09-12T01:36:57.330 INFO:teuthology.orchestra.run.smithi053.stdout: "reply": 3055, 2024-09-12T01:36:57.331 INFO:teuthology.orchestra.run.smithi053.stdout: "reply_latency": { 2024-09-12T01:36:57.331 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 3055, 2024-09-12T01:36:57.331 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 9.821179264, 2024-09-12T01:36:57.331 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.003214788 2024-09-12T01:36:57.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.331 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_reply": 0, 2024-09-12T01:36:57.331 INFO:teuthology.orchestra.run.smithi053.stdout: "forward": 114, 2024-09-12T01:36:57.331 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_complete": 0, 2024-09-12T01:36:57.331 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_keys": 0, 2024-09-12T01:36:57.331 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_commit": 341, 2024-09-12T01:36:57.331 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_split": 11, 2024-09-12T01:36:57.331 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_merge": 0, 2024-09-12T01:36:57.331 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes": 20, 2024-09-12T01:36:57.331 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_top": 3, 2024-09-12T01:36:57.332 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_bottom": 3, 2024-09-12T01:36:57.332 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pin_tail": 14, 2024-09-12T01:36:57.332 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pinned": 15, 2024-09-12T01:36:57.332 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_expired": 2949, 2024-09-12T01:36:57.332 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_with_caps": 3, 2024-09-12T01:36:57.332 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": 3, 2024-09-12T01:36:57.332 INFO:teuthology.orchestra.run.smithi053.stdout: "subtrees": 3, 2024-09-12T01:36:57.332 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse": 3296, 2024-09-12T01:36:57.332 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_hit": 2821, 2024-09-12T01:36:57.332 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_forward": 114, 2024-09-12T01:36:57.332 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_discover": 1, 2024-09-12T01:36:57.332 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_dir_fetch": 0, 2024-09-12T01:36:57.332 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_remote_ino": 0, 2024-09-12T01:36:57.332 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_lock": 3, 2024-09-12T01:36:57.333 INFO:teuthology.orchestra.run.smithi053.stdout: "load_cent": 0, 2024-09-12T01:36:57.333 INFO:teuthology.orchestra.run.smithi053.stdout: "q": 0, 2024-09-12T01:36:57.333 INFO:teuthology.orchestra.run.smithi053.stdout: "exported": 5, 2024-09-12T01:36:57.333 INFO:teuthology.orchestra.run.smithi053.stdout: "exported_inodes": 2921, 2024-09-12T01:36:57.333 INFO:teuthology.orchestra.run.smithi053.stdout: "imported": 2, 2024-09-12T01:36:57.333 INFO:teuthology.orchestra.run.smithi053.stdout: "imported_inodes": 11, 2024-09-12T01:36:57.333 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_dir_fetch": 0, 2024-09-12T01:36:57.333 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_backtrace_fetch": 0, 2024-09-12T01:36:57.333 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_peer_discover": 0, 2024-09-12T01:36:57.334 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rfiles": 4, 2024-09-12T01:36:57.334 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rbytes": 234, 2024-09-12T01:36:57.334 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rsnaps": 0, 2024-09-12T01:36:57.334 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T01:36:57.334 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_set_tag": 0, 2024-09-12T01:36:57.334 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T01:36:57.334 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_inotable_repaired": 0, 2024-09-12T01:36:57.334 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_inodes": 0, 2024-09-12T01:36:57.334 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T01:36:57.334 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T01:36:57.334 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_file_inodes": 0, 2024-09-12T01:36:57.335 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_inode_file_caps": 3, 2024-09-12T01:36:57.335 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_revoke": 28, 2024-09-12T01:36:57.335 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_grant": 2661, 2024-09-12T01:36:57.335 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T01:36:57.335 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T01:36:57.335 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T01:36:57.335 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps": 71, 2024-09-12T01:36:57.335 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps_dirty": 25, 2024-09-12T01:36:57.335 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_cap_release": 14, 2024-09-12T01:36:57.335 INFO:teuthology.orchestra.run.smithi053.stdout: "process_request_cap_release": 122 2024-09-12T01:36:57.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.336 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache": { 2024-09-12T01:36:57.336 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update": 0, 2024-09-12T01:36:57.336 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update_receipt": 0, 2024-09-12T01:36:57.336 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_try_discover": 0, 2024-09-12T01:36:57.336 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_send_discover": 1, 2024-09-12T01:36:57.336 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_handle_discover": 18, 2024-09-12T01:36:57.336 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays": 0, 2024-09-12T01:36:57.336 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_delayed": 0, 2024-09-12T01:36:57.336 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_enqueuing": 0, 2024-09-12T01:36:57.336 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_created": 29, 2024-09-12T01:36:57.336 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_enqueued": 26, 2024-09-12T01:36:57.336 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_reintegrated": 3, 2024-09-12T01:36:57.337 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_migrated": 0, 2024-09-12T01:36:57.337 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_processing": 0, 2024-09-12T01:36:57.337 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_enqueued": 0, 2024-09-12T01:36:57.337 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_prioritized": 0, 2024-09-12T01:36:57.337 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_started": 0, 2024-09-12T01:36:57.337 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_completed": 0, 2024-09-12T01:36:57.337 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_quiesce_path": 0, 2024-09-12T01:36:57.337 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_quiesce_inode": 0, 2024-09-12T01:36:57.337 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T01:36:57.337 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_exportdir": 5, 2024-09-12T01:36:57.337 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_flush": 0, 2024-09-12T01:36:57.337 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragmentdir": 11, 2024-09-12T01:36:57.337 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragstats": 0, 2024-09-12T01:36:57.337 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_inodestats": 0 2024-09-12T01:36:57.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.338 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log": { 2024-09-12T01:36:57.338 INFO:teuthology.orchestra.run.smithi053.stdout: "evlrg": 0, 2024-09-12T01:36:57.338 INFO:teuthology.orchestra.run.smithi053.stdout: "evadd": 2921, 2024-09-12T01:36:57.338 INFO:teuthology.orchestra.run.smithi053.stdout: "evex": 2888, 2024-09-12T01:36:57.338 INFO:teuthology.orchestra.run.smithi053.stdout: "evtrm": 2888, 2024-09-12T01:36:57.338 INFO:teuthology.orchestra.run.smithi053.stdout: "ev": 33, 2024-09-12T01:36:57.338 INFO:teuthology.orchestra.run.smithi053.stdout: "evexg": 0, 2024-09-12T01:36:57.338 INFO:teuthology.orchestra.run.smithi053.stdout: "evexd": 4, 2024-09-12T01:36:57.338 INFO:teuthology.orchestra.run.smithi053.stdout: "segadd": 35, 2024-09-12T01:36:57.338 INFO:teuthology.orchestra.run.smithi053.stdout: "segex": 34, 2024-09-12T01:36:57.338 INFO:teuthology.orchestra.run.smithi053.stdout: "segtrm": 34, 2024-09-12T01:36:57.338 INFO:teuthology.orchestra.run.smithi053.stdout: "seg": 1, 2024-09-12T01:36:57.338 INFO:teuthology.orchestra.run.smithi053.stdout: "segmjr": 1, 2024-09-12T01:36:57.339 INFO:teuthology.orchestra.run.smithi053.stdout: "segexg": 0, 2024-09-12T01:36:57.339 INFO:teuthology.orchestra.run.smithi053.stdout: "segexd": 1, 2024-09-12T01:36:57.339 INFO:teuthology.orchestra.run.smithi053.stdout: "expos": 10619715, 2024-09-12T01:36:57.339 INFO:teuthology.orchestra.run.smithi053.stdout: "wrpos": 10689199, 2024-09-12T01:36:57.339 INFO:teuthology.orchestra.run.smithi053.stdout: "rdpos": 0, 2024-09-12T01:36:57.339 INFO:teuthology.orchestra.run.smithi053.stdout: "jlat": { 2024-09-12T01:36:57.339 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 176, 2024-09-12T01:36:57.339 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 1.154379667, 2024-09-12T01:36:57.339 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.006558975 2024-09-12T01:36:57.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.339 INFO:teuthology.orchestra.run.smithi053.stdout: "replayed": 0 2024-09-12T01:36:57.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.339 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_mem": { 2024-09-12T01:36:57.339 INFO:teuthology.orchestra.run.smithi053.stdout: "ino": 23, 2024-09-12T01:36:57.340 INFO:teuthology.orchestra.run.smithi053.stdout: "ino+": 2678, 2024-09-12T01:36:57.340 INFO:teuthology.orchestra.run.smithi053.stdout: "ino-": 2655, 2024-09-12T01:36:57.340 INFO:teuthology.orchestra.run.smithi053.stdout: "dir": 22, 2024-09-12T01:36:57.340 INFO:teuthology.orchestra.run.smithi053.stdout: "dir+": 167, 2024-09-12T01:36:57.340 INFO:teuthology.orchestra.run.smithi053.stdout: "dir-": 145, 2024-09-12T01:36:57.340 INFO:teuthology.orchestra.run.smithi053.stdout: "dn": 20, 2024-09-12T01:36:57.340 INFO:teuthology.orchestra.run.smithi053.stdout: "dn+": 3011, 2024-09-12T01:36:57.340 INFO:teuthology.orchestra.run.smithi053.stdout: "dn-": 2991, 2024-09-12T01:36:57.340 INFO:teuthology.orchestra.run.smithi053.stdout: "cap": 3, 2024-09-12T01:36:57.340 INFO:teuthology.orchestra.run.smithi053.stdout: "cap+": 2668, 2024-09-12T01:36:57.340 INFO:teuthology.orchestra.run.smithi053.stdout: "cap-": 2665, 2024-09-12T01:36:57.340 INFO:teuthology.orchestra.run.smithi053.stdout: "rss": 78120, 2024-09-12T01:36:57.341 INFO:teuthology.orchestra.run.smithi053.stdout: "heap": 231712 2024-09-12T01:36:57.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.341 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_server": { 2024-09-12T01:36:57.341 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_client_request": 3256, 2024-09-12T01:36:57.341 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_server_request": 25, 2024-09-12T01:36:57.341 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_request": 3169, 2024-09-12T01:36:57.341 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_session": 26, 2024-09-12T01:36:57.341 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_peer_request": 69, 2024-09-12T01:36:57.341 INFO:teuthology.orchestra.run.smithi053.stdout: "req_create_latency": { 2024-09-12T01:36:57.341 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 2600, 2024-09-12T01:36:57.342 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 9.542997923, 2024-09-12T01:36:57.342 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.003670383 2024-09-12T01:36:57.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.342 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getattr_latency": { 2024-09-12T01:36:57.342 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 30, 2024-09-12T01:36:57.342 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.008076789, 2024-09-12T01:36:57.342 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000269226 2024-09-12T01:36:57.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.342 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getfilelock_latency": { 2024-09-12T01:36:57.342 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:57.342 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:57.343 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:57.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.343 INFO:teuthology.orchestra.run.smithi053.stdout: "req_link_latency": { 2024-09-12T01:36:57.343 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 3, 2024-09-12T01:36:57.343 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.003477631, 2024-09-12T01:36:57.343 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.001159210 2024-09-12T01:36:57.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.343 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookup_latency": { 2024-09-12T01:36:57.343 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 16, 2024-09-12T01:36:57.343 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.004458905, 2024-09-12T01:36:57.343 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000278681 2024-09-12T01:36:57.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.343 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookuphash_latency": { 2024-09-12T01:36:57.343 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:57.343 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:57.344 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:57.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.344 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupino_latency": { 2024-09-12T01:36:57.344 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 2, 2024-09-12T01:36:57.344 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000241165, 2024-09-12T01:36:57.344 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000120582 2024-09-12T01:36:57.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.344 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupname_latency": { 2024-09-12T01:36:57.344 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:57.344 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:57.344 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:57.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.345 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupparent_latency": { 2024-09-12T01:36:57.345 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:57.345 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:57.345 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:57.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.345 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupsnap_latency": { 2024-09-12T01:36:57.345 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:57.345 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:57.345 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:57.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.345 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lssnap_latency": { 2024-09-12T01:36:57.345 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:57.346 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:57.346 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:57.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.346 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mkdir_latency": { 2024-09-12T01:36:57.346 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 50, 2024-09-12T01:36:57.347 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.044493521, 2024-09-12T01:36:57.347 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000889870 2024-09-12T01:36:57.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.347 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mknod_latency": { 2024-09-12T01:36:57.347 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:57.347 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:57.347 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:57.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.347 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mksnap_latency": { 2024-09-12T01:36:57.347 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:57.347 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:57.348 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:57.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.348 INFO:teuthology.orchestra.run.smithi053.stdout: "req_open_latency": { 2024-09-12T01:36:57.348 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 294, 2024-09-12T01:36:57.348 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.049983680, 2024-09-12T01:36:57.348 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000170012 2024-09-12T01:36:57.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.348 INFO:teuthology.orchestra.run.smithi053.stdout: "req_readdir_latency": { 2024-09-12T01:36:57.348 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 8, 2024-09-12T01:36:57.348 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.015444768, 2024-09-12T01:36:57.348 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.001930596 2024-09-12T01:36:57.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.348 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rename_latency": { 2024-09-12T01:36:57.349 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 20, 2024-09-12T01:36:57.349 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.061031031, 2024-09-12T01:36:57.349 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.003051551 2024-09-12T01:36:57.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.349 INFO:teuthology.orchestra.run.smithi053.stdout: "req_renamesnap_latency": { 2024-09-12T01:36:57.349 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:57.349 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:57.349 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:57.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.349 INFO:teuthology.orchestra.run.smithi053.stdout: "req_snapdiff_latency": { 2024-09-12T01:36:57.349 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:57.349 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:57.349 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:57.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.350 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmdir_latency": { 2024-09-12T01:36:57.350 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 4, 2024-09-12T01:36:57.350 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.045049672, 2024-09-12T01:36:57.350 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.011262418 2024-09-12T01:36:57.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.350 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmsnap_latency": { 2024-09-12T01:36:57.350 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:57.350 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:57.350 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:57.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.350 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmxattr_latency": { 2024-09-12T01:36:57.350 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:57.350 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:57.351 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:57.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.351 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setattr_latency": { 2024-09-12T01:36:57.351 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 2, 2024-09-12T01:36:57.351 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.001554597, 2024-09-12T01:36:57.351 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000777298 2024-09-12T01:36:57.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.351 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setdirlayout_latency": { 2024-09-12T01:36:57.351 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:57.351 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:57.351 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:57.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.352 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setfilelock_latency": { 2024-09-12T01:36:57.352 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:57.352 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:57.352 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:57.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.352 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setlayout_latency": { 2024-09-12T01:36:57.352 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:57.352 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:57.352 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:57.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.352 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setxattr_latency": { 2024-09-12T01:36:57.352 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 9, 2024-09-12T01:36:57.352 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.023408664, 2024-09-12T01:36:57.353 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.002600962 2024-09-12T01:36:57.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.353 INFO:teuthology.orchestra.run.smithi053.stdout: "req_symlink_latency": { 2024-09-12T01:36:57.353 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1, 2024-09-12T01:36:57.353 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000581285, 2024-09-12T01:36:57.353 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000581285 2024-09-12T01:36:57.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.353 INFO:teuthology.orchestra.run.smithi053.stdout: "req_unlink_latency": { 2024-09-12T01:36:57.353 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 16, 2024-09-12T01:36:57.353 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.020379633, 2024-09-12T01:36:57.353 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.001273727 2024-09-12T01:36:57.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.353 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_revoke_eviction": 0, 2024-09-12T01:36:57.353 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_acquisition_throttle": 0, 2024-09-12T01:36:57.354 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getvxattr_latency": { 2024-09-12T01:36:57.354 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:57.354 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:57.354 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:57.354 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-12T01:36:57.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.354 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_sessions": { 2024-09-12T01:36:57.354 INFO:teuthology.orchestra.run.smithi053.stdout: "session_count": 1, 2024-09-12T01:36:57.354 INFO:teuthology.orchestra.run.smithi053.stdout: "session_add": 4, 2024-09-12T01:36:57.354 INFO:teuthology.orchestra.run.smithi053.stdout: "session_remove": 3, 2024-09-12T01:36:57.354 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_open": 1, 2024-09-12T01:36:57.354 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_stale": 0, 2024-09-12T01:36:57.354 INFO:teuthology.orchestra.run.smithi053.stdout: "total_load": 76, 2024-09-12T01:36:57.354 INFO:teuthology.orchestra.run.smithi053.stdout: "average_load": 76, 2024-09-12T01:36:57.355 INFO:teuthology.orchestra.run.smithi053.stdout: "avg_session_uptime": 426, 2024-09-12T01:36:57.355 INFO:teuthology.orchestra.run.smithi053.stdout: "mdthresh_evicted": 0 2024-09-12T01:36:57.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.355 INFO:teuthology.orchestra.run.smithi053.stdout: "mempool": { 2024-09-12T01:36:57.355 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_bytes": 360, 2024-09-12T01:36:57.355 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_items": 360, 2024-09-12T01:36:57.355 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T01:36:57.355 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_items": 0, 2024-09-12T01:36:57.355 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T01:36:57.355 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_items": 0, 2024-09-12T01:36:57.355 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T01:36:57.355 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T01:36:57.355 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T01:36:57.355 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T01:36:57.356 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T01:36:57.356 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_items": 0, 2024-09-12T01:36:57.356 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T01:36:57.356 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T01:36:57.356 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_bytes": 0, 2024-09-12T01:36:57.356 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_items": 0, 2024-09-12T01:36:57.356 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_bytes": 0, 2024-09-12T01:36:57.356 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_items": 0, 2024-09-12T01:36:57.356 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T01:36:57.356 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T01:36:57.356 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T01:36:57.356 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T01:36:57.356 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T01:36:57.357 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_items": 0, 2024-09-12T01:36:57.357 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_bytes": 0, 2024-09-12T01:36:57.357 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_items": 0, 2024-09-12T01:36:57.357 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T01:36:57.357 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T01:36:57.357 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_bytes": 0, 2024-09-12T01:36:57.357 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_items": 0, 2024-09-12T01:36:57.358 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_bytes": 0, 2024-09-12T01:36:57.358 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_items": 0, 2024-09-12T01:36:57.358 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T01:36:57.358 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_items": 0, 2024-09-12T01:36:57.358 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T01:36:57.358 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_items": 0, 2024-09-12T01:36:57.358 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_bytes": 299971, 2024-09-12T01:36:57.358 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_items": 138, 2024-09-12T01:36:57.358 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_bytes": 1672, 2024-09-12T01:36:57.358 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_items": 19, 2024-09-12T01:36:57.358 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bytes": 0, 2024-09-12T01:36:57.358 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_items": 0, 2024-09-12T01:36:57.358 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_bytes": 0, 2024-09-12T01:36:57.358 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_items": 0, 2024-09-12T01:36:57.359 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_bytes": 0, 2024-09-12T01:36:57.359 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_items": 0, 2024-09-12T01:36:57.359 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_bytes": 8856, 2024-09-12T01:36:57.359 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_items": 166, 2024-09-12T01:36:57.359 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T01:36:57.359 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_items": 0, 2024-09-12T01:36:57.359 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_bytes": 0, 2024-09-12T01:36:57.359 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_items": 0, 2024-09-12T01:36:57.359 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_bytes": 99165, 2024-09-12T01:36:57.359 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_items": 552, 2024-09-12T01:36:57.359 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_bytes": 0, 2024-09-12T01:36:57.359 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_items": 0, 2024-09-12T01:36:57.359 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_bytes": 0, 2024-09-12T01:36:57.359 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_items": 0 2024-09-12T01:36:57.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.360 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter": { 2024-09-12T01:36:57.360 INFO:teuthology.orchestra.run.smithi053.stdout: "op_active": 0, 2024-09-12T01:36:57.360 INFO:teuthology.orchestra.run.smithi053.stdout: "op_laggy": 0, 2024-09-12T01:36:57.360 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send": 5498, 2024-09-12T01:36:57.360 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send_bytes": 10291546, 2024-09-12T01:36:57.360 INFO:teuthology.orchestra.run.smithi053.stdout: "op_resend": 32, 2024-09-12T01:36:57.360 INFO:teuthology.orchestra.run.smithi053.stdout: "op_reply": 5498, 2024-09-12T01:36:57.360 INFO:teuthology.orchestra.run.smithi053.stdout: "op_latency": { 2024-09-12T01:36:57.360 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 5498, 2024-09-12T01:36:57.360 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 1367.318066134, 2024-09-12T01:36:57.360 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.248693718 2024-09-12T01:36:57.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.360 INFO:teuthology.orchestra.run.smithi053.stdout: "op_inflight": 0, 2024-09-12T01:36:57.360 INFO:teuthology.orchestra.run.smithi053.stdout: "oplen_avg": { 2024-09-12T01:36:57.361 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 5498, 2024-09-12T01:36:57.361 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 12412 2024-09-12T01:36:57.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.361 INFO:teuthology.orchestra.run.smithi053.stdout: "op": 5498, 2024-09-12T01:36:57.361 INFO:teuthology.orchestra.run.smithi053.stdout: "op_r": 10, 2024-09-12T01:36:57.361 INFO:teuthology.orchestra.run.smithi053.stdout: "op_w": 5488, 2024-09-12T01:36:57.361 INFO:teuthology.orchestra.run.smithi053.stdout: "op_rmw": 0, 2024-09-12T01:36:57.361 INFO:teuthology.orchestra.run.smithi053.stdout: "op_pg": 0, 2024-09-12T01:36:57.361 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_stat": 222, 2024-09-12T01:36:57.361 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_create": 4206, 2024-09-12T01:36:57.361 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_read": 10, 2024-09-12T01:36:57.361 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_write": 186, 2024-09-12T01:36:57.361 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writefull": 119, 2024-09-12T01:36:57.361 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writesame": 0, 2024-09-12T01:36:57.361 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_append": 0, 2024-09-12T01:36:57.362 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_zero": 0, 2024-09-12T01:36:57.362 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_truncate": 3, 2024-09-12T01:36:57.362 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_delete": 569, 2024-09-12T01:36:57.362 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_mapext": 0, 2024-09-12T01:36:57.362 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_sparse_read": 0, 2024-09-12T01:36:57.362 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_clonerange": 0, 2024-09-12T01:36:57.362 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_getxattr": 0, 2024-09-12T01:36:57.362 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_setxattr": 6324, 2024-09-12T01:36:57.362 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_cmpxattr": 0, 2024-09-12T01:36:57.362 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_rmxattr": 0, 2024-09-12T01:36:57.362 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_resetxattrs": 0, 2024-09-12T01:36:57.362 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_call": 0, 2024-09-12T01:36:57.362 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_watch": 0, 2024-09-12T01:36:57.362 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_notify": 0, 2024-09-12T01:36:57.363 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T01:36:57.363 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls": 0, 2024-09-12T01:36:57.363 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls_filter": 0, 2024-09-12T01:36:57.363 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_other": 0, 2024-09-12T01:36:57.363 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_active": 0, 2024-09-12T01:36:57.363 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_send": 0, 2024-09-12T01:36:57.363 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_resend": 0, 2024-09-12T01:36:57.363 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_ping": 0, 2024-09-12T01:36:57.363 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_active": 0, 2024-09-12T01:36:57.363 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_send": 0, 2024-09-12T01:36:57.363 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_resend": 0, 2024-09-12T01:36:57.363 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_active": 0, 2024-09-12T01:36:57.363 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_send": 0, 2024-09-12T01:36:57.363 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_resend": 0, 2024-09-12T01:36:57.363 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_active": 0, 2024-09-12T01:36:57.364 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_send": 0, 2024-09-12T01:36:57.364 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_resend": 0, 2024-09-12T01:36:57.364 INFO:teuthology.orchestra.run.smithi053.stdout: "command_active": 0, 2024-09-12T01:36:57.364 INFO:teuthology.orchestra.run.smithi053.stdout: "command_send": 0, 2024-09-12T01:36:57.364 INFO:teuthology.orchestra.run.smithi053.stdout: "command_resend": 0, 2024-09-12T01:36:57.364 INFO:teuthology.orchestra.run.smithi053.stdout: "map_epoch": 98, 2024-09-12T01:36:57.364 INFO:teuthology.orchestra.run.smithi053.stdout: "map_full": 0, 2024-09-12T01:36:57.364 INFO:teuthology.orchestra.run.smithi053.stdout: "map_inc": 11, 2024-09-12T01:36:57.364 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_sessions": 12, 2024-09-12T01:36:57.364 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_open": 12, 2024-09-12T01:36:57.364 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_close": 0, 2024-09-12T01:36:57.364 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_laggy": 0, 2024-09-12T01:36:57.364 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_wr": 750, 2024-09-12T01:36:57.364 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_rd": 0, 2024-09-12T01:36:57.365 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_del": 23 2024-09-12T01:36:57.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.365 INFO:teuthology.orchestra.run.smithi053.stdout: "oft": { 2024-09-12T01:36:57.365 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_objs": 1, 2024-09-12T01:36:57.365 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_kv_pairs": 0, 2024-09-12T01:36:57.365 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_updates": 0, 2024-09-12T01:36:57.365 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_removes": 0 2024-09-12T01:36:57.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.365 INFO:teuthology.orchestra.run.smithi053.stdout: "osd-slow-ops": { 2024-09-12T01:36:57.365 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_ops_count": 0 2024-09-12T01:36:57.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.365 INFO:teuthology.orchestra.run.smithi053.stdout: "purge_queue": { 2024-09-12T01:36:57.365 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops": 0, 2024-09-12T01:36:57.365 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops_high_water": 10, 2024-09-12T01:36:57.365 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing": 0, 2024-09-12T01:36:57.366 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_high_water": 5, 2024-09-12T01:36:57.366 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed_ops": 47, 2024-09-12T01:36:57.366 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed": 26, 2024-09-12T01:36:57.366 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_item_in_journal": 0 2024-09-12T01:36:57.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.366 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T01:36:57.366 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-09-12T01:36:57.366 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-09-12T01:36:57.366 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-09-12T01:36:57.366 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 12339, 2024-09-12T01:36:57.366 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 5436850, 2024-09-12T01:36:57.366 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-09-12T01:36:57.366 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 12339, 2024-09-12T01:36:57.366 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-09-12T01:36:57.366 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-09-12T01:36:57.367 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 12339, 2024-09-12T01:36:57.367 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 5436850, 2024-09-12T01:36:57.367 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-09-12T01:36:57.367 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:57.367 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:57.367 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:57.367 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-12T01:36:57.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.367 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_bytes": { 2024-09-12T01:36:57.367 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-09-12T01:36:57.367 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-09-12T01:36:57.367 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-09-12T01:36:57.367 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-09-12T01:36:57.367 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-09-12T01:36:57.367 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-09-12T01:36:57.368 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-09-12T01:36:57.368 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 5498, 2024-09-12T01:36:57.368 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 10294340, 2024-09-12T01:36:57.368 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 5498, 2024-09-12T01:36:57.368 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 10294340, 2024-09-12T01:36:57.368 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-09-12T01:36:57.368 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:57.368 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:57.368 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:57.368 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-12T01:36:57.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.368 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_ops": { 2024-09-12T01:36:57.368 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-09-12T01:36:57.368 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 1024, 2024-09-12T01:36:57.369 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-09-12T01:36:57.369 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-09-12T01:36:57.369 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-09-12T01:36:57.369 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-09-12T01:36:57.369 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-09-12T01:36:57.369 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 5498, 2024-09-12T01:36:57.369 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 5498, 2024-09-12T01:36:57.369 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 5498, 2024-09-12T01:36:57.369 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 5498, 2024-09-12T01:36:57.369 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-09-12T01:36:57.369 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:57.369 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:57.369 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:57.369 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-12T01:36:57.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.370 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle": { 2024-09-12T01:36:57.370 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-09-12T01:36:57.370 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-09-12T01:36:57.370 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-09-12T01:36:57.370 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 26, 2024-09-12T01:36:57.370 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 2794, 2024-09-12T01:36:57.370 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-09-12T01:36:57.370 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 26, 2024-09-12T01:36:57.370 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-09-12T01:36:57.370 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-09-12T01:36:57.371 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 10, 2024-09-12T01:36:57.371 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 2794, 2024-09-12T01:36:57.371 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-09-12T01:36:57.371 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:57.371 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:57.371 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:57.371 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-12T01:36:57.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:57.371 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle-0x564861406500": { 2024-09-12T01:36:57.371 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-09-12T01:36:57.372 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-09-12T01:36:57.372 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-09-12T01:36:57.372 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 2921, 2024-09-12T01:36:57.372 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 6494895, 2024-09-12T01:36:57.372 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-09-12T01:36:57.372 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 2921, 2024-09-12T01:36:57.372 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-09-12T01:36:57.372 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-09-12T01:36:57.372 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 176, 2024-09-12T01:36:57.372 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 6494895, 2024-09-12T01:36:57.372 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-09-12T01:36:57.372 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:57.372 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:57.373 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:57.373 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-12T01:36:57.373 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-12T01:36:57.373 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-12T01:36:58.247 INFO:tasks.check_counter:Daemon mds.b mds.exported=5 2024-09-12T01:36:58.247 INFO:tasks.check_counter:Daemon mds.b mds.imported=2 2024-09-12T01:36:58.247 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=11 2024-09-12T01:36:58.247 DEBUG:tasks.check_counter:Getting stats from e 2024-09-12T01:36:58.248 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-12T01:36:58.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:36:58 smithi053 ceph-mon[35158]: pgmap v619: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 386 KiB/s wr, 112 op/s 2024-09-12T01:36:58.501 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T01:36:58.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:36:58 smithi104 ceph-mon[35219]: pgmap v619: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 386 KiB/s wr, 112 op/s 2024-09-12T01:36:58.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:36:58 smithi001 ceph-mon[30387]: pgmap v619: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 386 KiB/s wr, 112 op/s 2024-09-12T01:36:58.999 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-12T01:36:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T01:36:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 827, 2024-09-12T01:36:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 809, 2024-09-12T01:36:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 3296934, 2024-09-12T01:36:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 307313, 2024-09-12T01:36:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 5, 2024-09-12T01:36:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 1, 2024-09-12T01:36:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.073516702, 2024-09-12T01:36:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.023579254, 2024-09-12T01:36:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.038167386, 2024-09-12T01:36:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.014775525, 2024-09-12T01:36:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:36:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 809, 2024-09-12T01:36:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.028808735, 2024-09-12T01:36:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000035610 2024-09-12T01:36:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:36:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 837184, 2024-09-12T01:36:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 108880 2024-09-12T01:36:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T01:36:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 1134, 2024-09-12T01:36:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 1265, 2024-09-12T01:36:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 4527177, 2024-09-12T01:36:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 649155, 2024-09-12T01:36:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 5, 2024-09-12T01:36:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 4, 2024-09-12T01:36:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.099667337, 2024-09-12T01:36:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.037521314, 2024-09-12T01:36:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.024240036, 2024-09-12T01:36:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.028929689, 2024-09-12T01:36:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:36:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1263, 2024-09-12T01:36:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.043365221, 2024-09-12T01:36:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000034335 2024-09-12T01:36:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:36:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-12T01:36:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 288800 2024-09-12T01:36:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T01:36:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 5, 2024-09-12T01:36:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 7, 2024-09-12T01:36:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 1873, 2024-09-12T01:36:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 1605, 2024-09-12T01:36:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 5, 2024-09-12T01:36:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 4, 2024-09-12T01:36:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.001259613, 2024-09-12T01:36:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.000057855, 2024-09-12T01:36:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.000972424, 2024-09-12T01:36:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.000105104, 2024-09-12T01:36:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:36:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 5, 2024-09-12T01:36:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.003439281, 2024-09-12T01:36:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000687856 2024-09-12T01:36:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:36:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T01:36:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T01:36:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: "cct": { 2024-09-12T01:36:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: "total_workers": 1, 2024-09-12T01:36:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: "unhealthy_workers": 0 2024-09-12T01:36:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-MDSRank": { 2024-09-12T01:36:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-09-12T01:36:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-09-12T01:36:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 2899, 2024-09-12T01:36:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.075814877, 2024-09-12T01:36:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000026152 2024-09-12T01:36:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-12T01:36:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-PurgeQueue": { 2024-09-12T01:36:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-09-12T01:36:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-09-12T01:36:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-12T01:36:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": { 2024-09-12T01:36:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "request": 0, 2024-09-12T01:36:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "reply": 0, 2024-09-12T01:36:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "reply_latency": { 2024-09-12T01:36:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_reply": 0, 2024-09-12T01:36:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: "forward": 0, 2024-09-12T01:36:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_complete": 0, 2024-09-12T01:36:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_keys": 0, 2024-09-12T01:36:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_commit": 0, 2024-09-12T01:36:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_split": 0, 2024-09-12T01:36:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_merge": 0, 2024-09-12T01:36:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes": 44, 2024-09-12T01:36:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_top": 18, 2024-09-12T01:36:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_bottom": 26, 2024-09-12T01:36:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pin_tail": 0, 2024-09-12T01:36:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pinned": 17, 2024-09-12T01:36:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_expired": 0, 2024-09-12T01:36:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_with_caps": 0, 2024-09-12T01:36:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": 0, 2024-09-12T01:36:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "subtrees": 2, 2024-09-12T01:36:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse": 0, 2024-09-12T01:36:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_hit": 0, 2024-09-12T01:36:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_forward": 0, 2024-09-12T01:36:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_discover": 0, 2024-09-12T01:36:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_dir_fetch": 0, 2024-09-12T01:36:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_remote_ino": 0, 2024-09-12T01:36:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_lock": 0, 2024-09-12T01:36:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "load_cent": 0, 2024-09-12T01:36:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "q": 0, 2024-09-12T01:36:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "exported": 0, 2024-09-12T01:36:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "exported_inodes": 0, 2024-09-12T01:36:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "imported": 0, 2024-09-12T01:36:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "imported_inodes": 0, 2024-09-12T01:36:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_dir_fetch": 0, 2024-09-12T01:36:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_backtrace_fetch": 0, 2024-09-12T01:36:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_peer_discover": 0, 2024-09-12T01:36:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rfiles": 4, 2024-09-12T01:36:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rbytes": 234, 2024-09-12T01:36:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rsnaps": 0, 2024-09-12T01:36:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T01:36:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_set_tag": 0, 2024-09-12T01:36:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T01:36:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_inotable_repaired": 0, 2024-09-12T01:36:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_inodes": 0, 2024-09-12T01:36:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T01:36:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T01:36:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_file_inodes": 0, 2024-09-12T01:36:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_inode_file_caps": 0, 2024-09-12T01:36:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T01:36:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_grant": 0, 2024-09-12T01:36:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T01:36:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T01:36:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T01:36:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps": 0, 2024-09-12T01:36:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps_dirty": 0, 2024-09-12T01:36:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_cap_release": 0, 2024-09-12T01:36:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "process_request_cap_release": 0 2024-09-12T01:36:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache": { 2024-09-12T01:36:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update": 0, 2024-09-12T01:36:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update_receipt": 0, 2024-09-12T01:36:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_try_discover": 0, 2024-09-12T01:36:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_send_discover": 0, 2024-09-12T01:36:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_handle_discover": 0, 2024-09-12T01:36:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays": 0, 2024-09-12T01:36:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_delayed": 0, 2024-09-12T01:36:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_enqueuing": 0, 2024-09-12T01:36:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_created": 0, 2024-09-12T01:36:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_enqueued": 0, 2024-09-12T01:36:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_reintegrated": 0, 2024-09-12T01:36:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_migrated": 0, 2024-09-12T01:36:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_processing": 0, 2024-09-12T01:36:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_enqueued": 0, 2024-09-12T01:36:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_prioritized": 0, 2024-09-12T01:36:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_started": 0, 2024-09-12T01:36:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_completed": 0, 2024-09-12T01:36:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_quiesce_path": 0, 2024-09-12T01:36:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_quiesce_inode": 0, 2024-09-12T01:36:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T01:36:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_exportdir": 0, 2024-09-12T01:36:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_flush": 0, 2024-09-12T01:36:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragmentdir": 0, 2024-09-12T01:36:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragstats": 0, 2024-09-12T01:36:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_inodestats": 0 2024-09-12T01:36:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log": { 2024-09-12T01:36:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "evlrg": 0, 2024-09-12T01:36:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "evadd": 0, 2024-09-12T01:36:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "evex": 0, 2024-09-12T01:36:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "evtrm": 0, 2024-09-12T01:36:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "ev": 2921, 2024-09-12T01:36:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "evexg": 0, 2024-09-12T01:36:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "evexd": 0, 2024-09-12T01:36:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "segadd": 0, 2024-09-12T01:36:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "segex": 0, 2024-09-12T01:36:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "segtrm": 0, 2024-09-12T01:36:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "seg": 2, 2024-09-12T01:36:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "segmjr": 33, 2024-09-12T01:36:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "segexg": 0, 2024-09-12T01:36:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "segexd": 0, 2024-09-12T01:36:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "expos": 10619715, 2024-09-12T01:36:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "wrpos": 10689199, 2024-09-12T01:36:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "rdpos": 10684562, 2024-09-12T01:36:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "jlat": { 2024-09-12T01:36:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "replayed": 2921 2024-09-12T01:36:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_mem": { 2024-09-12T01:36:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "ino": 15, 2024-09-12T01:36:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "ino+": 2666, 2024-09-12T01:36:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "ino-": 2651, 2024-09-12T01:36:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "dir": 10, 2024-09-12T01:36:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "dir+": 147, 2024-09-12T01:36:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "dir-": 137, 2024-09-12T01:36:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "dn": 44, 2024-09-12T01:36:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "dn+": 2694, 2024-09-12T01:36:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "dn-": 2650, 2024-09-12T01:36:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "cap": 0, 2024-09-12T01:36:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "cap+": 0, 2024-09-12T01:36:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "cap-": 0, 2024-09-12T01:36:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "rss": 57424, 2024-09-12T01:36:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "heap": 215328 2024-09-12T01:36:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_server": { 2024-09-12T01:36:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_client_request": 0, 2024-09-12T01:36:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_server_request": 0, 2024-09-12T01:36:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_request": 0, 2024-09-12T01:36:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_session": 0, 2024-09-12T01:36:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_peer_request": 0, 2024-09-12T01:36:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "req_create_latency": { 2024-09-12T01:36:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getattr_latency": { 2024-09-12T01:36:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getfilelock_latency": { 2024-09-12T01:36:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "req_link_latency": { 2024-09-12T01:36:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookup_latency": { 2024-09-12T01:36:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookuphash_latency": { 2024-09-12T01:36:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupino_latency": { 2024-09-12T01:36:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupname_latency": { 2024-09-12T01:36:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupparent_latency": { 2024-09-12T01:36:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupsnap_latency": { 2024-09-12T01:36:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lssnap_latency": { 2024-09-12T01:36:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mkdir_latency": { 2024-09-12T01:36:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mknod_latency": { 2024-09-12T01:36:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mksnap_latency": { 2024-09-12T01:36:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: "req_open_latency": { 2024-09-12T01:36:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: "req_readdir_latency": { 2024-09-12T01:36:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rename_latency": { 2024-09-12T01:36:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: "req_renamesnap_latency": { 2024-09-12T01:36:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "req_snapdiff_latency": { 2024-09-12T01:36:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmdir_latency": { 2024-09-12T01:36:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmsnap_latency": { 2024-09-12T01:36:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmxattr_latency": { 2024-09-12T01:36:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setattr_latency": { 2024-09-12T01:36:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setdirlayout_latency": { 2024-09-12T01:36:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setfilelock_latency": { 2024-09-12T01:36:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setlayout_latency": { 2024-09-12T01:36:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setxattr_latency": { 2024-09-12T01:36:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: "req_symlink_latency": { 2024-09-12T01:36:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: "req_unlink_latency": { 2024-09-12T01:36:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_revoke_eviction": 0, 2024-09-12T01:36:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_acquisition_throttle": 0, 2024-09-12T01:36:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getvxattr_latency": { 2024-09-12T01:36:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-12T01:36:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_sessions": { 2024-09-12T01:36:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: "session_count": 1, 2024-09-12T01:36:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: "session_add": 4, 2024-09-12T01:36:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: "session_remove": 3, 2024-09-12T01:36:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_open": 2, 2024-09-12T01:36:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_stale": 0, 2024-09-12T01:36:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "total_load": 0, 2024-09-12T01:36:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "average_load": 0, 2024-09-12T01:36:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "avg_session_uptime": 3396, 2024-09-12T01:36:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "mdthresh_evicted": 0 2024-09-12T01:36:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "mempool": { 2024-09-12T01:36:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_bytes": 0, 2024-09-12T01:36:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_items": 0, 2024-09-12T01:36:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T01:36:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_items": 0, 2024-09-12T01:36:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T01:36:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_items": 0, 2024-09-12T01:36:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T01:36:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T01:36:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T01:36:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T01:36:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T01:36:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_items": 0, 2024-09-12T01:36:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T01:36:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T01:36:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_bytes": 0, 2024-09-12T01:36:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_items": 0, 2024-09-12T01:36:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_bytes": 0, 2024-09-12T01:36:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_items": 0, 2024-09-12T01:36:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T01:36:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T01:36:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T01:36:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T01:36:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T01:36:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_items": 0, 2024-09-12T01:36:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_bytes": 0, 2024-09-12T01:36:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_items": 0, 2024-09-12T01:36:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T01:36:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T01:36:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_bytes": 0, 2024-09-12T01:36:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_items": 0, 2024-09-12T01:36:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_bytes": 0, 2024-09-12T01:36:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_items": 0, 2024-09-12T01:36:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T01:36:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_items": 0, 2024-09-12T01:36:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T01:36:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_items": 0, 2024-09-12T01:36:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_bytes": 197463, 2024-09-12T01:36:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_items": 54, 2024-09-12T01:36:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_bytes": 0, 2024-09-12T01:36:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_items": 0, 2024-09-12T01:36:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bytes": 0, 2024-09-12T01:36:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_items": 0, 2024-09-12T01:36:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_bytes": 0, 2024-09-12T01:36:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_items": 0, 2024-09-12T01:36:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_bytes": 0, 2024-09-12T01:36:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_items": 0, 2024-09-12T01:36:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_bytes": 8856, 2024-09-12T01:36:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_items": 166, 2024-09-12T01:36:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T01:36:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_items": 0, 2024-09-12T01:36:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_bytes": 0, 2024-09-12T01:36:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_items": 0, 2024-09-12T01:36:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_bytes": 77490, 2024-09-12T01:36:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_items": 380, 2024-09-12T01:36:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_bytes": 0, 2024-09-12T01:36:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_items": 0, 2024-09-12T01:36:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_bytes": 0, 2024-09-12T01:36:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_items": 0 2024-09-12T01:36:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter": { 2024-09-12T01:36:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: "op_active": 0, 2024-09-12T01:36:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: "op_laggy": 0, 2024-09-12T01:36:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send": 1762, 2024-09-12T01:36:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send_bytes": 26, 2024-09-12T01:36:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: "op_resend": 4, 2024-09-12T01:36:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: "op_reply": 1762, 2024-09-12T01:36:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: "op_latency": { 2024-09-12T01:36:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1762, 2024-09-12T01:36:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.954980581, 2024-09-12T01:36:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000541986 2024-09-12T01:36:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: "op_inflight": 0, 2024-09-12T01:36:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: "oplen_avg": { 2024-09-12T01:36:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1762, 2024-09-12T01:36:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 1763 2024-09-12T01:36:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: "op": 1762, 2024-09-12T01:36:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: "op_r": 1762, 2024-09-12T01:36:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: "op_w": 0, 2024-09-12T01:36:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: "op_rmw": 0, 2024-09-12T01:36:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: "op_pg": 0, 2024-09-12T01:36:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_stat": 1106, 2024-09-12T01:36:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_create": 0, 2024-09-12T01:36:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_read": 653, 2024-09-12T01:36:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_write": 0, 2024-09-12T01:36:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writefull": 0, 2024-09-12T01:36:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writesame": 0, 2024-09-12T01:36:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_append": 0, 2024-09-12T01:36:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_zero": 0, 2024-09-12T01:36:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_truncate": 0, 2024-09-12T01:36:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_delete": 0, 2024-09-12T01:36:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_mapext": 0, 2024-09-12T01:36:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_sparse_read": 0, 2024-09-12T01:36:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_clonerange": 0, 2024-09-12T01:36:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_getxattr": 2, 2024-09-12T01:36:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_setxattr": 0, 2024-09-12T01:36:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_cmpxattr": 0, 2024-09-12T01:36:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_rmxattr": 0, 2024-09-12T01:36:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_resetxattrs": 0, 2024-09-12T01:36:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_call": 0, 2024-09-12T01:36:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_watch": 0, 2024-09-12T01:36:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_notify": 0, 2024-09-12T01:36:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T01:36:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls": 0, 2024-09-12T01:36:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls_filter": 0, 2024-09-12T01:36:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_other": 0, 2024-09-12T01:36:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_active": 0, 2024-09-12T01:36:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_send": 0, 2024-09-12T01:36:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_resend": 0, 2024-09-12T01:36:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_ping": 0, 2024-09-12T01:36:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_active": 0, 2024-09-12T01:36:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_send": 0, 2024-09-12T01:36:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_resend": 0, 2024-09-12T01:36:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_active": 0, 2024-09-12T01:36:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_send": 0, 2024-09-12T01:36:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_resend": 0, 2024-09-12T01:36:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_active": 0, 2024-09-12T01:36:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_send": 0, 2024-09-12T01:36:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_resend": 0, 2024-09-12T01:36:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: "command_active": 0, 2024-09-12T01:36:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: "command_send": 0, 2024-09-12T01:36:59.050 INFO:teuthology.orchestra.run.smithi053.stdout: "command_resend": 0, 2024-09-12T01:36:59.050 INFO:teuthology.orchestra.run.smithi053.stdout: "map_epoch": 98, 2024-09-12T01:36:59.050 INFO:teuthology.orchestra.run.smithi053.stdout: "map_full": 0, 2024-09-12T01:36:59.050 INFO:teuthology.orchestra.run.smithi053.stdout: "map_inc": 2, 2024-09-12T01:36:59.050 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_sessions": 6, 2024-09-12T01:36:59.050 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_open": 6, 2024-09-12T01:36:59.050 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_close": 0, 2024-09-12T01:36:59.050 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_laggy": 0, 2024-09-12T01:36:59.050 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_wr": 0, 2024-09-12T01:36:59.050 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_rd": 2, 2024-09-12T01:36:59.050 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_del": 0 2024-09-12T01:36:59.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.050 INFO:teuthology.orchestra.run.smithi053.stdout: "oft": { 2024-09-12T01:36:59.050 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_objs": 0, 2024-09-12T01:36:59.051 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_kv_pairs": 0, 2024-09-12T01:36:59.051 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_updates": 0, 2024-09-12T01:36:59.051 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_removes": 0 2024-09-12T01:36:59.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.051 INFO:teuthology.orchestra.run.smithi053.stdout: "osd-slow-ops": { 2024-09-12T01:36:59.051 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_ops_count": 0 2024-09-12T01:36:59.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.051 INFO:teuthology.orchestra.run.smithi053.stdout: "purge_queue": { 2024-09-12T01:36:59.051 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops": 0, 2024-09-12T01:36:59.051 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T01:36:59.051 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing": 0, 2024-09-12T01:36:59.051 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_high_water": 0, 2024-09-12T01:36:59.051 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed_ops": 0, 2024-09-12T01:36:59.051 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed": 0, 2024-09-12T01:36:59.052 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_item_in_journal": 0 2024-09-12T01:36:59.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.052 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T01:36:59.052 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-09-12T01:36:59.052 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-09-12T01:36:59.052 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-09-12T01:36:59.052 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 1962, 2024-09-12T01:36:59.052 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 7636234, 2024-09-12T01:36:59.052 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-09-12T01:36:59.052 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 1962, 2024-09-12T01:36:59.052 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-09-12T01:36:59.052 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-09-12T01:36:59.052 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 1962, 2024-09-12T01:36:59.053 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 7636234, 2024-09-12T01:36:59.053 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-09-12T01:36:59.053 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:59.053 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:59.053 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:59.053 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-12T01:36:59.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.053 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_bytes": { 2024-09-12T01:36:59.053 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-09-12T01:36:59.053 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-09-12T01:36:59.053 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-09-12T01:36:59.053 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-09-12T01:36:59.054 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-09-12T01:36:59.054 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-09-12T01:36:59.054 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-09-12T01:36:59.054 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 1762, 2024-09-12T01:36:59.054 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 6495200, 2024-09-12T01:36:59.054 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 1762, 2024-09-12T01:36:59.054 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 6495200, 2024-09-12T01:36:59.054 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-09-12T01:36:59.054 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:59.054 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:59.054 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:59.055 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-12T01:36:59.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.055 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_ops": { 2024-09-12T01:36:59.055 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-09-12T01:36:59.055 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 1024, 2024-09-12T01:36:59.055 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-09-12T01:36:59.055 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-09-12T01:36:59.055 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-09-12T01:36:59.055 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-09-12T01:36:59.056 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-09-12T01:36:59.056 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 1762, 2024-09-12T01:36:59.056 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 1762, 2024-09-12T01:36:59.056 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 1762, 2024-09-12T01:36:59.056 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 1762, 2024-09-12T01:36:59.056 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-09-12T01:36:59.056 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:59.056 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:59.057 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:59.057 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-12T01:36:59.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.057 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle": { 2024-09-12T01:36:59.057 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-09-12T01:36:59.057 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-09-12T01:36:59.057 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-09-12T01:36:59.058 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-09-12T01:36:59.058 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-09-12T01:36:59.058 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-09-12T01:36:59.058 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-09-12T01:36:59.058 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-09-12T01:36:59.058 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-09-12T01:36:59.058 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 0, 2024-09-12T01:36:59.058 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 0, 2024-09-12T01:36:59.058 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-09-12T01:36:59.059 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:59.059 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:59.059 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:59.059 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-12T01:36:59.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:36:59.059 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle-0x55f28d7406e0": { 2024-09-12T01:36:59.059 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-09-12T01:36:59.059 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-09-12T01:36:59.059 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-09-12T01:36:59.060 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-09-12T01:36:59.060 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-09-12T01:36:59.060 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-09-12T01:36:59.060 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-09-12T01:36:59.060 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-09-12T01:36:59.060 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-09-12T01:36:59.060 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 0, 2024-09-12T01:36:59.060 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 0, 2024-09-12T01:36:59.061 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-09-12T01:36:59.061 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:36:59.061 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:36:59.061 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:36:59.061 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-12T01:36:59.061 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-12T01:36:59.061 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-12T01:36:59.737 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-12T01:36:59.738 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-12T01:36:59.738 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-09-12T01:36:59.738 DEBUG:tasks.check_counter:Getting stats from h 2024-09-12T01:36:59.739 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-12T01:36:59.988 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T01:37:00.506 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-12T01:37:00.506 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T01:37:00.506 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 37055, 2024-09-12T01:37:00.506 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 37679, 2024-09-12T01:37:00.506 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 12913533, 2024-09-12T01:37:00.506 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 39669594, 2024-09-12T01:37:00.506 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 9, 2024-09-12T01:37:00.506 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 5, 2024-09-12T01:37:00.506 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 2.353209055, 2024-09-12T01:37:00.507 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.664949306, 2024-09-12T01:37:00.507 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.738243630, 2024-09-12T01:37:00.507 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.818202217, 2024-09-12T01:37:00.507 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:37:00.507 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 37679, 2024-09-12T01:37:00.507 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.927732850, 2024-09-12T01:37:00.507 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000024622 2024-09-12T01:37:00.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.507 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:37:00.507 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 2444, 2024-09-12T01:37:00.507 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000462226, 2024-09-12T01:37:00.507 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000189 2024-09-12T01:37:00.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.507 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 860048, 2024-09-12T01:37:00.508 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 113104 2024-09-12T01:37:00.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.508 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T01:37:00.508 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 19527, 2024-09-12T01:37:00.508 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 19840, 2024-09-12T01:37:00.508 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 4939754, 2024-09-12T01:37:00.508 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 40381246, 2024-09-12T01:37:00.508 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 14, 2024-09-12T01:37:00.508 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 10, 2024-09-12T01:37:00.508 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 1.419938313, 2024-09-12T01:37:00.508 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.481376363, 2024-09-12T01:37:00.508 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.365689893, 2024-09-12T01:37:00.508 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.469969989, 2024-09-12T01:37:00.508 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:37:00.509 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 19838, 2024-09-12T01:37:00.509 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.753509286, 2024-09-12T01:37:00.509 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000037983 2024-09-12T01:37:00.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.509 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:37:00.509 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 612, 2024-09-12T01:37:00.509 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000107318, 2024-09-12T01:37:00.509 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000175 2024-09-12T01:37:00.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.509 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-09-12T01:37:00.510 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 294304 2024-09-12T01:37:00.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.510 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T01:37:00.510 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 65150, 2024-09-12T01:37:00.510 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 123765, 2024-09-12T01:37:00.510 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 20354111, 2024-09-12T01:37:00.510 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 74265577, 2024-09-12T01:37:00.510 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 10, 2024-09-12T01:37:00.510 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 6, 2024-09-12T01:37:00.510 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 4.198153393, 2024-09-12T01:37:00.510 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 1.860825867, 2024-09-12T01:37:00.510 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 2.252858360, 2024-09-12T01:37:00.510 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.532548835, 2024-09-12T01:37:00.510 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:37:00.510 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 123763, 2024-09-12T01:37:00.511 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 2.944843020, 2024-09-12T01:37:00.511 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000023794 2024-09-12T01:37:00.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.511 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:37:00.511 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 77512, 2024-09-12T01:37:00.511 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.023498567, 2024-09-12T01:37:00.511 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000303 2024-09-12T01:37:00.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.511 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T01:37:00.511 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T01:37:00.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.511 INFO:teuthology.orchestra.run.smithi053.stdout: "cct": { 2024-09-12T01:37:00.511 INFO:teuthology.orchestra.run.smithi053.stdout: "total_workers": 1, 2024-09-12T01:37:00.511 INFO:teuthology.orchestra.run.smithi053.stdout: "unhealthy_workers": 0 2024-09-12T01:37:00.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.512 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-MDSRank": { 2024-09-12T01:37:00.512 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-09-12T01:37:00.512 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-09-12T01:37:00.512 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 87960, 2024-09-12T01:37:00.512 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 23.507420181, 2024-09-12T01:37:00.512 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000267251 2024-09-12T01:37:00.512 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-12T01:37:00.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.512 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-PurgeQueue": { 2024-09-12T01:37:00.512 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-09-12T01:37:00.512 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-09-12T01:37:00.513 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 13515, 2024-09-12T01:37:00.513 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 2.474188267, 2024-09-12T01:37:00.513 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000183069 2024-09-12T01:37:00.513 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-12T01:37:00.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.513 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": { 2024-09-12T01:37:00.513 INFO:teuthology.orchestra.run.smithi053.stdout: "request": 26515, 2024-09-12T01:37:00.513 INFO:teuthology.orchestra.run.smithi053.stdout: "reply": 26512, 2024-09-12T01:37:00.513 INFO:teuthology.orchestra.run.smithi053.stdout: "reply_latency": { 2024-09-12T01:37:00.513 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 26512, 2024-09-12T01:37:00.513 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 64.817646735, 2024-09-12T01:37:00.513 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.002444841 2024-09-12T01:37:00.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.514 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_reply": 0, 2024-09-12T01:37:00.514 INFO:teuthology.orchestra.run.smithi053.stdout: "forward": 3, 2024-09-12T01:37:00.514 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_complete": 0, 2024-09-12T01:37:00.514 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_keys": 0, 2024-09-12T01:37:00.514 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_commit": 101, 2024-09-12T01:37:00.514 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_split": 11, 2024-09-12T01:37:00.514 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_merge": 0, 2024-09-12T01:37:00.514 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes": 22, 2024-09-12T01:37:00.514 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_top": 4, 2024-09-12T01:37:00.514 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_bottom": 2, 2024-09-12T01:37:00.514 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pin_tail": 16, 2024-09-12T01:37:00.514 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pinned": 16, 2024-09-12T01:37:00.515 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_expired": 24, 2024-09-12T01:37:00.515 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_with_caps": 0, 2024-09-12T01:37:00.515 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": 0, 2024-09-12T01:37:00.515 INFO:teuthology.orchestra.run.smithi053.stdout: "subtrees": 3, 2024-09-12T01:37:00.515 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse": 32783, 2024-09-12T01:37:00.515 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_hit": 32459, 2024-09-12T01:37:00.516 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_forward": 1, 2024-09-12T01:37:00.516 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_discover": 4, 2024-09-12T01:37:00.516 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_dir_fetch": 0, 2024-09-12T01:37:00.516 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_remote_ino": 0, 2024-09-12T01:37:00.516 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_lock": 0, 2024-09-12T01:37:00.516 INFO:teuthology.orchestra.run.smithi053.stdout: "load_cent": 0, 2024-09-12T01:37:00.516 INFO:teuthology.orchestra.run.smithi053.stdout: "q": 0, 2024-09-12T01:37:00.516 INFO:teuthology.orchestra.run.smithi053.stdout: "exported": 3, 2024-09-12T01:37:00.516 INFO:teuthology.orchestra.run.smithi053.stdout: "exported_inodes": 15, 2024-09-12T01:37:00.516 INFO:teuthology.orchestra.run.smithi053.stdout: "imported": 6, 2024-09-12T01:37:00.516 INFO:teuthology.orchestra.run.smithi053.stdout: "imported_inodes": 2921, 2024-09-12T01:37:00.516 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_dir_fetch": 0, 2024-09-12T01:37:00.516 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_backtrace_fetch": 0, 2024-09-12T01:37:00.517 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_peer_discover": 0, 2024-09-12T01:37:00.517 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rfiles": 4, 2024-09-12T01:37:00.517 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rbytes": 234, 2024-09-12T01:37:00.517 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rsnaps": 0, 2024-09-12T01:37:00.517 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T01:37:00.517 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_set_tag": 0, 2024-09-12T01:37:00.517 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T01:37:00.517 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_inotable_repaired": 0, 2024-09-12T01:37:00.517 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_inodes": 0, 2024-09-12T01:37:00.517 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T01:37:00.517 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T01:37:00.517 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_file_inodes": 0, 2024-09-12T01:37:00.518 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_inode_file_caps": 6, 2024-09-12T01:37:00.518 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_revoke": 8729, 2024-09-12T01:37:00.518 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_grant": 21531, 2024-09-12T01:37:00.518 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_trunc": 7863, 2024-09-12T01:37:00.518 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T01:37:00.518 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T01:37:00.518 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps": 16856, 2024-09-12T01:37:00.518 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps_dirty": 14497, 2024-09-12T01:37:00.518 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_cap_release": 535, 2024-09-12T01:37:00.518 INFO:teuthology.orchestra.run.smithi053.stdout: "process_request_cap_release": 16896 2024-09-12T01:37:00.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.518 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache": { 2024-09-12T01:37:00.518 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update": 2, 2024-09-12T01:37:00.518 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update_receipt": 0, 2024-09-12T01:37:00.519 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_try_discover": 0, 2024-09-12T01:37:00.519 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_send_discover": 7, 2024-09-12T01:37:00.519 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_handle_discover": 9, 2024-09-12T01:37:00.519 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays": 4, 2024-09-12T01:37:00.519 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_delayed": 0, 2024-09-12T01:37:00.519 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_enqueuing": 0, 2024-09-12T01:37:00.519 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_created": 7245, 2024-09-12T01:37:00.519 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_enqueued": 7241, 2024-09-12T01:37:00.519 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_reintegrated": 0, 2024-09-12T01:37:00.519 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_migrated": 0, 2024-09-12T01:37:00.519 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_processing": 0, 2024-09-12T01:37:00.519 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_enqueued": 0, 2024-09-12T01:37:00.519 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_prioritized": 0, 2024-09-12T01:37:00.520 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_started": 0, 2024-09-12T01:37:00.520 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_completed": 0, 2024-09-12T01:37:00.520 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_quiesce_path": 0, 2024-09-12T01:37:00.520 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_quiesce_inode": 0, 2024-09-12T01:37:00.520 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T01:37:00.520 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_exportdir": 3, 2024-09-12T01:37:00.520 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_flush": 0, 2024-09-12T01:37:00.520 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragmentdir": 11, 2024-09-12T01:37:00.520 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragstats": 0, 2024-09-12T01:37:00.520 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_inodestats": 0 2024-09-12T01:37:00.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.520 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log": { 2024-09-12T01:37:00.520 INFO:teuthology.orchestra.run.smithi053.stdout: "evlrg": 1, 2024-09-12T01:37:00.521 INFO:teuthology.orchestra.run.smithi053.stdout: "evadd": 57392, 2024-09-12T01:37:00.521 INFO:teuthology.orchestra.run.smithi053.stdout: "evex": 0, 2024-09-12T01:37:00.521 INFO:teuthology.orchestra.run.smithi053.stdout: "evtrm": 0, 2024-09-12T01:37:00.521 INFO:teuthology.orchestra.run.smithi053.stdout: "ev": 57392, 2024-09-12T01:37:00.521 INFO:teuthology.orchestra.run.smithi053.stdout: "evexg": 0, 2024-09-12T01:37:00.521 INFO:teuthology.orchestra.run.smithi053.stdout: "evexd": 0, 2024-09-12T01:37:00.521 INFO:teuthology.orchestra.run.smithi053.stdout: "segadd": 72, 2024-09-12T01:37:00.521 INFO:teuthology.orchestra.run.smithi053.stdout: "segex": 0, 2024-09-12T01:37:00.522 INFO:teuthology.orchestra.run.smithi053.stdout: "segtrm": 0, 2024-09-12T01:37:00.522 INFO:teuthology.orchestra.run.smithi053.stdout: "seg": 72, 2024-09-12T01:37:00.522 INFO:teuthology.orchestra.run.smithi053.stdout: "segmjr": 6, 2024-09-12T01:37:00.522 INFO:teuthology.orchestra.run.smithi053.stdout: "segexg": 0, 2024-09-12T01:37:00.522 INFO:teuthology.orchestra.run.smithi053.stdout: "segexd": 0, 2024-09-12T01:37:00.522 INFO:teuthology.orchestra.run.smithi053.stdout: "expos": 4194304, 2024-09-12T01:37:00.522 INFO:teuthology.orchestra.run.smithi053.stdout: "wrpos": 97136016, 2024-09-12T01:37:00.522 INFO:teuthology.orchestra.run.smithi053.stdout: "rdpos": 0, 2024-09-12T01:37:00.522 INFO:teuthology.orchestra.run.smithi053.stdout: "jlat": { 2024-09-12T01:37:00.522 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 24485, 2024-09-12T01:37:00.522 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 61.659200647, 2024-09-12T01:37:00.522 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.002518243 2024-09-12T01:37:00.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.522 INFO:teuthology.orchestra.run.smithi053.stdout: "replayed": 0 2024-09-12T01:37:00.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.523 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_mem": { 2024-09-12T01:37:00.523 INFO:teuthology.orchestra.run.smithi053.stdout: "ino": 22, 2024-09-12T01:37:00.523 INFO:teuthology.orchestra.run.smithi053.stdout: "ino+": 7276, 2024-09-12T01:37:00.523 INFO:teuthology.orchestra.run.smithi053.stdout: "ino-": 7254, 2024-09-12T01:37:00.524 INFO:teuthology.orchestra.run.smithi053.stdout: "dir": 21, 2024-09-12T01:37:00.524 INFO:teuthology.orchestra.run.smithi053.stdout: "dir+": 251, 2024-09-12T01:37:00.524 INFO:teuthology.orchestra.run.smithi053.stdout: "dir-": 230, 2024-09-12T01:37:00.524 INFO:teuthology.orchestra.run.smithi053.stdout: "dn": 22, 2024-09-12T01:37:00.524 INFO:teuthology.orchestra.run.smithi053.stdout: "dn+": 12466, 2024-09-12T01:37:00.524 INFO:teuthology.orchestra.run.smithi053.stdout: "dn-": 12444, 2024-09-12T01:37:00.524 INFO:teuthology.orchestra.run.smithi053.stdout: "cap": 0, 2024-09-12T01:37:00.524 INFO:teuthology.orchestra.run.smithi053.stdout: "cap+": 7606, 2024-09-12T01:37:00.524 INFO:teuthology.orchestra.run.smithi053.stdout: "cap-": 7606, 2024-09-12T01:37:00.524 INFO:teuthology.orchestra.run.smithi053.stdout: "rss": 93540, 2024-09-12T01:37:00.525 INFO:teuthology.orchestra.run.smithi053.stdout: "heap": 223520 2024-09-12T01:37:00.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.525 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_server": { 2024-09-12T01:37:00.525 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_client_request": 45241, 2024-09-12T01:37:00.525 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_server_request": 20, 2024-09-12T01:37:00.525 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_request": 26515, 2024-09-12T01:37:00.525 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_session": 16, 2024-09-12T01:37:00.525 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_peer_request": 69, 2024-09-12T01:37:00.525 INFO:teuthology.orchestra.run.smithi053.stdout: "req_create_latency": { 2024-09-12T01:37:00.525 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 4605, 2024-09-12T01:37:00.525 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 9.333403595, 2024-09-12T01:37:00.525 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.002026797 2024-09-12T01:37:00.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.526 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getattr_latency": { 2024-09-12T01:37:00.526 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 7, 2024-09-12T01:37:00.526 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.019829977, 2024-09-12T01:37:00.526 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.002832853 2024-09-12T01:37:00.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.526 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getfilelock_latency": { 2024-09-12T01:37:00.526 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:00.526 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:00.526 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:00.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.527 INFO:teuthology.orchestra.run.smithi053.stdout: "req_link_latency": { 2024-09-12T01:37:00.527 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:00.527 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:00.527 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:00.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.527 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookup_latency": { 2024-09-12T01:37:00.527 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 5144, 2024-09-12T01:37:00.527 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 2.742583656, 2024-09-12T01:37:00.527 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000533161 2024-09-12T01:37:00.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.527 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookuphash_latency": { 2024-09-12T01:37:00.527 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:00.527 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:00.527 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:00.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.528 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupino_latency": { 2024-09-12T01:37:00.528 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:00.528 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:00.528 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:00.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.528 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupname_latency": { 2024-09-12T01:37:00.528 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:00.528 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:00.528 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:00.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.528 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupparent_latency": { 2024-09-12T01:37:00.529 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:00.529 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:00.529 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:00.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.529 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupsnap_latency": { 2024-09-12T01:37:00.529 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:00.529 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:00.529 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:00.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.529 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lssnap_latency": { 2024-09-12T01:37:00.529 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:00.529 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:00.530 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:00.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.530 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mkdir_latency": { 2024-09-12T01:37:00.530 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 23, 2024-09-12T01:37:00.530 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.119000677, 2024-09-12T01:37:00.530 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.005173942 2024-09-12T01:37:00.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.530 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mknod_latency": { 2024-09-12T01:37:00.530 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:00.530 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:00.531 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:00.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.531 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mksnap_latency": { 2024-09-12T01:37:00.531 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:00.531 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:00.531 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:00.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.531 INFO:teuthology.orchestra.run.smithi053.stdout: "req_open_latency": { 2024-09-12T01:37:00.531 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 131, 2024-09-12T01:37:00.531 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.047587079, 2024-09-12T01:37:00.531 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000363260 2024-09-12T01:37:00.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.532 INFO:teuthology.orchestra.run.smithi053.stdout: "req_readdir_latency": { 2024-09-12T01:37:00.532 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 461, 2024-09-12T01:37:00.532 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.909037634, 2024-09-12T01:37:00.532 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.001971882 2024-09-12T01:37:00.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.532 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rename_latency": { 2024-09-12T01:37:00.532 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 74, 2024-09-12T01:37:00.532 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.106800375, 2024-09-12T01:37:00.532 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.001443248 2024-09-12T01:37:00.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.533 INFO:teuthology.orchestra.run.smithi053.stdout: "req_renamesnap_latency": { 2024-09-12T01:37:00.533 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:00.533 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:00.533 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:00.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.533 INFO:teuthology.orchestra.run.smithi053.stdout: "req_snapdiff_latency": { 2024-09-12T01:37:00.533 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:00.533 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:00.533 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:00.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.533 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmdir_latency": { 2024-09-12T01:37:00.533 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 69, 2024-09-12T01:37:00.533 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 2.259468968, 2024-09-12T01:37:00.534 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.032745927 2024-09-12T01:37:00.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.534 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmsnap_latency": { 2024-09-12T01:37:00.534 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:00.534 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:00.534 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:00.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.534 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmxattr_latency": { 2024-09-12T01:37:00.534 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:00.534 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:00.534 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:00.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.535 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setattr_latency": { 2024-09-12T01:37:00.535 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 8855, 2024-09-12T01:37:00.535 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 35.754285113, 2024-09-12T01:37:00.535 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.004037751 2024-09-12T01:37:00.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.535 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setdirlayout_latency": { 2024-09-12T01:37:00.535 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:00.535 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:00.535 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:00.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.535 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setfilelock_latency": { 2024-09-12T01:37:00.535 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 8, 2024-09-12T01:37:00.535 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.002541522, 2024-09-12T01:37:00.536 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000317690 2024-09-12T01:37:00.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.536 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setlayout_latency": { 2024-09-12T01:37:00.536 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:00.536 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:00.536 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:00.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.536 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setxattr_latency": { 2024-09-12T01:37:00.536 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:00.536 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:00.536 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:00.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.536 INFO:teuthology.orchestra.run.smithi053.stdout: "req_symlink_latency": { 2024-09-12T01:37:00.536 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 5, 2024-09-12T01:37:00.536 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.003786683, 2024-09-12T01:37:00.537 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000757336 2024-09-12T01:37:00.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.537 INFO:teuthology.orchestra.run.smithi053.stdout: "req_unlink_latency": { 2024-09-12T01:37:00.537 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 7130, 2024-09-12T01:37:00.537 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 13.519321456, 2024-09-12T01:37:00.537 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.001896118 2024-09-12T01:37:00.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.537 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_revoke_eviction": 0, 2024-09-12T01:37:00.537 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_acquisition_throttle": 0, 2024-09-12T01:37:00.537 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getvxattr_latency": { 2024-09-12T01:37:00.537 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:00.537 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:00.537 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:00.537 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-12T01:37:00.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.538 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_sessions": { 2024-09-12T01:37:00.538 INFO:teuthology.orchestra.run.smithi053.stdout: "session_count": 1, 2024-09-12T01:37:00.538 INFO:teuthology.orchestra.run.smithi053.stdout: "session_add": 3, 2024-09-12T01:37:00.538 INFO:teuthology.orchestra.run.smithi053.stdout: "session_remove": 2, 2024-09-12T01:37:00.538 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_open": 1, 2024-09-12T01:37:00.538 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_stale": 0, 2024-09-12T01:37:00.538 INFO:teuthology.orchestra.run.smithi053.stdout: "total_load": 9495, 2024-09-12T01:37:00.538 INFO:teuthology.orchestra.run.smithi053.stdout: "average_load": 9495, 2024-09-12T01:37:00.538 INFO:teuthology.orchestra.run.smithi053.stdout: "avg_session_uptime": 426, 2024-09-12T01:37:00.538 INFO:teuthology.orchestra.run.smithi053.stdout: "mdthresh_evicted": 0 2024-09-12T01:37:00.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.539 INFO:teuthology.orchestra.run.smithi053.stdout: "mempool": { 2024-09-12T01:37:00.539 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_bytes": 0, 2024-09-12T01:37:00.539 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_items": 0, 2024-09-12T01:37:00.539 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T01:37:00.539 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_items": 0, 2024-09-12T01:37:00.539 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T01:37:00.539 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_items": 0, 2024-09-12T01:37:00.539 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T01:37:00.539 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T01:37:00.540 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T01:37:00.540 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T01:37:00.540 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T01:37:00.540 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_items": 0, 2024-09-12T01:37:00.540 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T01:37:00.540 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T01:37:00.540 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_bytes": 0, 2024-09-12T01:37:00.540 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_items": 0, 2024-09-12T01:37:00.541 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_bytes": 0, 2024-09-12T01:37:00.541 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_items": 0, 2024-09-12T01:37:00.541 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T01:37:00.541 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T01:37:00.541 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T01:37:00.541 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T01:37:00.541 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T01:37:00.541 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_items": 0, 2024-09-12T01:37:00.541 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_bytes": 0, 2024-09-12T01:37:00.541 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_items": 0, 2024-09-12T01:37:00.541 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T01:37:00.541 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T01:37:00.542 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_bytes": 0, 2024-09-12T01:37:00.542 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_items": 0, 2024-09-12T01:37:00.542 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_bytes": 0, 2024-09-12T01:37:00.542 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_items": 0, 2024-09-12T01:37:00.542 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T01:37:00.542 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_items": 0, 2024-09-12T01:37:00.542 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T01:37:00.542 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_items": 0, 2024-09-12T01:37:00.542 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_bytes": 317470, 2024-09-12T01:37:00.542 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_items": 156, 2024-09-12T01:37:00.542 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_bytes": 1584, 2024-09-12T01:37:00.543 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_items": 18, 2024-09-12T01:37:00.544 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bytes": 0, 2024-09-12T01:37:00.544 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_items": 0, 2024-09-12T01:37:00.544 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_bytes": 0, 2024-09-12T01:37:00.544 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_items": 0, 2024-09-12T01:37:00.544 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_bytes": 0, 2024-09-12T01:37:00.544 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_items": 0, 2024-09-12T01:37:00.544 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_bytes": 8856, 2024-09-12T01:37:00.544 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_items": 166, 2024-09-12T01:37:00.544 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T01:37:00.544 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_items": 0, 2024-09-12T01:37:00.544 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_bytes": 0, 2024-09-12T01:37:00.544 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_items": 0, 2024-09-12T01:37:00.544 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_bytes": 96418, 2024-09-12T01:37:00.545 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_items": 1043, 2024-09-12T01:37:00.545 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_bytes": 0, 2024-09-12T01:37:00.545 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_items": 0, 2024-09-12T01:37:00.545 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_bytes": 0, 2024-09-12T01:37:00.545 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_items": 0 2024-09-12T01:37:00.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.545 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter": { 2024-09-12T01:37:00.545 INFO:teuthology.orchestra.run.smithi053.stdout: "op_active": 0, 2024-09-12T01:37:00.545 INFO:teuthology.orchestra.run.smithi053.stdout: "op_laggy": 0, 2024-09-12T01:37:00.545 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send": 76331, 2024-09-12T01:37:00.545 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send_bytes": 95529303, 2024-09-12T01:37:00.545 INFO:teuthology.orchestra.run.smithi053.stdout: "op_resend": 30, 2024-09-12T01:37:00.545 INFO:teuthology.orchestra.run.smithi053.stdout: "op_reply": 76331, 2024-09-12T01:37:00.545 INFO:teuthology.orchestra.run.smithi053.stdout: "op_latency": { 2024-09-12T01:37:00.545 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 76331, 2024-09-12T01:37:00.546 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 403.066777356, 2024-09-12T01:37:00.546 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.005280512 2024-09-12T01:37:00.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.546 INFO:teuthology.orchestra.run.smithi053.stdout: "op_inflight": 0, 2024-09-12T01:37:00.546 INFO:teuthology.orchestra.run.smithi053.stdout: "oplen_avg": { 2024-09-12T01:37:00.546 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 76331, 2024-09-12T01:37:00.546 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 76567 2024-09-12T01:37:00.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.546 INFO:teuthology.orchestra.run.smithi053.stdout: "op": 76331, 2024-09-12T01:37:00.546 INFO:teuthology.orchestra.run.smithi053.stdout: "op_r": 90, 2024-09-12T01:37:00.546 INFO:teuthology.orchestra.run.smithi053.stdout: "op_w": 76241, 2024-09-12T01:37:00.546 INFO:teuthology.orchestra.run.smithi053.stdout: "op_rmw": 0, 2024-09-12T01:37:00.546 INFO:teuthology.orchestra.run.smithi053.stdout: "op_pg": 0, 2024-09-12T01:37:00.546 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_stat": 0, 2024-09-12T01:37:00.546 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_create": 10, 2024-09-12T01:37:00.547 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_read": 90, 2024-09-12T01:37:00.547 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_write": 24575, 2024-09-12T01:37:00.547 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writefull": 67, 2024-09-12T01:37:00.547 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writesame": 0, 2024-09-12T01:37:00.547 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_append": 0, 2024-09-12T01:37:00.547 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_zero": 0, 2024-09-12T01:37:00.547 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_truncate": 3, 2024-09-12T01:37:00.547 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_delete": 14650, 2024-09-12T01:37:00.547 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_mapext": 0, 2024-09-12T01:37:00.547 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_sparse_read": 0, 2024-09-12T01:37:00.547 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_clonerange": 0, 2024-09-12T01:37:00.547 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_getxattr": 0, 2024-09-12T01:37:00.547 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_setxattr": 20, 2024-09-12T01:37:00.547 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_cmpxattr": 0, 2024-09-12T01:37:00.547 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_rmxattr": 0, 2024-09-12T01:37:00.547 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_resetxattrs": 0, 2024-09-12T01:37:00.548 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_call": 0, 2024-09-12T01:37:00.548 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_watch": 0, 2024-09-12T01:37:00.548 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_notify": 0, 2024-09-12T01:37:00.548 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T01:37:00.548 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls": 0, 2024-09-12T01:37:00.548 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls_filter": 0, 2024-09-12T01:37:00.548 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_other": 36717, 2024-09-12T01:37:00.548 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_active": 0, 2024-09-12T01:37:00.548 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_send": 0, 2024-09-12T01:37:00.548 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_resend": 0, 2024-09-12T01:37:00.548 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_ping": 0, 2024-09-12T01:37:00.548 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_active": 0, 2024-09-12T01:37:00.548 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_send": 0, 2024-09-12T01:37:00.548 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_resend": 0, 2024-09-12T01:37:00.548 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_active": 0, 2024-09-12T01:37:00.549 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_send": 0, 2024-09-12T01:37:00.549 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_resend": 0, 2024-09-12T01:37:00.549 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_active": 0, 2024-09-12T01:37:00.549 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_send": 0, 2024-09-12T01:37:00.549 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_resend": 0, 2024-09-12T01:37:00.549 INFO:teuthology.orchestra.run.smithi053.stdout: "command_active": 0, 2024-09-12T01:37:00.549 INFO:teuthology.orchestra.run.smithi053.stdout: "command_send": 0, 2024-09-12T01:37:00.549 INFO:teuthology.orchestra.run.smithi053.stdout: "command_resend": 0, 2024-09-12T01:37:00.549 INFO:teuthology.orchestra.run.smithi053.stdout: "map_epoch": 98, 2024-09-12T01:37:00.549 INFO:teuthology.orchestra.run.smithi053.stdout: "map_full": 0, 2024-09-12T01:37:00.549 INFO:teuthology.orchestra.run.smithi053.stdout: "map_inc": 2, 2024-09-12T01:37:00.549 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_sessions": 12, 2024-09-12T01:37:00.549 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_open": 12, 2024-09-12T01:37:00.549 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_close": 0, 2024-09-12T01:37:00.549 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_laggy": 0, 2024-09-12T01:37:00.550 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_wr": 387, 2024-09-12T01:37:00.550 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_rd": 0, 2024-09-12T01:37:00.550 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_del": 48 2024-09-12T01:37:00.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.550 INFO:teuthology.orchestra.run.smithi053.stdout: "oft": { 2024-09-12T01:37:00.550 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_objs": 1, 2024-09-12T01:37:00.550 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_kv_pairs": 0, 2024-09-12T01:37:00.550 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_updates": 0, 2024-09-12T01:37:00.550 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_removes": 0 2024-09-12T01:37:00.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.550 INFO:teuthology.orchestra.run.smithi053.stdout: "osd-slow-ops": { 2024-09-12T01:37:00.550 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_ops_count": 0 2024-09-12T01:37:00.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.550 INFO:teuthology.orchestra.run.smithi053.stdout: "purge_queue": { 2024-09-12T01:37:00.551 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops": 0, 2024-09-12T01:37:00.551 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops_high_water": 73, 2024-09-12T01:37:00.551 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing": 0, 2024-09-12T01:37:00.551 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_high_water": 7, 2024-09-12T01:37:00.551 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed_ops": 14608, 2024-09-12T01:37:00.551 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed": 7241, 2024-09-12T01:37:00.551 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_item_in_journal": 0 2024-09-12T01:37:00.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.551 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T01:37:00.551 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-09-12T01:37:00.551 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-09-12T01:37:00.551 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-09-12T01:37:00.551 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 121728, 2024-09-12T01:37:00.551 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 27940481, 2024-09-12T01:37:00.551 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-09-12T01:37:00.551 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 121728, 2024-09-12T01:37:00.552 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-09-12T01:37:00.552 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-09-12T01:37:00.552 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 121728, 2024-09-12T01:37:00.552 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 27940481, 2024-09-12T01:37:00.552 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-09-12T01:37:00.552 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:00.552 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:00.552 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:00.552 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-12T01:37:00.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.552 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_bytes": { 2024-09-12T01:37:00.552 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-09-12T01:37:00.553 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-09-12T01:37:00.553 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-09-12T01:37:00.553 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-09-12T01:37:00.553 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-09-12T01:37:00.553 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-09-12T01:37:00.554 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-09-12T01:37:00.554 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 76331, 2024-09-12T01:37:00.554 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 96318108, 2024-09-12T01:37:00.554 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 76331, 2024-09-12T01:37:00.554 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 96318108, 2024-09-12T01:37:00.554 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-09-12T01:37:00.554 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:00.554 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:00.554 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:00.554 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-12T01:37:00.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.554 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_ops": { 2024-09-12T01:37:00.554 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-09-12T01:37:00.555 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 1024, 2024-09-12T01:37:00.555 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-09-12T01:37:00.555 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-09-12T01:37:00.555 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-09-12T01:37:00.555 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-09-12T01:37:00.555 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-09-12T01:37:00.555 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 76331, 2024-09-12T01:37:00.555 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 76331, 2024-09-12T01:37:00.555 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 76331, 2024-09-12T01:37:00.555 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 76331, 2024-09-12T01:37:00.555 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-09-12T01:37:00.555 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:00.555 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:00.555 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:00.556 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-12T01:37:00.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.556 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle": { 2024-09-12T01:37:00.556 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-09-12T01:37:00.556 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-09-12T01:37:00.556 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-09-12T01:37:00.556 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 7241, 2024-09-12T01:37:00.556 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 788805, 2024-09-12T01:37:00.556 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-09-12T01:37:00.556 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 7241, 2024-09-12T01:37:00.556 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-09-12T01:37:00.557 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-09-12T01:37:00.557 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 90, 2024-09-12T01:37:00.557 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 788805, 2024-09-12T01:37:00.557 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-09-12T01:37:00.557 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:00.557 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:00.557 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:00.557 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-12T01:37:00.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:00.557 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle-0x558b80ea2500": { 2024-09-12T01:37:00.557 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-09-12T01:37:00.557 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-09-12T01:37:00.558 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-09-12T01:37:00.558 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 57392, 2024-09-12T01:37:00.558 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 92941712, 2024-09-12T01:37:00.558 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-09-12T01:37:00.558 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 57392, 2024-09-12T01:37:00.558 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-09-12T01:37:00.558 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-09-12T01:37:00.558 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 24485, 2024-09-12T01:37:00.558 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 92941712, 2024-09-12T01:37:00.558 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-09-12T01:37:00.558 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:00.558 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:00.558 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:00.559 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-12T01:37:00.559 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-12T01:37:00.559 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-12T01:37:00.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:00 smithi053 ceph-mon[35158]: pgmap v620: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 26 KiB/s wr, 25 op/s 2024-09-12T01:37:00.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:00 smithi104 ceph-mon[35219]: pgmap v620: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 26 KiB/s wr, 25 op/s 2024-09-12T01:37:00.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:00 smithi001 ceph-mon[30387]: pgmap v620: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 26 KiB/s wr, 25 op/s 2024-09-12T01:37:01.367 INFO:tasks.check_counter:Daemon mds.h mds.exported=3 2024-09-12T01:37:01.367 INFO:tasks.check_counter:Daemon mds.h mds.imported=6 2024-09-12T01:37:01.368 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=11 2024-09-12T01:37:01.368 DEBUG:tasks.check_counter:Getting stats from k 2024-09-12T01:37:01.368 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-12T01:37:01.621 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T01:37:01.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:01 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:37:01.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:01 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:37:01.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:01 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:37:02.256 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-12T01:37:02.256 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T01:37:02.256 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 1556, 2024-09-12T01:37:02.257 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 1556, 2024-09-12T01:37:02.257 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 445380, 2024-09-12T01:37:02.257 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 493991, 2024-09-12T01:37:02.257 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 6, 2024-09-12T01:37:02.257 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 2, 2024-09-12T01:37:02.257 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.105363995, 2024-09-12T01:37:02.257 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.034085795, 2024-09-12T01:37:02.257 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.027809768, 2024-09-12T01:37:02.257 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.035069066, 2024-09-12T01:37:02.257 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:37:02.257 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1555, 2024-09-12T01:37:02.257 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.045303407, 2024-09-12T01:37:02.257 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000029134 2024-09-12T01:37:02.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.258 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:37:02.258 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:02.258 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:02.258 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:02.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.258 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-12T01:37:02.258 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T01:37:02.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.258 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T01:37:02.258 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 203, 2024-09-12T01:37:02.258 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 187, 2024-09-12T01:37:02.258 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 835372, 2024-09-12T01:37:02.258 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 110890, 2024-09-12T01:37:02.259 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 3, 2024-09-12T01:37:02.259 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 2, 2024-09-12T01:37:02.259 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.028725263, 2024-09-12T01:37:02.259 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.010919710, 2024-09-12T01:37:02.259 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.025954470, 2024-09-12T01:37:02.259 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.000054230, 2024-09-12T01:37:02.259 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:37:02.259 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 186, 2024-09-12T01:37:02.259 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.010212486, 2024-09-12T01:37:02.259 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000054905 2024-09-12T01:37:02.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.259 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:37:02.259 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:02.259 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:02.260 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:02.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.260 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 834816, 2024-09-12T01:37:02.260 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 110192 2024-09-12T01:37:02.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.260 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T01:37:02.260 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 2, 2024-09-12T01:37:02.260 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 136, 2024-09-12T01:37:02.260 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 428, 2024-09-12T01:37:02.260 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 291262, 2024-09-12T01:37:02.260 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 4, 2024-09-12T01:37:02.261 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 3, 2024-09-12T01:37:02.261 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.007362244, 2024-09-12T01:37:02.261 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.005088566, 2024-09-12T01:37:02.261 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.001078714, 2024-09-12T01:37:02.261 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.000015055, 2024-09-12T01:37:02.261 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:37:02.261 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 134, 2024-09-12T01:37:02.261 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.006287351, 2024-09-12T01:37:02.262 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000046920 2024-09-12T01:37:02.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.262 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:37:02.262 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:02.262 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:02.262 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:02.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.262 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-12T01:37:02.262 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 290944 2024-09-12T01:37:02.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.262 INFO:teuthology.orchestra.run.smithi053.stdout: "cct": { 2024-09-12T01:37:02.262 INFO:teuthology.orchestra.run.smithi053.stdout: "total_workers": 1, 2024-09-12T01:37:02.262 INFO:teuthology.orchestra.run.smithi053.stdout: "unhealthy_workers": 0 2024-09-12T01:37:02.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.263 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-MDSRank": { 2024-09-12T01:37:02.263 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-09-12T01:37:02.263 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-09-12T01:37:02.263 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 2658, 2024-09-12T01:37:02.263 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.057477081, 2024-09-12T01:37:02.263 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000021624 2024-09-12T01:37:02.263 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-12T01:37:02.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.263 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-PurgeQueue": { 2024-09-12T01:37:02.263 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-09-12T01:37:02.263 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-09-12T01:37:02.263 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:02.263 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:02.264 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:02.264 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-12T01:37:02.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.264 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": { 2024-09-12T01:37:02.264 INFO:teuthology.orchestra.run.smithi053.stdout: "request": 0, 2024-09-12T01:37:02.264 INFO:teuthology.orchestra.run.smithi053.stdout: "reply": 0, 2024-09-12T01:37:02.264 INFO:teuthology.orchestra.run.smithi053.stdout: "reply_latency": { 2024-09-12T01:37:02.264 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:02.264 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:02.264 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:02.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.264 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_reply": 0, 2024-09-12T01:37:02.264 INFO:teuthology.orchestra.run.smithi053.stdout: "forward": 0, 2024-09-12T01:37:02.265 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_complete": 0, 2024-09-12T01:37:02.265 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_keys": 0, 2024-09-12T01:37:02.265 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_commit": 0, 2024-09-12T01:37:02.265 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_split": 0, 2024-09-12T01:37:02.265 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_merge": 0, 2024-09-12T01:37:02.265 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes": 0, 2024-09-12T01:37:02.265 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_top": 0, 2024-09-12T01:37:02.265 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_bottom": 0, 2024-09-12T01:37:02.265 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pin_tail": 0, 2024-09-12T01:37:02.265 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pinned": 0, 2024-09-12T01:37:02.265 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_expired": 0, 2024-09-12T01:37:02.265 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_with_caps": 0, 2024-09-12T01:37:02.265 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": 0, 2024-09-12T01:37:02.266 INFO:teuthology.orchestra.run.smithi053.stdout: "subtrees": 1, 2024-09-12T01:37:02.266 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse": 0, 2024-09-12T01:37:02.266 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_hit": 0, 2024-09-12T01:37:02.266 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_forward": 0, 2024-09-12T01:37:02.266 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_discover": 0, 2024-09-12T01:37:02.266 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_dir_fetch": 0, 2024-09-12T01:37:02.266 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_remote_ino": 0, 2024-09-12T01:37:02.266 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_lock": 0, 2024-09-12T01:37:02.266 INFO:teuthology.orchestra.run.smithi053.stdout: "load_cent": 0, 2024-09-12T01:37:02.266 INFO:teuthology.orchestra.run.smithi053.stdout: "q": 0, 2024-09-12T01:37:02.266 INFO:teuthology.orchestra.run.smithi053.stdout: "exported": 0, 2024-09-12T01:37:02.266 INFO:teuthology.orchestra.run.smithi053.stdout: "exported_inodes": 0, 2024-09-12T01:37:02.266 INFO:teuthology.orchestra.run.smithi053.stdout: "imported": 0, 2024-09-12T01:37:02.266 INFO:teuthology.orchestra.run.smithi053.stdout: "imported_inodes": 0, 2024-09-12T01:37:02.267 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_dir_fetch": 0, 2024-09-12T01:37:02.267 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_backtrace_fetch": 0, 2024-09-12T01:37:02.267 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_peer_discover": 0, 2024-09-12T01:37:02.267 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rfiles": 0, 2024-09-12T01:37:02.267 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rbytes": 0, 2024-09-12T01:37:02.267 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rsnaps": 0, 2024-09-12T01:37:02.267 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T01:37:02.267 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_set_tag": 0, 2024-09-12T01:37:02.267 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T01:37:02.267 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_inotable_repaired": 0, 2024-09-12T01:37:02.267 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_inodes": 0, 2024-09-12T01:37:02.267 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T01:37:02.267 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T01:37:02.268 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_file_inodes": 0, 2024-09-12T01:37:02.268 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_inode_file_caps": 0, 2024-09-12T01:37:02.268 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T01:37:02.268 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_grant": 0, 2024-09-12T01:37:02.268 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T01:37:02.268 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T01:37:02.268 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T01:37:02.268 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps": 0, 2024-09-12T01:37:02.268 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps_dirty": 0, 2024-09-12T01:37:02.268 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_cap_release": 0, 2024-09-12T01:37:02.268 INFO:teuthology.orchestra.run.smithi053.stdout: "process_request_cap_release": 0 2024-09-12T01:37:02.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.268 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache": { 2024-09-12T01:37:02.269 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update": 0, 2024-09-12T01:37:02.269 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update_receipt": 0, 2024-09-12T01:37:02.269 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_try_discover": 0, 2024-09-12T01:37:02.269 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_send_discover": 0, 2024-09-12T01:37:02.269 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_handle_discover": 0, 2024-09-12T01:37:02.269 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays": 0, 2024-09-12T01:37:02.269 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_delayed": 0, 2024-09-12T01:37:02.269 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_enqueuing": 0, 2024-09-12T01:37:02.269 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_created": 0, 2024-09-12T01:37:02.269 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_enqueued": 0, 2024-09-12T01:37:02.269 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_reintegrated": 0, 2024-09-12T01:37:02.269 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_migrated": 0, 2024-09-12T01:37:02.269 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_processing": 0, 2024-09-12T01:37:02.270 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_enqueued": 0, 2024-09-12T01:37:02.270 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_prioritized": 0, 2024-09-12T01:37:02.270 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_started": 0, 2024-09-12T01:37:02.270 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_completed": 0, 2024-09-12T01:37:02.270 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_quiesce_path": 0, 2024-09-12T01:37:02.270 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_quiesce_inode": 0, 2024-09-12T01:37:02.270 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T01:37:02.270 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_exportdir": 0, 2024-09-12T01:37:02.270 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_flush": 0, 2024-09-12T01:37:02.270 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragmentdir": 0, 2024-09-12T01:37:02.270 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragstats": 0, 2024-09-12T01:37:02.270 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_inodestats": 0 2024-09-12T01:37:02.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.271 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log": { 2024-09-12T01:37:02.271 INFO:teuthology.orchestra.run.smithi053.stdout: "evlrg": 0, 2024-09-12T01:37:02.271 INFO:teuthology.orchestra.run.smithi053.stdout: "evadd": 0, 2024-09-12T01:37:02.271 INFO:teuthology.orchestra.run.smithi053.stdout: "evex": 0, 2024-09-12T01:37:02.271 INFO:teuthology.orchestra.run.smithi053.stdout: "evtrm": 0, 2024-09-12T01:37:02.271 INFO:teuthology.orchestra.run.smithi053.stdout: "ev": 3, 2024-09-12T01:37:02.271 INFO:teuthology.orchestra.run.smithi053.stdout: "evexg": 0, 2024-09-12T01:37:02.271 INFO:teuthology.orchestra.run.smithi053.stdout: "evexd": 0, 2024-09-12T01:37:02.271 INFO:teuthology.orchestra.run.smithi053.stdout: "segadd": 0, 2024-09-12T01:37:02.271 INFO:teuthology.orchestra.run.smithi053.stdout: "segex": 0, 2024-09-12T01:37:02.271 INFO:teuthology.orchestra.run.smithi053.stdout: "segtrm": 0, 2024-09-12T01:37:02.271 INFO:teuthology.orchestra.run.smithi053.stdout: "seg": 2, 2024-09-12T01:37:02.271 INFO:teuthology.orchestra.run.smithi053.stdout: "segmjr": 2, 2024-09-12T01:37:02.271 INFO:teuthology.orchestra.run.smithi053.stdout: "segexg": 0, 2024-09-12T01:37:02.272 INFO:teuthology.orchestra.run.smithi053.stdout: "segexd": 0, 2024-09-12T01:37:02.272 INFO:teuthology.orchestra.run.smithi053.stdout: "expos": 4194304, 2024-09-12T01:37:02.272 INFO:teuthology.orchestra.run.smithi053.stdout: "wrpos": 4195203, 2024-09-12T01:37:02.272 INFO:teuthology.orchestra.run.smithi053.stdout: "rdpos": 4194892, 2024-09-12T01:37:02.272 INFO:teuthology.orchestra.run.smithi053.stdout: "jlat": { 2024-09-12T01:37:02.272 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:02.272 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:02.272 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:02.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.272 INFO:teuthology.orchestra.run.smithi053.stdout: "replayed": 3 2024-09-12T01:37:02.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.272 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_mem": { 2024-09-12T01:37:02.272 INFO:teuthology.orchestra.run.smithi053.stdout: "ino": 3, 2024-09-12T01:37:02.272 INFO:teuthology.orchestra.run.smithi053.stdout: "ino+": 3, 2024-09-12T01:37:02.273 INFO:teuthology.orchestra.run.smithi053.stdout: "ino-": 0, 2024-09-12T01:37:02.273 INFO:teuthology.orchestra.run.smithi053.stdout: "dir": 1, 2024-09-12T01:37:02.273 INFO:teuthology.orchestra.run.smithi053.stdout: "dir+": 1, 2024-09-12T01:37:02.273 INFO:teuthology.orchestra.run.smithi053.stdout: "dir-": 0, 2024-09-12T01:37:02.273 INFO:teuthology.orchestra.run.smithi053.stdout: "dn": 0, 2024-09-12T01:37:02.273 INFO:teuthology.orchestra.run.smithi053.stdout: "dn+": 0, 2024-09-12T01:37:02.273 INFO:teuthology.orchestra.run.smithi053.stdout: "dn-": 0, 2024-09-12T01:37:02.273 INFO:teuthology.orchestra.run.smithi053.stdout: "cap": 0, 2024-09-12T01:37:02.273 INFO:teuthology.orchestra.run.smithi053.stdout: "cap+": 0, 2024-09-12T01:37:02.273 INFO:teuthology.orchestra.run.smithi053.stdout: "cap-": 0, 2024-09-12T01:37:02.273 INFO:teuthology.orchestra.run.smithi053.stdout: "rss": 48740, 2024-09-12T01:37:02.273 INFO:teuthology.orchestra.run.smithi053.stdout: "heap": 215328 2024-09-12T01:37:02.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.273 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_server": { 2024-09-12T01:37:02.274 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_client_request": 0, 2024-09-12T01:37:02.274 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_server_request": 0, 2024-09-12T01:37:02.274 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_request": 0, 2024-09-12T01:37:02.274 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_session": 0, 2024-09-12T01:37:02.274 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_peer_request": 0, 2024-09-12T01:37:02.274 INFO:teuthology.orchestra.run.smithi053.stdout: "req_create_latency": { 2024-09-12T01:37:02.274 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:02.274 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:02.274 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:02.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.274 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getattr_latency": { 2024-09-12T01:37:02.274 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:02.274 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:02.274 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:02.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.275 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getfilelock_latency": { 2024-09-12T01:37:02.275 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:02.275 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:02.275 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:02.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.275 INFO:teuthology.orchestra.run.smithi053.stdout: "req_link_latency": { 2024-09-12T01:37:02.275 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:02.275 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:02.275 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:02.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.275 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookup_latency": { 2024-09-12T01:37:02.275 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:02.275 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:02.275 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:02.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.276 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookuphash_latency": { 2024-09-12T01:37:02.276 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:02.276 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:02.276 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:02.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.276 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupino_latency": { 2024-09-12T01:37:02.276 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:02.276 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:02.276 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:02.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.276 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupname_latency": { 2024-09-12T01:37:02.276 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:02.276 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:02.276 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:02.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.277 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupparent_latency": { 2024-09-12T01:37:02.277 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:02.277 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:02.277 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:02.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.277 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupsnap_latency": { 2024-09-12T01:37:02.277 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:02.277 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:02.277 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:02.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.277 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lssnap_latency": { 2024-09-12T01:37:02.277 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:02.278 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:02.278 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:02.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.278 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mkdir_latency": { 2024-09-12T01:37:02.278 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:02.278 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:02.278 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:02.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.279 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mknod_latency": { 2024-09-12T01:37:02.279 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:02.279 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:02.279 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:02.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.279 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mksnap_latency": { 2024-09-12T01:37:02.279 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:02.279 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:02.279 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:02.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.279 INFO:teuthology.orchestra.run.smithi053.stdout: "req_open_latency": { 2024-09-12T01:37:02.279 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:02.279 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:02.280 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:02.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.280 INFO:teuthology.orchestra.run.smithi053.stdout: "req_readdir_latency": { 2024-09-12T01:37:02.280 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:02.280 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:02.280 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:02.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.280 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rename_latency": { 2024-09-12T01:37:02.280 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:02.280 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:02.280 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:02.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.280 INFO:teuthology.orchestra.run.smithi053.stdout: "req_renamesnap_latency": { 2024-09-12T01:37:02.280 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:02.281 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:02.281 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:02.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.281 INFO:teuthology.orchestra.run.smithi053.stdout: "req_snapdiff_latency": { 2024-09-12T01:37:02.281 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:02.281 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:02.281 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:02.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.281 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmdir_latency": { 2024-09-12T01:37:02.281 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:02.281 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:02.281 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:02.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.282 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmsnap_latency": { 2024-09-12T01:37:02.282 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:02.282 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:02.282 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:02.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.282 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmxattr_latency": { 2024-09-12T01:37:02.282 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:02.282 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:02.282 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:02.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.282 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setattr_latency": { 2024-09-12T01:37:02.282 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:02.282 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:02.282 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:02.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.283 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setdirlayout_latency": { 2024-09-12T01:37:02.283 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:02.283 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:02.283 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:02.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.283 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setfilelock_latency": { 2024-09-12T01:37:02.283 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:02.283 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:02.283 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:02.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.283 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setlayout_latency": { 2024-09-12T01:37:02.283 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:02.283 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:02.284 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:02.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.284 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setxattr_latency": { 2024-09-12T01:37:02.284 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:02.284 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:02.284 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:02.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.284 INFO:teuthology.orchestra.run.smithi053.stdout: "req_symlink_latency": { 2024-09-12T01:37:02.284 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:02.284 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:02.284 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:02.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.285 INFO:teuthology.orchestra.run.smithi053.stdout: "req_unlink_latency": { 2024-09-12T01:37:02.285 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:02.285 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:02.285 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:02.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.285 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_revoke_eviction": 0, 2024-09-12T01:37:02.285 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_acquisition_throttle": 0, 2024-09-12T01:37:02.285 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getvxattr_latency": { 2024-09-12T01:37:02.285 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:02.285 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:02.285 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:02.285 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-12T01:37:02.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.285 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_sessions": { 2024-09-12T01:37:02.286 INFO:teuthology.orchestra.run.smithi053.stdout: "session_count": 1, 2024-09-12T01:37:02.286 INFO:teuthology.orchestra.run.smithi053.stdout: "session_add": 1, 2024-09-12T01:37:02.286 INFO:teuthology.orchestra.run.smithi053.stdout: "session_remove": 0, 2024-09-12T01:37:02.286 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_open": 1, 2024-09-12T01:37:02.286 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_stale": 0, 2024-09-12T01:37:02.286 INFO:teuthology.orchestra.run.smithi053.stdout: "total_load": 0, 2024-09-12T01:37:02.286 INFO:teuthology.orchestra.run.smithi053.stdout: "average_load": 0, 2024-09-12T01:37:02.286 INFO:teuthology.orchestra.run.smithi053.stdout: "avg_session_uptime": 1911, 2024-09-12T01:37:02.286 INFO:teuthology.orchestra.run.smithi053.stdout: "mdthresh_evicted": 0 2024-09-12T01:37:02.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.287 INFO:teuthology.orchestra.run.smithi053.stdout: "mempool": { 2024-09-12T01:37:02.287 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_bytes": 0, 2024-09-12T01:37:02.287 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_items": 0, 2024-09-12T01:37:02.287 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T01:37:02.287 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_items": 0, 2024-09-12T01:37:02.287 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T01:37:02.287 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_items": 0, 2024-09-12T01:37:02.287 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T01:37:02.287 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T01:37:02.287 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T01:37:02.287 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T01:37:02.288 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T01:37:02.288 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_items": 0, 2024-09-12T01:37:02.288 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T01:37:02.288 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T01:37:02.288 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_bytes": 0, 2024-09-12T01:37:02.288 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_items": 0, 2024-09-12T01:37:02.288 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_bytes": 0, 2024-09-12T01:37:02.288 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_items": 0, 2024-09-12T01:37:02.288 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T01:37:02.288 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T01:37:02.288 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T01:37:02.288 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T01:37:02.288 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T01:37:02.289 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_items": 0, 2024-09-12T01:37:02.289 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_bytes": 0, 2024-09-12T01:37:02.289 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_items": 0, 2024-09-12T01:37:02.289 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T01:37:02.289 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T01:37:02.289 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_bytes": 0, 2024-09-12T01:37:02.289 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_items": 0, 2024-09-12T01:37:02.289 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_bytes": 0, 2024-09-12T01:37:02.289 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_items": 0, 2024-09-12T01:37:02.289 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T01:37:02.289 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_items": 0, 2024-09-12T01:37:02.289 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T01:37:02.289 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_items": 0, 2024-09-12T01:37:02.290 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_bytes": 181320, 2024-09-12T01:37:02.290 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_items": 47, 2024-09-12T01:37:02.290 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_bytes": 0, 2024-09-12T01:37:02.290 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_items": 0, 2024-09-12T01:37:02.290 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bytes": 0, 2024-09-12T01:37:02.290 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_items": 0, 2024-09-12T01:37:02.290 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_bytes": 0, 2024-09-12T01:37:02.290 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_items": 0, 2024-09-12T01:37:02.290 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_bytes": 0, 2024-09-12T01:37:02.290 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_items": 0, 2024-09-12T01:37:02.290 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_bytes": 8856, 2024-09-12T01:37:02.290 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_items": 166, 2024-09-12T01:37:02.290 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T01:37:02.291 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_items": 0, 2024-09-12T01:37:02.291 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_bytes": 0, 2024-09-12T01:37:02.291 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_items": 0, 2024-09-12T01:37:02.291 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_bytes": 8512, 2024-09-12T01:37:02.291 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_items": 13, 2024-09-12T01:37:02.291 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_bytes": 0, 2024-09-12T01:37:02.291 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_items": 0, 2024-09-12T01:37:02.291 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_bytes": 0, 2024-09-12T01:37:02.291 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_items": 0 2024-09-12T01:37:02.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.291 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter": { 2024-09-12T01:37:02.291 INFO:teuthology.orchestra.run.smithi053.stdout: "op_active": 0, 2024-09-12T01:37:02.291 INFO:teuthology.orchestra.run.smithi053.stdout: "op_laggy": 0, 2024-09-12T01:37:02.292 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send": 1555, 2024-09-12T01:37:02.292 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send_bytes": 21, 2024-09-12T01:37:02.292 INFO:teuthology.orchestra.run.smithi053.stdout: "op_resend": 3, 2024-09-12T01:37:02.292 INFO:teuthology.orchestra.run.smithi053.stdout: "op_reply": 1555, 2024-09-12T01:37:02.292 INFO:teuthology.orchestra.run.smithi053.stdout: "op_latency": { 2024-09-12T01:37:02.292 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1555, 2024-09-12T01:37:02.292 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.762984479, 2024-09-12T01:37:02.292 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000490665 2024-09-12T01:37:02.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.292 INFO:teuthology.orchestra.run.smithi053.stdout: "op_inflight": 0, 2024-09-12T01:37:02.292 INFO:teuthology.orchestra.run.smithi053.stdout: "oplen_avg": { 2024-09-12T01:37:02.292 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1555, 2024-09-12T01:37:02.292 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 1556 2024-09-12T01:37:02.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.293 INFO:teuthology.orchestra.run.smithi053.stdout: "op": 1555, 2024-09-12T01:37:02.293 INFO:teuthology.orchestra.run.smithi053.stdout: "op_r": 1555, 2024-09-12T01:37:02.293 INFO:teuthology.orchestra.run.smithi053.stdout: "op_w": 0, 2024-09-12T01:37:02.293 INFO:teuthology.orchestra.run.smithi053.stdout: "op_rmw": 0, 2024-09-12T01:37:02.293 INFO:teuthology.orchestra.run.smithi053.stdout: "op_pg": 0, 2024-09-12T01:37:02.293 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_stat": 991, 2024-09-12T01:37:02.293 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_create": 0, 2024-09-12T01:37:02.293 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_read": 562, 2024-09-12T01:37:02.293 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_write": 0, 2024-09-12T01:37:02.293 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writefull": 0, 2024-09-12T01:37:02.293 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writesame": 0, 2024-09-12T01:37:02.293 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_append": 0, 2024-09-12T01:37:02.293 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_zero": 0, 2024-09-12T01:37:02.293 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_truncate": 0, 2024-09-12T01:37:02.294 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_delete": 0, 2024-09-12T01:37:02.294 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_mapext": 0, 2024-09-12T01:37:02.294 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_sparse_read": 0, 2024-09-12T01:37:02.294 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_clonerange": 0, 2024-09-12T01:37:02.294 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_getxattr": 1, 2024-09-12T01:37:02.294 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_setxattr": 0, 2024-09-12T01:37:02.294 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_cmpxattr": 0, 2024-09-12T01:37:02.294 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_rmxattr": 0, 2024-09-12T01:37:02.294 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_resetxattrs": 0, 2024-09-12T01:37:02.294 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_call": 0, 2024-09-12T01:37:02.294 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_watch": 0, 2024-09-12T01:37:02.294 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_notify": 0, 2024-09-12T01:37:02.294 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T01:37:02.294 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls": 0, 2024-09-12T01:37:02.295 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls_filter": 0, 2024-09-12T01:37:02.295 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_other": 0, 2024-09-12T01:37:02.295 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_active": 0, 2024-09-12T01:37:02.295 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_send": 0, 2024-09-12T01:37:02.295 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_resend": 0, 2024-09-12T01:37:02.295 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_ping": 0, 2024-09-12T01:37:02.295 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_active": 0, 2024-09-12T01:37:02.295 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_send": 0, 2024-09-12T01:37:02.295 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_resend": 0, 2024-09-12T01:37:02.295 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_active": 0, 2024-09-12T01:37:02.295 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_send": 0, 2024-09-12T01:37:02.295 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_resend": 0, 2024-09-12T01:37:02.295 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_active": 0, 2024-09-12T01:37:02.295 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_send": 0, 2024-09-12T01:37:02.296 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_resend": 0, 2024-09-12T01:37:02.296 INFO:teuthology.orchestra.run.smithi053.stdout: "command_active": 0, 2024-09-12T01:37:02.296 INFO:teuthology.orchestra.run.smithi053.stdout: "command_send": 0, 2024-09-12T01:37:02.296 INFO:teuthology.orchestra.run.smithi053.stdout: "command_resend": 0, 2024-09-12T01:37:02.296 INFO:teuthology.orchestra.run.smithi053.stdout: "map_epoch": 98, 2024-09-12T01:37:02.296 INFO:teuthology.orchestra.run.smithi053.stdout: "map_full": 0, 2024-09-12T01:37:02.296 INFO:teuthology.orchestra.run.smithi053.stdout: "map_inc": 2, 2024-09-12T01:37:02.296 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_sessions": 4, 2024-09-12T01:37:02.296 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_open": 4, 2024-09-12T01:37:02.296 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_close": 0, 2024-09-12T01:37:02.296 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_laggy": 0, 2024-09-12T01:37:02.296 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_wr": 0, 2024-09-12T01:37:02.296 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_rd": 2, 2024-09-12T01:37:02.297 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_del": 0 2024-09-12T01:37:02.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.297 INFO:teuthology.orchestra.run.smithi053.stdout: "oft": { 2024-09-12T01:37:02.297 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_objs": 0, 2024-09-12T01:37:02.297 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_kv_pairs": 0, 2024-09-12T01:37:02.297 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_updates": 0, 2024-09-12T01:37:02.297 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_removes": 0 2024-09-12T01:37:02.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.297 INFO:teuthology.orchestra.run.smithi053.stdout: "osd-slow-ops": { 2024-09-12T01:37:02.297 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_ops_count": 0 2024-09-12T01:37:02.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.297 INFO:teuthology.orchestra.run.smithi053.stdout: "purge_queue": { 2024-09-12T01:37:02.297 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops": 0, 2024-09-12T01:37:02.297 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T01:37:02.297 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing": 0, 2024-09-12T01:37:02.298 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_high_water": 0, 2024-09-12T01:37:02.298 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed_ops": 0, 2024-09-12T01:37:02.298 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed": 0, 2024-09-12T01:37:02.298 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_item_in_journal": 0 2024-09-12T01:37:02.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.298 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T01:37:02.298 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-09-12T01:37:02.298 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-09-12T01:37:02.298 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-09-12T01:37:02.298 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 1757, 2024-09-12T01:37:02.298 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 1109770, 2024-09-12T01:37:02.298 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-09-12T01:37:02.298 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 1757, 2024-09-12T01:37:02.298 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-09-12T01:37:02.299 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-09-12T01:37:02.299 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 1757, 2024-09-12T01:37:02.299 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 1109770, 2024-09-12T01:37:02.299 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-09-12T01:37:02.299 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:02.299 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:02.299 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:02.299 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-12T01:37:02.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.299 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_bytes": { 2024-09-12T01:37:02.299 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-09-12T01:37:02.299 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-09-12T01:37:02.299 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-09-12T01:37:02.299 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-09-12T01:37:02.300 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-09-12T01:37:02.300 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-09-12T01:37:02.300 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-09-12T01:37:02.300 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 1555, 2024-09-12T01:37:02.300 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 904, 2024-09-12T01:37:02.300 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 1555, 2024-09-12T01:37:02.300 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 904, 2024-09-12T01:37:02.300 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-09-12T01:37:02.300 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:02.300 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:02.300 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:02.300 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-12T01:37:02.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.300 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_ops": { 2024-09-12T01:37:02.301 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-09-12T01:37:02.301 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 1024, 2024-09-12T01:37:02.301 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-09-12T01:37:02.301 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-09-12T01:37:02.301 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-09-12T01:37:02.301 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-09-12T01:37:02.301 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-09-12T01:37:02.301 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 1555, 2024-09-12T01:37:02.301 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 1555, 2024-09-12T01:37:02.301 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 1555, 2024-09-12T01:37:02.301 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 1555, 2024-09-12T01:37:02.301 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-09-12T01:37:02.301 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:02.301 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:02.302 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:02.302 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-12T01:37:02.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.302 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle": { 2024-09-12T01:37:02.302 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-09-12T01:37:02.302 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-09-12T01:37:02.302 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-09-12T01:37:02.302 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-09-12T01:37:02.302 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-09-12T01:37:02.302 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-09-12T01:37:02.302 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-09-12T01:37:02.302 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-09-12T01:37:02.302 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-09-12T01:37:02.302 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 0, 2024-09-12T01:37:02.303 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 0, 2024-09-12T01:37:02.303 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-09-12T01:37:02.303 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:02.303 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:02.303 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:02.303 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-12T01:37:02.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-12T01:37:02.303 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle-0x557bcfae86e0": { 2024-09-12T01:37:02.303 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-09-12T01:37:02.303 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-09-12T01:37:02.303 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-09-12T01:37:02.303 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-09-12T01:37:02.303 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-09-12T01:37:02.303 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-09-12T01:37:02.304 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-09-12T01:37:02.304 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-09-12T01:37:02.304 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-09-12T01:37:02.304 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 0, 2024-09-12T01:37:02.304 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 0, 2024-09-12T01:37:02.304 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-09-12T01:37:02.304 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-12T01:37:02.304 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-12T01:37:02.304 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-12T01:37:02.304 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-12T01:37:02.304 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-12T01:37:02.304 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-12T01:37:02.515 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:02 smithi053 ceph-mon[35158]: pgmap v621: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 25 op/s 2024-09-12T01:37:02.515 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:37:02.515 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:37:02.515 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:02 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:37:02.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:02 smithi104 ceph-mon[35219]: pgmap v621: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 25 op/s 2024-09-12T01:37:02.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:02 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:37:02.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:02 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:37:02.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:02 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:37:02.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:02 smithi001 ceph-mon[30387]: pgmap v621: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 25 op/s 2024-09-12T01:37:02.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:02 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:37:02.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:02 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:37:02.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:02 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' 2024-09-12T01:37:03.179 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-12T01:37:03.179 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-12T01:37:03.179 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=0 2024-09-12T01:37:03.179 DEBUG:tasks.check_counter:Getting stats from c 2024-09-12T01:37:03.179 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-12T01:37:03.436 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:37:03.950 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-12T01:37:03.950 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T01:37:03.950 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 417, 2024-09-12T01:37:03.950 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 459, 2024-09-12T01:37:03.950 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 155456, 2024-09-12T01:37:03.950 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 280781, 2024-09-12T01:37:03.950 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 9, 2024-09-12T01:37:03.950 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 3, 2024-09-12T01:37:03.951 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.045720403, 2024-09-12T01:37:03.951 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.015967387, 2024-09-12T01:37:03.951 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.024122250, 2024-09-12T01:37:03.951 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.004753017, 2024-09-12T01:37:03.951 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:37:03.951 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 459, 2024-09-12T01:37:03.951 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.055270130, 2024-09-12T01:37:03.951 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000120414 2024-09-12T01:37:03.951 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.951 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:37:03.951 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 734, 2024-09-12T01:37:03.951 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000158170, 2024-09-12T01:37:03.952 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000215 2024-09-12T01:37:03.952 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.952 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T01:37:03.952 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T01:37:03.952 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.952 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T01:37:03.952 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 305, 2024-09-12T01:37:03.952 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 312, 2024-09-12T01:37:03.952 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 129459, 2024-09-12T01:37:03.952 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 394805, 2024-09-12T01:37:03.952 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 10, 2024-09-12T01:37:03.952 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 6, 2024-09-12T01:37:03.952 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.028064969, 2024-09-12T01:37:03.953 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.011928904, 2024-09-12T01:37:03.953 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.018481789, 2024-09-12T01:37:03.953 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.000458390, 2024-09-12T01:37:03.953 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:37:03.953 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 310, 2024-09-12T01:37:03.953 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.023917445, 2024-09-12T01:37:03.953 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000077153 2024-09-12T01:37:03.953 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.953 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:37:03.953 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 440, 2024-09-12T01:37:03.953 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000070471, 2024-09-12T01:37:03.953 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000160 2024-09-12T01:37:03.953 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.953 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-12T01:37:03.954 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 292032 2024-09-12T01:37:03.954 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.954 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T01:37:03.954 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 296, 2024-09-12T01:37:03.954 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 290, 2024-09-12T01:37:03.954 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 872496, 2024-09-12T01:37:03.954 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 174116, 2024-09-12T01:37:03.954 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 10, 2024-09-12T01:37:03.954 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 9, 2024-09-12T01:37:03.954 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.040180722, 2024-09-12T01:37:03.954 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.013773004, 2024-09-12T01:37:03.954 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.056458106, 2024-09-12T01:37:03.955 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.000385366, 2024-09-12T01:37:03.955 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:37:03.955 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 288, 2024-09-12T01:37:03.955 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.027308547, 2024-09-12T01:37:03.955 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000094821 2024-09-12T01:37:03.955 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.955 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:37:03.955 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 51, 2024-09-12T01:37:03.955 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000022162, 2024-09-12T01:37:03.956 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000434 2024-09-12T01:37:03.956 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.956 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 857776, 2024-09-12T01:37:03.956 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 115776 2024-09-12T01:37:03.956 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.956 INFO:teuthology.orchestra.run.smithi104.stdout: "cct": { 2024-09-12T01:37:03.956 INFO:teuthology.orchestra.run.smithi104.stdout: "total_workers": 1, 2024-09-12T01:37:03.956 INFO:teuthology.orchestra.run.smithi104.stdout: "unhealthy_workers": 0 2024-09-12T01:37:03.956 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.957 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-MDSRank": { 2024-09-12T01:37:03.957 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-09-12T01:37:03.957 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-09-12T01:37:03.957 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 106, 2024-09-12T01:37:03.957 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.009663611, 2024-09-12T01:37:03.957 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000091166 2024-09-12T01:37:03.957 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-12T01:37:03.957 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.957 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-PurgeQueue": { 2024-09-12T01:37:03.957 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-09-12T01:37:03.957 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-09-12T01:37:03.957 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 32, 2024-09-12T01:37:03.957 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.001056812, 2024-09-12T01:37:03.957 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000033025 2024-09-12T01:37:03.958 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-12T01:37:03.958 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.958 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": { 2024-09-12T01:37:03.958 INFO:teuthology.orchestra.run.smithi104.stdout: "request": 20, 2024-09-12T01:37:03.958 INFO:teuthology.orchestra.run.smithi104.stdout: "reply": 15, 2024-09-12T01:37:03.958 INFO:teuthology.orchestra.run.smithi104.stdout: "reply_latency": { 2024-09-12T01:37:03.958 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 15, 2024-09-12T01:37:03.958 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.016441491, 2024-09-12T01:37:03.958 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.001096099 2024-09-12T01:37:03.958 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.958 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_reply": 0, 2024-09-12T01:37:03.958 INFO:teuthology.orchestra.run.smithi104.stdout: "forward": 5, 2024-09-12T01:37:03.959 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_complete": 0, 2024-09-12T01:37:03.959 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_keys": 0, 2024-09-12T01:37:03.959 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_commit": 11, 2024-09-12T01:37:03.959 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_split": 0, 2024-09-12T01:37:03.959 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_merge": 0, 2024-09-12T01:37:03.959 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes": 18, 2024-09-12T01:37:03.959 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_top": 2, 2024-09-12T01:37:03.959 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_bottom": 16, 2024-09-12T01:37:03.959 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pin_tail": 0, 2024-09-12T01:37:03.959 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pinned": 13, 2024-09-12T01:37:03.959 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_expired": 10, 2024-09-12T01:37:03.959 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_with_caps": 0, 2024-09-12T01:37:03.959 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": 0, 2024-09-12T01:37:03.960 INFO:teuthology.orchestra.run.smithi104.stdout: "subtrees": 4, 2024-09-12T01:37:03.960 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse": 42, 2024-09-12T01:37:03.960 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_hit": 21, 2024-09-12T01:37:03.960 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_forward": 3, 2024-09-12T01:37:03.960 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_discover": 4, 2024-09-12T01:37:03.960 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_dir_fetch": 0, 2024-09-12T01:37:03.960 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_remote_ino": 0, 2024-09-12T01:37:03.960 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_lock": 0, 2024-09-12T01:37:03.960 INFO:teuthology.orchestra.run.smithi104.stdout: "load_cent": 0, 2024-09-12T01:37:03.960 INFO:teuthology.orchestra.run.smithi104.stdout: "q": 0, 2024-09-12T01:37:03.960 INFO:teuthology.orchestra.run.smithi104.stdout: "exported": 1, 2024-09-12T01:37:03.960 INFO:teuthology.orchestra.run.smithi104.stdout: "exported_inodes": 0, 2024-09-12T01:37:03.960 INFO:teuthology.orchestra.run.smithi104.stdout: "imported": 1, 2024-09-12T01:37:03.960 INFO:teuthology.orchestra.run.smithi104.stdout: "imported_inodes": 4, 2024-09-12T01:37:03.961 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_dir_fetch": 0, 2024-09-12T01:37:03.961 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_backtrace_fetch": 0, 2024-09-12T01:37:03.961 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_peer_discover": 1, 2024-09-12T01:37:03.961 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rfiles": 4, 2024-09-12T01:37:03.961 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rbytes": 234, 2024-09-12T01:37:03.961 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rsnaps": 0, 2024-09-12T01:37:03.961 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T01:37:03.961 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_set_tag": 0, 2024-09-12T01:37:03.961 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T01:37:03.961 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_inotable_repaired": 0, 2024-09-12T01:37:03.961 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_inodes": 0, 2024-09-12T01:37:03.962 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T01:37:03.962 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T01:37:03.962 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_file_inodes": 0, 2024-09-12T01:37:03.962 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_inode_file_caps": 0, 2024-09-12T01:37:03.962 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_revoke": 1, 2024-09-12T01:37:03.962 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_grant": 8, 2024-09-12T01:37:03.962 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T01:37:03.962 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T01:37:03.962 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T01:37:03.962 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps": 6, 2024-09-12T01:37:03.962 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps_dirty": 5, 2024-09-12T01:37:03.962 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_cap_release": 4, 2024-09-12T01:37:03.962 INFO:teuthology.orchestra.run.smithi104.stdout: "process_request_cap_release": 14 2024-09-12T01:37:03.962 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.963 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_cache": { 2024-09-12T01:37:03.963 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update": 0, 2024-09-12T01:37:03.963 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update_receipt": 0, 2024-09-12T01:37:03.963 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_try_discover": 0, 2024-09-12T01:37:03.963 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_send_discover": 6, 2024-09-12T01:37:03.963 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_handle_discover": 0, 2024-09-12T01:37:03.963 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays": 0, 2024-09-12T01:37:03.963 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_delayed": 0, 2024-09-12T01:37:03.963 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_enqueuing": 0, 2024-09-12T01:37:03.963 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_created": 5, 2024-09-12T01:37:03.963 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_enqueued": 5, 2024-09-12T01:37:03.963 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_reintegrated": 0, 2024-09-12T01:37:03.963 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_migrated": 0, 2024-09-12T01:37:03.963 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_processing": 0, 2024-09-12T01:37:03.963 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_enqueued": 0, 2024-09-12T01:37:03.963 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_prioritized": 0, 2024-09-12T01:37:03.964 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_started": 0, 2024-09-12T01:37:03.964 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_completed": 0, 2024-09-12T01:37:03.964 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_path": 0, 2024-09-12T01:37:03.964 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_inode": 0, 2024-09-12T01:37:03.964 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T01:37:03.964 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_exportdir": 1, 2024-09-12T01:37:03.964 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_flush": 0, 2024-09-12T01:37:03.964 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragmentdir": 0, 2024-09-12T01:37:03.964 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragstats": 0, 2024-09-12T01:37:03.964 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_inodestats": 0 2024-09-12T01:37:03.964 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.964 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_log": { 2024-09-12T01:37:03.964 INFO:teuthology.orchestra.run.smithi104.stdout: "evlrg": 0, 2024-09-12T01:37:03.964 INFO:teuthology.orchestra.run.smithi104.stdout: "evadd": 37, 2024-09-12T01:37:03.965 INFO:teuthology.orchestra.run.smithi104.stdout: "evex": 0, 2024-09-12T01:37:03.965 INFO:teuthology.orchestra.run.smithi104.stdout: "evtrm": 0, 2024-09-12T01:37:03.965 INFO:teuthology.orchestra.run.smithi104.stdout: "ev": 37, 2024-09-12T01:37:03.965 INFO:teuthology.orchestra.run.smithi104.stdout: "evexg": 0, 2024-09-12T01:37:03.965 INFO:teuthology.orchestra.run.smithi104.stdout: "evexd": 0, 2024-09-12T01:37:03.965 INFO:teuthology.orchestra.run.smithi104.stdout: "segadd": 2, 2024-09-12T01:37:03.965 INFO:teuthology.orchestra.run.smithi104.stdout: "segex": 0, 2024-09-12T01:37:03.965 INFO:teuthology.orchestra.run.smithi104.stdout: "segtrm": 0, 2024-09-12T01:37:03.965 INFO:teuthology.orchestra.run.smithi104.stdout: "seg": 2, 2024-09-12T01:37:03.965 INFO:teuthology.orchestra.run.smithi104.stdout: "segmjr": 2, 2024-09-12T01:37:03.965 INFO:teuthology.orchestra.run.smithi104.stdout: "segexg": 0, 2024-09-12T01:37:03.965 INFO:teuthology.orchestra.run.smithi104.stdout: "segexd": 0, 2024-09-12T01:37:03.966 INFO:teuthology.orchestra.run.smithi104.stdout: "expos": 4194304, 2024-09-12T01:37:03.966 INFO:teuthology.orchestra.run.smithi104.stdout: "wrpos": 4243135, 2024-09-12T01:37:03.966 INFO:teuthology.orchestra.run.smithi104.stdout: "rdpos": 0, 2024-09-12T01:37:03.966 INFO:teuthology.orchestra.run.smithi104.stdout: "jlat": { 2024-09-12T01:37:03.966 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 23, 2024-09-12T01:37:03.966 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.052475419, 2024-09-12T01:37:03.966 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.002281539 2024-09-12T01:37:03.966 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.966 INFO:teuthology.orchestra.run.smithi104.stdout: "replayed": 0 2024-09-12T01:37:03.967 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.967 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_mem": { 2024-09-12T01:37:03.967 INFO:teuthology.orchestra.run.smithi104.stdout: "ino": 23, 2024-09-12T01:37:03.967 INFO:teuthology.orchestra.run.smithi104.stdout: "ino+": 33, 2024-09-12T01:37:03.967 INFO:teuthology.orchestra.run.smithi104.stdout: "ino-": 10, 2024-09-12T01:37:03.967 INFO:teuthology.orchestra.run.smithi104.stdout: "dir": 22, 2024-09-12T01:37:03.967 INFO:teuthology.orchestra.run.smithi104.stdout: "dir+": 28, 2024-09-12T01:37:03.967 INFO:teuthology.orchestra.run.smithi104.stdout: "dir-": 6, 2024-09-12T01:37:03.967 INFO:teuthology.orchestra.run.smithi104.stdout: "dn": 18, 2024-09-12T01:37:03.967 INFO:teuthology.orchestra.run.smithi104.stdout: "dn+": 38, 2024-09-12T01:37:03.967 INFO:teuthology.orchestra.run.smithi104.stdout: "dn-": 20, 2024-09-12T01:37:03.968 INFO:teuthology.orchestra.run.smithi104.stdout: "cap": 0, 2024-09-12T01:37:03.968 INFO:teuthology.orchestra.run.smithi104.stdout: "cap+": 11, 2024-09-12T01:37:03.968 INFO:teuthology.orchestra.run.smithi104.stdout: "cap-": 11, 2024-09-12T01:37:03.968 INFO:teuthology.orchestra.run.smithi104.stdout: "rss": 51384, 2024-09-12T01:37:03.968 INFO:teuthology.orchestra.run.smithi104.stdout: "heap": 223520 2024-09-12T01:37:03.968 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.968 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_server": { 2024-09-12T01:37:03.968 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_client_request": 32, 2024-09-12T01:37:03.968 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_server_request": 2, 2024-09-12T01:37:03.968 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_request": 20, 2024-09-12T01:37:03.968 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_session": 10, 2024-09-12T01:37:03.968 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_peer_request": 12, 2024-09-12T01:37:03.968 INFO:teuthology.orchestra.run.smithi104.stdout: "req_create_latency": { 2024-09-12T01:37:03.968 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 2, 2024-09-12T01:37:03.969 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.006248720, 2024-09-12T01:37:03.969 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.003124360 2024-09-12T01:37:03.969 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.969 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getattr_latency": { 2024-09-12T01:37:03.969 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1, 2024-09-12T01:37:03.969 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000179776, 2024-09-12T01:37:03.969 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000179776 2024-09-12T01:37:03.969 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.969 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getfilelock_latency": { 2024-09-12T01:37:03.969 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:03.969 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:03.969 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:03.969 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.970 INFO:teuthology.orchestra.run.smithi104.stdout: "req_link_latency": { 2024-09-12T01:37:03.970 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:03.970 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:03.970 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:03.970 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.970 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookup_latency": { 2024-09-12T01:37:03.970 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 5, 2024-09-12T01:37:03.970 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.001249744, 2024-09-12T01:37:03.970 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000249948 2024-09-12T01:37:03.970 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.970 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookuphash_latency": { 2024-09-12T01:37:03.970 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:03.970 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:03.970 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:03.970 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.971 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupino_latency": { 2024-09-12T01:37:03.971 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:03.971 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:03.971 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:03.971 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.971 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupname_latency": { 2024-09-12T01:37:03.971 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:03.971 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:03.971 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:03.971 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.971 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupparent_latency": { 2024-09-12T01:37:03.971 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:03.972 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:03.972 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:03.972 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.972 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupsnap_latency": { 2024-09-12T01:37:03.972 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:03.972 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:03.972 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:03.972 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.972 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lssnap_latency": { 2024-09-12T01:37:03.972 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:03.972 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:03.973 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:03.973 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.973 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mkdir_latency": { 2024-09-12T01:37:03.974 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:03.974 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:03.974 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:03.974 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.975 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mknod_latency": { 2024-09-12T01:37:03.975 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:03.975 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:03.975 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:03.975 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.975 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mksnap_latency": { 2024-09-12T01:37:03.976 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:03.976 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:03.976 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:03.976 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.976 INFO:teuthology.orchestra.run.smithi104.stdout: "req_open_latency": { 2024-09-12T01:37:03.976 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:03.977 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:03.977 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:03.977 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.977 INFO:teuthology.orchestra.run.smithi104.stdout: "req_readdir_latency": { 2024-09-12T01:37:03.977 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 2, 2024-09-12T01:37:03.977 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000960966, 2024-09-12T01:37:03.978 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000480483 2024-09-12T01:37:03.978 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.978 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rename_latency": { 2024-09-12T01:37:03.978 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:03.978 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:03.978 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:03.978 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.979 INFO:teuthology.orchestra.run.smithi104.stdout: "req_renamesnap_latency": { 2024-09-12T01:37:03.979 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:03.979 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:03.979 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:03.979 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.979 INFO:teuthology.orchestra.run.smithi104.stdout: "req_snapdiff_latency": { 2024-09-12T01:37:03.980 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:03.980 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:03.980 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:03.980 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.980 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmdir_latency": { 2024-09-12T01:37:03.980 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:03.981 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:03.981 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:03.981 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.981 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmsnap_latency": { 2024-09-12T01:37:03.981 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:03.982 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:03.982 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:03.982 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.982 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmxattr_latency": { 2024-09-12T01:37:03.982 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:03.982 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:03.983 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:03.983 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.983 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setattr_latency": { 2024-09-12T01:37:03.983 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:03.983 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:03.983 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:03.984 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.984 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setdirlayout_latency": { 2024-09-12T01:37:03.984 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:03.984 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:03.984 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:03.984 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.985 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setfilelock_latency": { 2024-09-12T01:37:03.985 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:03.985 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:03.985 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:03.985 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.986 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setlayout_latency": { 2024-09-12T01:37:03.986 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:03.986 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:03.986 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:03.986 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.986 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setxattr_latency": { 2024-09-12T01:37:03.987 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:03.987 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:03.987 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:03.987 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.987 INFO:teuthology.orchestra.run.smithi104.stdout: "req_symlink_latency": { 2024-09-12T01:37:03.988 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:03.988 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:03.988 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:03.988 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.988 INFO:teuthology.orchestra.run.smithi104.stdout: "req_unlink_latency": { 2024-09-12T01:37:03.988 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 5, 2024-09-12T01:37:03.989 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.007802285, 2024-09-12T01:37:03.989 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.001560457 2024-09-12T01:37:03.989 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.989 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_revoke_eviction": 0, 2024-09-12T01:37:03.989 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_acquisition_throttle": 0, 2024-09-12T01:37:03.989 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getvxattr_latency": { 2024-09-12T01:37:03.990 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:03.990 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:03.990 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:03.990 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-12T01:37:03.990 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.990 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_sessions": { 2024-09-12T01:37:03.991 INFO:teuthology.orchestra.run.smithi104.stdout: "session_count": 1, 2024-09-12T01:37:03.991 INFO:teuthology.orchestra.run.smithi104.stdout: "session_add": 2, 2024-09-12T01:37:03.991 INFO:teuthology.orchestra.run.smithi104.stdout: "session_remove": 1, 2024-09-12T01:37:03.991 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_open": 1, 2024-09-12T01:37:03.991 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_stale": 0, 2024-09-12T01:37:03.991 INFO:teuthology.orchestra.run.smithi104.stdout: "total_load": 8, 2024-09-12T01:37:03.992 INFO:teuthology.orchestra.run.smithi104.stdout: "average_load": 8, 2024-09-12T01:37:03.992 INFO:teuthology.orchestra.run.smithi104.stdout: "avg_session_uptime": 433, 2024-09-12T01:37:03.992 INFO:teuthology.orchestra.run.smithi104.stdout: "mdthresh_evicted": 0 2024-09-12T01:37:03.992 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:03.992 INFO:teuthology.orchestra.run.smithi104.stdout: "mempool": { 2024-09-12T01:37:03.992 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_bytes": 0, 2024-09-12T01:37:03.993 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_items": 0, 2024-09-12T01:37:03.993 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T01:37:03.993 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_items": 0, 2024-09-12T01:37:03.993 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T01:37:03.993 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_items": 0, 2024-09-12T01:37:03.994 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T01:37:03.994 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T01:37:03.994 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T01:37:03.994 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T01:37:03.994 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T01:37:03.994 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_items": 0, 2024-09-12T01:37:03.995 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T01:37:03.995 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T01:37:03.995 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_bytes": 0, 2024-09-12T01:37:03.995 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_items": 0, 2024-09-12T01:37:03.995 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_bytes": 0, 2024-09-12T01:37:03.995 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_items": 0, 2024-09-12T01:37:03.996 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T01:37:03.996 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T01:37:03.996 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T01:37:03.996 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T01:37:03.996 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T01:37:03.996 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_items": 0, 2024-09-12T01:37:03.997 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_bytes": 0, 2024-09-12T01:37:03.997 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_items": 0, 2024-09-12T01:37:03.997 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T01:37:03.997 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T01:37:03.997 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_bytes": 0, 2024-09-12T01:37:03.998 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_items": 0, 2024-09-12T01:37:03.998 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_bytes": 0, 2024-09-12T01:37:03.998 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_items": 0, 2024-09-12T01:37:03.998 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T01:37:03.998 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_items": 0, 2024-09-12T01:37:03.998 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T01:37:03.999 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_items": 0, 2024-09-12T01:37:03.999 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_bytes": 283638, 2024-09-12T01:37:03.999 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_items": 116, 2024-09-12T01:37:03.999 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_bytes": 176, 2024-09-12T01:37:03.999 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_items": 2, 2024-09-12T01:37:03.999 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_bytes": 0, 2024-09-12T01:37:04.000 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_items": 0, 2024-09-12T01:37:04.000 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_bytes": 0, 2024-09-12T01:37:04.000 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_items": 0, 2024-09-12T01:37:04.000 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_bytes": 0, 2024-09-12T01:37:04.000 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_items": 0, 2024-09-12T01:37:04.000 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_bytes": 8856, 2024-09-12T01:37:04.001 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_items": 166, 2024-09-12T01:37:04.001 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T01:37:04.001 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_items": 0, 2024-09-12T01:37:04.001 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_bytes": 0, 2024-09-12T01:37:04.001 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_items": 0, 2024-09-12T01:37:04.001 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_bytes": 92821, 2024-09-12T01:37:04.002 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_items": 360, 2024-09-12T01:37:04.002 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_bytes": 0, 2024-09-12T01:37:04.002 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_items": 0, 2024-09-12T01:37:04.002 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_bytes": 0, 2024-09-12T01:37:04.002 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_items": 0 2024-09-12T01:37:04.002 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:04.002 INFO:teuthology.orchestra.run.smithi104.stdout: "objecter": { 2024-09-12T01:37:04.002 INFO:teuthology.orchestra.run.smithi104.stdout: "op_active": 0, 2024-09-12T01:37:04.003 INFO:teuthology.orchestra.run.smithi104.stdout: "op_laggy": 0, 2024-09-12T01:37:04.003 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send": 99, 2024-09-12T01:37:04.003 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send_bytes": 62453, 2024-09-12T01:37:04.003 INFO:teuthology.orchestra.run.smithi104.stdout: "op_resend": 33, 2024-09-12T01:37:04.003 INFO:teuthology.orchestra.run.smithi104.stdout: "op_reply": 99, 2024-09-12T01:37:04.003 INFO:teuthology.orchestra.run.smithi104.stdout: "op_latency": { 2024-09-12T01:37:04.003 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 99, 2024-09-12T01:37:04.003 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.493002994, 2024-09-12T01:37:04.003 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.004979828 2024-09-12T01:37:04.003 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:04.003 INFO:teuthology.orchestra.run.smithi104.stdout: "op_inflight": 0, 2024-09-12T01:37:04.003 INFO:teuthology.orchestra.run.smithi104.stdout: "oplen_avg": { 2024-09-12T01:37:04.003 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 99, 2024-09-12T01:37:04.004 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 131 2024-09-12T01:37:04.004 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:04.004 INFO:teuthology.orchestra.run.smithi104.stdout: "op": 99, 2024-09-12T01:37:04.004 INFO:teuthology.orchestra.run.smithi104.stdout: "op_r": 3, 2024-09-12T01:37:04.004 INFO:teuthology.orchestra.run.smithi104.stdout: "op_w": 96, 2024-09-12T01:37:04.004 INFO:teuthology.orchestra.run.smithi104.stdout: "op_rmw": 0, 2024-09-12T01:37:04.004 INFO:teuthology.orchestra.run.smithi104.stdout: "op_pg": 0, 2024-09-12T01:37:04.004 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_stat": 0, 2024-09-12T01:37:04.004 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_create": 10, 2024-09-12T01:37:04.004 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_read": 3, 2024-09-12T01:37:04.004 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_write": 26, 2024-09-12T01:37:04.004 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writefull": 16, 2024-09-12T01:37:04.004 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writesame": 0, 2024-09-12T01:37:04.004 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_append": 0, 2024-09-12T01:37:04.005 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_zero": 0, 2024-09-12T01:37:04.005 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_truncate": 0, 2024-09-12T01:37:04.005 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_delete": 22, 2024-09-12T01:37:04.005 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_mapext": 0, 2024-09-12T01:37:04.005 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_sparse_read": 0, 2024-09-12T01:37:04.005 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_clonerange": 0, 2024-09-12T01:37:04.005 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_getxattr": 0, 2024-09-12T01:37:04.005 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_setxattr": 20, 2024-09-12T01:37:04.005 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_cmpxattr": 0, 2024-09-12T01:37:04.005 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_rmxattr": 0, 2024-09-12T01:37:04.005 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_resetxattrs": 0, 2024-09-12T01:37:04.005 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_call": 0, 2024-09-12T01:37:04.005 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_watch": 0, 2024-09-12T01:37:04.005 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_notify": 0, 2024-09-12T01:37:04.006 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T01:37:04.006 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls": 0, 2024-09-12T01:37:04.006 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls_filter": 0, 2024-09-12T01:37:04.006 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_other": 0, 2024-09-12T01:37:04.006 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_active": 0, 2024-09-12T01:37:04.006 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_send": 0, 2024-09-12T01:37:04.006 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_resend": 0, 2024-09-12T01:37:04.006 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_ping": 0, 2024-09-12T01:37:04.006 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_active": 0, 2024-09-12T01:37:04.006 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_send": 0, 2024-09-12T01:37:04.006 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_resend": 0, 2024-09-12T01:37:04.006 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_active": 0, 2024-09-12T01:37:04.006 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_send": 0, 2024-09-12T01:37:04.006 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_resend": 0, 2024-09-12T01:37:04.007 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_active": 0, 2024-09-12T01:37:04.007 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_send": 0, 2024-09-12T01:37:04.007 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_resend": 0, 2024-09-12T01:37:04.007 INFO:teuthology.orchestra.run.smithi104.stdout: "command_active": 0, 2024-09-12T01:37:04.007 INFO:teuthology.orchestra.run.smithi104.stdout: "command_send": 0, 2024-09-12T01:37:04.007 INFO:teuthology.orchestra.run.smithi104.stdout: "command_resend": 0, 2024-09-12T01:37:04.007 INFO:teuthology.orchestra.run.smithi104.stdout: "map_epoch": 98, 2024-09-12T01:37:04.007 INFO:teuthology.orchestra.run.smithi104.stdout: "map_full": 0, 2024-09-12T01:37:04.007 INFO:teuthology.orchestra.run.smithi104.stdout: "map_inc": 2, 2024-09-12T01:37:04.007 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_sessions": 11, 2024-09-12T01:37:04.007 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_open": 11, 2024-09-12T01:37:04.007 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_close": 0, 2024-09-12T01:37:04.007 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_laggy": 0, 2024-09-12T01:37:04.007 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_wr": 28, 2024-09-12T01:37:04.007 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_rd": 0, 2024-09-12T01:37:04.008 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_del": 6 2024-09-12T01:37:04.008 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:04.008 INFO:teuthology.orchestra.run.smithi104.stdout: "oft": { 2024-09-12T01:37:04.008 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_objs": 1, 2024-09-12T01:37:04.008 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_kv_pairs": 0, 2024-09-12T01:37:04.008 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_updates": 0, 2024-09-12T01:37:04.008 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_removes": 0 2024-09-12T01:37:04.008 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:04.008 INFO:teuthology.orchestra.run.smithi104.stdout: "osd-slow-ops": { 2024-09-12T01:37:04.008 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_ops_count": 0 2024-09-12T01:37:04.008 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:04.008 INFO:teuthology.orchestra.run.smithi104.stdout: "purge_queue": { 2024-09-12T01:37:04.008 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops": 0, 2024-09-12T01:37:04.009 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops_high_water": 8, 2024-09-12T01:37:04.009 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing": 0, 2024-09-12T01:37:04.009 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_high_water": 4, 2024-09-12T01:37:04.009 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed_ops": 10, 2024-09-12T01:37:04.009 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed": 5, 2024-09-12T01:37:04.009 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_item_in_journal": 0 2024-09-12T01:37:04.009 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:04.009 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T01:37:04.009 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-12T01:37:04.009 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-09-12T01:37:04.010 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-12T01:37:04.010 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 1014, 2024-09-12T01:37:04.010 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 1051718, 2024-09-12T01:37:04.010 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-12T01:37:04.010 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 1014, 2024-09-12T01:37:04.010 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-09-12T01:37:04.010 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-09-12T01:37:04.010 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 1014, 2024-09-12T01:37:04.010 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 1051718, 2024-09-12T01:37:04.010 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-12T01:37:04.010 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:04.010 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:04.010 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:04.010 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-12T01:37:04.010 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:04.011 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_bytes": { 2024-09-12T01:37:04.011 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-12T01:37:04.011 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-09-12T01:37:04.011 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-12T01:37:04.011 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-09-12T01:37:04.011 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-09-12T01:37:04.011 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-12T01:37:04.011 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-09-12T01:37:04.011 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 99, 2024-09-12T01:37:04.011 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 62998, 2024-09-12T01:37:04.011 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 99, 2024-09-12T01:37:04.011 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 62998, 2024-09-12T01:37:04.011 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-12T01:37:04.011 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:04.012 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:04.012 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:04.012 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-12T01:37:04.012 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:04.012 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_ops": { 2024-09-12T01:37:04.012 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-12T01:37:04.012 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 1024, 2024-09-12T01:37:04.012 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-12T01:37:04.012 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-09-12T01:37:04.012 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-09-12T01:37:04.012 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-12T01:37:04.012 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-09-12T01:37:04.013 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 99, 2024-09-12T01:37:04.013 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 99, 2024-09-12T01:37:04.013 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 99, 2024-09-12T01:37:04.013 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 99, 2024-09-12T01:37:04.013 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-12T01:37:04.013 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:04.013 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:04.013 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:04.013 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-12T01:37:04.013 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:04.013 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle": { 2024-09-12T01:37:04.013 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-12T01:37:04.013 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-09-12T01:37:04.014 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-12T01:37:04.014 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 5, 2024-09-12T01:37:04.014 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 545, 2024-09-12T01:37:04.014 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-12T01:37:04.014 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 5, 2024-09-12T01:37:04.014 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-09-12T01:37:04.014 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-09-12T01:37:04.014 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 3, 2024-09-12T01:37:04.014 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 545, 2024-09-12T01:37:04.014 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-12T01:37:04.014 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:04.014 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:04.014 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:04.014 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-12T01:37:04.015 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:04.015 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle-0x55ce753bc500": { 2024-09-12T01:37:04.015 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-12T01:37:04.015 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-09-12T01:37:04.015 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-12T01:37:04.015 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 37, 2024-09-12T01:37:04.015 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 48831, 2024-09-12T01:37:04.015 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-12T01:37:04.015 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 37, 2024-09-12T01:37:04.015 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-09-12T01:37:04.015 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-09-12T01:37:04.015 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 23, 2024-09-12T01:37:04.015 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 48831, 2024-09-12T01:37:04.015 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-12T01:37:04.016 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:04.016 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:04.016 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:04.016 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-12T01:37:04.016 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-12T01:37:04.016 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-12T01:37:04.574 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:04 smithi104 ceph-mon[35219]: pgmap v622: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 682 B/s wr, 13 op/s 2024-09-12T01:37:04.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:04 smithi053 ceph-mon[35158]: pgmap v622: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 682 B/s wr, 13 op/s 2024-09-12T01:37:04.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:04 smithi001 ceph-mon[30387]: pgmap v622: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 682 B/s wr, 13 op/s 2024-09-12T01:37:04.964 INFO:tasks.check_counter:Daemon mds.c mds.exported=1 2024-09-12T01:37:04.964 INFO:tasks.check_counter:Daemon mds.c mds.imported=1 2024-09-12T01:37:04.964 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-09-12T01:37:04.964 DEBUG:tasks.check_counter:Getting stats from f 2024-09-12T01:37:04.965 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-12T01:37:05.207 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:37:05.721 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-12T01:37:05.722 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T01:37:05.722 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 20, 2024-09-12T01:37:05.722 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 20, 2024-09-12T01:37:05.722 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 7006, 2024-09-12T01:37:05.722 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 32258, 2024-09-12T01:37:05.722 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 10, 2024-09-12T01:37:05.722 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 3, 2024-09-12T01:37:05.722 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.006436779, 2024-09-12T01:37:05.722 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.000286868, 2024-09-12T01:37:05.722 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.004384792, 2024-09-12T01:37:05.722 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.000294113, 2024-09-12T01:37:05.722 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:37:05.722 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 19, 2024-09-12T01:37:05.723 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.018562965, 2024-09-12T01:37:05.723 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000976998 2024-09-12T01:37:05.723 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.723 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:37:05.723 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 2, 2024-09-12T01:37:05.723 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000128, 2024-09-12T01:37:05.723 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000064 2024-09-12T01:37:05.723 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.723 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-12T01:37:05.723 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T01:37:05.723 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.723 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T01:37:05.723 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 564, 2024-09-12T01:37:05.723 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 606, 2024-09-12T01:37:05.723 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 981312, 2024-09-12T01:37:05.724 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 254996, 2024-09-12T01:37:05.724 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 6, 2024-09-12T01:37:05.724 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 5, 2024-09-12T01:37:05.724 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.062458832, 2024-09-12T01:37:05.724 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.024805949, 2024-09-12T01:37:05.724 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.042422621, 2024-09-12T01:37:05.724 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.002913224, 2024-09-12T01:37:05.724 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:37:05.724 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 605, 2024-09-12T01:37:05.724 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.033675820, 2024-09-12T01:37:05.724 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000055662 2024-09-12T01:37:05.724 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.724 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:37:05.724 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 696, 2024-09-12T01:37:05.725 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000169678, 2024-09-12T01:37:05.725 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000243 2024-09-12T01:37:05.725 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.725 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 846736, 2024-09-12T01:37:05.725 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 113712 2024-09-12T01:37:05.725 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.725 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T01:37:05.725 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 136, 2024-09-12T01:37:05.725 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 209, 2024-09-12T01:37:05.725 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 70283, 2024-09-12T01:37:05.725 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 367450, 2024-09-12T01:37:05.725 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 10, 2024-09-12T01:37:05.725 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 7, 2024-09-12T01:37:05.725 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.018306917, 2024-09-12T01:37:05.725 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.008518698, 2024-09-12T01:37:05.726 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.010100667, 2024-09-12T01:37:05.726 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.000320061, 2024-09-12T01:37:05.726 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:37:05.726 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 207, 2024-09-12T01:37:05.726 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.020083263, 2024-09-12T01:37:05.726 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000097020 2024-09-12T01:37:05.726 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.726 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:37:05.726 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 173, 2024-09-12T01:37:05.726 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000029376, 2024-09-12T01:37:05.726 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000169 2024-09-12T01:37:05.726 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.726 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-09-12T01:37:05.726 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 292944 2024-09-12T01:37:05.726 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.727 INFO:teuthology.orchestra.run.smithi104.stdout: "cct": { 2024-09-12T01:37:05.727 INFO:teuthology.orchestra.run.smithi104.stdout: "total_workers": 1, 2024-09-12T01:37:05.727 INFO:teuthology.orchestra.run.smithi104.stdout: "unhealthy_workers": 0 2024-09-12T01:37:05.727 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.727 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-MDSRank": { 2024-09-12T01:37:05.727 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-09-12T01:37:05.727 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-09-12T01:37:05.727 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 46, 2024-09-12T01:37:05.727 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.001264075, 2024-09-12T01:37:05.727 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000027479 2024-09-12T01:37:05.728 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-12T01:37:05.728 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.728 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-PurgeQueue": { 2024-09-12T01:37:05.728 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-09-12T01:37:05.728 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-09-12T01:37:05.728 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 2, 2024-09-12T01:37:05.728 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000010284, 2024-09-12T01:37:05.729 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000005142 2024-09-12T01:37:05.729 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-12T01:37:05.729 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.729 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": { 2024-09-12T01:37:05.729 INFO:teuthology.orchestra.run.smithi104.stdout: "request": 0, 2024-09-12T01:37:05.729 INFO:teuthology.orchestra.run.smithi104.stdout: "reply": 0, 2024-09-12T01:37:05.729 INFO:teuthology.orchestra.run.smithi104.stdout: "reply_latency": { 2024-09-12T01:37:05.729 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:05.729 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:05.730 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:05.730 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.730 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_reply": 0, 2024-09-12T01:37:05.730 INFO:teuthology.orchestra.run.smithi104.stdout: "forward": 0, 2024-09-12T01:37:05.730 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_complete": 0, 2024-09-12T01:37:05.730 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_keys": 0, 2024-09-12T01:37:05.731 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_commit": 11, 2024-09-12T01:37:05.731 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_split": 0, 2024-09-12T01:37:05.731 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_merge": 0, 2024-09-12T01:37:05.731 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes": 16, 2024-09-12T01:37:05.731 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_top": 0, 2024-09-12T01:37:05.731 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_bottom": 16, 2024-09-12T01:37:05.731 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pin_tail": 0, 2024-09-12T01:37:05.731 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pinned": 13, 2024-09-12T01:37:05.732 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_expired": 6, 2024-09-12T01:37:05.732 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_with_caps": 0, 2024-09-12T01:37:05.732 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": 0, 2024-09-12T01:37:05.732 INFO:teuthology.orchestra.run.smithi104.stdout: "subtrees": 4, 2024-09-12T01:37:05.732 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse": 5, 2024-09-12T01:37:05.732 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_hit": 2, 2024-09-12T01:37:05.732 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_forward": 0, 2024-09-12T01:37:05.732 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_discover": 3, 2024-09-12T01:37:05.732 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_dir_fetch": 0, 2024-09-12T01:37:05.732 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_remote_ino": 0, 2024-09-12T01:37:05.733 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_lock": 0, 2024-09-12T01:37:05.733 INFO:teuthology.orchestra.run.smithi104.stdout: "load_cent": 0, 2024-09-12T01:37:05.733 INFO:teuthology.orchestra.run.smithi104.stdout: "q": 0, 2024-09-12T01:37:05.733 INFO:teuthology.orchestra.run.smithi104.stdout: "exported": 0, 2024-09-12T01:37:05.733 INFO:teuthology.orchestra.run.smithi104.stdout: "exported_inodes": 0, 2024-09-12T01:37:05.733 INFO:teuthology.orchestra.run.smithi104.stdout: "imported": 0, 2024-09-12T01:37:05.733 INFO:teuthology.orchestra.run.smithi104.stdout: "imported_inodes": 0, 2024-09-12T01:37:05.733 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_dir_fetch": 0, 2024-09-12T01:37:05.733 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_backtrace_fetch": 0, 2024-09-12T01:37:05.733 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_peer_discover": 0, 2024-09-12T01:37:05.733 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rfiles": 4, 2024-09-12T01:37:05.733 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rbytes": 234, 2024-09-12T01:37:05.733 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rsnaps": 0, 2024-09-12T01:37:05.733 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T01:37:05.733 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_set_tag": 0, 2024-09-12T01:37:05.734 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T01:37:05.734 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_inotable_repaired": 0, 2024-09-12T01:37:05.734 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_inodes": 0, 2024-09-12T01:37:05.734 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T01:37:05.734 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T01:37:05.734 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_file_inodes": 0, 2024-09-12T01:37:05.734 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_inode_file_caps": 0, 2024-09-12T01:37:05.734 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T01:37:05.734 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_grant": 0, 2024-09-12T01:37:05.734 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T01:37:05.734 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T01:37:05.734 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T01:37:05.734 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps": 0, 2024-09-12T01:37:05.734 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps_dirty": 0, 2024-09-12T01:37:05.735 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_cap_release": 0, 2024-09-12T01:37:05.735 INFO:teuthology.orchestra.run.smithi104.stdout: "process_request_cap_release": 0 2024-09-12T01:37:05.735 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.735 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_cache": { 2024-09-12T01:37:05.735 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update": 0, 2024-09-12T01:37:05.735 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update_receipt": 5, 2024-09-12T01:37:05.735 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_try_discover": 4, 2024-09-12T01:37:05.735 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_send_discover": 6, 2024-09-12T01:37:05.735 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_handle_discover": 0, 2024-09-12T01:37:05.735 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays": 0, 2024-09-12T01:37:05.735 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_delayed": 0, 2024-09-12T01:37:05.735 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_enqueuing": 0, 2024-09-12T01:37:05.736 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_created": 0, 2024-09-12T01:37:05.736 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_enqueued": 0, 2024-09-12T01:37:05.736 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_reintegrated": 0, 2024-09-12T01:37:05.736 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_migrated": 0, 2024-09-12T01:37:05.736 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_processing": 0, 2024-09-12T01:37:05.736 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_enqueued": 0, 2024-09-12T01:37:05.736 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_prioritized": 0, 2024-09-12T01:37:05.736 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_started": 0, 2024-09-12T01:37:05.736 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_completed": 0, 2024-09-12T01:37:05.736 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_path": 0, 2024-09-12T01:37:05.736 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_inode": 0, 2024-09-12T01:37:05.736 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T01:37:05.736 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_exportdir": 0, 2024-09-12T01:37:05.737 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_flush": 0, 2024-09-12T01:37:05.737 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragmentdir": 0, 2024-09-12T01:37:05.737 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragstats": 0, 2024-09-12T01:37:05.737 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_inodestats": 0 2024-09-12T01:37:05.737 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.737 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_log": { 2024-09-12T01:37:05.737 INFO:teuthology.orchestra.run.smithi104.stdout: "evlrg": 0, 2024-09-12T01:37:05.737 INFO:teuthology.orchestra.run.smithi104.stdout: "evadd": 4, 2024-09-12T01:37:05.737 INFO:teuthology.orchestra.run.smithi104.stdout: "evex": 0, 2024-09-12T01:37:05.737 INFO:teuthology.orchestra.run.smithi104.stdout: "evtrm": 0, 2024-09-12T01:37:05.737 INFO:teuthology.orchestra.run.smithi104.stdout: "ev": 4, 2024-09-12T01:37:05.737 INFO:teuthology.orchestra.run.smithi104.stdout: "evexg": 0, 2024-09-12T01:37:05.737 INFO:teuthology.orchestra.run.smithi104.stdout: "evexd": 0, 2024-09-12T01:37:05.737 INFO:teuthology.orchestra.run.smithi104.stdout: "segadd": 2, 2024-09-12T01:37:05.737 INFO:teuthology.orchestra.run.smithi104.stdout: "segex": 0, 2024-09-12T01:37:05.738 INFO:teuthology.orchestra.run.smithi104.stdout: "segtrm": 0, 2024-09-12T01:37:05.738 INFO:teuthology.orchestra.run.smithi104.stdout: "seg": 2, 2024-09-12T01:37:05.738 INFO:teuthology.orchestra.run.smithi104.stdout: "segmjr": 2, 2024-09-12T01:37:05.738 INFO:teuthology.orchestra.run.smithi104.stdout: "segexg": 0, 2024-09-12T01:37:05.738 INFO:teuthology.orchestra.run.smithi104.stdout: "segexd": 0, 2024-09-12T01:37:05.738 INFO:teuthology.orchestra.run.smithi104.stdout: "expos": 4194304, 2024-09-12T01:37:05.738 INFO:teuthology.orchestra.run.smithi104.stdout: "wrpos": 4195426, 2024-09-12T01:37:05.738 INFO:teuthology.orchestra.run.smithi104.stdout: "rdpos": 0, 2024-09-12T01:37:05.738 INFO:teuthology.orchestra.run.smithi104.stdout: "jlat": { 2024-09-12T01:37:05.738 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 3, 2024-09-12T01:37:05.738 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.006482188, 2024-09-12T01:37:05.738 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.002160729 2024-09-12T01:37:05.738 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.738 INFO:teuthology.orchestra.run.smithi104.stdout: "replayed": 0 2024-09-12T01:37:05.738 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.739 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_mem": { 2024-09-12T01:37:05.739 INFO:teuthology.orchestra.run.smithi104.stdout: "ino": 21, 2024-09-12T01:37:05.739 INFO:teuthology.orchestra.run.smithi104.stdout: "ino+": 24, 2024-09-12T01:37:05.739 INFO:teuthology.orchestra.run.smithi104.stdout: "ino-": 3, 2024-09-12T01:37:05.739 INFO:teuthology.orchestra.run.smithi104.stdout: "dir": 20, 2024-09-12T01:37:05.739 INFO:teuthology.orchestra.run.smithi104.stdout: "dir+": 24, 2024-09-12T01:37:05.739 INFO:teuthology.orchestra.run.smithi104.stdout: "dir-": 4, 2024-09-12T01:37:05.739 INFO:teuthology.orchestra.run.smithi104.stdout: "dn": 16, 2024-09-12T01:37:05.739 INFO:teuthology.orchestra.run.smithi104.stdout: "dn+": 22, 2024-09-12T01:37:05.739 INFO:teuthology.orchestra.run.smithi104.stdout: "dn-": 6, 2024-09-12T01:37:05.739 INFO:teuthology.orchestra.run.smithi104.stdout: "cap": 0, 2024-09-12T01:37:05.739 INFO:teuthology.orchestra.run.smithi104.stdout: "cap+": 0, 2024-09-12T01:37:05.739 INFO:teuthology.orchestra.run.smithi104.stdout: "cap-": 0, 2024-09-12T01:37:05.739 INFO:teuthology.orchestra.run.smithi104.stdout: "rss": 49244, 2024-09-12T01:37:05.739 INFO:teuthology.orchestra.run.smithi104.stdout: "heap": 223520 2024-09-12T01:37:05.740 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.740 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_server": { 2024-09-12T01:37:05.740 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_client_request": 0, 2024-09-12T01:37:05.740 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_server_request": 1, 2024-09-12T01:37:05.740 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_request": 0, 2024-09-12T01:37:05.740 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_session": 4, 2024-09-12T01:37:05.740 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_peer_request": 2, 2024-09-12T01:37:05.740 INFO:teuthology.orchestra.run.smithi104.stdout: "req_create_latency": { 2024-09-12T01:37:05.740 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:05.740 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:05.740 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:05.740 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.740 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getattr_latency": { 2024-09-12T01:37:05.740 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:05.740 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:05.741 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:05.741 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.741 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getfilelock_latency": { 2024-09-12T01:37:05.741 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:05.741 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:05.741 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:05.741 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.741 INFO:teuthology.orchestra.run.smithi104.stdout: "req_link_latency": { 2024-09-12T01:37:05.741 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:05.741 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:05.741 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:05.741 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.742 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookup_latency": { 2024-09-12T01:37:05.742 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:05.742 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:05.742 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:05.742 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.742 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookuphash_latency": { 2024-09-12T01:37:05.742 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:05.742 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:05.742 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:05.742 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.742 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupino_latency": { 2024-09-12T01:37:05.742 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:05.742 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:05.742 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:05.743 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.743 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupname_latency": { 2024-09-12T01:37:05.743 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:05.743 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:05.743 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:05.743 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.743 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupparent_latency": { 2024-09-12T01:37:05.743 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:05.743 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:05.743 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:05.743 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.743 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupsnap_latency": { 2024-09-12T01:37:05.743 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:05.743 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:05.743 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:05.744 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.744 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lssnap_latency": { 2024-09-12T01:37:05.744 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:05.744 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:05.744 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:05.744 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.744 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mkdir_latency": { 2024-09-12T01:37:05.744 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:05.745 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:05.745 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:05.745 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.745 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mknod_latency": { 2024-09-12T01:37:05.745 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:05.745 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:05.745 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:05.745 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.745 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mksnap_latency": { 2024-09-12T01:37:05.745 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:05.745 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:05.746 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:05.746 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.746 INFO:teuthology.orchestra.run.smithi104.stdout: "req_open_latency": { 2024-09-12T01:37:05.746 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:05.746 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:05.746 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:05.746 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.746 INFO:teuthology.orchestra.run.smithi104.stdout: "req_readdir_latency": { 2024-09-12T01:37:05.746 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:05.746 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:05.746 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:05.746 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.746 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rename_latency": { 2024-09-12T01:37:05.746 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:05.747 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:05.747 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:05.747 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.747 INFO:teuthology.orchestra.run.smithi104.stdout: "req_renamesnap_latency": { 2024-09-12T01:37:05.747 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:05.747 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:05.747 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:05.747 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.747 INFO:teuthology.orchestra.run.smithi104.stdout: "req_snapdiff_latency": { 2024-09-12T01:37:05.747 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:05.747 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:05.747 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:05.747 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.747 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmdir_latency": { 2024-09-12T01:37:05.748 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:05.748 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:05.748 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:05.748 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.748 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmsnap_latency": { 2024-09-12T01:37:05.748 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:05.748 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:05.748 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:05.748 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.748 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmxattr_latency": { 2024-09-12T01:37:05.748 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:05.748 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:05.748 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:05.748 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.749 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setattr_latency": { 2024-09-12T01:37:05.749 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:05.749 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:05.749 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:05.749 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.749 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setdirlayout_latency": { 2024-09-12T01:37:05.749 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:05.749 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:05.749 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:05.749 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.749 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setfilelock_latency": { 2024-09-12T01:37:05.750 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:05.750 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:05.750 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:05.750 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.750 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setlayout_latency": { 2024-09-12T01:37:05.750 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:05.750 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:05.750 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:05.750 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.750 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setxattr_latency": { 2024-09-12T01:37:05.750 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:05.750 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:05.750 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:05.750 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.751 INFO:teuthology.orchestra.run.smithi104.stdout: "req_symlink_latency": { 2024-09-12T01:37:05.751 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:05.751 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:05.751 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:05.751 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.751 INFO:teuthology.orchestra.run.smithi104.stdout: "req_unlink_latency": { 2024-09-12T01:37:05.751 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:05.751 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:05.751 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:05.751 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.751 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_revoke_eviction": 0, 2024-09-12T01:37:05.751 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_acquisition_throttle": 0, 2024-09-12T01:37:05.751 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getvxattr_latency": { 2024-09-12T01:37:05.751 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:05.752 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:05.752 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:05.752 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-12T01:37:05.752 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.752 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_sessions": { 2024-09-12T01:37:05.752 INFO:teuthology.orchestra.run.smithi104.stdout: "session_count": 0, 2024-09-12T01:37:05.752 INFO:teuthology.orchestra.run.smithi104.stdout: "session_add": 1, 2024-09-12T01:37:05.752 INFO:teuthology.orchestra.run.smithi104.stdout: "session_remove": 1, 2024-09-12T01:37:05.752 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_open": 0, 2024-09-12T01:37:05.752 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_stale": 0, 2024-09-12T01:37:05.752 INFO:teuthology.orchestra.run.smithi104.stdout: "total_load": 0, 2024-09-12T01:37:05.752 INFO:teuthology.orchestra.run.smithi104.stdout: "average_load": 0, 2024-09-12T01:37:05.752 INFO:teuthology.orchestra.run.smithi104.stdout: "avg_session_uptime": 82, 2024-09-12T01:37:05.752 INFO:teuthology.orchestra.run.smithi104.stdout: "mdthresh_evicted": 0 2024-09-12T01:37:05.753 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.753 INFO:teuthology.orchestra.run.smithi104.stdout: "mempool": { 2024-09-12T01:37:05.753 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_bytes": 0, 2024-09-12T01:37:05.753 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_items": 0, 2024-09-12T01:37:05.753 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T01:37:05.753 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_items": 0, 2024-09-12T01:37:05.753 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T01:37:05.753 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_items": 0, 2024-09-12T01:37:05.753 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T01:37:05.753 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T01:37:05.753 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T01:37:05.753 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T01:37:05.753 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T01:37:05.753 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_items": 0, 2024-09-12T01:37:05.754 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T01:37:05.754 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T01:37:05.754 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_bytes": 0, 2024-09-12T01:37:05.754 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_items": 0, 2024-09-12T01:37:05.754 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_bytes": 0, 2024-09-12T01:37:05.754 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_items": 0, 2024-09-12T01:37:05.754 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T01:37:05.754 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T01:37:05.754 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T01:37:05.754 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T01:37:05.754 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T01:37:05.754 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_items": 0, 2024-09-12T01:37:05.754 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_bytes": 0, 2024-09-12T01:37:05.754 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_items": 0, 2024-09-12T01:37:05.755 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T01:37:05.755 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T01:37:05.755 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_bytes": 0, 2024-09-12T01:37:05.755 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_items": 0, 2024-09-12T01:37:05.755 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_bytes": 0, 2024-09-12T01:37:05.755 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_items": 0, 2024-09-12T01:37:05.755 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T01:37:05.755 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_items": 0, 2024-09-12T01:37:05.755 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T01:37:05.755 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_items": 0, 2024-09-12T01:37:05.755 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_bytes": 242351, 2024-09-12T01:37:05.755 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_items": 81, 2024-09-12T01:37:05.755 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_bytes": 0, 2024-09-12T01:37:05.755 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_items": 0, 2024-09-12T01:37:05.756 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_bytes": 0, 2024-09-12T01:37:05.756 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_items": 0, 2024-09-12T01:37:05.756 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_bytes": 0, 2024-09-12T01:37:05.756 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_items": 0, 2024-09-12T01:37:05.756 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_bytes": 0, 2024-09-12T01:37:05.756 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_items": 0, 2024-09-12T01:37:05.756 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_bytes": 8856, 2024-09-12T01:37:05.756 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_items": 166, 2024-09-12T01:37:05.756 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T01:37:05.756 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_items": 0, 2024-09-12T01:37:05.756 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_bytes": 0, 2024-09-12T01:37:05.756 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_items": 0, 2024-09-12T01:37:05.756 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_bytes": 84333, 2024-09-12T01:37:05.756 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_items": 321, 2024-09-12T01:37:05.757 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_bytes": 0, 2024-09-12T01:37:05.757 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_items": 0, 2024-09-12T01:37:05.757 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_bytes": 0, 2024-09-12T01:37:05.757 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_items": 0 2024-09-12T01:37:05.757 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.757 INFO:teuthology.orchestra.run.smithi104.stdout: "objecter": { 2024-09-12T01:37:05.757 INFO:teuthology.orchestra.run.smithi104.stdout: "op_active": 0, 2024-09-12T01:37:05.757 INFO:teuthology.orchestra.run.smithi104.stdout: "op_laggy": 0, 2024-09-12T01:37:05.757 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send": 38, 2024-09-12T01:37:05.757 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send_bytes": 11334, 2024-09-12T01:37:05.757 INFO:teuthology.orchestra.run.smithi104.stdout: "op_resend": 33, 2024-09-12T01:37:05.757 INFO:teuthology.orchestra.run.smithi104.stdout: "op_reply": 38, 2024-09-12T01:37:05.757 INFO:teuthology.orchestra.run.smithi104.stdout: "op_latency": { 2024-09-12T01:37:05.757 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 38, 2024-09-12T01:37:05.758 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.225001348, 2024-09-12T01:37:05.758 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.005921088 2024-09-12T01:37:05.758 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.758 INFO:teuthology.orchestra.run.smithi104.stdout: "op_inflight": 0, 2024-09-12T01:37:05.758 INFO:teuthology.orchestra.run.smithi104.stdout: "oplen_avg": { 2024-09-12T01:37:05.758 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 38, 2024-09-12T01:37:05.758 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 60 2024-09-12T01:37:05.758 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.758 INFO:teuthology.orchestra.run.smithi104.stdout: "op": 38, 2024-09-12T01:37:05.758 INFO:teuthology.orchestra.run.smithi104.stdout: "op_r": 0, 2024-09-12T01:37:05.758 INFO:teuthology.orchestra.run.smithi104.stdout: "op_w": 38, 2024-09-12T01:37:05.758 INFO:teuthology.orchestra.run.smithi104.stdout: "op_rmw": 0, 2024-09-12T01:37:05.758 INFO:teuthology.orchestra.run.smithi104.stdout: "op_pg": 0, 2024-09-12T01:37:05.758 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_stat": 0, 2024-09-12T01:37:05.759 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_create": 10, 2024-09-12T01:37:05.759 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_read": 0, 2024-09-12T01:37:05.759 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_write": 3, 2024-09-12T01:37:05.759 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writefull": 6, 2024-09-12T01:37:05.759 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writesame": 0, 2024-09-12T01:37:05.759 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_append": 0, 2024-09-12T01:37:05.759 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_zero": 0, 2024-09-12T01:37:05.759 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_truncate": 0, 2024-09-12T01:37:05.759 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_delete": 6, 2024-09-12T01:37:05.759 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_mapext": 0, 2024-09-12T01:37:05.759 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_sparse_read": 0, 2024-09-12T01:37:05.759 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_clonerange": 0, 2024-09-12T01:37:05.759 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_getxattr": 0, 2024-09-12T01:37:05.759 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_setxattr": 20, 2024-09-12T01:37:05.759 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_cmpxattr": 0, 2024-09-12T01:37:05.760 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_rmxattr": 0, 2024-09-12T01:37:05.760 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_resetxattrs": 0, 2024-09-12T01:37:05.760 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_call": 0, 2024-09-12T01:37:05.760 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_watch": 0, 2024-09-12T01:37:05.760 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_notify": 0, 2024-09-12T01:37:05.760 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T01:37:05.760 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls": 0, 2024-09-12T01:37:05.760 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls_filter": 0, 2024-09-12T01:37:05.760 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_other": 0, 2024-09-12T01:37:05.760 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_active": 0, 2024-09-12T01:37:05.760 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_send": 0, 2024-09-12T01:37:05.760 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_resend": 0, 2024-09-12T01:37:05.760 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_ping": 0, 2024-09-12T01:37:05.760 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_active": 0, 2024-09-12T01:37:05.761 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_send": 0, 2024-09-12T01:37:05.761 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_resend": 0, 2024-09-12T01:37:05.761 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_active": 0, 2024-09-12T01:37:05.761 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_send": 0, 2024-09-12T01:37:05.761 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_resend": 0, 2024-09-12T01:37:05.761 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_active": 0, 2024-09-12T01:37:05.761 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_send": 0, 2024-09-12T01:37:05.761 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_resend": 0, 2024-09-12T01:37:05.761 INFO:teuthology.orchestra.run.smithi104.stdout: "command_active": 0, 2024-09-12T01:37:05.761 INFO:teuthology.orchestra.run.smithi104.stdout: "command_send": 0, 2024-09-12T01:37:05.761 INFO:teuthology.orchestra.run.smithi104.stdout: "command_resend": 0, 2024-09-12T01:37:05.761 INFO:teuthology.orchestra.run.smithi104.stdout: "map_epoch": 98, 2024-09-12T01:37:05.761 INFO:teuthology.orchestra.run.smithi104.stdout: "map_full": 0, 2024-09-12T01:37:05.761 INFO:teuthology.orchestra.run.smithi104.stdout: "map_inc": 2, 2024-09-12T01:37:05.761 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_sessions": 10, 2024-09-12T01:37:05.762 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_open": 10, 2024-09-12T01:37:05.762 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_close": 0, 2024-09-12T01:37:05.762 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_laggy": 0, 2024-09-12T01:37:05.762 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_wr": 14, 2024-09-12T01:37:05.762 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_rd": 0, 2024-09-12T01:37:05.762 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_del": 1 2024-09-12T01:37:05.762 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.762 INFO:teuthology.orchestra.run.smithi104.stdout: "oft": { 2024-09-12T01:37:05.762 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_objs": 1, 2024-09-12T01:37:05.762 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_kv_pairs": 0, 2024-09-12T01:37:05.762 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_updates": 0, 2024-09-12T01:37:05.762 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_removes": 0 2024-09-12T01:37:05.762 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.762 INFO:teuthology.orchestra.run.smithi104.stdout: "osd-slow-ops": { 2024-09-12T01:37:05.763 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_ops_count": 0 2024-09-12T01:37:05.763 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.763 INFO:teuthology.orchestra.run.smithi104.stdout: "purge_queue": { 2024-09-12T01:37:05.763 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops": 0, 2024-09-12T01:37:05.763 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T01:37:05.763 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing": 0, 2024-09-12T01:37:05.763 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_high_water": 0, 2024-09-12T01:37:05.763 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed_ops": 0, 2024-09-12T01:37:05.763 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed": 0, 2024-09-12T01:37:05.763 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_item_in_journal": 0 2024-09-12T01:37:05.763 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.763 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T01:37:05.763 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-12T01:37:05.763 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-09-12T01:37:05.763 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-12T01:37:05.764 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 716, 2024-09-12T01:37:05.764 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 979343, 2024-09-12T01:37:05.764 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-12T01:37:05.764 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 716, 2024-09-12T01:37:05.764 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-09-12T01:37:05.764 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-09-12T01:37:05.764 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 716, 2024-09-12T01:37:05.764 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 979343, 2024-09-12T01:37:05.764 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-12T01:37:05.764 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:05.764 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:05.764 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:05.764 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-12T01:37:05.764 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.765 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_bytes": { 2024-09-12T01:37:05.765 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-12T01:37:05.765 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-09-12T01:37:05.765 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-12T01:37:05.765 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-09-12T01:37:05.765 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-09-12T01:37:05.765 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-12T01:37:05.765 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-09-12T01:37:05.765 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 38, 2024-09-12T01:37:05.765 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 11334, 2024-09-12T01:37:05.765 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 38, 2024-09-12T01:37:05.765 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 11334, 2024-09-12T01:37:05.765 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-12T01:37:05.765 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:05.765 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:05.766 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:05.766 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-12T01:37:05.766 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.766 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_ops": { 2024-09-12T01:37:05.766 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-12T01:37:05.766 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 1024, 2024-09-12T01:37:05.766 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-12T01:37:05.766 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-09-12T01:37:05.766 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-09-12T01:37:05.766 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-12T01:37:05.766 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-09-12T01:37:05.766 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 38, 2024-09-12T01:37:05.766 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 38, 2024-09-12T01:37:05.766 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 38, 2024-09-12T01:37:05.767 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 38, 2024-09-12T01:37:05.767 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-12T01:37:05.767 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:05.767 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:05.767 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:05.767 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-12T01:37:05.767 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.767 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle": { 2024-09-12T01:37:05.767 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-12T01:37:05.767 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-09-12T01:37:05.767 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-12T01:37:05.767 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-09-12T01:37:05.767 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-09-12T01:37:05.767 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-12T01:37:05.767 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-09-12T01:37:05.768 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-09-12T01:37:05.768 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-09-12T01:37:05.768 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 0, 2024-09-12T01:37:05.768 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 0, 2024-09-12T01:37:05.768 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-12T01:37:05.768 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:05.768 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:05.768 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:05.768 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-12T01:37:05.768 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:05.768 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle-0x5577b84b2500": { 2024-09-12T01:37:05.768 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-12T01:37:05.768 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-09-12T01:37:05.768 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-12T01:37:05.768 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 4, 2024-09-12T01:37:05.769 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 1122, 2024-09-12T01:37:05.769 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-12T01:37:05.769 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 4, 2024-09-12T01:37:05.769 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-09-12T01:37:05.769 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-09-12T01:37:05.769 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 3, 2024-09-12T01:37:05.769 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 1122, 2024-09-12T01:37:05.769 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-12T01:37:05.769 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:05.769 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:05.769 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:05.769 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-12T01:37:05.769 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-12T01:37:05.769 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-12T01:37:06.502 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-12T01:37:06.502 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-12T01:37:06.502 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-09-12T01:37:06.502 DEBUG:tasks.check_counter:Getting stats from i 2024-09-12T01:37:06.503 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-12T01:37:06.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:06 smithi053 ceph-mon[35158]: pgmap v623: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-12T01:37:06.747 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:37:06.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:06 smithi104 ceph-mon[35219]: pgmap v623: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-12T01:37:06.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:06 smithi001 ceph-mon[30387]: pgmap v623: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-12T01:37:07.271 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-12T01:37:07.271 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T01:37:07.271 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 354, 2024-09-12T01:37:07.271 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 271, 2024-09-12T01:37:07.271 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 928468, 2024-09-12T01:37:07.271 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 159062, 2024-09-12T01:37:07.271 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 8, 2024-09-12T01:37:07.271 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 4, 2024-09-12T01:37:07.271 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.038397265, 2024-09-12T01:37:07.271 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.014005173, 2024-09-12T01:37:07.271 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.031647167, 2024-09-12T01:37:07.272 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.000264536, 2024-09-12T01:37:07.272 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:37:07.272 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 270, 2024-09-12T01:37:07.272 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.020130155, 2024-09-12T01:37:07.272 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000074556 2024-09-12T01:37:07.272 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.272 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:37:07.272 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 164, 2024-09-12T01:37:07.272 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000028143, 2024-09-12T01:37:07.272 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000171 2024-09-12T01:37:07.272 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.272 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 861120, 2024-09-12T01:37:07.272 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 112272 2024-09-12T01:37:07.272 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.273 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T01:37:07.273 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 32, 2024-09-12T01:37:07.273 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 166, 2024-09-12T01:37:07.273 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 8328, 2024-09-12T01:37:07.273 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 354198, 2024-09-12T01:37:07.273 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 12, 2024-09-12T01:37:07.273 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 6, 2024-09-12T01:37:07.273 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.015195728, 2024-09-12T01:37:07.273 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.006821149, 2024-09-12T01:37:07.273 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.006501915, 2024-09-12T01:37:07.273 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.000404403, 2024-09-12T01:37:07.273 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:37:07.273 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 164, 2024-09-12T01:37:07.273 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.022732096, 2024-09-12T01:37:07.274 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000138610 2024-09-12T01:37:07.274 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.274 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:37:07.274 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 18, 2024-09-12T01:37:07.274 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000009571, 2024-09-12T01:37:07.274 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000531 2024-09-12T01:37:07.274 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.274 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-12T01:37:07.274 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 293648 2024-09-12T01:37:07.274 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.274 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T01:37:07.274 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 360, 2024-09-12T01:37:07.274 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 415, 2024-09-12T01:37:07.274 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 141015, 2024-09-12T01:37:07.275 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 199763, 2024-09-12T01:37:07.275 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 7, 2024-09-12T01:37:07.275 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 5, 2024-09-12T01:37:07.275 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.041161246, 2024-09-12T01:37:07.275 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.014847658, 2024-09-12T01:37:07.275 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.023593634, 2024-09-12T01:37:07.275 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.003256080, 2024-09-12T01:37:07.275 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:37:07.275 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 414, 2024-09-12T01:37:07.275 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.036354630, 2024-09-12T01:37:07.275 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000087813 2024-09-12T01:37:07.275 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.275 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:37:07.275 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 727, 2024-09-12T01:37:07.276 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000163923, 2024-09-12T01:37:07.276 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000225 2024-09-12T01:37:07.276 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.276 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-12T01:37:07.276 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T01:37:07.276 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.276 INFO:teuthology.orchestra.run.smithi104.stdout: "cct": { 2024-09-12T01:37:07.276 INFO:teuthology.orchestra.run.smithi104.stdout: "total_workers": 1, 2024-09-12T01:37:07.276 INFO:teuthology.orchestra.run.smithi104.stdout: "unhealthy_workers": 0 2024-09-12T01:37:07.276 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.276 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-MDSRank": { 2024-09-12T01:37:07.276 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-09-12T01:37:07.276 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-09-12T01:37:07.276 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 41, 2024-09-12T01:37:07.276 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.001345973, 2024-09-12T01:37:07.277 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000032828 2024-09-12T01:37:07.277 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-12T01:37:07.277 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.277 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-PurgeQueue": { 2024-09-12T01:37:07.277 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-09-12T01:37:07.277 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-09-12T01:37:07.277 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 2, 2024-09-12T01:37:07.277 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000053682, 2024-09-12T01:37:07.277 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000026841 2024-09-12T01:37:07.277 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-12T01:37:07.277 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.277 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": { 2024-09-12T01:37:07.277 INFO:teuthology.orchestra.run.smithi104.stdout: "request": 3, 2024-09-12T01:37:07.277 INFO:teuthology.orchestra.run.smithi104.stdout: "reply": 1, 2024-09-12T01:37:07.278 INFO:teuthology.orchestra.run.smithi104.stdout: "reply_latency": { 2024-09-12T01:37:07.278 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1, 2024-09-12T01:37:07.278 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000877884, 2024-09-12T01:37:07.278 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000877884 2024-09-12T01:37:07.278 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.278 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_reply": 0, 2024-09-12T01:37:07.278 INFO:teuthology.orchestra.run.smithi104.stdout: "forward": 2, 2024-09-12T01:37:07.278 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_complete": 0, 2024-09-12T01:37:07.278 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_keys": 0, 2024-09-12T01:37:07.278 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_commit": 11, 2024-09-12T01:37:07.278 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_split": 0, 2024-09-12T01:37:07.278 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_merge": 0, 2024-09-12T01:37:07.278 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes": 16, 2024-09-12T01:37:07.278 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_top": 0, 2024-09-12T01:37:07.279 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_bottom": 16, 2024-09-12T01:37:07.279 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pin_tail": 0, 2024-09-12T01:37:07.279 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pinned": 14, 2024-09-12T01:37:07.279 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_expired": 6, 2024-09-12T01:37:07.279 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_with_caps": 1, 2024-09-12T01:37:07.279 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": 1, 2024-09-12T01:37:07.279 INFO:teuthology.orchestra.run.smithi104.stdout: "subtrees": 4, 2024-09-12T01:37:07.279 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse": 9, 2024-09-12T01:37:07.279 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_hit": 3, 2024-09-12T01:37:07.279 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_forward": 1, 2024-09-12T01:37:07.279 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_discover": 4, 2024-09-12T01:37:07.279 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_dir_fetch": 0, 2024-09-12T01:37:07.279 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_remote_ino": 0, 2024-09-12T01:37:07.279 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_lock": 0, 2024-09-12T01:37:07.279 INFO:teuthology.orchestra.run.smithi104.stdout: "load_cent": 0, 2024-09-12T01:37:07.280 INFO:teuthology.orchestra.run.smithi104.stdout: "q": 0, 2024-09-12T01:37:07.280 INFO:teuthology.orchestra.run.smithi104.stdout: "exported": 0, 2024-09-12T01:37:07.280 INFO:teuthology.orchestra.run.smithi104.stdout: "exported_inodes": 0, 2024-09-12T01:37:07.280 INFO:teuthology.orchestra.run.smithi104.stdout: "imported": 0, 2024-09-12T01:37:07.280 INFO:teuthology.orchestra.run.smithi104.stdout: "imported_inodes": 0, 2024-09-12T01:37:07.280 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_dir_fetch": 0, 2024-09-12T01:37:07.280 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_backtrace_fetch": 0, 2024-09-12T01:37:07.280 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_peer_discover": 0, 2024-09-12T01:37:07.280 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rfiles": 4, 2024-09-12T01:37:07.280 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rbytes": 234, 2024-09-12T01:37:07.280 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rsnaps": 0, 2024-09-12T01:37:07.280 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T01:37:07.280 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_set_tag": 0, 2024-09-12T01:37:07.280 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T01:37:07.281 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_inotable_repaired": 0, 2024-09-12T01:37:07.281 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_inodes": 0, 2024-09-12T01:37:07.281 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T01:37:07.281 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T01:37:07.281 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_file_inodes": 0, 2024-09-12T01:37:07.281 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_inode_file_caps": 0, 2024-09-12T01:37:07.281 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_revoke": 2, 2024-09-12T01:37:07.281 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_grant": 3, 2024-09-12T01:37:07.281 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T01:37:07.281 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T01:37:07.281 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T01:37:07.281 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps": 2, 2024-09-12T01:37:07.281 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps_dirty": 0, 2024-09-12T01:37:07.281 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_cap_release": 0, 2024-09-12T01:37:07.281 INFO:teuthology.orchestra.run.smithi104.stdout: "process_request_cap_release": 1 2024-09-12T01:37:07.282 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.282 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_cache": { 2024-09-12T01:37:07.282 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update": 0, 2024-09-12T01:37:07.282 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update_receipt": 5, 2024-09-12T01:37:07.282 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_try_discover": 4, 2024-09-12T01:37:07.282 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_send_discover": 7, 2024-09-12T01:37:07.282 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_handle_discover": 0, 2024-09-12T01:37:07.282 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays": 0, 2024-09-12T01:37:07.282 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_delayed": 0, 2024-09-12T01:37:07.282 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_enqueuing": 0, 2024-09-12T01:37:07.282 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_created": 0, 2024-09-12T01:37:07.282 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_enqueued": 0, 2024-09-12T01:37:07.282 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_reintegrated": 0, 2024-09-12T01:37:07.282 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_migrated": 0, 2024-09-12T01:37:07.283 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_processing": 0, 2024-09-12T01:37:07.283 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_enqueued": 0, 2024-09-12T01:37:07.283 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_prioritized": 0, 2024-09-12T01:37:07.283 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_started": 0, 2024-09-12T01:37:07.283 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_completed": 0, 2024-09-12T01:37:07.283 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_path": 0, 2024-09-12T01:37:07.283 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_inode": 0, 2024-09-12T01:37:07.283 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T01:37:07.283 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_exportdir": 0, 2024-09-12T01:37:07.283 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_flush": 0, 2024-09-12T01:37:07.283 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragmentdir": 0, 2024-09-12T01:37:07.283 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragstats": 0, 2024-09-12T01:37:07.283 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_inodestats": 0 2024-09-12T01:37:07.283 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.283 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_log": { 2024-09-12T01:37:07.284 INFO:teuthology.orchestra.run.smithi104.stdout: "evlrg": 0, 2024-09-12T01:37:07.284 INFO:teuthology.orchestra.run.smithi104.stdout: "evadd": 3, 2024-09-12T01:37:07.284 INFO:teuthology.orchestra.run.smithi104.stdout: "evex": 0, 2024-09-12T01:37:07.284 INFO:teuthology.orchestra.run.smithi104.stdout: "evtrm": 0, 2024-09-12T01:37:07.284 INFO:teuthology.orchestra.run.smithi104.stdout: "ev": 3, 2024-09-12T01:37:07.284 INFO:teuthology.orchestra.run.smithi104.stdout: "evexg": 0, 2024-09-12T01:37:07.284 INFO:teuthology.orchestra.run.smithi104.stdout: "evexd": 0, 2024-09-12T01:37:07.284 INFO:teuthology.orchestra.run.smithi104.stdout: "segadd": 2, 2024-09-12T01:37:07.284 INFO:teuthology.orchestra.run.smithi104.stdout: "segex": 0, 2024-09-12T01:37:07.284 INFO:teuthology.orchestra.run.smithi104.stdout: "segtrm": 0, 2024-09-12T01:37:07.284 INFO:teuthology.orchestra.run.smithi104.stdout: "seg": 2, 2024-09-12T01:37:07.284 INFO:teuthology.orchestra.run.smithi104.stdout: "segmjr": 2, 2024-09-12T01:37:07.284 INFO:teuthology.orchestra.run.smithi104.stdout: "segexg": 0, 2024-09-12T01:37:07.284 INFO:teuthology.orchestra.run.smithi104.stdout: "segexd": 0, 2024-09-12T01:37:07.284 INFO:teuthology.orchestra.run.smithi104.stdout: "expos": 4194304, 2024-09-12T01:37:07.285 INFO:teuthology.orchestra.run.smithi104.stdout: "wrpos": 4195203, 2024-09-12T01:37:07.285 INFO:teuthology.orchestra.run.smithi104.stdout: "rdpos": 0, 2024-09-12T01:37:07.285 INFO:teuthology.orchestra.run.smithi104.stdout: "jlat": { 2024-09-12T01:37:07.285 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 2, 2024-09-12T01:37:07.285 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.003149229, 2024-09-12T01:37:07.285 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.001574614 2024-09-12T01:37:07.285 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.285 INFO:teuthology.orchestra.run.smithi104.stdout: "replayed": 0 2024-09-12T01:37:07.285 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.285 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_mem": { 2024-09-12T01:37:07.285 INFO:teuthology.orchestra.run.smithi104.stdout: "ino": 21, 2024-09-12T01:37:07.285 INFO:teuthology.orchestra.run.smithi104.stdout: "ino+": 24, 2024-09-12T01:37:07.285 INFO:teuthology.orchestra.run.smithi104.stdout: "ino-": 3, 2024-09-12T01:37:07.285 INFO:teuthology.orchestra.run.smithi104.stdout: "dir": 20, 2024-09-12T01:37:07.285 INFO:teuthology.orchestra.run.smithi104.stdout: "dir+": 24, 2024-09-12T01:37:07.286 INFO:teuthology.orchestra.run.smithi104.stdout: "dir-": 4, 2024-09-12T01:37:07.286 INFO:teuthology.orchestra.run.smithi104.stdout: "dn": 16, 2024-09-12T01:37:07.286 INFO:teuthology.orchestra.run.smithi104.stdout: "dn+": 22, 2024-09-12T01:37:07.286 INFO:teuthology.orchestra.run.smithi104.stdout: "dn-": 6, 2024-09-12T01:37:07.286 INFO:teuthology.orchestra.run.smithi104.stdout: "cap": 1, 2024-09-12T01:37:07.286 INFO:teuthology.orchestra.run.smithi104.stdout: "cap+": 1, 2024-09-12T01:37:07.286 INFO:teuthology.orchestra.run.smithi104.stdout: "cap-": 0, 2024-09-12T01:37:07.286 INFO:teuthology.orchestra.run.smithi104.stdout: "rss": 50188, 2024-09-12T01:37:07.286 INFO:teuthology.orchestra.run.smithi104.stdout: "heap": 223520 2024-09-12T01:37:07.286 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.286 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_server": { 2024-09-12T01:37:07.286 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_client_request": 4, 2024-09-12T01:37:07.286 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_server_request": 1, 2024-09-12T01:37:07.286 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_request": 3, 2024-09-12T01:37:07.286 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_session": 6, 2024-09-12T01:37:07.287 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_peer_request": 2, 2024-09-12T01:37:07.287 INFO:teuthology.orchestra.run.smithi104.stdout: "req_create_latency": { 2024-09-12T01:37:07.287 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:07.287 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:07.287 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:07.287 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.287 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getattr_latency": { 2024-09-12T01:37:07.287 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1, 2024-09-12T01:37:07.287 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000877884, 2024-09-12T01:37:07.287 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000877884 2024-09-12T01:37:07.287 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.287 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getfilelock_latency": { 2024-09-12T01:37:07.287 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:07.287 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:07.287 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:07.288 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.288 INFO:teuthology.orchestra.run.smithi104.stdout: "req_link_latency": { 2024-09-12T01:37:07.288 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:07.288 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:07.288 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:07.288 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.288 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookup_latency": { 2024-09-12T01:37:07.288 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:07.288 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:07.288 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:07.288 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.288 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookuphash_latency": { 2024-09-12T01:37:07.288 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:07.288 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:07.289 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:07.289 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.289 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupino_latency": { 2024-09-12T01:37:07.289 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:07.289 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:07.289 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:07.289 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.289 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupname_latency": { 2024-09-12T01:37:07.289 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:07.289 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:07.289 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:07.289 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.289 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupparent_latency": { 2024-09-12T01:37:07.289 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:07.289 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:07.290 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:07.290 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.290 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupsnap_latency": { 2024-09-12T01:37:07.290 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:07.290 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:07.290 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:07.290 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.290 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lssnap_latency": { 2024-09-12T01:37:07.290 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:07.290 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:07.290 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:07.290 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.290 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mkdir_latency": { 2024-09-12T01:37:07.290 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:07.291 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:07.291 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:07.291 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.291 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mknod_latency": { 2024-09-12T01:37:07.291 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:07.291 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:07.291 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:07.292 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.292 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mksnap_latency": { 2024-09-12T01:37:07.292 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:07.292 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:07.292 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:07.292 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.292 INFO:teuthology.orchestra.run.smithi104.stdout: "req_open_latency": { 2024-09-12T01:37:07.292 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:07.292 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:07.292 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:07.292 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.292 INFO:teuthology.orchestra.run.smithi104.stdout: "req_readdir_latency": { 2024-09-12T01:37:07.292 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:07.292 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:07.293 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:07.293 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.293 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rename_latency": { 2024-09-12T01:37:07.293 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:07.293 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:07.293 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:07.293 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.293 INFO:teuthology.orchestra.run.smithi104.stdout: "req_renamesnap_latency": { 2024-09-12T01:37:07.293 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:07.293 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:07.293 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:07.293 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.293 INFO:teuthology.orchestra.run.smithi104.stdout: "req_snapdiff_latency": { 2024-09-12T01:37:07.293 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:07.294 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:07.294 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:07.294 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.294 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmdir_latency": { 2024-09-12T01:37:07.294 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:07.294 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:07.294 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:07.294 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.294 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmsnap_latency": { 2024-09-12T01:37:07.294 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:07.294 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:07.294 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:07.294 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.294 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmxattr_latency": { 2024-09-12T01:37:07.295 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:07.295 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:07.295 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:07.295 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.295 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setattr_latency": { 2024-09-12T01:37:07.295 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:07.295 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:07.295 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:07.295 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.295 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setdirlayout_latency": { 2024-09-12T01:37:07.295 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:07.295 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:07.295 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:07.295 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.295 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setfilelock_latency": { 2024-09-12T01:37:07.296 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:07.296 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:07.296 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:07.296 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.296 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setlayout_latency": { 2024-09-12T01:37:07.296 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:07.296 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:07.296 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:07.296 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.296 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setxattr_latency": { 2024-09-12T01:37:07.296 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:07.296 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:07.296 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:07.296 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.297 INFO:teuthology.orchestra.run.smithi104.stdout: "req_symlink_latency": { 2024-09-12T01:37:07.297 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:07.297 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:07.297 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:07.297 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.297 INFO:teuthology.orchestra.run.smithi104.stdout: "req_unlink_latency": { 2024-09-12T01:37:07.297 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:07.297 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:07.297 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:07.297 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.297 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_revoke_eviction": 0, 2024-09-12T01:37:07.298 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_acquisition_throttle": 0, 2024-09-12T01:37:07.298 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getvxattr_latency": { 2024-09-12T01:37:07.298 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:07.298 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:07.298 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:07.298 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-12T01:37:07.298 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.298 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_sessions": { 2024-09-12T01:37:07.298 INFO:teuthology.orchestra.run.smithi104.stdout: "session_count": 1, 2024-09-12T01:37:07.298 INFO:teuthology.orchestra.run.smithi104.stdout: "session_add": 1, 2024-09-12T01:37:07.298 INFO:teuthology.orchestra.run.smithi104.stdout: "session_remove": 0, 2024-09-12T01:37:07.298 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_open": 1, 2024-09-12T01:37:07.298 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_stale": 0, 2024-09-12T01:37:07.299 INFO:teuthology.orchestra.run.smithi104.stdout: "total_load": 1, 2024-09-12T01:37:07.299 INFO:teuthology.orchestra.run.smithi104.stdout: "average_load": 1, 2024-09-12T01:37:07.299 INFO:teuthology.orchestra.run.smithi104.stdout: "avg_session_uptime": 436, 2024-09-12T01:37:07.299 INFO:teuthology.orchestra.run.smithi104.stdout: "mdthresh_evicted": 0 2024-09-12T01:37:07.299 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.299 INFO:teuthology.orchestra.run.smithi104.stdout: "mempool": { 2024-09-12T01:37:07.299 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_bytes": 0, 2024-09-12T01:37:07.299 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_items": 0, 2024-09-12T01:37:07.299 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T01:37:07.299 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_items": 0, 2024-09-12T01:37:07.299 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T01:37:07.299 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_items": 0, 2024-09-12T01:37:07.299 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T01:37:07.300 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T01:37:07.300 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T01:37:07.300 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T01:37:07.300 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T01:37:07.300 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_items": 0, 2024-09-12T01:37:07.300 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T01:37:07.300 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T01:37:07.300 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_bytes": 0, 2024-09-12T01:37:07.300 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_items": 0, 2024-09-12T01:37:07.300 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_bytes": 0, 2024-09-12T01:37:07.300 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_items": 0, 2024-09-12T01:37:07.300 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T01:37:07.300 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T01:37:07.301 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T01:37:07.301 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T01:37:07.301 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T01:37:07.301 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_items": 0, 2024-09-12T01:37:07.301 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_bytes": 0, 2024-09-12T01:37:07.301 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_items": 0, 2024-09-12T01:37:07.301 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T01:37:07.301 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T01:37:07.301 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_bytes": 0, 2024-09-12T01:37:07.301 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_items": 0, 2024-09-12T01:37:07.301 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_bytes": 0, 2024-09-12T01:37:07.301 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_items": 0, 2024-09-12T01:37:07.301 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T01:37:07.302 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_items": 0, 2024-09-12T01:37:07.302 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T01:37:07.302 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_items": 0, 2024-09-12T01:37:07.302 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_bytes": 267280, 2024-09-12T01:37:07.302 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_items": 93, 2024-09-12T01:37:07.302 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_bytes": 0, 2024-09-12T01:37:07.302 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_items": 0, 2024-09-12T01:37:07.302 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_bytes": 0, 2024-09-12T01:37:07.302 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_items": 0, 2024-09-12T01:37:07.302 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_bytes": 0, 2024-09-12T01:37:07.303 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_items": 0, 2024-09-12T01:37:07.303 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_bytes": 0, 2024-09-12T01:37:07.303 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_items": 0, 2024-09-12T01:37:07.303 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_bytes": 8856, 2024-09-12T01:37:07.303 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_items": 166, 2024-09-12T01:37:07.303 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T01:37:07.303 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_items": 0, 2024-09-12T01:37:07.303 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_bytes": 0, 2024-09-12T01:37:07.303 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_items": 0, 2024-09-12T01:37:07.303 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_bytes": 84781, 2024-09-12T01:37:07.303 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_items": 333, 2024-09-12T01:37:07.303 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_bytes": 0, 2024-09-12T01:37:07.304 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_items": 0, 2024-09-12T01:37:07.304 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_bytes": 0, 2024-09-12T01:37:07.304 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_items": 0 2024-09-12T01:37:07.304 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.304 INFO:teuthology.orchestra.run.smithi104.stdout: "objecter": { 2024-09-12T01:37:07.304 INFO:teuthology.orchestra.run.smithi104.stdout: "op_active": 0, 2024-09-12T01:37:07.304 INFO:teuthology.orchestra.run.smithi104.stdout: "op_laggy": 0, 2024-09-12T01:37:07.304 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send": 39, 2024-09-12T01:37:07.304 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send_bytes": 11516, 2024-09-12T01:37:07.304 INFO:teuthology.orchestra.run.smithi104.stdout: "op_resend": 29, 2024-09-12T01:37:07.304 INFO:teuthology.orchestra.run.smithi104.stdout: "op_reply": 39, 2024-09-12T01:37:07.304 INFO:teuthology.orchestra.run.smithi104.stdout: "op_latency": { 2024-09-12T01:37:07.304 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 39, 2024-09-12T01:37:07.304 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.235001443, 2024-09-12T01:37:07.304 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.006025678 2024-09-12T01:37:07.305 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.305 INFO:teuthology.orchestra.run.smithi104.stdout: "op_inflight": 0, 2024-09-12T01:37:07.305 INFO:teuthology.orchestra.run.smithi104.stdout: "oplen_avg": { 2024-09-12T01:37:07.305 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 39, 2024-09-12T01:37:07.305 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 63 2024-09-12T01:37:07.305 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.305 INFO:teuthology.orchestra.run.smithi104.stdout: "op": 39, 2024-09-12T01:37:07.305 INFO:teuthology.orchestra.run.smithi104.stdout: "op_r": 0, 2024-09-12T01:37:07.305 INFO:teuthology.orchestra.run.smithi104.stdout: "op_w": 39, 2024-09-12T01:37:07.305 INFO:teuthology.orchestra.run.smithi104.stdout: "op_rmw": 0, 2024-09-12T01:37:07.305 INFO:teuthology.orchestra.run.smithi104.stdout: "op_pg": 0, 2024-09-12T01:37:07.305 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_stat": 0, 2024-09-12T01:37:07.305 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_create": 10, 2024-09-12T01:37:07.305 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_read": 0, 2024-09-12T01:37:07.306 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_write": 2, 2024-09-12T01:37:07.306 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writefull": 6, 2024-09-12T01:37:07.306 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writesame": 0, 2024-09-12T01:37:07.306 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_append": 0, 2024-09-12T01:37:07.306 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_zero": 0, 2024-09-12T01:37:07.306 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_truncate": 0, 2024-09-12T01:37:07.306 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_delete": 6, 2024-09-12T01:37:07.306 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_mapext": 0, 2024-09-12T01:37:07.306 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_sparse_read": 0, 2024-09-12T01:37:07.306 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_clonerange": 0, 2024-09-12T01:37:07.306 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_getxattr": 0, 2024-09-12T01:37:07.306 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_setxattr": 20, 2024-09-12T01:37:07.306 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_cmpxattr": 0, 2024-09-12T01:37:07.306 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_rmxattr": 0, 2024-09-12T01:37:07.307 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_resetxattrs": 0, 2024-09-12T01:37:07.307 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_call": 0, 2024-09-12T01:37:07.307 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_watch": 0, 2024-09-12T01:37:07.307 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_notify": 0, 2024-09-12T01:37:07.307 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T01:37:07.307 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls": 0, 2024-09-12T01:37:07.307 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls_filter": 0, 2024-09-12T01:37:07.307 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_other": 0, 2024-09-12T01:37:07.307 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_active": 0, 2024-09-12T01:37:07.307 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_send": 0, 2024-09-12T01:37:07.307 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_resend": 0, 2024-09-12T01:37:07.307 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_ping": 0, 2024-09-12T01:37:07.307 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_active": 0, 2024-09-12T01:37:07.307 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_send": 0, 2024-09-12T01:37:07.307 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_resend": 0, 2024-09-12T01:37:07.308 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_active": 0, 2024-09-12T01:37:07.308 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_send": 0, 2024-09-12T01:37:07.308 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_resend": 0, 2024-09-12T01:37:07.308 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_active": 0, 2024-09-12T01:37:07.308 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_send": 0, 2024-09-12T01:37:07.308 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_resend": 0, 2024-09-12T01:37:07.308 INFO:teuthology.orchestra.run.smithi104.stdout: "command_active": 0, 2024-09-12T01:37:07.308 INFO:teuthology.orchestra.run.smithi104.stdout: "command_send": 0, 2024-09-12T01:37:07.308 INFO:teuthology.orchestra.run.smithi104.stdout: "command_resend": 0, 2024-09-12T01:37:07.308 INFO:teuthology.orchestra.run.smithi104.stdout: "map_epoch": 98, 2024-09-12T01:37:07.308 INFO:teuthology.orchestra.run.smithi104.stdout: "map_full": 0, 2024-09-12T01:37:07.308 INFO:teuthology.orchestra.run.smithi104.stdout: "map_inc": 2, 2024-09-12T01:37:07.308 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_sessions": 10, 2024-09-12T01:37:07.308 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_open": 10, 2024-09-12T01:37:07.308 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_close": 0, 2024-09-12T01:37:07.309 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_laggy": 0, 2024-09-12T01:37:07.309 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_wr": 18, 2024-09-12T01:37:07.309 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_rd": 0, 2024-09-12T01:37:07.309 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_del": 1 2024-09-12T01:37:07.309 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.309 INFO:teuthology.orchestra.run.smithi104.stdout: "oft": { 2024-09-12T01:37:07.309 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_objs": 1, 2024-09-12T01:37:07.309 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_kv_pairs": 0, 2024-09-12T01:37:07.309 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_updates": 0, 2024-09-12T01:37:07.309 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_removes": 0 2024-09-12T01:37:07.309 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.309 INFO:teuthology.orchestra.run.smithi104.stdout: "osd-slow-ops": { 2024-09-12T01:37:07.309 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_ops_count": 0 2024-09-12T01:37:07.309 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.310 INFO:teuthology.orchestra.run.smithi104.stdout: "purge_queue": { 2024-09-12T01:37:07.310 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops": 0, 2024-09-12T01:37:07.310 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T01:37:07.310 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing": 0, 2024-09-12T01:37:07.310 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_high_water": 0, 2024-09-12T01:37:07.310 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed_ops": 0, 2024-09-12T01:37:07.310 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed": 0, 2024-09-12T01:37:07.310 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_item_in_journal": 0 2024-09-12T01:37:07.310 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.310 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T01:37:07.310 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-12T01:37:07.310 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-09-12T01:37:07.310 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-12T01:37:07.310 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 742, 2024-09-12T01:37:07.310 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 996575, 2024-09-12T01:37:07.311 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-12T01:37:07.311 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 742, 2024-09-12T01:37:07.311 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-09-12T01:37:07.311 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-09-12T01:37:07.311 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 742, 2024-09-12T01:37:07.311 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 996575, 2024-09-12T01:37:07.311 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-12T01:37:07.311 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:07.311 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:07.311 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:07.311 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-12T01:37:07.311 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.311 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_bytes": { 2024-09-12T01:37:07.311 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-12T01:37:07.311 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-09-12T01:37:07.312 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-12T01:37:07.312 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-09-12T01:37:07.312 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-09-12T01:37:07.312 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-12T01:37:07.312 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-09-12T01:37:07.312 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 39, 2024-09-12T01:37:07.312 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 11516, 2024-09-12T01:37:07.312 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 39, 2024-09-12T01:37:07.312 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 11516, 2024-09-12T01:37:07.312 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-12T01:37:07.312 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:07.312 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:07.312 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:07.312 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-12T01:37:07.313 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.313 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_ops": { 2024-09-12T01:37:07.313 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-12T01:37:07.313 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 1024, 2024-09-12T01:37:07.313 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-12T01:37:07.313 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-09-12T01:37:07.313 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-09-12T01:37:07.313 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-12T01:37:07.313 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-09-12T01:37:07.313 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 39, 2024-09-12T01:37:07.313 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 39, 2024-09-12T01:37:07.313 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 39, 2024-09-12T01:37:07.313 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 39, 2024-09-12T01:37:07.314 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-12T01:37:07.314 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:07.314 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:07.314 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:07.314 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-12T01:37:07.314 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.314 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle": { 2024-09-12T01:37:07.314 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-12T01:37:07.314 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-09-12T01:37:07.314 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-12T01:37:07.314 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-09-12T01:37:07.314 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-09-12T01:37:07.314 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-12T01:37:07.314 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-09-12T01:37:07.314 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-09-12T01:37:07.315 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-09-12T01:37:07.315 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 0, 2024-09-12T01:37:07.315 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 0, 2024-09-12T01:37:07.315 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-12T01:37:07.315 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:07.315 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:07.315 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:07.315 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-12T01:37:07.315 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:07.315 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle-0x55c4e9c0e500": { 2024-09-12T01:37:07.315 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-12T01:37:07.315 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-09-12T01:37:07.315 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-12T01:37:07.315 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 3, 2024-09-12T01:37:07.315 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 899, 2024-09-12T01:37:07.316 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-12T01:37:07.316 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 3, 2024-09-12T01:37:07.316 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-09-12T01:37:07.316 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-09-12T01:37:07.316 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 2, 2024-09-12T01:37:07.316 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 899, 2024-09-12T01:37:07.316 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-12T01:37:07.316 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:07.316 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:07.316 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:07.316 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-12T01:37:07.316 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-12T01:37:07.316 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-12T01:37:08.156 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-09-12T01:37:08.157 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-09-12T01:37:08.157 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2024-09-12T01:37:08.157 DEBUG:tasks.check_counter:Getting stats from l 2024-09-12T01:37:08.157 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-12T01:37:08.400 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:37:08.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:08 smithi053 ceph-mon[35158]: pgmap v624: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-12T01:37:08.797 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:08 smithi104 ceph-mon[35219]: pgmap v624: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-12T01:37:08.900 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-12T01:37:08.900 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T01:37:08.900 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 584, 2024-09-12T01:37:08.900 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 584, 2024-09-12T01:37:08.900 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 30185599, 2024-09-12T01:37:08.900 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 184952, 2024-09-12T01:37:08.901 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 7, 2024-09-12T01:37:08.901 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 3, 2024-09-12T01:37:08.901 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.064680056, 2024-09-12T01:37:08.901 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.013307475, 2024-09-12T01:37:08.901 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.032692248, 2024-09-12T01:37:08.901 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.013824361, 2024-09-12T01:37:08.901 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:37:08.901 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 583, 2024-09-12T01:37:08.901 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.023827943, 2024-09-12T01:37:08.901 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000040871 2024-09-12T01:37:08.901 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.901 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:37:08.902 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:08.902 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:08.902 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:08.902 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.902 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-12T01:37:08.902 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T01:37:08.902 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.902 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T01:37:08.902 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 578, 2024-09-12T01:37:08.902 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 714, 2024-09-12T01:37:08.902 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 37902272, 2024-09-12T01:37:08.902 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 476403, 2024-09-12T01:37:08.902 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 6, 2024-09-12T01:37:08.903 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 5, 2024-09-12T01:37:08.903 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.068993905, 2024-09-12T01:37:08.903 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.019492239, 2024-09-12T01:37:08.903 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.029728169, 2024-09-12T01:37:08.903 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.014822465, 2024-09-12T01:37:08.903 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:37:08.903 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 712, 2024-09-12T01:37:08.903 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.024278606, 2024-09-12T01:37:08.903 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000034099 2024-09-12T01:37:08.903 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.903 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:37:08.903 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:08.903 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:08.903 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:08.904 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.904 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-12T01:37:08.904 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 292912 2024-09-12T01:37:08.904 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.904 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T01:37:08.904 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 1076, 2024-09-12T01:37:08.904 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 1060, 2024-09-12T01:37:08.904 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 26272398, 2024-09-12T01:37:08.904 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 389408, 2024-09-12T01:37:08.904 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 5, 2024-09-12T01:37:08.904 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 4, 2024-09-12T01:37:08.904 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.119797335, 2024-09-12T01:37:08.904 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.034250990, 2024-09-12T01:37:08.904 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.060298773, 2024-09-12T01:37:08.905 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.025144290, 2024-09-12T01:37:08.905 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:37:08.905 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1059, 2024-09-12T01:37:08.905 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.044517523, 2024-09-12T01:37:08.905 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000042037 2024-09-12T01:37:08.905 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.905 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:37:08.905 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:08.905 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:08.905 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:08.905 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.905 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 835632, 2024-09-12T01:37:08.905 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 112112 2024-09-12T01:37:08.906 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.906 INFO:teuthology.orchestra.run.smithi104.stdout: "cct": { 2024-09-12T01:37:08.906 INFO:teuthology.orchestra.run.smithi104.stdout: "total_workers": 1, 2024-09-12T01:37:08.906 INFO:teuthology.orchestra.run.smithi104.stdout: "unhealthy_workers": 0 2024-09-12T01:37:08.906 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.906 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-MDSRank": { 2024-09-12T01:37:08.906 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-09-12T01:37:08.906 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-09-12T01:37:08.906 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 3430, 2024-09-12T01:37:08.906 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.078365596, 2024-09-12T01:37:08.906 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000022847 2024-09-12T01:37:08.906 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-12T01:37:08.906 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.906 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-PurgeQueue": { 2024-09-12T01:37:08.906 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-09-12T01:37:08.907 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-09-12T01:37:08.907 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:08.907 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:08.907 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:08.907 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-12T01:37:08.907 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.907 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": { 2024-09-12T01:37:08.907 INFO:teuthology.orchestra.run.smithi104.stdout: "request": 0, 2024-09-12T01:37:08.907 INFO:teuthology.orchestra.run.smithi104.stdout: "reply": 0, 2024-09-12T01:37:08.907 INFO:teuthology.orchestra.run.smithi104.stdout: "reply_latency": { 2024-09-12T01:37:08.907 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:08.907 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:08.907 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:08.907 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.907 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_reply": 0, 2024-09-12T01:37:08.908 INFO:teuthology.orchestra.run.smithi104.stdout: "forward": 0, 2024-09-12T01:37:08.908 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_complete": 0, 2024-09-12T01:37:08.908 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_keys": 0, 2024-09-12T01:37:08.908 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_commit": 0, 2024-09-12T01:37:08.908 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_split": 0, 2024-09-12T01:37:08.908 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_merge": 0, 2024-09-12T01:37:08.908 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes": 7617, 2024-09-12T01:37:08.908 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_top": 179, 2024-09-12T01:37:08.908 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_bottom": 7438, 2024-09-12T01:37:08.908 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pin_tail": 0, 2024-09-12T01:37:08.908 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pinned": 7360, 2024-09-12T01:37:08.908 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_expired": 0, 2024-09-12T01:37:08.908 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_with_caps": 0, 2024-09-12T01:37:08.909 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": 0, 2024-09-12T01:37:08.909 INFO:teuthology.orchestra.run.smithi104.stdout: "subtrees": 1, 2024-09-12T01:37:08.909 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse": 0, 2024-09-12T01:37:08.909 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_hit": 0, 2024-09-12T01:37:08.909 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_forward": 0, 2024-09-12T01:37:08.909 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_discover": 0, 2024-09-12T01:37:08.909 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_dir_fetch": 0, 2024-09-12T01:37:08.909 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_remote_ino": 0, 2024-09-12T01:37:08.909 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_lock": 0, 2024-09-12T01:37:08.909 INFO:teuthology.orchestra.run.smithi104.stdout: "load_cent": 0, 2024-09-12T01:37:08.909 INFO:teuthology.orchestra.run.smithi104.stdout: "q": 0, 2024-09-12T01:37:08.909 INFO:teuthology.orchestra.run.smithi104.stdout: "exported": 0, 2024-09-12T01:37:08.909 INFO:teuthology.orchestra.run.smithi104.stdout: "exported_inodes": 0, 2024-09-12T01:37:08.909 INFO:teuthology.orchestra.run.smithi104.stdout: "imported": 0, 2024-09-12T01:37:08.910 INFO:teuthology.orchestra.run.smithi104.stdout: "imported_inodes": 0, 2024-09-12T01:37:08.910 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_dir_fetch": 0, 2024-09-12T01:37:08.910 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_backtrace_fetch": 0, 2024-09-12T01:37:08.910 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_peer_discover": 0, 2024-09-12T01:37:08.910 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rfiles": 0, 2024-09-12T01:37:08.910 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rbytes": 0, 2024-09-12T01:37:08.910 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rsnaps": 0, 2024-09-12T01:37:08.910 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T01:37:08.910 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_set_tag": 0, 2024-09-12T01:37:08.910 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T01:37:08.910 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_inotable_repaired": 0, 2024-09-12T01:37:08.910 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_inodes": 0, 2024-09-12T01:37:08.910 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T01:37:08.910 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T01:37:08.911 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_file_inodes": 0, 2024-09-12T01:37:08.911 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_inode_file_caps": 0, 2024-09-12T01:37:08.911 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T01:37:08.911 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_grant": 0, 2024-09-12T01:37:08.911 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T01:37:08.911 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T01:37:08.911 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T01:37:08.911 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps": 0, 2024-09-12T01:37:08.911 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps_dirty": 0, 2024-09-12T01:37:08.911 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_cap_release": 0, 2024-09-12T01:37:08.911 INFO:teuthology.orchestra.run.smithi104.stdout: "process_request_cap_release": 0 2024-09-12T01:37:08.911 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.911 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_cache": { 2024-09-12T01:37:08.911 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update": 0, 2024-09-12T01:37:08.912 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update_receipt": 0, 2024-09-12T01:37:08.912 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_try_discover": 0, 2024-09-12T01:37:08.912 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_send_discover": 0, 2024-09-12T01:37:08.912 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_handle_discover": 0, 2024-09-12T01:37:08.912 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays": 0, 2024-09-12T01:37:08.912 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_delayed": 0, 2024-09-12T01:37:08.912 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_enqueuing": 0, 2024-09-12T01:37:08.912 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_created": 0, 2024-09-12T01:37:08.912 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_enqueued": 0, 2024-09-12T01:37:08.912 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_reintegrated": 0, 2024-09-12T01:37:08.912 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_migrated": 0, 2024-09-12T01:37:08.912 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_processing": 0, 2024-09-12T01:37:08.912 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_enqueued": 0, 2024-09-12T01:37:08.912 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_prioritized": 0, 2024-09-12T01:37:08.913 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_started": 0, 2024-09-12T01:37:08.913 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_completed": 0, 2024-09-12T01:37:08.913 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_path": 0, 2024-09-12T01:37:08.913 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_inode": 0, 2024-09-12T01:37:08.913 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T01:37:08.913 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_exportdir": 0, 2024-09-12T01:37:08.913 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_flush": 0, 2024-09-12T01:37:08.913 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragmentdir": 0, 2024-09-12T01:37:08.913 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragstats": 0, 2024-09-12T01:37:08.913 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_inodestats": 0 2024-09-12T01:37:08.913 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.913 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_log": { 2024-09-12T01:37:08.913 INFO:teuthology.orchestra.run.smithi104.stdout: "evlrg": 0, 2024-09-12T01:37:08.913 INFO:teuthology.orchestra.run.smithi104.stdout: "evadd": 0, 2024-09-12T01:37:08.913 INFO:teuthology.orchestra.run.smithi104.stdout: "evex": 0, 2024-09-12T01:37:08.914 INFO:teuthology.orchestra.run.smithi104.stdout: "evtrm": 0, 2024-09-12T01:37:08.914 INFO:teuthology.orchestra.run.smithi104.stdout: "ev": 57392, 2024-09-12T01:37:08.914 INFO:teuthology.orchestra.run.smithi104.stdout: "evexg": 0, 2024-09-12T01:37:08.914 INFO:teuthology.orchestra.run.smithi104.stdout: "evexd": 0, 2024-09-12T01:37:08.914 INFO:teuthology.orchestra.run.smithi104.stdout: "segadd": 0, 2024-09-12T01:37:08.914 INFO:teuthology.orchestra.run.smithi104.stdout: "segex": 0, 2024-09-12T01:37:08.914 INFO:teuthology.orchestra.run.smithi104.stdout: "segtrm": 0, 2024-09-12T01:37:08.914 INFO:teuthology.orchestra.run.smithi104.stdout: "seg": 72, 2024-09-12T01:37:08.914 INFO:teuthology.orchestra.run.smithi104.stdout: "segmjr": 6, 2024-09-12T01:37:08.914 INFO:teuthology.orchestra.run.smithi104.stdout: "segexg": 0, 2024-09-12T01:37:08.914 INFO:teuthology.orchestra.run.smithi104.stdout: "segexd": 0, 2024-09-12T01:37:08.914 INFO:teuthology.orchestra.run.smithi104.stdout: "expos": 4194304, 2024-09-12T01:37:08.914 INFO:teuthology.orchestra.run.smithi104.stdout: "wrpos": 97136016, 2024-09-12T01:37:08.914 INFO:teuthology.orchestra.run.smithi104.stdout: "rdpos": 97134878, 2024-09-12T01:37:08.915 INFO:teuthology.orchestra.run.smithi104.stdout: "jlat": { 2024-09-12T01:37:08.915 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:08.915 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:08.915 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:08.915 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.915 INFO:teuthology.orchestra.run.smithi104.stdout: "replayed": 57392 2024-09-12T01:37:08.915 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.915 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_mem": { 2024-09-12T01:37:08.915 INFO:teuthology.orchestra.run.smithi104.stdout: "ino": 17, 2024-09-12T01:37:08.915 INFO:teuthology.orchestra.run.smithi104.stdout: "ino+": 7276, 2024-09-12T01:37:08.915 INFO:teuthology.orchestra.run.smithi104.stdout: "ino-": 7259, 2024-09-12T01:37:08.915 INFO:teuthology.orchestra.run.smithi104.stdout: "dir": 15, 2024-09-12T01:37:08.915 INFO:teuthology.orchestra.run.smithi104.stdout: "dir+": 250, 2024-09-12T01:37:08.915 INFO:teuthology.orchestra.run.smithi104.stdout: "dir-": 235, 2024-09-12T01:37:08.916 INFO:teuthology.orchestra.run.smithi104.stdout: "dn": 7617, 2024-09-12T01:37:08.916 INFO:teuthology.orchestra.run.smithi104.stdout: "dn+": 12337, 2024-09-12T01:37:08.916 INFO:teuthology.orchestra.run.smithi104.stdout: "dn-": 4720, 2024-09-12T01:37:08.916 INFO:teuthology.orchestra.run.smithi104.stdout: "cap": 0, 2024-09-12T01:37:08.916 INFO:teuthology.orchestra.run.smithi104.stdout: "cap+": 0, 2024-09-12T01:37:08.916 INFO:teuthology.orchestra.run.smithi104.stdout: "cap-": 0, 2024-09-12T01:37:08.916 INFO:teuthology.orchestra.run.smithi104.stdout: "rss": 69048, 2024-09-12T01:37:08.916 INFO:teuthology.orchestra.run.smithi104.stdout: "heap": 215328 2024-09-12T01:37:08.916 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.916 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_server": { 2024-09-12T01:37:08.916 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_client_request": 0, 2024-09-12T01:37:08.916 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_server_request": 0, 2024-09-12T01:37:08.916 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_request": 0, 2024-09-12T01:37:08.916 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_session": 0, 2024-09-12T01:37:08.917 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_peer_request": 0, 2024-09-12T01:37:08.917 INFO:teuthology.orchestra.run.smithi104.stdout: "req_create_latency": { 2024-09-12T01:37:08.917 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:08.917 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:08.917 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:08.917 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.917 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getattr_latency": { 2024-09-12T01:37:08.917 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:08.917 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:08.917 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:08.918 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.918 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getfilelock_latency": { 2024-09-12T01:37:08.918 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:08.918 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:08.918 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:08.918 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.918 INFO:teuthology.orchestra.run.smithi104.stdout: "req_link_latency": { 2024-09-12T01:37:08.918 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:08.918 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:08.918 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:08.918 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.918 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookup_latency": { 2024-09-12T01:37:08.918 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:08.918 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:08.919 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:08.919 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.919 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookuphash_latency": { 2024-09-12T01:37:08.919 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:08.919 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:08.919 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:08.919 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.919 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupino_latency": { 2024-09-12T01:37:08.919 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:08.919 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:08.919 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:08.919 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.919 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupname_latency": { 2024-09-12T01:37:08.920 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:08.920 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:08.920 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:08.920 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.920 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupparent_latency": { 2024-09-12T01:37:08.920 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:08.920 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:08.920 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:08.920 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.920 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupsnap_latency": { 2024-09-12T01:37:08.920 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:08.920 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:08.920 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:08.920 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.921 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lssnap_latency": { 2024-09-12T01:37:08.921 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:08.921 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:08.921 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:08.921 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.921 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mkdir_latency": { 2024-09-12T01:37:08.922 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:08.922 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:08.922 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:08.922 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.922 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mknod_latency": { 2024-09-12T01:37:08.922 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:08.922 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:08.922 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:08.922 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.922 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mksnap_latency": { 2024-09-12T01:37:08.922 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:08.923 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:08.923 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:08.923 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.923 INFO:teuthology.orchestra.run.smithi104.stdout: "req_open_latency": { 2024-09-12T01:37:08.923 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:08.923 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:08.923 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:08.923 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.923 INFO:teuthology.orchestra.run.smithi104.stdout: "req_readdir_latency": { 2024-09-12T01:37:08.923 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:08.923 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:08.923 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:08.923 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.924 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rename_latency": { 2024-09-12T01:37:08.924 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:08.924 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:08.924 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:08.924 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.924 INFO:teuthology.orchestra.run.smithi104.stdout: "req_renamesnap_latency": { 2024-09-12T01:37:08.924 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:08.924 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:08.924 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:08.924 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.924 INFO:teuthology.orchestra.run.smithi104.stdout: "req_snapdiff_latency": { 2024-09-12T01:37:08.924 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:08.924 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:08.924 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:08.925 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.925 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmdir_latency": { 2024-09-12T01:37:08.925 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:08.925 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:08.925 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:08.925 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.925 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmsnap_latency": { 2024-09-12T01:37:08.925 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:08.925 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:08.925 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:08.925 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.925 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmxattr_latency": { 2024-09-12T01:37:08.925 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:08.926 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:08.926 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:08.926 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.926 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setattr_latency": { 2024-09-12T01:37:08.926 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:08.926 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:08.926 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:08.926 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.926 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setdirlayout_latency": { 2024-09-12T01:37:08.926 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:08.926 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:08.926 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:08.926 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.926 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setfilelock_latency": { 2024-09-12T01:37:08.927 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:08.927 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:08.927 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:08.927 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.927 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setlayout_latency": { 2024-09-12T01:37:08.927 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:08.927 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:08.927 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:08.927 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.927 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setxattr_latency": { 2024-09-12T01:37:08.927 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:08.927 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:08.927 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:08.928 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.928 INFO:teuthology.orchestra.run.smithi104.stdout: "req_symlink_latency": { 2024-09-12T01:37:08.928 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:08.928 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:08.928 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:08.928 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.928 INFO:teuthology.orchestra.run.smithi104.stdout: "req_unlink_latency": { 2024-09-12T01:37:08.928 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:08.928 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:08.928 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:08.928 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.928 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_revoke_eviction": 0, 2024-09-12T01:37:08.928 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_acquisition_throttle": 0, 2024-09-12T01:37:08.928 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getvxattr_latency": { 2024-09-12T01:37:08.929 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:08.929 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:08.929 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:08.929 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-12T01:37:08.929 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.929 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_sessions": { 2024-09-12T01:37:08.929 INFO:teuthology.orchestra.run.smithi104.stdout: "session_count": 1, 2024-09-12T01:37:08.929 INFO:teuthology.orchestra.run.smithi104.stdout: "session_add": 3, 2024-09-12T01:37:08.929 INFO:teuthology.orchestra.run.smithi104.stdout: "session_remove": 2, 2024-09-12T01:37:08.929 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_open": 2, 2024-09-12T01:37:08.929 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_stale": 0, 2024-09-12T01:37:08.929 INFO:teuthology.orchestra.run.smithi104.stdout: "total_load": 0, 2024-09-12T01:37:08.929 INFO:teuthology.orchestra.run.smithi104.stdout: "average_load": 0, 2024-09-12T01:37:08.930 INFO:teuthology.orchestra.run.smithi104.stdout: "avg_session_uptime": 3334, 2024-09-12T01:37:08.930 INFO:teuthology.orchestra.run.smithi104.stdout: "mdthresh_evicted": 0 2024-09-12T01:37:08.930 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.930 INFO:teuthology.orchestra.run.smithi104.stdout: "mempool": { 2024-09-12T01:37:08.930 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_bytes": 0, 2024-09-12T01:37:08.930 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_items": 0, 2024-09-12T01:37:08.930 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T01:37:08.930 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_items": 0, 2024-09-12T01:37:08.930 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T01:37:08.930 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_items": 0, 2024-09-12T01:37:08.930 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T01:37:08.930 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T01:37:08.930 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T01:37:08.930 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T01:37:08.931 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T01:37:08.931 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_items": 0, 2024-09-12T01:37:08.931 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T01:37:08.931 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T01:37:08.931 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_bytes": 0, 2024-09-12T01:37:08.931 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_items": 0, 2024-09-12T01:37:08.931 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_bytes": 0, 2024-09-12T01:37:08.931 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_items": 0, 2024-09-12T01:37:08.931 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T01:37:08.931 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T01:37:08.931 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T01:37:08.931 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T01:37:08.931 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T01:37:08.931 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_items": 0, 2024-09-12T01:37:08.931 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_bytes": 0, 2024-09-12T01:37:08.932 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_items": 0, 2024-09-12T01:37:08.932 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T01:37:08.932 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T01:37:08.932 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_bytes": 0, 2024-09-12T01:37:08.932 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_items": 0, 2024-09-12T01:37:08.932 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_bytes": 0, 2024-09-12T01:37:08.932 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_items": 0, 2024-09-12T01:37:08.932 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T01:37:08.932 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_items": 0, 2024-09-12T01:37:08.932 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T01:37:08.932 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_items": 0, 2024-09-12T01:37:08.932 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_bytes": 222635, 2024-09-12T01:37:08.932 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_items": 72, 2024-09-12T01:37:08.932 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_bytes": 0, 2024-09-12T01:37:08.932 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_items": 0, 2024-09-12T01:37:08.933 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_bytes": 0, 2024-09-12T01:37:08.933 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_items": 0, 2024-09-12T01:37:08.933 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_bytes": 0, 2024-09-12T01:37:08.933 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_items": 0, 2024-09-12T01:37:08.933 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_bytes": 0, 2024-09-12T01:37:08.933 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_items": 0, 2024-09-12T01:37:08.933 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_bytes": 8856, 2024-09-12T01:37:08.933 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_items": 166, 2024-09-12T01:37:08.933 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T01:37:08.933 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_items": 0, 2024-09-12T01:37:08.933 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_bytes": 0, 2024-09-12T01:37:08.933 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_items": 0, 2024-09-12T01:37:08.933 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_bytes": 5417287, 2024-09-12T01:37:08.934 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_items": 30866, 2024-09-12T01:37:08.934 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_bytes": 0, 2024-09-12T01:37:08.934 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_items": 0, 2024-09-12T01:37:08.934 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_bytes": 0, 2024-09-12T01:37:08.934 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_items": 0 2024-09-12T01:37:08.934 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.934 INFO:teuthology.orchestra.run.smithi104.stdout: "objecter": { 2024-09-12T01:37:08.934 INFO:teuthology.orchestra.run.smithi104.stdout: "op_active": 0, 2024-09-12T01:37:08.934 INFO:teuthology.orchestra.run.smithi104.stdout: "op_laggy": 0, 2024-09-12T01:37:08.934 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send": 2029, 2024-09-12T01:37:08.934 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send_bytes": 21, 2024-09-12T01:37:08.935 INFO:teuthology.orchestra.run.smithi104.stdout: "op_resend": 3, 2024-09-12T01:37:08.936 INFO:teuthology.orchestra.run.smithi104.stdout: "op_reply": 2029, 2024-09-12T01:37:08.936 INFO:teuthology.orchestra.run.smithi104.stdout: "op_latency": { 2024-09-12T01:37:08.936 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 2029, 2024-09-12T01:37:08.936 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 1.767010889, 2024-09-12T01:37:08.936 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000870877 2024-09-12T01:37:08.936 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.936 INFO:teuthology.orchestra.run.smithi104.stdout: "op_inflight": 0, 2024-09-12T01:37:08.936 INFO:teuthology.orchestra.run.smithi104.stdout: "oplen_avg": { 2024-09-12T01:37:08.936 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 2029, 2024-09-12T01:37:08.936 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 2030 2024-09-12T01:37:08.936 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.936 INFO:teuthology.orchestra.run.smithi104.stdout: "op": 2029, 2024-09-12T01:37:08.937 INFO:teuthology.orchestra.run.smithi104.stdout: "op_r": 2029, 2024-09-12T01:37:08.937 INFO:teuthology.orchestra.run.smithi104.stdout: "op_w": 0, 2024-09-12T01:37:08.937 INFO:teuthology.orchestra.run.smithi104.stdout: "op_rmw": 0, 2024-09-12T01:37:08.937 INFO:teuthology.orchestra.run.smithi104.stdout: "op_pg": 0, 2024-09-12T01:37:08.937 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_stat": 1117, 2024-09-12T01:37:08.937 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_create": 0, 2024-09-12T01:37:08.937 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_read": 910, 2024-09-12T01:37:08.937 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_write": 0, 2024-09-12T01:37:08.937 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writefull": 0, 2024-09-12T01:37:08.937 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writesame": 0, 2024-09-12T01:37:08.937 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_append": 0, 2024-09-12T01:37:08.937 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_zero": 0, 2024-09-12T01:37:08.937 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_truncate": 0, 2024-09-12T01:37:08.937 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_delete": 0, 2024-09-12T01:37:08.938 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_mapext": 0, 2024-09-12T01:37:08.938 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_sparse_read": 0, 2024-09-12T01:37:08.938 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_clonerange": 0, 2024-09-12T01:37:08.938 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_getxattr": 1, 2024-09-12T01:37:08.938 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_setxattr": 0, 2024-09-12T01:37:08.938 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_cmpxattr": 0, 2024-09-12T01:37:08.938 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_rmxattr": 0, 2024-09-12T01:37:08.938 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_resetxattrs": 0, 2024-09-12T01:37:08.938 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_call": 0, 2024-09-12T01:37:08.938 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_watch": 0, 2024-09-12T01:37:08.938 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_notify": 0, 2024-09-12T01:37:08.938 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T01:37:08.938 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls": 0, 2024-09-12T01:37:08.938 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls_filter": 0, 2024-09-12T01:37:08.939 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_other": 0, 2024-09-12T01:37:08.939 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_active": 0, 2024-09-12T01:37:08.939 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_send": 0, 2024-09-12T01:37:08.939 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_resend": 0, 2024-09-12T01:37:08.939 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_ping": 0, 2024-09-12T01:37:08.939 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_active": 0, 2024-09-12T01:37:08.939 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_send": 0, 2024-09-12T01:37:08.939 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_resend": 0, 2024-09-12T01:37:08.939 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_active": 0, 2024-09-12T01:37:08.939 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_send": 0, 2024-09-12T01:37:08.939 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_resend": 0, 2024-09-12T01:37:08.939 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_active": 0, 2024-09-12T01:37:08.939 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_send": 0, 2024-09-12T01:37:08.939 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_resend": 0, 2024-09-12T01:37:08.940 INFO:teuthology.orchestra.run.smithi104.stdout: "command_active": 0, 2024-09-12T01:37:08.940 INFO:teuthology.orchestra.run.smithi104.stdout: "command_send": 0, 2024-09-12T01:37:08.940 INFO:teuthology.orchestra.run.smithi104.stdout: "command_resend": 0, 2024-09-12T01:37:08.940 INFO:teuthology.orchestra.run.smithi104.stdout: "map_epoch": 98, 2024-09-12T01:37:08.940 INFO:teuthology.orchestra.run.smithi104.stdout: "map_full": 0, 2024-09-12T01:37:08.940 INFO:teuthology.orchestra.run.smithi104.stdout: "map_inc": 2, 2024-09-12T01:37:08.940 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_sessions": 9, 2024-09-12T01:37:08.940 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_open": 9, 2024-09-12T01:37:08.940 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_close": 0, 2024-09-12T01:37:08.940 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_laggy": 0, 2024-09-12T01:37:08.940 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_wr": 0, 2024-09-12T01:37:08.940 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_rd": 2, 2024-09-12T01:37:08.940 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_del": 0 2024-09-12T01:37:08.940 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.940 INFO:teuthology.orchestra.run.smithi104.stdout: "oft": { 2024-09-12T01:37:08.941 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_objs": 0, 2024-09-12T01:37:08.941 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_kv_pairs": 0, 2024-09-12T01:37:08.941 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_updates": 0, 2024-09-12T01:37:08.941 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_removes": 0 2024-09-12T01:37:08.941 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.941 INFO:teuthology.orchestra.run.smithi104.stdout: "osd-slow-ops": { 2024-09-12T01:37:08.941 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_ops_count": 0 2024-09-12T01:37:08.941 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.941 INFO:teuthology.orchestra.run.smithi104.stdout: "purge_queue": { 2024-09-12T01:37:08.941 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops": 0, 2024-09-12T01:37:08.941 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T01:37:08.941 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing": 0, 2024-09-12T01:37:08.941 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_high_water": 0, 2024-09-12T01:37:08.941 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed_ops": 0, 2024-09-12T01:37:08.942 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed": 0, 2024-09-12T01:37:08.942 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_item_in_journal": 0 2024-09-12T01:37:08.942 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.942 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T01:37:08.942 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-12T01:37:08.942 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-09-12T01:37:08.942 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-12T01:37:08.942 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 2234, 2024-09-12T01:37:08.942 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 94145773, 2024-09-12T01:37:08.942 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-12T01:37:08.942 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 2234, 2024-09-12T01:37:08.942 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-09-12T01:37:08.942 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-09-12T01:37:08.942 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 2234, 2024-09-12T01:37:08.943 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 94145773, 2024-09-12T01:37:08.943 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-12T01:37:08.943 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:08.943 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:08.943 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:08.943 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-12T01:37:08.943 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.943 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_bytes": { 2024-09-12T01:37:08.943 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-12T01:37:08.943 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-09-12T01:37:08.943 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-12T01:37:08.943 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-09-12T01:37:08.943 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-09-12T01:37:08.943 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-12T01:37:08.943 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-09-12T01:37:08.944 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 2029, 2024-09-12T01:37:08.944 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 92959113, 2024-09-12T01:37:08.944 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 2029, 2024-09-12T01:37:08.944 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 92959113, 2024-09-12T01:37:08.944 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-12T01:37:08.944 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:08.944 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:08.944 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:08.944 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-12T01:37:08.944 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.944 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_ops": { 2024-09-12T01:37:08.944 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-12T01:37:08.944 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 1024, 2024-09-12T01:37:08.944 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-12T01:37:08.945 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-09-12T01:37:08.945 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-09-12T01:37:08.945 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-12T01:37:08.945 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-09-12T01:37:08.945 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 2029, 2024-09-12T01:37:08.945 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 2029, 2024-09-12T01:37:08.945 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 2029, 2024-09-12T01:37:08.945 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 2029, 2024-09-12T01:37:08.945 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-12T01:37:08.945 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:08.945 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:08.945 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:08.945 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-12T01:37:08.945 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.946 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle": { 2024-09-12T01:37:08.946 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-12T01:37:08.946 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-09-12T01:37:08.946 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-12T01:37:08.946 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-09-12T01:37:08.946 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-09-12T01:37:08.946 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-12T01:37:08.946 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-09-12T01:37:08.946 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-09-12T01:37:08.946 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-09-12T01:37:08.946 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 0, 2024-09-12T01:37:08.946 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 0, 2024-09-12T01:37:08.946 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-12T01:37:08.946 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:08.946 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:08.947 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:08.947 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-12T01:37:08.947 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-12T01:37:08.947 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle-0x56331ab526e0": { 2024-09-12T01:37:08.947 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-12T01:37:08.947 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-09-12T01:37:08.947 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-12T01:37:08.947 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-09-12T01:37:08.947 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-09-12T01:37:08.947 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-12T01:37:08.947 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-09-12T01:37:08.947 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-09-12T01:37:08.947 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-09-12T01:37:08.947 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 0, 2024-09-12T01:37:08.948 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 0, 2024-09-12T01:37:08.948 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-12T01:37:08.948 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-12T01:37:08.948 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-12T01:37:08.948 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-12T01:37:08.948 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-12T01:37:08.948 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-12T01:37:08.948 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-12T01:37:08.949 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:08 smithi001 ceph-mon[30387]: pgmap v624: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-12T01:37:09.705 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-12T01:37:09.706 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-12T01:37:09.706 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=0 2024-09-12T01:37:09.706 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-12T01:37:09.716 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-12T01:37:09.716 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-12T01:37:09.716 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T01:37:09.757 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-12T01:37:09.757 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T01:37:09.836 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-12T01:37:10.260 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:37:10.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:10 smithi001 ceph-mon[30387]: pgmap v625: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-12T01:37:10.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:10 smithi053 ceph-mon[35158]: pgmap v625: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-12T01:37:10.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:10 smithi104 ceph-mon[35219]: pgmap v625: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-12T01:37:10.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:10.989+0000 7fd7dac00640 1 -- 172.21.15.1:0/2499451603 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7d4076720 msgr2=0x7fd7d4076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:10.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:10.989+0000 7fd7dac00640 1 --2- 172.21.15.1:0/2499451603 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7d4076720 0x7fd7d4076b00 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4009900 tx=0x7fd7c402a3f0 comp rx=0 tx=0).stop 2024-09-12T01:37:10.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:10.989+0000 7fd7dac00640 1 -- 172.21.15.1:0/2499451603 shutdown_connections 2024-09-12T01:37:10.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:10.989+0000 7fd7dac00640 1 --2- 172.21.15.1:0/2499451603 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7d4112a30 0x7fd7d4114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:10.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:10.989+0000 7fd7dac00640 1 --2- 172.21.15.1:0/2499451603 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7d4077040 0x7fd7d41124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:10.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:10.989+0000 7fd7dac00640 1 --2- 172.21.15.1:0/2499451603 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7d4076720 0x7fd7d4076b00 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:10.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:10.989+0000 7fd7dac00640 1 -- 172.21.15.1:0/2499451603 >> 172.21.15.1:0/2499451603 conn(0x7fd7d40ffa90 msgr2=0x7fd7d4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:10.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:10.990+0000 7fd7dac00640 1 -- 172.21.15.1:0/2499451603 shutdown_connections 2024-09-12T01:37:10.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:10.990+0000 7fd7dac00640 1 -- 172.21.15.1:0/2499451603 wait complete. 2024-09-12T01:37:10.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:10.990+0000 7fd7dac00640 1 Processor -- start 2024-09-12T01:37:10.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:10.991+0000 7fd7dac00640 1 -- start start 2024-09-12T01:37:10.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:10.991+0000 7fd7dac00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7d4076720 0x7fd7d41a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:10.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:10.991+0000 7fd7dac00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7d4077040 0x7fd7d41a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:10.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:10.991+0000 7fd7dac00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7d4112a30 0x7fd7d419e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:10.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:10.991+0000 7fd7dac00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d41185c0 con 0x7fd7d4077040 2024-09-12T01:37:10.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:10.991+0000 7fd7dac00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d4118440 con 0x7fd7d4112a30 2024-09-12T01:37:10.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:10.991+0000 7fd7dac00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d4118740 con 0x7fd7d4076720 2024-09-12T01:37:10.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:10.991+0000 7fd7d3400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7d4077040 0x7fd7d41a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:10.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:10.991+0000 7fd7d3e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7d4076720 0x7fd7d41a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:10.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:10.991+0000 7fd7d3400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7d4077040 0x7fd7d41a4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53320/0 (socket says 172.21.15.1:53320) 2024-09-12T01:37:10.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:10.991+0000 7fd7d3400640 1 -- 172.21.15.1:0/3186970400 learned_addr learned my addr 172.21.15.1:0/3186970400 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:37:10.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:10.992+0000 7fd7d8c00640 1 --2- 172.21.15.1:0/3186970400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7d4112a30 0x7fd7d419e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:10.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:10.992+0000 7fd7d3400640 1 -- 172.21.15.1:0/3186970400 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7d4076720 msgr2=0x7fd7d41a4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:10.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:10.992+0000 7fd7d3400640 1 --2- 172.21.15.1:0/3186970400 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7d4076720 0x7fd7d41a4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:10.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:10.992+0000 7fd7d3400640 1 -- 172.21.15.1:0/3186970400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7d4112a30 msgr2=0x7fd7d419e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:10.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:10.992+0000 7fd7d3400640 1 --2- 172.21.15.1:0/3186970400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7d4112a30 0x7fd7d419e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:10.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:10.992+0000 7fd7d3400640 1 -- 172.21.15.1:0/3186970400 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7d419f000 con 0x7fd7d4077040 2024-09-12T01:37:10.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:10.992+0000 7fd7d3e00640 1 --2- 172.21.15.1:0/3186970400 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7d4076720 0x7fd7d41a4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:37:10.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:10.992+0000 7fd7d3400640 1 --2- 172.21.15.1:0/3186970400 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7d4077040 0x7fd7d41a4c50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd7c000e9f0 tx=0x7fd7c000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:10.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:10.993+0000 7fd7d0c00640 1 -- 172.21.15.1:0/3186970400 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c000cd80 con 0x7fd7d4077040 2024-09-12T01:37:10.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:10.993+0000 7fd7d0c00640 1 -- 172.21.15.1:0/3186970400 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7c0005500 con 0x7fd7d4077040 2024-09-12T01:37:10.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:10.993+0000 7fd7dac00640 1 -- 172.21.15.1:0/3186970400 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7d419f2f0 con 0x7fd7d4077040 2024-09-12T01:37:10.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:10.993+0000 7fd7d0c00640 1 -- 172.21.15.1:0/3186970400 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c0010690 con 0x7fd7d4077040 2024-09-12T01:37:10.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:10.993+0000 7fd7dac00640 1 -- 172.21.15.1:0/3186970400 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7d41ab5e0 con 0x7fd7d4077040 2024-09-12T01:37:10.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:10.994+0000 7fd7ca000640 1 -- 172.21.15.1:0/3186970400 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7d4076b00 con 0x7fd7d4077040 2024-09-12T01:37:10.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:10.996+0000 7fd7d0c00640 1 -- 172.21.15.1:0/3186970400 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fd7c00046c0 con 0x7fd7d4077040 2024-09-12T01:37:10.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:10.997+0000 7fd7d0c00640 1 --2- 172.21.15.1:0/3186970400 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fd7b80a2c00 0x7fd7b80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:10.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:10.997+0000 7fd7d0c00640 1 -- 172.21.15.1:0/3186970400 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fd7c00cf800 con 0x7fd7d4077040 2024-09-12T01:37:10.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:10.998+0000 7fd7d3e00640 1 --2- 172.21.15.1:0/3186970400 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fd7b80a2c00 0x7fd7b80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:10.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:10.998+0000 7fd7d3e00640 1 --2- 172.21.15.1:0/3186970400 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fd7b80a2c00 0x7fd7b80a50c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd7d419fa10 tx=0x7fd7c40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:11.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:11.002+0000 7fd7d0c00640 1 -- 172.21.15.1:0/3186970400 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd7c009c370 con 0x7fd7d4077040 2024-09-12T01:37:11.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:11.201+0000 7fd7ca000640 1 -- 172.21.15.1:0/3186970400 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd7d4062df0 con 0x7fd7d4077040 2024-09-12T01:37:11.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:11.201+0000 7fd7d0c00640 1 -- 172.21.15.1:0/3186970400 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v98) ==== 81+0+580 (secure 0 0 0) 0x7fd7c009bc70 con 0x7fd7d4077040 2024-09-12T01:37:11.202 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/1373984726 2024-09-13T01:19:14.580637+0000 2024-09-12T01:37:11.202 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/1429963718 2024-09-13T01:19:14.580637+0000 2024-09-12T01:37:11.202 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6801/4101281615 2024-09-13T01:19:14.580637+0000 2024-09-12T01:37:11.202 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6800/4101281615 2024-09-13T01:19:14.580637+0000 2024-09-12T01:37:11.202 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/2564759586 2024-09-13T01:19:14.580637+0000 2024-09-12T01:37:11.202 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6801/2935968453 2024-09-13T01:18:45.882682+0000 2024-09-12T01:37:11.202 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6800/2935968453 2024-09-13T01:18:45.882682+0000 2024-09-12T01:37:11.202 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/1459848813 2024-09-13T01:18:45.882682+0000 2024-09-12T01:37:11.202 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/578605922 2024-09-13T01:18:45.882682+0000 2024-09-12T01:37:11.202 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/738035035 2024-09-13T01:18:45.882682+0000 2024-09-12T01:37:11.202 INFO:teuthology.orchestra.run.smithi001.stderr:listed 10 entries 2024-09-12T01:37:11.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:11.203+0000 7fd7ca000640 1 -- 172.21.15.1:0/3186970400 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fd7b80a2c00 msgr2=0x7fd7b80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:11.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:11.203+0000 7fd7ca000640 1 --2- 172.21.15.1:0/3186970400 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fd7b80a2c00 0x7fd7b80a50c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd7d419fa10 tx=0x7fd7c40058d0 comp rx=0 tx=0).stop 2024-09-12T01:37:11.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:11.203+0000 7fd7ca000640 1 -- 172.21.15.1:0/3186970400 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7d4077040 msgr2=0x7fd7d41a4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:11.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:11.203+0000 7fd7ca000640 1 --2- 172.21.15.1:0/3186970400 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7d4077040 0x7fd7d41a4c50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd7c000e9f0 tx=0x7fd7c000edf0 comp rx=0 tx=0).stop 2024-09-12T01:37:11.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:11.203+0000 7fd7ca000640 1 -- 172.21.15.1:0/3186970400 shutdown_connections 2024-09-12T01:37:11.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:11.203+0000 7fd7ca000640 1 --2- 172.21.15.1:0/3186970400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7d4112a30 0x7fd7d419e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:11.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:11.203+0000 7fd7ca000640 1 --2- 172.21.15.1:0/3186970400 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7d4077040 0x7fd7d41a4c50 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:11.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:11.203+0000 7fd7ca000640 1 --2- 172.21.15.1:0/3186970400 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7fd7b80a2c00 0x7fd7b80a50c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:11.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:11.203+0000 7fd7ca000640 1 --2- 172.21.15.1:0/3186970400 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7d4076720 0x7fd7d41a4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:11.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:11.203+0000 7fd7ca000640 1 -- 172.21.15.1:0/3186970400 >> 172.21.15.1:0/3186970400 conn(0x7fd7d40ffa90 msgr2=0x7fd7d4101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:11.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:11.204+0000 7fd7ca000640 1 -- 172.21.15.1:0/3186970400 shutdown_connections 2024-09-12T01:37:11.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:11.204+0000 7fd7ca000640 1 -- 172.21.15.1:0/3186970400 wait complete. 2024-09-12T01:37:11.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:11 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/3186970400' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T01:37:11.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:11 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/3186970400' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T01:37:11.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:11 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/3186970400' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T01:37:12.030 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-12T01:37:12.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:37:12.030 DEBUG:teuthology.orchestra.run.smithi001:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-12T01:37:12.147 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi001.front.sepia.ceph.com 2024-09-12T01:37:12.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:37:12.147 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-12T01:37:12.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:37:12.226 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-09-12T01:37:12.268 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-12T01:37:12.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:37:12.268 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T01:37:12.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:37:12.367 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link delete ceph-brx 2024-09-12T01:37:12.627 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:12 smithi001 ceph-mon[30387]: pgmap v626: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-12T01:37:12.672 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-12T01:37:12.682 INFO:tasks.cephadm:Teardown begin 2024-09-12T01:37:12.682 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-12T01:37:12.682 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:37:12.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:12 smithi104 ceph-mon[35219]: pgmap v626: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-12T01:37:12.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:12 smithi053 ceph-mon[35158]: pgmap v626: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-12T01:37:13.064 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:37:13.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.683+0000 7f495b200640 1 -- 172.21.15.1:0/2096002785 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4954106f10 msgr2=0x7f495410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:13.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.683+0000 7f495b200640 1 --2- 172.21.15.1:0/2096002785 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4954106f10 0x7f495410ea90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f494c009ca0 tx=0x7f494c02a8c0 comp rx=0 tx=0).stop 2024-09-12T01:37:13.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.683+0000 7f495b200640 1 -- 172.21.15.1:0/2096002785 shutdown_connections 2024-09-12T01:37:13.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.683+0000 7f495b200640 1 --2- 172.21.15.1:0/2096002785 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4954106f10 0x7f495410ea90 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:13.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.683+0000 7f495b200640 1 --2- 172.21.15.1:0/2096002785 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4954102790 0x7f49541067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:13.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.683+0000 7f495b200640 1 --2- 172.21.15.1:0/2096002785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4954101de0 0x7f49541021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:13.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.683+0000 7f495b200640 1 -- 172.21.15.1:0/2096002785 >> 172.21.15.1:0/2096002785 conn(0x7f49540fd8c0 msgr2=0x7f49540ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:13.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.683+0000 7f495b200640 1 -- 172.21.15.1:0/2096002785 shutdown_connections 2024-09-12T01:37:13.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.683+0000 7f495b200640 1 -- 172.21.15.1:0/2096002785 wait complete. 2024-09-12T01:37:13.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.684+0000 7f495b200640 1 Processor -- start 2024-09-12T01:37:13.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.684+0000 7f495b200640 1 -- start start 2024-09-12T01:37:13.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.685+0000 7f495b200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4954101de0 0x7f49540744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:13.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.685+0000 7f495b200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4954102790 0x7f4954074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:13.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.685+0000 7f495b200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4954106f10 0x7f4954074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:13.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.685+0000 7f495b200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4954111f70 con 0x7f4954106f10 2024-09-12T01:37:13.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.685+0000 7f495b200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4954111df0 con 0x7f4954101de0 2024-09-12T01:37:13.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.685+0000 7f495b200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f49541120f0 con 0x7f4954102790 2024-09-12T01:37:13.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.685+0000 7f4953e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4954101de0 0x7f49540744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:13.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.685+0000 7f4953e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4954101de0 0x7f49540744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:41768/0 (socket says 172.21.15.1:41768) 2024-09-12T01:37:13.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.685+0000 7f4953e00640 1 -- 172.21.15.1:0/2417118993 learned_addr learned my addr 172.21.15.1:0/2417118993 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:37:13.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.685+0000 7f4959200640 1 --2- 172.21.15.1:0/2417118993 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4954106f10 0x7f4954074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:13.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.685+0000 7f494be00640 1 --2- 172.21.15.1:0/2417118993 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4954102790 0x7f4954074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:13.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.685+0000 7f4953e00640 1 -- 172.21.15.1:0/2417118993 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4954102790 msgr2=0x7f4954074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:13.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.685+0000 7f4953e00640 1 --2- 172.21.15.1:0/2417118993 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4954102790 0x7f4954074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:13.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.685+0000 7f4953e00640 1 -- 172.21.15.1:0/2417118993 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4954106f10 msgr2=0x7f4954074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:13.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.685+0000 7f4953e00640 1 --2- 172.21.15.1:0/2417118993 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4954106f10 0x7f4954074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:13.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.685+0000 7f4953e00640 1 -- 172.21.15.1:0/2417118993 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49541a68b0 con 0x7f4954101de0 2024-09-12T01:37:13.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.685+0000 7f494be00640 1 --2- 172.21.15.1:0/2417118993 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4954102790 0x7f4954074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:37:13.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.686+0000 7f4953e00640 1 --2- 172.21.15.1:0/2417118993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4954101de0 0x7f49540744f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f494400e9f0 tx=0x7f494400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:13.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.686+0000 7f4951600640 1 -- 172.21.15.1:0/2417118993 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494400cd80 con 0x7f4954101de0 2024-09-12T01:37:13.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.686+0000 7f4951600640 1 -- 172.21.15.1:0/2417118993 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4944010910 con 0x7f4954101de0 2024-09-12T01:37:13.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.686+0000 7f4951600640 1 -- 172.21.15.1:0/2417118993 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4944018990 con 0x7f4954101de0 2024-09-12T01:37:13.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.686+0000 7f495b200640 1 -- 172.21.15.1:0/2417118993 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49541a6ba0 con 0x7f4954101de0 2024-09-12T01:37:13.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.686+0000 7f495b200640 1 -- 172.21.15.1:0/2417118993 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49541a7130 con 0x7f4954101de0 2024-09-12T01:37:13.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.687+0000 7f495b200640 1 -- 172.21.15.1:0/2417118993 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49541021c0 con 0x7f4954101de0 2024-09-12T01:37:13.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.689+0000 7f4951600640 1 -- 172.21.15.1:0/2417118993 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f4944010ac0 con 0x7f4954101de0 2024-09-12T01:37:13.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.690+0000 7f4951600640 1 --2- 172.21.15.1:0/2417118993 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f49380a2c00 0x7f49380a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:13.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.690+0000 7f4951600640 1 -- 172.21.15.1:0/2417118993 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f49440cfcf0 con 0x7f4954101de0 2024-09-12T01:37:13.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.691+0000 7f494be00640 1 --2- 172.21.15.1:0/2417118993 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f49380a2c00 0x7f49380a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:13.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.691+0000 7f494be00640 1 --2- 172.21.15.1:0/2417118993 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f49380a2c00 0x7f49380a50c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4954075f80 tx=0x7f493c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:13.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.692+0000 7f4951600640 1 -- 172.21.15.1:0/2417118993 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f494409c860 con 0x7f4954101de0 2024-09-12T01:37:13.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.915+0000 7f495b200640 1 -- 172.21.15.1:0/2417118993 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4954075a20 con 0x7f4954101de0 2024-09-12T01:37:13.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.916+0000 7f4951600640 1 -- 172.21.15.1:0/2417118993 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11965 (secure 0 0 0) 0x7f4954075a20 con 0x7f4954101de0 2024-09-12T01:37:13.917 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:37:13.917 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":38,"btime":"2024-09-12T01:36:47:440990+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14430,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/572192306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":572192306},{"type":"v1","addr":"172.21.15.1:6835","nonce":572192306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14442,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/1050734277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":1050734277},{"type":"v1","addr":"172.21.15.1:6837","nonce":1050734277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:26:48.555447+0000","modified":"2024-09-12T01:36:46.713424+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24545,"mds_1":24511,"mds_2":24506,"mds_3":24493,"mds_4":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14481":{"gid":14481,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/3651849993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":3651849993},{"type":"v1","addr":"172.21.15.1:6839","nonce":3651849993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14493":{"gid":14493,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2464781973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2464781973},{"type":"v1","addr":"172.21.15.1:6841","nonce":2464781973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2071825633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2071825633},{"type":"v1","addr":"172.21.15.104:6833","nonce":2071825633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/2930310711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2930310711},{"type":"v1","addr":"172.21.15.53:6833","nonce":2930310711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2032381725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2032381725},{"type":"v1","addr":"172.21.15.53:6835","nonce":2032381725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.104:6835/1547038540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1547038540},{"type":"v1","addr":"172.21.15.104:6835","nonce":1547038540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":23,"addr":"172.21.15.104:6837/464720247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":464720247},{"type":"v1","addr":"172.21.15.104:6837","nonce":464720247}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":22,"addr":"172.21.15.53:6837/2848743754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2848743754},{"type":"v1","addr":"172.21.15.53:6837","nonce":2848743754}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":21,"addr":"172.21.15.104:6839/1459976041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1459976041},{"type":"v1","addr":"172.21.15.104:6839","nonce":1459976041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.53:6839/4019389301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":4019389301},{"type":"v1","addr":"172.21.15.53:6839","nonce":4019389301}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24487,24493,24506,24545,24511]},"id":1}]} 2024-09-12T01:37:13.917 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 38 2024-09-12T01:37:13.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.919+0000 7f495b200640 1 -- 172.21.15.1:0/2417118993 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f49380a2c00 msgr2=0x7f49380a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:13.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.919+0000 7f495b200640 1 --2- 172.21.15.1:0/2417118993 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f49380a2c00 0x7f49380a50c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4954075f80 tx=0x7f493c005f90 comp rx=0 tx=0).stop 2024-09-12T01:37:13.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.919+0000 7f495b200640 1 -- 172.21.15.1:0/2417118993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4954101de0 msgr2=0x7f49540744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:13.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.919+0000 7f495b200640 1 --2- 172.21.15.1:0/2417118993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4954101de0 0x7f49540744f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f494400e9f0 tx=0x7f494400eeb0 comp rx=0 tx=0).stop 2024-09-12T01:37:13.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.919+0000 7f495b200640 1 -- 172.21.15.1:0/2417118993 shutdown_connections 2024-09-12T01:37:13.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.919+0000 7f495b200640 1 --2- 172.21.15.1:0/2417118993 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4954106f10 0x7f4954074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:13.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.919+0000 7f495b200640 1 --2- 172.21.15.1:0/2417118993 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f49380a2c00 0x7f49380a50c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:13.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.919+0000 7f495b200640 1 --2- 172.21.15.1:0/2417118993 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4954102790 0x7f4954074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:13.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.919+0000 7f495b200640 1 --2- 172.21.15.1:0/2417118993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4954101de0 0x7f49540744f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:13.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.919+0000 7f495b200640 1 -- 172.21.15.1:0/2417118993 >> 172.21.15.1:0/2417118993 conn(0x7f49540fd8c0 msgr2=0x7f49540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:13.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.919+0000 7f495b200640 1 -- 172.21.15.1:0/2417118993 shutdown_connections 2024-09-12T01:37:13.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:13.919+0000 7f495b200640 1 -- 172.21.15.1:0/2417118993 wait complete. 2024-09-12T01:37:14.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:14 smithi001 ceph-mon[30387]: pgmap v627: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-12T01:37:14.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:14 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/2417118993' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:37:14.794 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:37:14.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:14 smithi104 ceph-mon[35219]: pgmap v627: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-12T01:37:14.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:14 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/2417118993' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:37:14.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:14 smithi053 ceph-mon[35158]: pgmap v627: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-12T01:37:14.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:14 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/2417118993' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:37:15.184 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:37:15.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:15.864+0000 7f0babe00640 1 -- 172.21.15.1:0/4214705672 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ba4076720 msgr2=0x7f0ba4076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:15.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:15.864+0000 7f0babe00640 1 --2- 172.21.15.1:0/4214705672 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ba4076720 0x7f0ba4076b00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0b98009900 tx=0x7f0b9802a3f0 comp rx=0 tx=0).stop 2024-09-12T01:37:15.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:15.864+0000 7f0babe00640 1 -- 172.21.15.1:0/4214705672 shutdown_connections 2024-09-12T01:37:15.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:15.864+0000 7f0babe00640 1 --2- 172.21.15.1:0/4214705672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ba4112a30 0x7f0ba4114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:15.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:15.864+0000 7f0babe00640 1 --2- 172.21.15.1:0/4214705672 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ba4077040 0x7f0ba41124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:15.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:15.864+0000 7f0babe00640 1 --2- 172.21.15.1:0/4214705672 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ba4076720 0x7f0ba4076b00 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:15.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:15.864+0000 7f0babe00640 1 -- 172.21.15.1:0/4214705672 >> 172.21.15.1:0/4214705672 conn(0x7f0ba40ffa90 msgr2=0x7f0ba4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:15.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:15.864+0000 7f0babe00640 1 -- 172.21.15.1:0/4214705672 shutdown_connections 2024-09-12T01:37:15.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:15.865+0000 7f0babe00640 1 -- 172.21.15.1:0/4214705672 wait complete. 2024-09-12T01:37:15.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:15.866+0000 7f0babe00640 1 Processor -- start 2024-09-12T01:37:15.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:15.866+0000 7f0babe00640 1 -- start start 2024-09-12T01:37:15.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:15.866+0000 7f0babe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ba4076720 0x7f0ba41a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:15.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:15.866+0000 7f0babe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ba4077040 0x7f0ba41a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:15.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:15.866+0000 7f0babe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ba4112a30 0x7f0ba419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:15.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:15.866+0000 7f0babe00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba41185a0 con 0x7f0ba4077040 2024-09-12T01:37:15.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:15.866+0000 7f0babe00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba4118420 con 0x7f0ba4112a30 2024-09-12T01:37:15.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:15.866+0000 7f0babe00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba4118720 con 0x7f0ba4076720 2024-09-12T01:37:15.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:15.867+0000 7f0ba8a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ba4077040 0x7f0ba41a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:15.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:15.867+0000 7f0ba9400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ba4076720 0x7f0ba41a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:15.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:15.867+0000 7f0ba8a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ba4077040 0x7f0ba41a4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53350/0 (socket says 172.21.15.1:53350) 2024-09-12T01:37:15.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:15.867+0000 7f0ba8a00640 1 -- 172.21.15.1:0/341683716 learned_addr learned my addr 172.21.15.1:0/341683716 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:37:15.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:15.867+0000 7f0ba9e00640 1 --2- 172.21.15.1:0/341683716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ba4112a30 0x7f0ba419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:15.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:15.867+0000 7f0ba8a00640 1 -- 172.21.15.1:0/341683716 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ba4076720 msgr2=0x7f0ba41a46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:15.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:15.867+0000 7f0ba8a00640 1 --2- 172.21.15.1:0/341683716 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ba4076720 0x7f0ba41a46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:15.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:15.867+0000 7f0ba8a00640 1 -- 172.21.15.1:0/341683716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ba4112a30 msgr2=0x7f0ba419e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:15.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:15.867+0000 7f0ba8a00640 1 --2- 172.21.15.1:0/341683716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ba4112a30 0x7f0ba419e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:15.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:15.867+0000 7f0ba8a00640 1 -- 172.21.15.1:0/341683716 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ba419efe0 con 0x7f0ba4077040 2024-09-12T01:37:15.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:15.867+0000 7f0ba8a00640 1 --2- 172.21.15.1:0/341683716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ba4077040 0x7f0ba41a4c30 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0b9000d950 tx=0x7f0b9000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:15.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:15.868+0000 7f0ba2000640 1 -- 172.21.15.1:0/341683716 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b90017070 con 0x7f0ba4077040 2024-09-12T01:37:15.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:15.868+0000 7f0ba2000640 1 -- 172.21.15.1:0/341683716 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b90009c30 con 0x7f0ba4077040 2024-09-12T01:37:15.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:15.868+0000 7f0babe00640 1 -- 172.21.15.1:0/341683716 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ba419f2d0 con 0x7f0ba4077040 2024-09-12T01:37:15.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:15.868+0000 7f0ba2000640 1 -- 172.21.15.1:0/341683716 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9000f6b0 con 0x7f0ba4077040 2024-09-12T01:37:15.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:15.868+0000 7f0babe00640 1 -- 172.21.15.1:0/341683716 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ba41ab5c0 con 0x7f0ba4077040 2024-09-12T01:37:15.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:15.870+0000 7f0babe00640 1 -- 172.21.15.1:0/341683716 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ba4076b00 con 0x7f0ba4077040 2024-09-12T01:37:15.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:15.872+0000 7f0ba2000640 1 -- 172.21.15.1:0/341683716 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f0b900109d0 con 0x7f0ba4077040 2024-09-12T01:37:15.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:15.875+0000 7f0ba2000640 1 --2- 172.21.15.1:0/341683716 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f0b8c0a2b80 0x7f0b8c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:15.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:15.875+0000 7f0ba2000640 1 -- 172.21.15.1:0/341683716 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f0b900cefb0 con 0x7f0ba4077040 2024-09-12T01:37:15.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:15.875+0000 7f0ba9400640 1 --2- 172.21.15.1:0/341683716 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f0b8c0a2b80 0x7f0b8c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:15.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:15.876+0000 7f0ba9400640 1 --2- 172.21.15.1:0/341683716 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f0b8c0a2b80 0x7f0b8c0a5040 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0ba419f9f0 tx=0x7f0b980058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:15.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:15.877+0000 7f0ba2000640 1 -- 172.21.15.1:0/341683716 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0b90009de0 con 0x7f0ba4077040 2024-09-12T01:37:16.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:16.100+0000 7f0babe00640 1 -- 172.21.15.1:0/341683716 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ba402d7c0 con 0x7f0ba4077040 2024-09-12T01:37:16.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:16.101+0000 7f0ba2000640 1 -- 172.21.15.1:0/341683716 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11965 (secure 0 0 0) 0x7f0b9009b4d0 con 0x7f0ba4077040 2024-09-12T01:37:16.101 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:37:16.102 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":38,"btime":"2024-09-12T01:36:47:440990+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14430,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/572192306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":572192306},{"type":"v1","addr":"172.21.15.1:6835","nonce":572192306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14442,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/1050734277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":1050734277},{"type":"v1","addr":"172.21.15.1:6837","nonce":1050734277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:26:48.555447+0000","modified":"2024-09-12T01:36:46.713424+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24545,"mds_1":24511,"mds_2":24506,"mds_3":24493,"mds_4":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14481":{"gid":14481,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/3651849993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":3651849993},{"type":"v1","addr":"172.21.15.1:6839","nonce":3651849993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14493":{"gid":14493,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2464781973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2464781973},{"type":"v1","addr":"172.21.15.1:6841","nonce":2464781973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2071825633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2071825633},{"type":"v1","addr":"172.21.15.104:6833","nonce":2071825633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/2930310711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2930310711},{"type":"v1","addr":"172.21.15.53:6833","nonce":2930310711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2032381725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2032381725},{"type":"v1","addr":"172.21.15.53:6835","nonce":2032381725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.104:6835/1547038540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1547038540},{"type":"v1","addr":"172.21.15.104:6835","nonce":1547038540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":23,"addr":"172.21.15.104:6837/464720247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":464720247},{"type":"v1","addr":"172.21.15.104:6837","nonce":464720247}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":22,"addr":"172.21.15.53:6837/2848743754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2848743754},{"type":"v1","addr":"172.21.15.53:6837","nonce":2848743754}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":21,"addr":"172.21.15.104:6839/1459976041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1459976041},{"type":"v1","addr":"172.21.15.104:6839","nonce":1459976041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.53:6839/4019389301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":4019389301},{"type":"v1","addr":"172.21.15.53:6839","nonce":4019389301}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24487,24493,24506,24545,24511]},"id":1}]} 2024-09-12T01:37:16.102 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 38 2024-09-12T01:37:16.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:16.105+0000 7f0babe00640 1 -- 172.21.15.1:0/341683716 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f0b8c0a2b80 msgr2=0x7f0b8c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:16.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:16.105+0000 7f0babe00640 1 --2- 172.21.15.1:0/341683716 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f0b8c0a2b80 0x7f0b8c0a5040 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0ba419f9f0 tx=0x7f0b980058d0 comp rx=0 tx=0).stop 2024-09-12T01:37:16.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:16.105+0000 7f0babe00640 1 -- 172.21.15.1:0/341683716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ba4077040 msgr2=0x7f0ba41a4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:16.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:16.105+0000 7f0babe00640 1 --2- 172.21.15.1:0/341683716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ba4077040 0x7f0ba41a4c30 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0b9000d950 tx=0x7f0b9000de10 comp rx=0 tx=0).stop 2024-09-12T01:37:16.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:16.105+0000 7f0babe00640 1 -- 172.21.15.1:0/341683716 shutdown_connections 2024-09-12T01:37:16.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:16.105+0000 7f0babe00640 1 --2- 172.21.15.1:0/341683716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ba4112a30 0x7f0ba419e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:16.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:16.105+0000 7f0babe00640 1 --2- 172.21.15.1:0/341683716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ba4077040 0x7f0ba41a4c30 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:16.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:16.105+0000 7f0babe00640 1 --2- 172.21.15.1:0/341683716 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f0b8c0a2b80 0x7f0b8c0a5040 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:16.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:16.105+0000 7f0babe00640 1 --2- 172.21.15.1:0/341683716 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ba4076720 0x7f0ba41a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:16.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:16.105+0000 7f0babe00640 1 -- 172.21.15.1:0/341683716 >> 172.21.15.1:0/341683716 conn(0x7f0ba40ffa90 msgr2=0x7f0ba4101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:16.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:16.105+0000 7f0babe00640 1 -- 172.21.15.1:0/341683716 shutdown_connections 2024-09-12T01:37:16.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:16.106+0000 7f0babe00640 1 -- 172.21.15.1:0/341683716 wait complete. 2024-09-12T01:37:16.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:16 smithi001 ceph-mon[30387]: pgmap v628: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-12T01:37:16.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:16 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/341683716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:37:16.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:16 smithi104 ceph-mon[35219]: pgmap v628: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-12T01:37:16.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:16 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/341683716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:37:16.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:16 smithi053 ceph-mon[35158]: pgmap v628: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-12T01:37:16.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:16 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/341683716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:37:17.026 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T01:37:17.407 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:37:18.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.009+0000 7f3046e00640 1 -- 172.21.15.1:0/399994307 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30401049c0 msgr2=0x7f3040108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:18.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.009+0000 7f3046e00640 1 --2- 172.21.15.1:0/399994307 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30401049c0 0x7f3040108a10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3028009900 tx=0x7f302802a3f0 comp rx=0 tx=0).stop 2024-09-12T01:37:18.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.009+0000 7f3046e00640 1 -- 172.21.15.1:0/399994307 shutdown_connections 2024-09-12T01:37:18.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.009+0000 7f3046e00640 1 --2- 172.21.15.1:0/399994307 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3040109140 0x7f3040110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:18.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.009+0000 7f3046e00640 1 --2- 172.21.15.1:0/399994307 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30401049c0 0x7f3040108a10 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:18.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.009+0000 7f3046e00640 1 --2- 172.21.15.1:0/399994307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3040104010 0x7f30401043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:18.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.009+0000 7f3046e00640 1 -- 172.21.15.1:0/399994307 >> 172.21.15.1:0/399994307 conn(0x7f30400ffa50 msgr2=0x7f3040101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:18.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.010+0000 7f3046e00640 1 -- 172.21.15.1:0/399994307 shutdown_connections 2024-09-12T01:37:18.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.010+0000 7f3046e00640 1 -- 172.21.15.1:0/399994307 wait complete. 2024-09-12T01:37:18.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.010+0000 7f3046e00640 1 Processor -- start 2024-09-12T01:37:18.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.011+0000 7f3046e00640 1 -- start start 2024-09-12T01:37:18.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.011+0000 7f3046e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3040104010 0x7f304019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:18.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.011+0000 7f3046e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f30401049c0 0x7f304019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:18.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.011+0000 7f3046e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3040109140 0x7f304019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:18.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.011+0000 7f3046e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3040114140 con 0x7f3040104010 2024-09-12T01:37:18.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.011+0000 7f3046e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3040113fc0 con 0x7f3040109140 2024-09-12T01:37:18.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.011+0000 7f3046e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f30401142c0 con 0x7f30401049c0 2024-09-12T01:37:18.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.011+0000 7f303fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3040104010 0x7f304019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:18.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.011+0000 7f3037400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f30401049c0 0x7f304019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:18.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.011+0000 7f303fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3040104010 0x7f304019a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53366/0 (socket says 172.21.15.1:53366) 2024-09-12T01:37:18.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.011+0000 7f303fe00640 1 -- 172.21.15.1:0/990652925 learned_addr learned my addr 172.21.15.1:0/990652925 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:37:18.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.011+0000 7f3044e00640 1 --2- 172.21.15.1:0/990652925 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3040109140 0x7f304019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:18.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.012+0000 7f303fe00640 1 -- 172.21.15.1:0/990652925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f30401049c0 msgr2=0x7f304019ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:18.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.012+0000 7f303fe00640 1 --2- 172.21.15.1:0/990652925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f30401049c0 0x7f304019ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:18.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.012+0000 7f303fe00640 1 -- 172.21.15.1:0/990652925 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3040109140 msgr2=0x7f304019eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:18.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.012+0000 7f303fe00640 1 --2- 172.21.15.1:0/990652925 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3040109140 0x7f304019eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:18.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.012+0000 7f303fe00640 1 -- 172.21.15.1:0/990652925 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f304019f5d0 con 0x7f3040104010 2024-09-12T01:37:18.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.012+0000 7f303fe00640 1 --2- 172.21.15.1:0/990652925 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3040104010 0x7f304019a620 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f303000e9f0 tx=0x7f303000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:18.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.012+0000 7f303d600640 1 -- 172.21.15.1:0/990652925 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303000cd80 con 0x7f3040104010 2024-09-12T01:37:18.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.013+0000 7f3046e00640 1 -- 172.21.15.1:0/990652925 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f304019f8c0 con 0x7f3040104010 2024-09-12T01:37:18.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.013+0000 7f303d600640 1 -- 172.21.15.1:0/990652925 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3030010910 con 0x7f3040104010 2024-09-12T01:37:18.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.013+0000 7f3046e00640 1 -- 172.21.15.1:0/990652925 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30401a71a0 con 0x7f3040104010 2024-09-12T01:37:18.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.013+0000 7f303d600640 1 -- 172.21.15.1:0/990652925 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3030018910 con 0x7f3040104010 2024-09-12T01:37:18.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.014+0000 7f3046e00640 1 -- 172.21.15.1:0/990652925 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30401043f0 con 0x7f3040104010 2024-09-12T01:37:18.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.016+0000 7f303d600640 1 -- 172.21.15.1:0/990652925 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f3030010ac0 con 0x7f3040104010 2024-09-12T01:37:18.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.017+0000 7f303d600640 1 --2- 172.21.15.1:0/990652925 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f30240a2b80 0x7f30240a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:18.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.017+0000 7f303d600640 1 -- 172.21.15.1:0/990652925 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f30300cf7b0 con 0x7f3040104010 2024-09-12T01:37:18.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.017+0000 7f3037400640 1 --2- 172.21.15.1:0/990652925 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f30240a2b80 0x7f30240a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:18.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.018+0000 7f3037400640 1 --2- 172.21.15.1:0/990652925 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f30240a2b80 0x7f30240a5040 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3028009230 tx=0x7f30280058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:18.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.021+0000 7f303d600640 1 -- 172.21.15.1:0/990652925 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f303009bcd0 con 0x7f3040104010 2024-09-12T01:37:18.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.208+0000 7f3046e00640 1 -- 172.21.15.1:0/990652925 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f304010c3e0 con 0x7f3040104010 2024-09-12T01:37:18.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.209+0000 7f303d600640 1 -- 172.21.15.1:0/990652925 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+24751 (secure 0 0 0) 0x7f303009ffd0 con 0x7f3040104010 2024-09-12T01:37:18.210 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-12T01:37:18.210 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":98,"fsid":"c0e4f180-70a4-11ef-bcea-c7b262605968","created":"2024-09-12T01:18:19.337406+0000","modified":"2024-09-12T01:28:17.407774+0000","last_up_change":"2024-09-12T01:25:50.601477+0000","last_in_change":"2024-09-12T01:25:27.674600+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:21:41.204242+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:26:43.066475+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:26:46.112771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T01:26:56.040130+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b9a396ae-bf97-462f-9ef9-b10ef6b9d337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6803","nonce":1125335136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6805","nonce":1125335136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6809","nonce":1125335136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":1125335136},{"type":"v1","addr":"172.21.15.1:6807","nonce":1125335136}]},"public_addr":"172.21.15.1:6803/1125335136","cluster_addr":"172.21.15.1:6805/1125335136","heartbeat_back_addr":"172.21.15.1:6809/1125335136","heartbeat_front_addr":"172.21.15.1:6807/1125335136","state":["exists","up"]},{"osd":1,"uuid":"dfe7a8a3-3700-4789-ac64-88660de9e0d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6801","nonce":3131049165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6803","nonce":3131049165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6807","nonce":3131049165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3131049165},{"type":"v1","addr":"172.21.15.53:6805","nonce":3131049165}]},"public_addr":"172.21.15.53:6801/3131049165","cluster_addr":"172.21.15.53:6803/3131049165","heartbeat_back_addr":"172.21.15.53:6807/3131049165","heartbeat_front_addr":"172.21.15.53:6805/3131049165","state":["exists","up"]},{"osd":2,"uuid":"5d9d3603-ba7c-4115-9267-12abcb20d55b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6801","nonce":1133285636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6803","nonce":1133285636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6807","nonce":1133285636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1133285636},{"type":"v1","addr":"172.21.15.104:6805","nonce":1133285636}]},"public_addr":"172.21.15.104:6801/1133285636","cluster_addr":"172.21.15.104:6803/1133285636","heartbeat_back_addr":"172.21.15.104:6807/1133285636","heartbeat_front_addr":"172.21.15.104:6805/1133285636","state":["exists","up"]},{"osd":3,"uuid":"1e063c66-41b4-4db1-a964-28e122ce628e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6811","nonce":1247468803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6813","nonce":1247468803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6817","nonce":1247468803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1247468803},{"type":"v1","addr":"172.21.15.1:6815","nonce":1247468803}]},"public_addr":"172.21.15.1:6811/1247468803","cluster_addr":"172.21.15.1:6813/1247468803","heartbeat_back_addr":"172.21.15.1:6817/1247468803","heartbeat_front_addr":"172.21.15.1:6815/1247468803","state":["exists","up"]},{"osd":4,"uuid":"fe89ece0-2171-45d5-a06d-e400f77233e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6809","nonce":2673413639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6811","nonce":2673413639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6815","nonce":2673413639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2673413639},{"type":"v1","addr":"172.21.15.53:6813","nonce":2673413639}]},"public_addr":"172.21.15.53:6809/2673413639","cluster_addr":"172.21.15.53:6811/2673413639","heartbeat_back_addr":"172.21.15.53:6815/2673413639","heartbeat_front_addr":"172.21.15.53:6813/2673413639","state":["exists","up"]},{"osd":5,"uuid":"daab795c-9942-4d16-aa34-8bba30a4b99b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6809","nonce":3131498790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6811","nonce":3131498790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6815","nonce":3131498790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3131498790},{"type":"v1","addr":"172.21.15.104:6813","nonce":3131498790}]},"public_addr":"172.21.15.104:6809/3131498790","cluster_addr":"172.21.15.104:6811/3131498790","heartbeat_back_addr":"172.21.15.104:6815/3131498790","heartbeat_front_addr":"172.21.15.104:6813/3131498790","state":["exists","up"]},{"osd":6,"uuid":"f31d8633-e435-499b-9d46-0aa11a2481cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6819","nonce":3413067918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6821","nonce":3413067918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6825","nonce":3413067918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":3413067918},{"type":"v1","addr":"172.21.15.1:6823","nonce":3413067918}]},"public_addr":"172.21.15.1:6819/3413067918","cluster_addr":"172.21.15.1:6821/3413067918","heartbeat_back_addr":"172.21.15.1:6825/3413067918","heartbeat_front_addr":"172.21.15.1:6823/3413067918","state":["exists","up"]},{"osd":7,"uuid":"e64da49e-d31e-4801-b6cc-634a233b4761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6817","nonce":2841484136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6819","nonce":2841484136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6823","nonce":2841484136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2841484136},{"type":"v1","addr":"172.21.15.53:6821","nonce":2841484136}]},"public_addr":"172.21.15.53:6817/2841484136","cluster_addr":"172.21.15.53:6819/2841484136","heartbeat_back_addr":"172.21.15.53:6823/2841484136","heartbeat_front_addr":"172.21.15.53:6821/2841484136","state":["exists","up"]},{"osd":8,"uuid":"fdfb5a42-dca7-4874-b479-176b4a191e21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6817","nonce":941453547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6819","nonce":941453547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6823","nonce":941453547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":941453547},{"type":"v1","addr":"172.21.15.104:6821","nonce":941453547}]},"public_addr":"172.21.15.104:6817/941453547","cluster_addr":"172.21.15.104:6819/941453547","heartbeat_back_addr":"172.21.15.104:6823/941453547","heartbeat_front_addr":"172.21.15.104:6821/941453547","state":["exists","up"]},{"osd":9,"uuid":"7f81537a-cae5-4bb8-aec2-4eb07deb7958","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6827","nonce":958287586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6829","nonce":958287586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6833","nonce":958287586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":958287586},{"type":"v1","addr":"172.21.15.1:6831","nonce":958287586}]},"public_addr":"172.21.15.1:6827/958287586","cluster_addr":"172.21.15.1:6829/958287586","heartbeat_back_addr":"172.21.15.1:6833/958287586","heartbeat_front_addr":"172.21.15.1:6831/958287586","state":["exists","up"]},{"osd":10,"uuid":"e3c9c37d-46b9-4585-990d-69c447453bd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6825","nonce":3357909498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6827","nonce":3357909498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6831","nonce":3357909498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3357909498},{"type":"v1","addr":"172.21.15.53:6829","nonce":3357909498}]},"public_addr":"172.21.15.53:6825/3357909498","cluster_addr":"172.21.15.53:6827/3357909498","heartbeat_back_addr":"172.21.15.53:6831/3357909498","heartbeat_front_addr":"172.21.15.53:6829/3357909498","state":["exists","up"]},{"osd":11,"uuid":"474b5345-f2ae-4e0e-a197-53ab11f58c04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6825","nonce":3381427987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6827","nonce":3381427987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6831","nonce":3381427987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3381427987},{"type":"v1","addr":"172.21.15.104:6829","nonce":3381427987}]},"public_addr":"172.21.15.104:6825/3381427987","cluster_addr":"172.21.15.104:6827/3381427987","heartbeat_back_addr":"172.21.15.104:6831/3381427987","heartbeat_front_addr":"172.21.15.104:6829/3381427987","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:20:46.903181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:21:12.735556+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:21:37.868739+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:22:09.141228+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:22:36.438563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:23:02.220948+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:23:33.505881+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:24:00.982176+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:24:26.010444+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:24:56.711121+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:25:23.846495+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:25:48.815427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1373984726":"2024-09-13T01:19:14.580637+0000","172.21.15.1:0/1429963718":"2024-09-13T01:19:14.580637+0000","172.21.15.1:6801/4101281615":"2024-09-13T01:19:14.580637+0000","172.21.15.1:6800/4101281615":"2024-09-13T01:19:14.580637+0000","172.21.15.1:0/2564759586":"2024-09-13T01:19:14.580637+0000","172.21.15.1:6801/2935968453":"2024-09-13T01:18:45.882682+0000","172.21.15.1:6800/2935968453":"2024-09-13T01:18:45.882682+0000","172.21.15.1:0/1459848813":"2024-09-13T01:18:45.882682+0000","172.21.15.1:0/578605922":"2024-09-13T01:18:45.882682+0000","172.21.15.1:0/738035035":"2024-09-13T01:18:45.882682+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:37:18.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.212+0000 7f3046e00640 1 -- 172.21.15.1:0/990652925 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f30240a2b80 msgr2=0x7f30240a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:18.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.212+0000 7f3046e00640 1 --2- 172.21.15.1:0/990652925 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f30240a2b80 0x7f30240a5040 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3028009230 tx=0x7f30280058d0 comp rx=0 tx=0).stop 2024-09-12T01:37:18.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.212+0000 7f3046e00640 1 -- 172.21.15.1:0/990652925 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3040104010 msgr2=0x7f304019a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:18.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.212+0000 7f3046e00640 1 --2- 172.21.15.1:0/990652925 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3040104010 0x7f304019a620 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f303000e9f0 tx=0x7f303000eeb0 comp rx=0 tx=0).stop 2024-09-12T01:37:18.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.212+0000 7f3046e00640 1 -- 172.21.15.1:0/990652925 shutdown_connections 2024-09-12T01:37:18.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.212+0000 7f3046e00640 1 --2- 172.21.15.1:0/990652925 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3040109140 0x7f304019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:18.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.212+0000 7f3046e00640 1 --2- 172.21.15.1:0/990652925 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f30240a2b80 0x7f30240a5040 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:18.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.212+0000 7f3046e00640 1 --2- 172.21.15.1:0/990652925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f30401049c0 0x7f304019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:18.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.212+0000 7f3046e00640 1 --2- 172.21.15.1:0/990652925 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3040104010 0x7f304019a620 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:18.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.212+0000 7f3046e00640 1 -- 172.21.15.1:0/990652925 >> 172.21.15.1:0/990652925 conn(0x7f30400ffa50 msgr2=0x7f3040100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:18.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.212+0000 7f3046e00640 1 -- 172.21.15.1:0/990652925 shutdown_connections 2024-09-12T01:37:18.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:18.212+0000 7f3046e00640 1 -- 172.21.15.1:0/990652925 wait complete. 2024-09-12T01:37:18.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:18 smithi104 ceph-mon[35219]: pgmap v629: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-12T01:37:18.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:18 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/990652925' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:37:18.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:18 smithi001 ceph-mon[30387]: pgmap v629: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-12T01:37:18.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:18 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/990652925' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:37:18.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:18 smithi053 ceph-mon[35158]: pgmap v629: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-12T01:37:18.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:18 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/990652925' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:37:19.103 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph fs fail cephfs 2024-09-12T01:37:19.482 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:37:20.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.087+0000 7f8f75400640 1 -- 172.21.15.1:0/1828758911 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f70103ff0 msgr2=0x7f8f701043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:20.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.087+0000 7f8f75400640 1 --2- 172.21.15.1:0/1828758911 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f70103ff0 0x7f8f701043d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8f60009be0 tx=0x7f8f6002a6c0 comp rx=0 tx=0).stop 2024-09-12T01:37:20.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.087+0000 7f8f75400640 1 -- 172.21.15.1:0/1828758911 shutdown_connections 2024-09-12T01:37:20.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.087+0000 7f8f75400640 1 --2- 172.21.15.1:0/1828758911 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f70109120 0x7f8f70110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:20.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.087+0000 7f8f75400640 1 --2- 172.21.15.1:0/1828758911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f701049a0 0x7f8f701089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:20.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.087+0000 7f8f75400640 1 --2- 172.21.15.1:0/1828758911 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f70103ff0 0x7f8f701043d0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:20.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.087+0000 7f8f75400640 1 -- 172.21.15.1:0/1828758911 >> 172.21.15.1:0/1828758911 conn(0x7f8f700ffa90 msgr2=0x7f8f70101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:20.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.087+0000 7f8f75400640 1 -- 172.21.15.1:0/1828758911 shutdown_connections 2024-09-12T01:37:20.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.087+0000 7f8f75400640 1 -- 172.21.15.1:0/1828758911 wait complete. 2024-09-12T01:37:20.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.088+0000 7f8f75400640 1 Processor -- start 2024-09-12T01:37:20.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.088+0000 7f8f75400640 1 -- start start 2024-09-12T01:37:20.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.089+0000 7f8f75400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f70103ff0 0x7f8f7019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:20.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.089+0000 7f8f75400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f701049a0 0x7f8f7019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:20.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.089+0000 7f8f75400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f70109120 0x7f8f7019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:20.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.089+0000 7f8f75400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8f70114150 con 0x7f8f701049a0 2024-09-12T01:37:20.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.089+0000 7f8f75400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8f70113fd0 con 0x7f8f70109120 2024-09-12T01:37:20.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.089+0000 7f8f75400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8f701142d0 con 0x7f8f70103ff0 2024-09-12T01:37:20.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.089+0000 7f8f65e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f701049a0 0x7f8f7019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:20.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.089+0000 7f8f65e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f701049a0 0x7f8f7019ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51204/0 (socket says 172.21.15.1:51204) 2024-09-12T01:37:20.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.089+0000 7f8f65e00640 1 -- 172.21.15.1:0/875288081 learned_addr learned my addr 172.21.15.1:0/875288081 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:37:20.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.089+0000 7f8f6e800640 1 --2- 172.21.15.1:0/875288081 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f70103ff0 0x7f8f7019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:20.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.089+0000 7f8f6f200640 1 --2- 172.21.15.1:0/875288081 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f70109120 0x7f8f7019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:20.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.090+0000 7f8f65e00640 1 -- 172.21.15.1:0/875288081 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f70103ff0 msgr2=0x7f8f7019a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:20.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.090+0000 7f8f65e00640 1 --2- 172.21.15.1:0/875288081 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f70103ff0 0x7f8f7019a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:20.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.090+0000 7f8f65e00640 1 -- 172.21.15.1:0/875288081 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f70109120 msgr2=0x7f8f7019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:20.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.090+0000 7f8f65e00640 1 --2- 172.21.15.1:0/875288081 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f70109120 0x7f8f7019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:20.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.090+0000 7f8f65e00640 1 -- 172.21.15.1:0/875288081 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f7019f5e0 con 0x7f8f701049a0 2024-09-12T01:37:20.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.090+0000 7f8f6f200640 1 --2- 172.21.15.1:0/875288081 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f70109120 0x7f8f7019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:37:20.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.090+0000 7f8f6e800640 1 --2- 172.21.15.1:0/875288081 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f70103ff0 0x7f8f7019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:37:20.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.090+0000 7f8f65e00640 1 --2- 172.21.15.1:0/875288081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f701049a0 0x7f8f7019ab70 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8f5800e9c0 tx=0x7f8f5800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:20.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.090+0000 7f8f67e00640 1 -- 172.21.15.1:0/875288081 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5800cde0 con 0x7f8f701049a0 2024-09-12T01:37:20.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.090+0000 7f8f75400640 1 -- 172.21.15.1:0/875288081 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f7019f8d0 con 0x7f8f701049a0 2024-09-12T01:37:20.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.090+0000 7f8f67e00640 1 -- 172.21.15.1:0/875288081 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f580035f0 con 0x7f8f701049a0 2024-09-12T01:37:20.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.090+0000 7f8f67e00640 1 -- 172.21.15.1:0/875288081 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f58010650 con 0x7f8f701049a0 2024-09-12T01:37:20.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.090+0000 7f8f75400640 1 -- 172.21.15.1:0/875288081 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f701a71b0 con 0x7f8f701049a0 2024-09-12T01:37:20.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.091+0000 7f8f75400640 1 -- 172.21.15.1:0/875288081 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f701043d0 con 0x7f8f701049a0 2024-09-12T01:37:20.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.094+0000 7f8f67e00640 1 -- 172.21.15.1:0/875288081 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f8f580037a0 con 0x7f8f701049a0 2024-09-12T01:37:20.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.095+0000 7f8f67e00640 1 --2- 172.21.15.1:0/875288081 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f8f540a2b80 0x7f8f540a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:20.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.095+0000 7f8f67e00640 1 -- 172.21.15.1:0/875288081 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f8f580cf800 con 0x7f8f701049a0 2024-09-12T01:37:20.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.095+0000 7f8f6e800640 1 --2- 172.21.15.1:0/875288081 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f8f540a2b80 0x7f8f540a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:20.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.096+0000 7f8f6e800640 1 --2- 172.21.15.1:0/875288081 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f8f540a2b80 0x7f8f540a5040 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8f60000f80 tx=0x7f8f60005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:20.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.099+0000 7f8f67e00640 1 -- 172.21.15.1:0/875288081 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8f5809c370 con 0x7f8f701049a0 2024-09-12T01:37:20.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.322+0000 7f8f75400640 1 -- 172.21.15.1:0/875288081 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f8f7002d7c0 con 0x7f8f701049a0 2024-09-12T01:37:20.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.325+0000 7f8f67e00640 1 -- 172.21.15.1:0/875288081 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v39) ==== 161+0+0 (secure 0 0 0) 0x7f8f5809bc70 con 0x7f8f701049a0 2024-09-12T01:37:20.325 INFO:teuthology.orchestra.run.smithi001.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-12T01:37:20.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.331+0000 7f8f75400640 1 -- 172.21.15.1:0/875288081 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f8f540a2b80 msgr2=0x7f8f540a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:20.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.331+0000 7f8f75400640 1 --2- 172.21.15.1:0/875288081 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f8f540a2b80 0x7f8f540a5040 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8f60000f80 tx=0x7f8f60005fd0 comp rx=0 tx=0).stop 2024-09-12T01:37:20.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.331+0000 7f8f75400640 1 -- 172.21.15.1:0/875288081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f701049a0 msgr2=0x7f8f7019ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:20.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.331+0000 7f8f75400640 1 --2- 172.21.15.1:0/875288081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f701049a0 0x7f8f7019ab70 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8f5800e9c0 tx=0x7f8f5800ee80 comp rx=0 tx=0).stop 2024-09-12T01:37:20.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.331+0000 7f8f75400640 1 -- 172.21.15.1:0/875288081 shutdown_connections 2024-09-12T01:37:20.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.331+0000 7f8f75400640 1 --2- 172.21.15.1:0/875288081 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f70109120 0x7f8f7019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:20.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.331+0000 7f8f75400640 1 --2- 172.21.15.1:0/875288081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f701049a0 0x7f8f7019ab70 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:20.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.331+0000 7f8f75400640 1 --2- 172.21.15.1:0/875288081 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f8f540a2b80 0x7f8f540a5040 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:20.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.331+0000 7f8f75400640 1 --2- 172.21.15.1:0/875288081 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f70103ff0 0x7f8f7019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:20.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.331+0000 7f8f75400640 1 -- 172.21.15.1:0/875288081 >> 172.21.15.1:0/875288081 conn(0x7f8f700ffa90 msgr2=0x7f8f70100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:20.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.331+0000 7f8f75400640 1 -- 172.21.15.1:0/875288081 shutdown_connections 2024-09-12T01:37:20.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:20.331+0000 7f8f75400640 1 -- 172.21.15.1:0/875288081 wait complete. 2024-09-12T01:37:20.660 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:20 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mon-a[30383]: 2024-09-12T01:37:20.323+0000 7f3326e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T01:37:20.660 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:20 smithi001 ceph-mon[30387]: pgmap v630: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-12T01:37:20.660 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:20 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/875288081' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T01:37:20.660 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:20 smithi001 ceph-mon[30387]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T01:37:20.660 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:20 smithi001 ceph-mon[30387]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T01:37:20.660 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:20 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/875288081' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T01:37:20.661 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:20 smithi001 ceph-mon[30387]: osdmap e99: 12 total, 12 up, 12 in 2024-09-12T01:37:20.661 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:20 smithi001 ceph-mon[30387]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-12T01:37:20.661 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:20 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T01:37:20.661 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 12 01:37:20 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-g[71269]: ignoring --setuser ceph since I am not root 2024-09-12T01:37:20.661 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 12 01:37:20 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-g[71269]: ignoring --setgroup ceph since I am not root 2024-09-12T01:37:20.661 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 12 01:37:20 smithi001 ceph-mds[71277]: -- 172.21.15.1:0/4137450390 <== mon.1 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d905f87860 con 0x55d905f64800 2024-09-12T01:37:20.661 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 12 01:37:20 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-g[71269]: starting mds.g at 2024-09-12T01:37:20.661 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 12 01:37:20 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-a[71628]: ignoring --setuser ceph since I am not root 2024-09-12T01:37:20.662 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 12 01:37:20 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-a[71628]: ignoring --setgroup ceph since I am not root 2024-09-12T01:37:20.662 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 12 01:37:20 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-a[71628]: starting mds.a at 2024-09-12T01:37:20.740 INFO:journalctl@ceph.mds.e.smithi053.stdout:Sep 12 01:37:20 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-e[64882]: ignoring --setuser ceph since I am not root 2024-09-12T01:37:20.740 INFO:journalctl@ceph.mds.e.smithi053.stdout:Sep 12 01:37:20 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-e[64882]: ignoring --setgroup ceph since I am not root 2024-09-12T01:37:20.741 INFO:journalctl@ceph.mds.e.smithi053.stdout:Sep 12 01:37:20 smithi053 ceph-mds[64886]: -- 172.21.15.53:0/3496331825 <== mon.1 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561d2d1cdc20 con 0x561d2d1ab000 2024-09-12T01:37:20.741 INFO:journalctl@ceph.mds.e.smithi053.stdout:Sep 12 01:37:20 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-e[64882]: starting mds.e at 2024-09-12T01:37:20.741 INFO:journalctl@ceph.mds.h.smithi053.stdout:Sep 12 01:37:20 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-h[65182]: ignoring --setuser ceph since I am not root 2024-09-12T01:37:20.741 INFO:journalctl@ceph.mds.h.smithi053.stdout:Sep 12 01:37:20 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-h[65182]: ignoring --setgroup ceph since I am not root 2024-09-12T01:37:20.741 INFO:journalctl@ceph.mds.h.smithi053.stdout:Sep 12 01:37:20 smithi053 ceph-mds[65186]: -- 172.21.15.53:0/3333905950 <== mon.2 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56234479b860 con 0x562344778800 2024-09-12T01:37:20.742 INFO:journalctl@ceph.mds.h.smithi053.stdout:Sep 12 01:37:20 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-h[65182]: starting mds.h at 2024-09-12T01:37:20.742 INFO:journalctl@ceph.mds.k.smithi053.stdout:Sep 12 01:37:20 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-k[64570]: ignoring --setuser ceph since I am not root 2024-09-12T01:37:20.742 INFO:journalctl@ceph.mds.k.smithi053.stdout:Sep 12 01:37:20 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-k[64570]: ignoring --setgroup ceph since I am not root 2024-09-12T01:37:20.742 INFO:journalctl@ceph.mds.k.smithi053.stdout:Sep 12 01:37:20 smithi053 ceph-mds[64574]: -- 172.21.15.53:0/1960346408 <== mon.1 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d79fe15860 con 0x55d79fdf3400 2024-09-12T01:37:20.742 INFO:journalctl@ceph.mds.k.smithi053.stdout:Sep 12 01:37:20 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-k[64570]: starting mds.k at 2024-09-12T01:37:20.743 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:20 smithi053 ceph-mon[35158]: pgmap v630: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-12T01:37:20.743 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:20 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/875288081' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T01:37:20.743 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:20 smithi053 ceph-mon[35158]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T01:37:20.743 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:20 smithi053 ceph-mon[35158]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T01:37:20.743 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:20 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/875288081' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T01:37:20.743 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:20 smithi053 ceph-mon[35158]: osdmap e99: 12 total, 12 up, 12 in 2024-09-12T01:37:20.743 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:20 smithi053 ceph-mon[35158]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-12T01:37:20.743 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:20 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T01:37:20.744 INFO:journalctl@ceph.mds.b.smithi053.stdout:Sep 12 01:37:20 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-b[65497]: ignoring --setuser ceph since I am not root 2024-09-12T01:37:20.744 INFO:journalctl@ceph.mds.b.smithi053.stdout:Sep 12 01:37:20 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-b[65497]: ignoring --setgroup ceph since I am not root 2024-09-12T01:37:20.744 INFO:journalctl@ceph.mds.b.smithi053.stdout:Sep 12 01:37:20 smithi053 ceph-mds[65501]: -- 172.21.15.53:0/1874857737 <== mon.2 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d2cedb7860 con 0x55d2ced95000 2024-09-12T01:37:20.744 INFO:journalctl@ceph.mds.b.smithi053.stdout:Sep 12 01:37:20 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-b[65497]: starting mds.b at 2024-09-12T01:37:20.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:20 smithi104 ceph-mon[35219]: pgmap v630: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-12T01:37:20.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:20 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/875288081' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T01:37:20.826 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:20 smithi104 ceph-mon[35219]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T01:37:20.826 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:20 smithi104 ceph-mon[35219]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T01:37:20.826 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:20 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/875288081' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T01:37:20.826 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:20 smithi104 ceph-mon[35219]: osdmap e99: 12 total, 12 up, 12 in 2024-09-12T01:37:20.826 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:20 smithi104 ceph-mon[35219]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-12T01:37:20.826 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:20 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T01:37:20.826 INFO:journalctl@ceph.mds.c.smithi104.stdout:Sep 12 01:37:20 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-c[65214]: ignoring --setuser ceph since I am not root 2024-09-12T01:37:20.826 INFO:journalctl@ceph.mds.c.smithi104.stdout:Sep 12 01:37:20 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-c[65214]: ignoring --setgroup ceph since I am not root 2024-09-12T01:37:20.826 INFO:journalctl@ceph.mds.c.smithi104.stdout:Sep 12 01:37:20 smithi104 ceph-mds[65218]: -- 172.21.15.104:0/338697126 <== mon.2 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56321ad57860 con 0x56321ad35000 2024-09-12T01:37:20.826 INFO:journalctl@ceph.mds.c.smithi104.stdout:Sep 12 01:37:20 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-c[65214]: starting mds.c at 2024-09-12T01:37:20.827 INFO:journalctl@ceph.mds.f.smithi104.stdout:Sep 12 01:37:20 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-f[64914]: ignoring --setuser ceph since I am not root 2024-09-12T01:37:20.827 INFO:journalctl@ceph.mds.f.smithi104.stdout:Sep 12 01:37:20 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-f[64914]: ignoring --setgroup ceph since I am not root 2024-09-12T01:37:20.827 INFO:journalctl@ceph.mds.f.smithi104.stdout:Sep 12 01:37:20 smithi104 ceph-mds[64918]: -- 172.21.15.104:0/1950426580 <== mon.1 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56392d863c20 con 0x56392d841000 2024-09-12T01:37:20.827 INFO:journalctl@ceph.mds.f.smithi104.stdout:Sep 12 01:37:20 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-f[64914]: starting mds.f at 2024-09-12T01:37:20.827 INFO:journalctl@ceph.mds.i.smithi104.stdout:Sep 12 01:37:20 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-i[65514]: ignoring --setuser ceph since I am not root 2024-09-12T01:37:20.827 INFO:journalctl@ceph.mds.i.smithi104.stdout:Sep 12 01:37:20 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-i[65514]: ignoring --setgroup ceph since I am not root 2024-09-12T01:37:20.827 INFO:journalctl@ceph.mds.i.smithi104.stdout:Sep 12 01:37:20 smithi104 ceph-mds[65518]: -- 172.21.15.104:0/274042060 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564e3d450680 con 0x564e3d425400 2024-09-12T01:37:20.827 INFO:journalctl@ceph.mds.i.smithi104.stdout:Sep 12 01:37:20 smithi104 ceph-mds[65518]: -- 172.21.15.104:0/274042060 <== mon.2 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564e3d447860 con 0x564e3d425400 2024-09-12T01:37:20.828 INFO:journalctl@ceph.mds.i.smithi104.stdout:Sep 12 01:37:20 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-i[65514]: starting mds.i at 2024-09-12T01:37:20.828 INFO:journalctl@ceph.mds.l.smithi104.stdout:Sep 12 01:37:20 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-l[64617]: ignoring --setuser ceph since I am not root 2024-09-12T01:37:20.828 INFO:journalctl@ceph.mds.l.smithi104.stdout:Sep 12 01:37:20 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-l[64617]: ignoring --setgroup ceph since I am not root 2024-09-12T01:37:20.828 INFO:journalctl@ceph.mds.l.smithi104.stdout:Sep 12 01:37:20 smithi104 ceph-mds[64621]: -- 172.21.15.104:0/2268277366 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55be7a8f4d00 con 0x55be7a8c8800 2024-09-12T01:37:20.828 INFO:journalctl@ceph.mds.l.smithi104.stdout:Sep 12 01:37:20 smithi104 ceph-mds[64621]: -- 172.21.15.104:0/2268277366 <== mon.1 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55be7a8eb860 con 0x55be7a8c8800 2024-09-12T01:37:20.828 INFO:journalctl@ceph.mds.l.smithi104.stdout:Sep 12 01:37:20 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-l[64617]: starting mds.l at 2024-09-12T01:37:21.047 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-12T01:37:21.440 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:37:22.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.029+0000 7f5b50e00640 1 -- 172.21.15.1:0/3898846148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b4c106f10 msgr2=0x7f5b4c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:22.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.029+0000 7f5b50e00640 1 --2- 172.21.15.1:0/3898846148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b4c106f10 0x7f5b4c10ea90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5b44009ca0 tx=0x7f5b4402a8c0 comp rx=0 tx=0).stop 2024-09-12T01:37:22.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.030+0000 7f5b50e00640 1 -- 172.21.15.1:0/3898846148 shutdown_connections 2024-09-12T01:37:22.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.030+0000 7f5b50e00640 1 --2- 172.21.15.1:0/3898846148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b4c106f10 0x7f5b4c10ea90 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:22.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.030+0000 7f5b50e00640 1 --2- 172.21.15.1:0/3898846148 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b4c102790 0x7f5b4c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:22.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.030+0000 7f5b50e00640 1 --2- 172.21.15.1:0/3898846148 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b4c101de0 0x7f5b4c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:22.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.030+0000 7f5b50e00640 1 -- 172.21.15.1:0/3898846148 >> 172.21.15.1:0/3898846148 conn(0x7f5b4c0fd8c0 msgr2=0x7f5b4c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:22.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.030+0000 7f5b50e00640 1 -- 172.21.15.1:0/3898846148 shutdown_connections 2024-09-12T01:37:22.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.030+0000 7f5b50e00640 1 -- 172.21.15.1:0/3898846148 wait complete. 2024-09-12T01:37:22.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.031+0000 7f5b50e00640 1 Processor -- start 2024-09-12T01:37:22.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.031+0000 7f5b50e00640 1 -- start start 2024-09-12T01:37:22.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.031+0000 7f5b50e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b4c101de0 0x7f5b4c19a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:22.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.031+0000 7f5b50e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b4c102790 0x7f5b4c19ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:22.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.031+0000 7f5b50e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b4c106f10 0x7f5b4c19eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:22.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.031+0000 7f5b50e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4c111f70 con 0x7f5b4c106f10 2024-09-12T01:37:22.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.031+0000 7f5b50e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4c111df0 con 0x7f5b4c101de0 2024-09-12T01:37:22.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.031+0000 7f5b50e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4c1120f0 con 0x7f5b4c102790 2024-09-12T01:37:22.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.032+0000 7f5b4a800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b4c106f10 0x7f5b4c19eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:22.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.032+0000 7f5b41400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b4c102790 0x7f5b4c19ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:22.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.032+0000 7f5b4a800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b4c106f10 0x7f5b4c19eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51256/0 (socket says 172.21.15.1:51256) 2024-09-12T01:37:22.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.032+0000 7f5b4a800640 1 -- 172.21.15.1:0/1104503183 learned_addr learned my addr 172.21.15.1:0/1104503183 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:37:22.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.032+0000 7f5b49e00640 1 --2- 172.21.15.1:0/1104503183 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b4c101de0 0x7f5b4c19a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:22.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.032+0000 7f5b4a800640 1 -- 172.21.15.1:0/1104503183 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b4c102790 msgr2=0x7f5b4c19ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:22.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.032+0000 7f5b4a800640 1 --2- 172.21.15.1:0/1104503183 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b4c102790 0x7f5b4c19ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:22.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.032+0000 7f5b4a800640 1 -- 172.21.15.1:0/1104503183 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b4c101de0 msgr2=0x7f5b4c19a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:22.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.032+0000 7f5b4a800640 1 --2- 172.21.15.1:0/1104503183 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b4c101de0 0x7f5b4c19a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:22.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.032+0000 7f5b4a800640 1 -- 172.21.15.1:0/1104503183 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b4c19f580 con 0x7f5b4c106f10 2024-09-12T01:37:22.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.032+0000 7f5b4a800640 1 --2- 172.21.15.1:0/1104503183 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b4c106f10 0x7f5b4c19eea0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5b44031a20 tx=0x7f5b44031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:22.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.033+0000 7f5b43400640 1 -- 172.21.15.1:0/1104503183 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b44003b70 con 0x7f5b4c106f10 2024-09-12T01:37:22.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.033+0000 7f5b43400640 1 -- 172.21.15.1:0/1104503183 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b44003d10 con 0x7f5b4c106f10 2024-09-12T01:37:22.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.033+0000 7f5b50e00640 1 -- 172.21.15.1:0/1104503183 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b4c19f810 con 0x7f5b4c106f10 2024-09-12T01:37:22.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.033+0000 7f5b43400640 1 -- 172.21.15.1:0/1104503183 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4402fc40 con 0x7f5b4c106f10 2024-09-12T01:37:22.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.033+0000 7f5b50e00640 1 -- 172.21.15.1:0/1104503183 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b4c1a7150 con 0x7f5b4c106f10 2024-09-12T01:37:22.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.034+0000 7f5b50e00640 1 -- 172.21.15.1:0/1104503183 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b4c1021c0 con 0x7f5b4c106f10 2024-09-12T01:37:22.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.036+0000 7f5b43400640 1 -- 172.21.15.1:0/1104503183 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f5b44004050 con 0x7f5b4c106f10 2024-09-12T01:37:22.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.037+0000 7f5b43400640 1 --2- 172.21.15.1:0/1104503183 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f5b300a2c00 0x7f5b300a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:22.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.037+0000 7f5b43400640 1 -- 172.21.15.1:0/1104503183 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9580+0+0 (secure 0 0 0) 0x7f5b440f05c0 con 0x7f5b4c106f10 2024-09-12T01:37:22.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.037+0000 7f5b49e00640 1 --2- 172.21.15.1:0/1104503183 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f5b300a2c00 0x7f5b300a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:22.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.038+0000 7f5b49e00640 1 --2- 172.21.15.1:0/1104503183 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f5b300a2c00 0x7f5b300a50c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c009bd0 tx=0x7f5b3c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:22.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.042+0000 7f5b43400640 1 -- 172.21.15.1:0/1104503183 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5b440bd190 con 0x7f5b4c106f10 2024-09-12T01:37:22.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.265+0000 7f5b50e00640 1 -- 172.21.15.1:0/1104503183 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f5b4c1a73b0 con 0x7f5b4c106f10 2024-09-12T01:37:22.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.512+0000 7f5b43400640 1 -- 172.21.15.1:0/1104503183 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v41) ==== 104+0+0 (secure 0 0 0) 0x7f5b440bca90 con 0x7f5b4c106f10 2024-09-12T01:37:22.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.514+0000 7f5b50e00640 1 -- 172.21.15.1:0/1104503183 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f5b300a2c00 msgr2=0x7f5b300a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:22.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.515+0000 7f5b50e00640 1 --2- 172.21.15.1:0/1104503183 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f5b300a2c00 0x7f5b300a50c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c009bd0 tx=0x7f5b3c005f90 comp rx=0 tx=0).stop 2024-09-12T01:37:22.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.515+0000 7f5b50e00640 1 -- 172.21.15.1:0/1104503183 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b4c106f10 msgr2=0x7f5b4c19eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:22.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.515+0000 7f5b50e00640 1 --2- 172.21.15.1:0/1104503183 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b4c106f10 0x7f5b4c19eea0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5b44031a20 tx=0x7f5b44031a50 comp rx=0 tx=0).stop 2024-09-12T01:37:22.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.515+0000 7f5b50e00640 1 -- 172.21.15.1:0/1104503183 shutdown_connections 2024-09-12T01:37:22.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.515+0000 7f5b50e00640 1 --2- 172.21.15.1:0/1104503183 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b4c106f10 0x7f5b4c19eea0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:22.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.515+0000 7f5b50e00640 1 --2- 172.21.15.1:0/1104503183 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f5b300a2c00 0x7f5b300a50c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:22.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.515+0000 7f5b50e00640 1 --2- 172.21.15.1:0/1104503183 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b4c102790 0x7f5b4c19ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:22.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.515+0000 7f5b50e00640 1 --2- 172.21.15.1:0/1104503183 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b4c101de0 0x7f5b4c19a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:22.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.515+0000 7f5b50e00640 1 -- 172.21.15.1:0/1104503183 >> 172.21.15.1:0/1104503183 conn(0x7f5b4c0fd8c0 msgr2=0x7f5b4c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:22.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.515+0000 7f5b50e00640 1 -- 172.21.15.1:0/1104503183 shutdown_connections 2024-09-12T01:37:22.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:22.515+0000 7f5b50e00640 1 -- 172.21.15.1:0/1104503183 wait complete. 2024-09-12T01:37:22.516 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:22 smithi001 ceph-mon[30387]: pgmap v632: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-12T01:37:22.516 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:22 smithi001 ceph-mon[30387]: mds.? [v2:172.21.15.104:6832/2524163979,v1:172.21.15.104:6836/2524163979] up:boot 2024-09-12T01:37:22.516 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:22 smithi001 ceph-mon[30387]: mds.? [v2:172.21.15.104:6833/3023824676,v1:172.21.15.104:6838/3023824676] up:boot 2024-09-12T01:37:22.516 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:22 smithi001 ceph-mon[30387]: mds.? [v2:172.21.15.104:6835/1299789911,v1:172.21.15.104:6839/1299789911] up:boot 2024-09-12T01:37:22.516 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:22 smithi001 ceph-mon[30387]: mds.? [v2:172.21.15.104:6834/2614161635,v1:172.21.15.104:6837/2614161635] up:boot 2024-09-12T01:37:22.516 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:22 smithi001 ceph-mon[30387]: mds.? [v2:172.21.15.53:6833/3435621088,v1:172.21.15.53:6835/3435621088] up:boot 2024-09-12T01:37:22.517 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:22 smithi001 ceph-mon[30387]: mds.? [v2:172.21.15.53:6832/490594966,v1:172.21.15.53:6834/490594966] up:boot 2024-09-12T01:37:22.517 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:22 smithi001 ceph-mon[30387]: mds.? [v2:172.21.15.53:6836/1232269731,v1:172.21.15.53:6838/1232269731] up:boot 2024-09-12T01:37:22.517 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:22 smithi001 ceph-mon[30387]: mds.? [v2:172.21.15.53:6837/923993848,v1:172.21.15.53:6839/923993848] up:boot 2024-09-12T01:37:22.517 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:22 smithi001 ceph-mon[30387]: mds.? [v2:172.21.15.1:6839/751109303,v1:172.21.15.1:6840/751109303] up:boot 2024-09-12T01:37:22.517 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:22 smithi001 ceph-mon[30387]: mds.? [v2:172.21.15.1:6838/3183033113,v1:172.21.15.1:6841/3183033113] up:boot 2024-09-12T01:37:22.517 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:22 smithi001 ceph-mon[30387]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-12T01:37:22.517 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:22 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T01:37:22.517 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:22 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T01:37:22.517 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:22 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T01:37:22.517 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:22 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T01:37:22.517 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:22 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T01:37:22.517 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:22 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T01:37:22.517 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:22 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T01:37:22.518 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:22 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T01:37:22.518 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:22 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T01:37:22.518 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:22 smithi001 ceph-mon[30387]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T01:37:22.518 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:22 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/1104503183' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T01:37:22.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:22 smithi053 ceph-mon[35158]: pgmap v632: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-12T01:37:22.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:22 smithi053 ceph-mon[35158]: mds.? [v2:172.21.15.104:6832/2524163979,v1:172.21.15.104:6836/2524163979] up:boot 2024-09-12T01:37:22.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:22 smithi053 ceph-mon[35158]: mds.? [v2:172.21.15.104:6833/3023824676,v1:172.21.15.104:6838/3023824676] up:boot 2024-09-12T01:37:22.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:22 smithi053 ceph-mon[35158]: mds.? [v2:172.21.15.104:6835/1299789911,v1:172.21.15.104:6839/1299789911] up:boot 2024-09-12T01:37:22.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:22 smithi053 ceph-mon[35158]: mds.? [v2:172.21.15.104:6834/2614161635,v1:172.21.15.104:6837/2614161635] up:boot 2024-09-12T01:37:22.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:22 smithi053 ceph-mon[35158]: mds.? [v2:172.21.15.53:6833/3435621088,v1:172.21.15.53:6835/3435621088] up:boot 2024-09-12T01:37:22.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:22 smithi053 ceph-mon[35158]: mds.? [v2:172.21.15.53:6832/490594966,v1:172.21.15.53:6834/490594966] up:boot 2024-09-12T01:37:22.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:22 smithi053 ceph-mon[35158]: mds.? [v2:172.21.15.53:6836/1232269731,v1:172.21.15.53:6838/1232269731] up:boot 2024-09-12T01:37:22.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:22 smithi053 ceph-mon[35158]: mds.? [v2:172.21.15.53:6837/923993848,v1:172.21.15.53:6839/923993848] up:boot 2024-09-12T01:37:22.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:22 smithi053 ceph-mon[35158]: mds.? [v2:172.21.15.1:6839/751109303,v1:172.21.15.1:6840/751109303] up:boot 2024-09-12T01:37:22.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:22 smithi053 ceph-mon[35158]: mds.? [v2:172.21.15.1:6838/3183033113,v1:172.21.15.1:6841/3183033113] up:boot 2024-09-12T01:37:22.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:22 smithi053 ceph-mon[35158]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-12T01:37:22.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:22 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T01:37:22.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:22 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T01:37:22.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:22 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T01:37:22.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:22 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T01:37:22.742 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:22 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T01:37:22.742 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:22 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T01:37:22.742 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:22 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T01:37:22.742 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:22 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T01:37:22.742 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:22 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T01:37:22.742 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:22 smithi053 ceph-mon[35158]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T01:37:22.742 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:22 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/1104503183' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T01:37:22.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:22 smithi104 ceph-mon[35219]: pgmap v632: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-12T01:37:22.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:22 smithi104 ceph-mon[35219]: mds.? [v2:172.21.15.104:6832/2524163979,v1:172.21.15.104:6836/2524163979] up:boot 2024-09-12T01:37:22.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:22 smithi104 ceph-mon[35219]: mds.? [v2:172.21.15.104:6833/3023824676,v1:172.21.15.104:6838/3023824676] up:boot 2024-09-12T01:37:22.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:22 smithi104 ceph-mon[35219]: mds.? [v2:172.21.15.104:6835/1299789911,v1:172.21.15.104:6839/1299789911] up:boot 2024-09-12T01:37:22.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:22 smithi104 ceph-mon[35219]: mds.? [v2:172.21.15.104:6834/2614161635,v1:172.21.15.104:6837/2614161635] up:boot 2024-09-12T01:37:22.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:22 smithi104 ceph-mon[35219]: mds.? [v2:172.21.15.53:6833/3435621088,v1:172.21.15.53:6835/3435621088] up:boot 2024-09-12T01:37:22.826 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:22 smithi104 ceph-mon[35219]: mds.? [v2:172.21.15.53:6832/490594966,v1:172.21.15.53:6834/490594966] up:boot 2024-09-12T01:37:22.826 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:22 smithi104 ceph-mon[35219]: mds.? [v2:172.21.15.53:6836/1232269731,v1:172.21.15.53:6838/1232269731] up:boot 2024-09-12T01:37:22.826 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:22 smithi104 ceph-mon[35219]: mds.? [v2:172.21.15.53:6837/923993848,v1:172.21.15.53:6839/923993848] up:boot 2024-09-12T01:37:22.826 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:22 smithi104 ceph-mon[35219]: mds.? [v2:172.21.15.1:6839/751109303,v1:172.21.15.1:6840/751109303] up:boot 2024-09-12T01:37:22.826 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:22 smithi104 ceph-mon[35219]: mds.? [v2:172.21.15.1:6838/3183033113,v1:172.21.15.1:6841/3183033113] up:boot 2024-09-12T01:37:22.826 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:22 smithi104 ceph-mon[35219]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-12T01:37:22.826 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:22 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T01:37:22.826 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:22 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T01:37:22.826 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:22 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T01:37:22.826 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:22 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T01:37:22.826 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:22 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T01:37:22.826 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:22 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T01:37:22.826 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:22 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T01:37:22.826 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:22 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T01:37:22.826 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:22 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T01:37:22.827 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:22 smithi104 ceph-mon[35219]: from='mgr.14150 172.21.15.1:0/510836250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T01:37:22.827 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:22 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/1104503183' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T01:37:23.327 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-12T01:37:23.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:23 smithi001 ceph-mon[30387]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T01:37:23.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:23 smithi001 ceph-mon[30387]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T01:37:23.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:23 smithi001 ceph-mon[30387]: Cluster is now healthy 2024-09-12T01:37:23.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:23 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/1104503183' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T01:37:23.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:23 smithi001 ceph-mon[30387]: fsmap 12 up:standby 2024-09-12T01:37:23.706 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:37:23.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:23 smithi053 ceph-mon[35158]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T01:37:23.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:23 smithi053 ceph-mon[35158]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T01:37:23.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:23 smithi053 ceph-mon[35158]: Cluster is now healthy 2024-09-12T01:37:23.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:23 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/1104503183' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T01:37:23.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:23 smithi053 ceph-mon[35158]: fsmap 12 up:standby 2024-09-12T01:37:23.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:23 smithi104 ceph-mon[35219]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T01:37:23.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:23 smithi104 ceph-mon[35219]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T01:37:23.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:23 smithi104 ceph-mon[35219]: Cluster is now healthy 2024-09-12T01:37:23.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:23 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/1104503183' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T01:37:23.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:23 smithi104 ceph-mon[35219]: fsmap 12 up:standby 2024-09-12T01:37:24.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.264+0000 7f2370c00640 1 -- 172.21.15.1:0/2576735083 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f236c1049c0 msgr2=0x7f236c108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:24.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.264+0000 7f2370c00640 1 --2- 172.21.15.1:0/2576735083 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f236c1049c0 0x7f236c108a10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2358009900 tx=0x7f235802a410 comp rx=0 tx=0).stop 2024-09-12T01:37:24.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.264+0000 7f2370c00640 1 -- 172.21.15.1:0/2576735083 shutdown_connections 2024-09-12T01:37:24.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.264+0000 7f2370c00640 1 --2- 172.21.15.1:0/2576735083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f236c109140 0x7f236c110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:24.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.264+0000 7f2370c00640 1 --2- 172.21.15.1:0/2576735083 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f236c1049c0 0x7f236c108a10 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:24.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.264+0000 7f2370c00640 1 --2- 172.21.15.1:0/2576735083 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f236c104010 0x7f236c1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:24.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.264+0000 7f2370c00640 1 -- 172.21.15.1:0/2576735083 >> 172.21.15.1:0/2576735083 conn(0x7f236c0ffa50 msgr2=0x7f236c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:24.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.264+0000 7f2370c00640 1 -- 172.21.15.1:0/2576735083 shutdown_connections 2024-09-12T01:37:24.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.264+0000 7f2370c00640 1 -- 172.21.15.1:0/2576735083 wait complete. 2024-09-12T01:37:24.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.265+0000 7f2370c00640 1 Processor -- start 2024-09-12T01:37:24.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.266+0000 7f2370c00640 1 -- start start 2024-09-12T01:37:24.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.266+0000 7f2370c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f236c104010 0x7f236c19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:24.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.266+0000 7f2370c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f236c1049c0 0x7f236c19ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:24.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.266+0000 7f2370c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f236c109140 0x7f236c19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:24.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.266+0000 7f2370c00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f236c1141b0 con 0x7f236c1049c0 2024-09-12T01:37:24.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.266+0000 7f2370c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f236c114030 con 0x7f236c104010 2024-09-12T01:37:24.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.266+0000 7f2370c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f236c114330 con 0x7f236c109140 2024-09-12T01:37:24.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.266+0000 7f2369400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f236c1049c0 0x7f236c19ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:24.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.266+0000 7f2369400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f236c1049c0 0x7f236c19ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51274/0 (socket says 172.21.15.1:51274) 2024-09-12T01:37:24.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.266+0000 7f2369400640 1 -- 172.21.15.1:0/622345364 learned_addr learned my addr 172.21.15.1:0/622345364 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:37:24.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.266+0000 7f2369e00640 1 --2- 172.21.15.1:0/622345364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f236c104010 0x7f236c19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:24.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.267+0000 7f236a800640 1 --2- 172.21.15.1:0/622345364 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f236c109140 0x7f236c19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:24.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.267+0000 7f2369400640 1 -- 172.21.15.1:0/622345364 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f236c109140 msgr2=0x7f236c19efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:24.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.267+0000 7f2369400640 1 --2- 172.21.15.1:0/622345364 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f236c109140 0x7f236c19efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:24.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.267+0000 7f2369400640 1 -- 172.21.15.1:0/622345364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f236c104010 msgr2=0x7f236c19a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:24.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.267+0000 7f2369400640 1 --2- 172.21.15.1:0/622345364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f236c104010 0x7f236c19a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:24.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.267+0000 7f2369400640 1 -- 172.21.15.1:0/622345364 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f236c19f6c0 con 0x7f236c1049c0 2024-09-12T01:37:24.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.267+0000 7f2369e00640 1 --2- 172.21.15.1:0/622345364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f236c104010 0x7f236c19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:37:24.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.267+0000 7f2369400640 1 --2- 172.21.15.1:0/622345364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f236c1049c0 0x7f236c19ac50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2358004770 tx=0x7f2358003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:24.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.267+0000 7f2362a00640 1 -- 172.21.15.1:0/622345364 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235803f070 con 0x7f236c1049c0 2024-09-12T01:37:24.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.268+0000 7f2362a00640 1 -- 172.21.15.1:0/622345364 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2358031ca0 con 0x7f236c1049c0 2024-09-12T01:37:24.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.268+0000 7f2362a00640 1 -- 172.21.15.1:0/622345364 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2358035030 con 0x7f236c1049c0 2024-09-12T01:37:24.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.268+0000 7f2370c00640 1 -- 172.21.15.1:0/622345364 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f236c19f950 con 0x7f236c1049c0 2024-09-12T01:37:24.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.268+0000 7f2370c00640 1 -- 172.21.15.1:0/622345364 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f236c1a7290 con 0x7f236c1049c0 2024-09-12T01:37:24.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.269+0000 7f2370c00640 1 -- 172.21.15.1:0/622345364 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f236c1054f0 con 0x7f236c1049c0 2024-09-12T01:37:24.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.271+0000 7f2362a00640 1 -- 172.21.15.1:0/622345364 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f2358042070 con 0x7f236c1049c0 2024-09-12T01:37:24.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.272+0000 7f2362a00640 1 --2- 172.21.15.1:0/622345364 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f23500a2b80 0x7f23500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:24.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.272+0000 7f2362a00640 1 -- 172.21.15.1:0/622345364 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9580+0+0 (secure 0 0 0) 0x7f23580efbc0 con 0x7f236c1049c0 2024-09-12T01:37:24.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.272+0000 7f2369e00640 1 --2- 172.21.15.1:0/622345364 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f23500a2b80 0x7f23500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:24.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.273+0000 7f2369e00640 1 --2- 172.21.15.1:0/622345364 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f23500a2b80 0x7f23500a5040 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f235c009fc0 tx=0x7f235c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:24.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.276+0000 7f2362a00640 1 -- 172.21.15.1:0/622345364 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f23580bbff0 con 0x7f236c1049c0 2024-09-12T01:37:24.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.465+0000 7f2370c00640 1 -- 172.21.15.1:0/622345364 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f236c10c380 con 0x7f236c1049c0 2024-09-12T01:37:24.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.515+0000 7f2362a00640 1 -- 172.21.15.1:0/622345364 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v100) ==== 181+0+0 (secure 0 0 0) 0x7f23580c02f0 con 0x7f236c1049c0 2024-09-12T01:37:24.515 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_metadata' removed 2024-09-12T01:37:24.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.520+0000 7f2370c00640 1 -- 172.21.15.1:0/622345364 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f23500a2b80 msgr2=0x7f23500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:24.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.520+0000 7f2370c00640 1 --2- 172.21.15.1:0/622345364 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f23500a2b80 0x7f23500a5040 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f235c009fc0 tx=0x7f235c005f90 comp rx=0 tx=0).stop 2024-09-12T01:37:24.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.520+0000 7f2370c00640 1 -- 172.21.15.1:0/622345364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f236c1049c0 msgr2=0x7f236c19ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:24.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.520+0000 7f2370c00640 1 --2- 172.21.15.1:0/622345364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f236c1049c0 0x7f236c19ac50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2358004770 tx=0x7f2358003c10 comp rx=0 tx=0).stop 2024-09-12T01:37:24.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.521+0000 7f2370c00640 1 -- 172.21.15.1:0/622345364 shutdown_connections 2024-09-12T01:37:24.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.521+0000 7f2370c00640 1 --2- 172.21.15.1:0/622345364 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f23500a2b80 0x7f23500a5040 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:24.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.521+0000 7f2370c00640 1 --2- 172.21.15.1:0/622345364 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f236c109140 0x7f236c19efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:24.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.521+0000 7f2370c00640 1 --2- 172.21.15.1:0/622345364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f236c1049c0 0x7f236c19ac50 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:24.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.521+0000 7f2370c00640 1 --2- 172.21.15.1:0/622345364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f236c104010 0x7f236c19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:24.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.521+0000 7f2370c00640 1 -- 172.21.15.1:0/622345364 >> 172.21.15.1:0/622345364 conn(0x7f236c0ffa50 msgr2=0x7f236c100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:24.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.521+0000 7f2370c00640 1 -- 172.21.15.1:0/622345364 shutdown_connections 2024-09-12T01:37:24.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:24.521+0000 7f2370c00640 1 -- 172.21.15.1:0/622345364 wait complete. 2024-09-12T01:37:24.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:24 smithi104 ceph-mon[35219]: pgmap v633: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-12T01:37:24.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:24 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/622345364' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T01:37:24.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:24 smithi001 ceph-mon[30387]: pgmap v633: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-12T01:37:24.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:24 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/622345364' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T01:37:24.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:24 smithi053 ceph-mon[35158]: pgmap v633: 129 pgs: 129 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-12T01:37:24.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:24 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/622345364' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T01:37:25.361 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-12T01:37:25.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:25 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/622345364' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T01:37:25.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:25 smithi001 ceph-mon[30387]: osdmap e100: 12 total, 12 up, 12 in 2024-09-12T01:37:25.738 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:37:25.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:25 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/622345364' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T01:37:25.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:25 smithi104 ceph-mon[35219]: osdmap e100: 12 total, 12 up, 12 in 2024-09-12T01:37:25.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:25 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/622345364' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T01:37:25.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:25 smithi053 ceph-mon[35158]: osdmap e100: 12 total, 12 up, 12 in 2024-09-12T01:37:26.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.299+0000 7ffb05200640 1 -- 172.21.15.1:0/2040852341 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb00109120 msgr2=0x7ffb00110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:26.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.299+0000 7ffb05200640 1 --2- 172.21.15.1:0/2040852341 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb00109120 0x7ffb00110ca0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8009ca0 tx=0x7ffaf802a8c0 comp rx=0 tx=0).stop 2024-09-12T01:37:26.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.299+0000 7ffb05200640 1 -- 172.21.15.1:0/2040852341 shutdown_connections 2024-09-12T01:37:26.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.299+0000 7ffb05200640 1 --2- 172.21.15.1:0/2040852341 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb00109120 0x7ffb00110ca0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:26.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.299+0000 7ffb05200640 1 --2- 172.21.15.1:0/2040852341 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb001049a0 0x7ffb001089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:26.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.299+0000 7ffb05200640 1 --2- 172.21.15.1:0/2040852341 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb00103ff0 0x7ffb001043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:26.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.299+0000 7ffb05200640 1 -- 172.21.15.1:0/2040852341 >> 172.21.15.1:0/2040852341 conn(0x7ffb000ffa90 msgr2=0x7ffb00101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:26.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.300+0000 7ffb05200640 1 -- 172.21.15.1:0/2040852341 shutdown_connections 2024-09-12T01:37:26.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.300+0000 7ffb05200640 1 -- 172.21.15.1:0/2040852341 wait complete. 2024-09-12T01:37:26.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.300+0000 7ffb05200640 1 Processor -- start 2024-09-12T01:37:26.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.301+0000 7ffb05200640 1 -- start start 2024-09-12T01:37:26.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.301+0000 7ffb05200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb00103ff0 0x7ffb0019a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:26.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.301+0000 7ffb05200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb001049a0 0x7ffb0019aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:26.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.301+0000 7ffb05200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb00109120 0x7ffb0019ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:26.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.301+0000 7ffb05200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ffb00114150 con 0x7ffb00109120 2024-09-12T01:37:26.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.301+0000 7ffb05200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ffb00113fd0 con 0x7ffb00103ff0 2024-09-12T01:37:26.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.301+0000 7ffb05200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ffb001142d0 con 0x7ffb001049a0 2024-09-12T01:37:26.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.301+0000 7ffafe800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb00109120 0x7ffb0019ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:26.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.301+0000 7ffafde00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb00103ff0 0x7ffb0019a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:26.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.301+0000 7ffaf7e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb001049a0 0x7ffb0019aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:26.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.301+0000 7ffafde00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb00103ff0 0x7ffb0019a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:40460/0 (socket says 172.21.15.1:40460) 2024-09-12T01:37:26.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.301+0000 7ffafde00640 1 -- 172.21.15.1:0/204694524 learned_addr learned my addr 172.21.15.1:0/204694524 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:37:26.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.301+0000 7ffafe800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb00109120 0x7ffb0019ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51302/0 (socket says 172.21.15.1:51302) 2024-09-12T01:37:26.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.302+0000 7ffafde00640 1 -- 172.21.15.1:0/204694524 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb001049a0 msgr2=0x7ffb0019aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:26.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.302+0000 7ffafde00640 1 --2- 172.21.15.1:0/204694524 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb001049a0 0x7ffb0019aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:26.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.302+0000 7ffafde00640 1 -- 172.21.15.1:0/204694524 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb00109120 msgr2=0x7ffb0019ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:26.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.302+0000 7ffafde00640 1 --2- 172.21.15.1:0/204694524 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb00109120 0x7ffb0019ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:26.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.302+0000 7ffafde00640 1 -- 172.21.15.1:0/204694524 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb0019f560 con 0x7ffb00103ff0 2024-09-12T01:37:26.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.302+0000 7ffaf7e00640 1 --2- 172.21.15.1:0/204694524 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb001049a0 0x7ffb0019aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:37:26.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.302+0000 7ffafe800640 1 --2- 172.21.15.1:0/204694524 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb00109120 0x7ffb0019ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:37:26.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.302+0000 7ffafde00640 1 --2- 172.21.15.1:0/204694524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb00103ff0 0x7ffb0019a5b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ffaf000e9f0 tx=0x7ffaf000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:26.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.302+0000 7ffaf6a00640 1 -- 172.21.15.1:0/204694524 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf000cd80 con 0x7ffb00103ff0 2024-09-12T01:37:26.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.302+0000 7ffb05200640 1 -- 172.21.15.1:0/204694524 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb0019f850 con 0x7ffb00103ff0 2024-09-12T01:37:26.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.303+0000 7ffaf6a00640 1 -- 172.21.15.1:0/204694524 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffaf0010910 con 0x7ffb00103ff0 2024-09-12T01:37:26.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.303+0000 7ffaf6a00640 1 -- 172.21.15.1:0/204694524 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf00188f0 con 0x7ffb00103ff0 2024-09-12T01:37:26.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.303+0000 7ffb05200640 1 -- 172.21.15.1:0/204694524 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb001a7130 con 0x7ffb00103ff0 2024-09-12T01:37:26.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.304+0000 7ffb05200640 1 -- 172.21.15.1:0/204694524 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb001043d0 con 0x7ffb00103ff0 2024-09-12T01:37:26.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.306+0000 7ffaf6a00640 1 -- 172.21.15.1:0/204694524 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ffaf0010ac0 con 0x7ffb00103ff0 2024-09-12T01:37:26.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.307+0000 7ffaf6a00640 1 --2- 172.21.15.1:0/204694524 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ffae40a2b80 0x7ffae40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:26.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.307+0000 7ffaf6a00640 1 -- 172.21.15.1:0/204694524 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8997+0+0 (secure 0 0 0) 0x7ffaf00cfb30 con 0x7ffb00103ff0 2024-09-12T01:37:26.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.307+0000 7ffaf7e00640 1 --2- 172.21.15.1:0/204694524 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ffae40a2b80 0x7ffae40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:26.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.308+0000 7ffaf7e00640 1 --2- 172.21.15.1:0/204694524 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ffae40a2b80 0x7ffae40a5040 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ffb000ffeb0 tx=0x7ffae8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:26.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.309+0000 7ffaf6a00640 1 -- 172.21.15.1:0/204694524 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffaf009c0e0 con 0x7ffb00103ff0 2024-09-12T01:37:26.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.499+0000 7ffb05200640 1 -- 172.21.15.1:0/204694524 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ffb001a73a0 con 0x7ffb00103ff0 2024-09-12T01:37:26.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.534+0000 7ffaf6a00640 1 -- 172.21.15.1:0/204694524 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v101) ==== 169+0+0 (secure 0 0 0) 0x7ffaf00a03e0 con 0x7ffb00103ff0 2024-09-12T01:37:26.534 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_data' removed 2024-09-12T01:37:26.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.539+0000 7ffb05200640 1 -- 172.21.15.1:0/204694524 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ffae40a2b80 msgr2=0x7ffae40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:26.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.539+0000 7ffb05200640 1 --2- 172.21.15.1:0/204694524 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ffae40a2b80 0x7ffae40a5040 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ffb000ffeb0 tx=0x7ffae8005f90 comp rx=0 tx=0).stop 2024-09-12T01:37:26.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.539+0000 7ffb05200640 1 -- 172.21.15.1:0/204694524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb00103ff0 msgr2=0x7ffb0019a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:26.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.539+0000 7ffb05200640 1 --2- 172.21.15.1:0/204694524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb00103ff0 0x7ffb0019a5b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ffaf000e9f0 tx=0x7ffaf000eeb0 comp rx=0 tx=0).stop 2024-09-12T01:37:26.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.540+0000 7ffb05200640 1 -- 172.21.15.1:0/204694524 shutdown_connections 2024-09-12T01:37:26.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.540+0000 7ffb05200640 1 --2- 172.21.15.1:0/204694524 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb00109120 0x7ffb0019ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:26.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.540+0000 7ffb05200640 1 --2- 172.21.15.1:0/204694524 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7ffae40a2b80 0x7ffae40a5040 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:26.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.540+0000 7ffb05200640 1 --2- 172.21.15.1:0/204694524 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb001049a0 0x7ffb0019aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:26.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.540+0000 7ffb05200640 1 --2- 172.21.15.1:0/204694524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb00103ff0 0x7ffb0019a5b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:26.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.540+0000 7ffb05200640 1 -- 172.21.15.1:0/204694524 >> 172.21.15.1:0/204694524 conn(0x7ffb000ffa90 msgr2=0x7ffb00101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:26.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.541+0000 7ffb05200640 1 -- 172.21.15.1:0/204694524 shutdown_connections 2024-09-12T01:37:26.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:26.541+0000 7ffb05200640 1 -- 172.21.15.1:0/204694524 wait complete. 2024-09-12T01:37:26.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:26 smithi104 ceph-mon[35219]: pgmap v635: 97 pgs: 97 active+clean; 577 KiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:37:26.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:26 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/204694524' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T01:37:26.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:26 smithi104 ceph-mon[35219]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T01:37:26.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:26 smithi001 ceph-mon[30387]: pgmap v635: 97 pgs: 97 active+clean; 577 KiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:37:26.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:26 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/204694524' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T01:37:26.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:26 smithi001 ceph-mon[30387]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T01:37:26.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:26 smithi053 ceph-mon[35158]: pgmap v635: 97 pgs: 97 active+clean; 577 KiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:37:26.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:26 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/204694524' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T01:37:26.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:26 smithi053 ceph-mon[35158]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T01:37:27.483 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-12T01:37:27.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:27 smithi104 ceph-mon[35219]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T01:37:27.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:27 smithi104 ceph-mon[35219]: osdmap e101: 12 total, 12 up, 12 in 2024-09-12T01:37:27.860 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:37:27.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:27 smithi001 ceph-mon[30387]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T01:37:27.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:27 smithi001 ceph-mon[30387]: osdmap e101: 12 total, 12 up, 12 in 2024-09-12T01:37:27.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:27 smithi053 ceph-mon[35158]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T01:37:27.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:27 smithi053 ceph-mon[35158]: osdmap e101: 12 total, 12 up, 12 in 2024-09-12T01:37:28.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:28.530+0000 7f49a0e00640 1 -- 172.21.15.1:0/588169742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f499c1049a0 msgr2=0x7f499c1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:28.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:28.530+0000 7f49a0e00640 1 --2- 172.21.15.1:0/588169742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f499c1049a0 0x7f499c1089f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4988009900 tx=0x7f498802a410 comp rx=0 tx=0).stop 2024-09-12T01:37:28.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:28.530+0000 7f49a0e00640 1 -- 172.21.15.1:0/588169742 shutdown_connections 2024-09-12T01:37:28.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:28.530+0000 7f49a0e00640 1 --2- 172.21.15.1:0/588169742 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f499c109120 0x7f499c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:28.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:28.530+0000 7f49a0e00640 1 --2- 172.21.15.1:0/588169742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f499c1049a0 0x7f499c1089f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:28.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:28.530+0000 7f49a0e00640 1 --2- 172.21.15.1:0/588169742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f499c103ff0 0x7f499c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:28.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:28.530+0000 7f49a0e00640 1 -- 172.21.15.1:0/588169742 >> 172.21.15.1:0/588169742 conn(0x7f499c0ffa90 msgr2=0x7f499c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:28.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:28.530+0000 7f49a0e00640 1 -- 172.21.15.1:0/588169742 shutdown_connections 2024-09-12T01:37:28.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:28.530+0000 7f49a0e00640 1 -- 172.21.15.1:0/588169742 wait complete. 2024-09-12T01:37:28.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:28.531+0000 7f49a0e00640 1 Processor -- start 2024-09-12T01:37:28.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:28.531+0000 7f49a0e00640 1 -- start start 2024-09-12T01:37:28.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:28.531+0000 7f49a0e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f499c103ff0 0x7f499c110c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:28.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:28.531+0000 7f49a0e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f499c1049a0 0x7f499c10dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:28.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:28.531+0000 7f49a0e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f499c109120 0x7f499c10e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:28.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:28.531+0000 7f49a0e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f499c114120 con 0x7f499c103ff0 2024-09-12T01:37:28.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:28.531+0000 7f49a0e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f499c113fa0 con 0x7f499c1049a0 2024-09-12T01:37:28.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:28.531+0000 7f49a0e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f499c1142a0 con 0x7f499c109120 2024-09-12T01:37:28.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:28.532+0000 7f4999e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f499c103ff0 0x7f499c110c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:28.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:28.532+0000 7f4999400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f499c1049a0 0x7f499c10dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:28.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:28.532+0000 7f499a800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f499c109120 0x7f499c10e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:28.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:28.532+0000 7f4999e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f499c103ff0 0x7f499c110c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51310/0 (socket says 172.21.15.1:51310) 2024-09-12T01:37:28.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:28.532+0000 7f4999400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f499c1049a0 0x7f499c10dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:40490/0 (socket says 172.21.15.1:40490) 2024-09-12T01:37:28.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:28.532+0000 7f4999e00640 1 -- 172.21.15.1:0/1259306707 learned_addr learned my addr 172.21.15.1:0/1259306707 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-12T01:37:28.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:28.532+0000 7f499a800640 1 -- 172.21.15.1:0/1259306707 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f499c1049a0 msgr2=0x7f499c10dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:28.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:28.532+0000 7f499a800640 1 --2- 172.21.15.1:0/1259306707 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f499c1049a0 0x7f499c10dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:28.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:28.532+0000 7f499a800640 1 -- 172.21.15.1:0/1259306707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f499c103ff0 msgr2=0x7f499c110c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:28.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:28.532+0000 7f499a800640 1 --2- 172.21.15.1:0/1259306707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f499c103ff0 0x7f499c110c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:28.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:28.532+0000 7f499a800640 1 -- 172.21.15.1:0/1259306707 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f499c1aae60 con 0x7f499c109120 2024-09-12T01:37:28.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:28.532+0000 7f499a800640 1 --2- 172.21.15.1:0/1259306707 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f499c109120 0x7f499c10e2a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4994009fd0 tx=0x7f499400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:28.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:28.533+0000 7f4992a00640 1 -- 172.21.15.1:0/1259306707 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499400ce20 con 0x7f499c109120 2024-09-12T01:37:28.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:28.533+0000 7f49a0e00640 1 -- 172.21.15.1:0/1259306707 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f499c1ab030 con 0x7f499c109120 2024-09-12T01:37:28.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:28.533+0000 7f4992a00640 1 -- 172.21.15.1:0/1259306707 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4994005020 con 0x7f499c109120 2024-09-12T01:37:28.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:28.533+0000 7f4992a00640 1 -- 172.21.15.1:0/1259306707 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4994010720 con 0x7f499c109120 2024-09-12T01:37:28.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:28.533+0000 7f49a0e00640 1 -- 172.21.15.1:0/1259306707 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f499c1ab540 con 0x7f499c109120 2024-09-12T01:37:28.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:28.534+0000 7f49a0e00640 1 -- 172.21.15.1:0/1259306707 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f499c1054d0 con 0x7f499c109120 2024-09-12T01:37:28.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:28.536+0000 7f4992a00640 1 -- 172.21.15.1:0/1259306707 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f4994005540 con 0x7f499c109120 2024-09-12T01:37:28.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:28.537+0000 7f4992a00640 1 --2- 172.21.15.1:0/1259306707 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f49800a2c00 0x7f49800a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:28.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:28.537+0000 7f4992a00640 1 -- 172.21.15.1:0/1259306707 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8496+0+0 (secure 0 0 0) 0x7f49940cf8a0 con 0x7f499c109120 2024-09-12T01:37:28.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:28.537+0000 7f4999e00640 1 --2- 172.21.15.1:0/1259306707 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f49800a2c00 0x7f49800a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:28.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:28.538+0000 7f4999e00640 1 --2- 172.21.15.1:0/1259306707 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f49800a2c00 0x7f49800a50c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f498c009c90 tx=0x7f498c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:28.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:28.539+0000 7f4992a00640 1 -- 172.21.15.1:0/1259306707 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f499409c6c0 con 0x7f499c109120 2024-09-12T01:37:28.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:28.724+0000 7f49a0e00640 1 -- 172.21.15.1:0/1259306707 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f499c077d50 con 0x7f499c109120 2024-09-12T01:37:28.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:28 smithi104 ceph-mon[35219]: pgmap v637: 33 pgs: 33 active+clean; 577 KiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:37:28.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:28 smithi001 ceph-mon[30387]: pgmap v637: 33 pgs: 33 active+clean; 577 KiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:37:28.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:28 smithi053 ceph-mon[35158]: pgmap v637: 33 pgs: 33 active+clean; 577 KiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:37:29.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:29.554+0000 7f4992a00640 1 -- 172.21.15.1:0/1259306707 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v102) ==== 178+0+0 (secure 0 0 0) 0x7f499409bfc0 con 0x7f499c109120 2024-09-12T01:37:29.555 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_data_ec' removed 2024-09-12T01:37:29.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:29.559+0000 7f49a0e00640 1 -- 172.21.15.1:0/1259306707 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f49800a2c00 msgr2=0x7f49800a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:29.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:29.559+0000 7f49a0e00640 1 --2- 172.21.15.1:0/1259306707 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f49800a2c00 0x7f49800a50c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f498c009c90 tx=0x7f498c005f90 comp rx=0 tx=0).stop 2024-09-12T01:37:29.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:29.559+0000 7f49a0e00640 1 -- 172.21.15.1:0/1259306707 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f499c109120 msgr2=0x7f499c10e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:29.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:29.559+0000 7f49a0e00640 1 --2- 172.21.15.1:0/1259306707 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f499c109120 0x7f499c10e2a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4994009fd0 tx=0x7f499400ef10 comp rx=0 tx=0).stop 2024-09-12T01:37:29.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:29.561+0000 7f49a0e00640 1 -- 172.21.15.1:0/1259306707 shutdown_connections 2024-09-12T01:37:29.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:29.561+0000 7f49a0e00640 1 --2- 172.21.15.1:0/1259306707 >> [v2:172.21.15.1:6800/1928662456,v1:172.21.15.1:6801/1928662456] conn(0x7f49800a2c00 0x7f49800a50c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:29.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:29.561+0000 7f49a0e00640 1 --2- 172.21.15.1:0/1259306707 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f499c109120 0x7f499c10e2a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:29.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:29.561+0000 7f49a0e00640 1 --2- 172.21.15.1:0/1259306707 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f499c1049a0 0x7f499c10dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:29.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:29.561+0000 7f49a0e00640 1 --2- 172.21.15.1:0/1259306707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f499c103ff0 0x7f499c110c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:29.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:29.561+0000 7f49a0e00640 1 -- 172.21.15.1:0/1259306707 >> 172.21.15.1:0/1259306707 conn(0x7f499c0ffa90 msgr2=0x7f499c100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:29.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:29.562+0000 7f49a0e00640 1 -- 172.21.15.1:0/1259306707 shutdown_connections 2024-09-12T01:37:29.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-12T01:37:29.563+0000 7f49a0e00640 1 -- 172.21.15.1:0/1259306707 wait complete. 2024-09-12T01:37:29.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:29 smithi104 ceph-mon[35219]: from='client.? 172.21.15.1:0/1259306707' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T01:37:29.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:29 smithi104 ceph-mon[35219]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T01:37:29.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:29 smithi001 ceph-mon[30387]: from='client.? 172.21.15.1:0/1259306707' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T01:37:29.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:29 smithi001 ceph-mon[30387]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T01:37:29.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:29 smithi053 ceph-mon[35158]: from='client.? 172.21.15.1:0/1259306707' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T01:37:29.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:29 smithi053 ceph-mon[35158]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T01:37:30.367 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T01:37:30.426 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T01:37:30.460 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T01:37:30.513 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-12T01:37:30.513 DEBUG:teuthology.orchestra.run.smithi001:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T01:37:30.565 DEBUG:teuthology.orchestra.run.smithi053:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T01:37:30.581 DEBUG:teuthology.orchestra.run.smithi104:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T01:37:30.597 INFO:tasks.cephadm:Stopping all daemons... 2024-09-12T01:37:30.597 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-12T01:37:30.597 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mon.a 2024-09-12T01:37:30.646 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:30 smithi001 ceph-mon[30387]: pgmap v638: 33 pgs: 33 active+clean; 577 KiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:37:30.646 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:30 smithi001 ceph-mon[30387]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T01:37:30.647 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:30 smithi001 ceph-mon[30387]: osdmap e102: 12 total, 12 up, 12 in 2024-09-12T01:37:30.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:30 smithi053 ceph-mon[35158]: pgmap v638: 33 pgs: 33 active+clean; 577 KiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:37:30.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:30 smithi053 ceph-mon[35158]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T01:37:30.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:30 smithi053 ceph-mon[35158]: osdmap e102: 12 total, 12 up, 12 in 2024-09-12T01:37:30.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:30 smithi104 ceph-mon[35219]: pgmap v638: 33 pgs: 33 active+clean; 577 KiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:37:30.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:30 smithi104 ceph-mon[35219]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T01:37:30.825 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:30 smithi104 ceph-mon[35219]: osdmap e102: 12 total, 12 up, 12 in 2024-09-12T01:37:30.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:30 smithi001 systemd[1]: Stopping Ceph mon.a for c0e4f180-70a4-11ef-bcea-c7b262605968... 2024-09-12T01:37:31.429 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:31 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mon-a[30383]: 2024-09-12T01:37:31.021+0000 7f332dc00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:37:31.429 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:31 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mon-a[30383]: 2024-09-12T01:37:31.021+0000 7f332dc00640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-12T01:37:31.429 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:31 smithi001 podman[108852]: 2024-09-12 01:37:31.062761448 +0000 UTC m=+0.177007299 container died 72a843457ce67b7e125e8b9fb9e94d917efcc71d6d6ce00727dbb3f1b9e267b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=) 2024-09-12T01:37:31.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:31 smithi001 podman[108852]: 2024-09-12 01:37:31.601129691 +0000 UTC m=+0.715375545 container cleanup 72a843457ce67b7e125e8b9fb9e94d917efcc71d6d6ce00727dbb3f1b9e267b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T01:37:31.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:31 smithi001 bash[108852]: ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mon-a 2024-09-12T01:37:32.335 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 12 01:37:32 smithi001 podman[108864]: 2024-09-12 01:37:32.014563507 +0000 UTC m=+0.940264610 container remove 72a843457ce67b7e125e8b9fb9e94d917efcc71d6d6ce00727dbb3f1b9e267b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mon-a, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-12T01:37:32.359 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mon.a.service' 2024-09-12T01:37:32.455 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:37:32.456 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-12T01:37:32.456 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-12T01:37:32.456 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mon.b 2024-09-12T01:37:32.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:32 smithi053 systemd[1]: Stopping Ceph mon.b for c0e4f180-70a4-11ef-bcea-c7b262605968... 2024-09-12T01:37:33.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:32 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mon-b[35154]: 2024-09-12T01:37:32.806+0000 7fe53bc30640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:37:33.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:32 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mon-b[35154]: 2024-09-12T01:37:32.806+0000 7fe53bc30640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-12T01:37:33.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:32 smithi053 podman[69067]: 2024-09-12 01:37:32.848103438 +0000 UTC m=+0.178657827 container stop a344fc8ebfb66e31006082877ac60329fe62e38c30afe02d745d460e84c957cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mon-b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS) 2024-09-12T01:37:33.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:32 smithi053 podman[69067]: 2024-09-12 01:37:32.873950922 +0000 UTC m=+0.204505310 container died a344fc8ebfb66e31006082877ac60329fe62e38c30afe02d745d460e84c957cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mon-b, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-12T01:37:33.682 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:33 smithi053 podman[69067]: 2024-09-12 01:37:33.398302183 +0000 UTC m=+0.728856571 container cleanup a344fc8ebfb66e31006082877ac60329fe62e38c30afe02d745d460e84c957cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mon-b, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2) 2024-09-12T01:37:33.683 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:33 smithi053 bash[69067]: ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mon-b 2024-09-12T01:37:33.966 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 12 01:37:33 smithi053 podman[69081]: 2024-09-12 01:37:33.683527519 +0000 UTC m=+0.829868584 container remove a344fc8ebfb66e31006082877ac60329fe62e38c30afe02d745d460e84c957cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mon-b, org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T01:37:33.986 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mon.b.service' 2024-09-12T01:37:34.032 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:37:34.032 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-12T01:37:34.033 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-12T01:37:34.033 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mon.c 2024-09-12T01:37:34.325 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:34 smithi104 systemd[1]: Stopping Ceph mon.c for c0e4f180-70a4-11ef-bcea-c7b262605968... 2024-09-12T01:37:34.806 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:34 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mon-c[35215]: 2024-09-12T01:37:34.366+0000 7f9d85b2f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:37:34.806 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:34 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mon-c[35215]: 2024-09-12T01:37:34.366+0000 7f9d85b2f640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-12T01:37:34.806 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:34 smithi104 podman[69333]: 2024-09-12 01:37:34.510202629 +0000 UTC m=+0.274296980 container died b8b8b09583ac35a38f447711ee391c25f3ed43b218ee97ece37d4dbfba0ce28c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mon-c, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-12T01:37:35.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:34 smithi104 podman[69333]: 2024-09-12 01:37:34.889719142 +0000 UTC m=+0.653813534 container cleanup b8b8b09583ac35a38f447711ee391c25f3ed43b218ee97ece37d4dbfba0ce28c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mon-c, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-12T01:37:35.075 INFO:journalctl@ceph.mon.c.smithi104.stdout:Sep 12 01:37:34 smithi104 bash[69333]: ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mon-c 2024-09-12T01:37:35.314 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mon.c.service' 2024-09-12T01:37:35.355 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:37:35.355 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-12T01:37:35.355 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-12T01:37:35.355 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mgr.x 2024-09-12T01:37:35.678 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:37:35 smithi001 systemd[1]: Stopping Ceph mgr.x for c0e4f180-70a4-11ef-bcea-c7b262605968... 2024-09-12T01:37:36.178 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:37:35 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:37:35.807+0000 7feb9a000640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:37:36.178 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:37:35 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x[30611]: 2024-09-12T01:37:35.807+0000 7feb9a000640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-12T01:37:36.178 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:37:35 smithi001 podman[108974]: 2024-09-12 01:37:35.89190365 +0000 UTC m=+0.248215919 container died 37d4931d977d617b131dfa8fdec6437092701d746dec92db2ab59df09b2d5822 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-12T01:37:36.641 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:37:36 smithi001 podman[108974]: 2024-09-12 01:37:36.413752477 +0000 UTC m=+0.770064745 container cleanup 37d4931d977d617b131dfa8fdec6437092701d746dec92db2ab59df09b2d5822 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3) 2024-09-12T01:37:36.641 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:37:36 smithi001 bash[108974]: ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x 2024-09-12T01:37:36.903 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 12 01:37:36 smithi001 podman[108988]: 2024-09-12 01:37:36.641460342 +0000 UTC m=+0.738784321 container remove 37d4931d977d617b131dfa8fdec6437092701d746dec92db2ab59df09b2d5822 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-x, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-12T01:37:36.912 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-12T01:37:36.984 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:37:36.984 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-12T01:37:36.985 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-12T01:37:36.985 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mgr.y 2024-09-12T01:37:37.323 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 12 01:37:37 smithi053 systemd[1]: Stopping Ceph mgr.y for c0e4f180-70a4-11ef-bcea-c7b262605968... 2024-09-12T01:37:37.638 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 12 01:37:37 smithi053 podman[69202]: 2024-09-12 01:37:37.32446936 +0000 UTC m=+0.139866524 container died d40466187460d25b017b58541ba5974f7c2626c107cbc66d861b3c0a8cd985f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-12T01:37:37.988 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 12 01:37:37 smithi053 podman[69202]: 2024-09-12 01:37:37.746360435 +0000 UTC m=+0.561757590 container cleanup d40466187460d25b017b58541ba5974f7c2626c107cbc66d861b3c0a8cd985f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-y, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-12T01:37:37.989 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 12 01:37:37 smithi053 bash[69202]: ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-y 2024-09-12T01:37:38.225 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-12T01:37:38.240 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 12 01:37:37 smithi053 podman[69214]: 2024-09-12 01:37:37.989803636 +0000 UTC m=+0.660923045 container remove d40466187460d25b017b58541ba5974f7c2626c107cbc66d861b3c0a8cd985f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-12T01:37:38.240 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 12 01:37:37 smithi053 systemd[1]: ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-09-12T01:37:38.240 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 12 01:37:38 smithi053 systemd[1]: ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-09-12T01:37:38.240 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 12 01:37:38 smithi053 systemd[1]: Stopped Ceph mgr.y for c0e4f180-70a4-11ef-bcea-c7b262605968. 2024-09-12T01:37:38.240 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 12 01:37:38 smithi053 systemd[1]: ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mgr.y.service: Consumed 6.680s CPU time. 2024-09-12T01:37:38.262 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:37:38.262 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-12T01:37:38.262 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-12T01:37:38.262 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mgr.z 2024-09-12T01:37:38.575 INFO:journalctl@ceph.mgr.z.smithi104.stdout:Sep 12 01:37:38 smithi104 systemd[1]: Stopping Ceph mgr.z for c0e4f180-70a4-11ef-bcea-c7b262605968... 2024-09-12T01:37:38.954 INFO:journalctl@ceph.mgr.z.smithi104.stdout:Sep 12 01:37:38 smithi104 podman[69456]: 2024-09-12 01:37:38.649000423 +0000 UTC m=+0.195661973 container died 240430cd19ba97a027b8a4e3c60b392ac5bde645c55be8485eea1850a6e37a17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-z, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-12T01:37:39.279 INFO:journalctl@ceph.mgr.z.smithi104.stdout:Sep 12 01:37:39 smithi104 podman[69456]: 2024-09-12 01:37:39.07051892 +0000 UTC m=+0.617180470 container cleanup 240430cd19ba97a027b8a4e3c60b392ac5bde645c55be8485eea1850a6e37a17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-z, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T01:37:39.279 INFO:journalctl@ceph.mgr.z.smithi104.stdout:Sep 12 01:37:39 smithi104 bash[69456]: ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mgr-z 2024-09-12T01:37:39.494 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mgr.z.service' 2024-09-12T01:37:39.528 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:37:39.528 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-12T01:37:39.528 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-12T01:37:39.528 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@osd.0 2024-09-12T01:37:39.928 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 12 01:37:39 smithi001 systemd[1]: Stopping Ceph osd.0 for c0e4f180-70a4-11ef-bcea-c7b262605968... 2024-09-12T01:37:40.428 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 12 01:37:39 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-0[44328]: 2024-09-12T01:37:39.947+0000 7f0aac200640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:37:40.429 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 12 01:37:39 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-0[44328]: 2024-09-12T01:37:39.947+0000 7f0aac200640 -1 osd.0 102 *** Got signal Terminated *** 2024-09-12T01:37:40.429 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 12 01:37:39 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-0[44328]: 2024-09-12T01:37:39.947+0000 7f0aac200640 -1 osd.0 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T01:37:45.913 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 12 01:37:45 smithi001 podman[109095]: 2024-09-12 01:37:45.566375474 +0000 UTC m=+5.748901010 container died 5f728dbef0a61b94f8955797c55bedfb5f55cfef85dbe50c8ba72ac642063972 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T01:37:46.178 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 12 01:37:46 smithi001 podman[109095]: 2024-09-12 01:37:46.094679678 +0000 UTC m=+6.277205207 container cleanup 5f728dbef0a61b94f8955797c55bedfb5f55cfef85dbe50c8ba72ac642063972 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T01:37:46.179 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 12 01:37:46 smithi001 bash[109095]: ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-0 2024-09-12T01:37:46.678 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 12 01:37:46 smithi001 podman[109111]: 2024-09-12 01:37:46.304648509 +0000 UTC m=+0.726923251 container remove 5f728dbef0a61b94f8955797c55bedfb5f55cfef85dbe50c8ba72ac642063972 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3) 2024-09-12T01:37:47.125 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 12 01:37:46 smithi001 podman[109206]: 2024-09-12 01:37:46.753512733 +0000 UTC m=+0.027288933 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:37:47.125 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 12 01:37:46 smithi001 podman[109206]: 2024-09-12 01:37:46.889156908 +0000 UTC m=+0.162933103 container create 30a0fabc7f47f685235210bf20f90e1046074ed5ebec5db7844115a4eea29608 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-12T01:37:47.395 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 12 01:37:47 smithi001 podman[109206]: 2024-09-12 01:37:47.12539103 +0000 UTC m=+0.399167218 container init 30a0fabc7f47f685235210bf20f90e1046074ed5ebec5db7844115a4eea29608 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T01:37:47.395 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 12 01:37:47 smithi001 podman[109206]: 2024-09-12 01:37:47.132982945 +0000 UTC m=+0.406759133 container start 30a0fabc7f47f685235210bf20f90e1046074ed5ebec5db7844115a4eea29608 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-0-deactivate, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T01:37:47.395 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 12 01:37:47 smithi001 podman[109206]: 2024-09-12 01:37:47.225189528 +0000 UTC m=+0.498965733 container attach 30a0fabc7f47f685235210bf20f90e1046074ed5ebec5db7844115a4eea29608 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0) 2024-09-12T01:37:47.655 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 12 01:37:47 smithi001 podman[109206]: 2024-09-12 01:37:47.39651692 +0000 UTC m=+0.670293115 container died 30a0fabc7f47f685235210bf20f90e1046074ed5ebec5db7844115a4eea29608 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T01:37:48.189 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@osd.0.service' 2024-09-12T01:37:48.267 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:37:48.268 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-12T01:37:48.268 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-12T01:37:48.268 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@osd.1 2024-09-12T01:37:48.634 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 12 01:37:48 smithi053 systemd[1]: Stopping Ceph osd.1 for c0e4f180-70a4-11ef-bcea-c7b262605968... 2024-09-12T01:37:48.990 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 12 01:37:48 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-1[39630]: 2024-09-12T01:37:48.634+0000 7fdf76ed7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:37:48.990 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 12 01:37:48 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-1[39630]: 2024-09-12T01:37:48.634+0000 7fdf76ed7640 -1 osd.1 102 *** Got signal Terminated *** 2024-09-12T01:37:48.990 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 12 01:37:48 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-1[39630]: 2024-09-12T01:37:48.634+0000 7fdf76ed7640 -1 osd.1 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T01:37:54.240 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 12 01:37:53 smithi053 podman[69326]: 2024-09-12 01:37:53.983478159 +0000 UTC m=+5.516553038 container died 24dedaea265f751b96b78a389f7a983b3ea1b3480fbcd21f1e74f99b8ef2c874 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-1, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-12T01:37:54.636 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 12 01:37:54 smithi053 podman[69326]: 2024-09-12 01:37:54.393951961 +0000 UTC m=+5.927026840 container cleanup 24dedaea265f751b96b78a389f7a983b3ea1b3480fbcd21f1e74f99b8ef2c874 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-1, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-12T01:37:54.636 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 12 01:37:54 smithi053 bash[69326]: ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-1 2024-09-12T01:37:54.990 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 12 01:37:54 smithi053 podman[69341]: 2024-09-12 01:37:54.637203789 +0000 UTC m=+0.650669198 container remove 24dedaea265f751b96b78a389f7a983b3ea1b3480fbcd21f1e74f99b8ef2c874 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-12T01:37:55.373 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 12 01:37:55 smithi053 podman[69431]: 2024-09-12 01:37:54.970803364 +0000 UTC m=+0.015616583 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:37:55.373 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 12 01:37:55 smithi053 podman[69431]: 2024-09-12 01:37:55.113308583 +0000 UTC m=+0.158121833 container create 66bd7a88f4f593ab2eb686e762a05dfc39e0ecd86c1d032be1c95028a7cc2309 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-12T01:37:55.740 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 12 01:37:55 smithi053 podman[69431]: 2024-09-12 01:37:55.374252015 +0000 UTC m=+0.419065236 container init 66bd7a88f4f593ab2eb686e762a05dfc39e0ecd86c1d032be1c95028a7cc2309 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS) 2024-09-12T01:37:55.740 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 12 01:37:55 smithi053 podman[69431]: 2024-09-12 01:37:55.377786169 +0000 UTC m=+0.422599382 container start 66bd7a88f4f593ab2eb686e762a05dfc39e0ecd86c1d032be1c95028a7cc2309 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T01:37:55.740 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 12 01:37:55 smithi053 podman[69431]: 2024-09-12 01:37:55.482778126 +0000 UTC m=+0.527591335 container attach 66bd7a88f4f593ab2eb686e762a05dfc39e0ecd86c1d032be1c95028a7cc2309 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-12T01:37:55.740 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 12 01:37:55 smithi053 podman[69431]: 2024-09-12 01:37:55.566595646 +0000 UTC m=+0.611408864 container died 66bd7a88f4f593ab2eb686e762a05dfc39e0ecd86c1d032be1c95028a7cc2309 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-12T01:37:56.232 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@osd.1.service' 2024-09-12T01:37:56.274 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:37:56.274 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-12T01:37:56.274 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-12T01:37:56.275 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@osd.2 2024-09-12T01:37:56.575 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 12 01:37:56 smithi104 systemd[1]: Stopping Ceph osd.2 for c0e4f180-70a4-11ef-bcea-c7b262605968... 2024-09-12T01:37:56.950 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 12 01:37:56 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-2[40211]: 2024-09-12T01:37:56.606+0000 7f056395d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:37:56.950 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 12 01:37:56 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-2[40211]: 2024-09-12T01:37:56.606+0000 7f056395d640 -1 osd.2 102 *** Got signal Terminated *** 2024-09-12T01:37:56.950 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 12 01:37:56 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-2[40211]: 2024-09-12T01:37:56.606+0000 7f056395d640 -1 osd.2 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T01:38:02.256 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 12 01:38:01 smithi104 podman[69584]: 2024-09-12 01:38:01.993598996 +0000 UTC m=+5.527935938 container died 7efa77d25e68f00c6a7c19c80e0c540888c57ad80870acec6d3b71da63cb98c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-2, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-12T01:38:02.575 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 12 01:38:02 smithi104 podman[69584]: 2024-09-12 01:38:02.356404357 +0000 UTC m=+5.890741298 container cleanup 7efa77d25e68f00c6a7c19c80e0c540888c57ad80870acec6d3b71da63cb98c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-12T01:38:02.575 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 12 01:38:02 smithi104 bash[69584]: ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-2 2024-09-12T01:38:02.972 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 12 01:38:02 smithi104 podman[69613]: 2024-09-12 01:38:02.658174977 +0000 UTC m=+0.658736573 container remove 7efa77d25e68f00c6a7c19c80e0c540888c57ad80870acec6d3b71da63cb98c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731) 2024-09-12T01:38:03.260 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 12 01:38:02 smithi104 podman[69699]: 2024-09-12 01:38:02.875740689 +0000 UTC m=+0.015829283 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:38:03.261 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 12 01:38:02 smithi104 podman[69699]: 2024-09-12 01:38:02.992382924 +0000 UTC m=+0.132471509 container create ea6a4ae6fa519608d944dc28f44d723826d52183ef8b3c985f8d0df0666a8731 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731) 2024-09-12T01:38:03.261 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 12 01:38:03 smithi104 podman[69699]: 2024-09-12 01:38:03.261776672 +0000 UTC m=+0.401865260 container init ea6a4ae6fa519608d944dc28f44d723826d52183ef8b3c985f8d0df0666a8731 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-b53b6d3) 2024-09-12T01:38:03.575 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 12 01:38:03 smithi104 podman[69699]: 2024-09-12 01:38:03.265242564 +0000 UTC m=+0.405331157 container start ea6a4ae6fa519608d944dc28f44d723826d52183ef8b3c985f8d0df0666a8731 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-12T01:38:03.576 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 12 01:38:03 smithi104 podman[69699]: 2024-09-12 01:38:03.370121167 +0000 UTC m=+0.510209765 container attach ea6a4ae6fa519608d944dc28f44d723826d52183ef8b3c985f8d0df0666a8731 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T01:38:03.576 INFO:journalctl@ceph.osd.2.smithi104.stdout:Sep 12 01:38:03 smithi104 podman[69699]: 2024-09-12 01:38:03.450799068 +0000 UTC m=+0.590887656 container died ea6a4ae6fa519608d944dc28f44d723826d52183ef8b3c985f8d0df0666a8731 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-12T01:38:04.102 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@osd.2.service' 2024-09-12T01:38:04.145 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:38:04.145 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-12T01:38:04.145 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-12T01:38:04.145 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@osd.3 2024-09-12T01:38:04.644 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 12 01:38:04 smithi001 systemd[1]: Stopping Ceph osd.3 for c0e4f180-70a4-11ef-bcea-c7b262605968... 2024-09-12T01:38:04.928 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 12 01:38:04 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-3[50823]: 2024-09-12T01:38:04.642+0000 7f6b5c400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:38:04.928 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 12 01:38:04 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-3[50823]: 2024-09-12T01:38:04.642+0000 7f6b5c400640 -1 osd.3 102 *** Got signal Terminated *** 2024-09-12T01:38:04.928 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 12 01:38:04 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-3[50823]: 2024-09-12T01:38:04.642+0000 7f6b5c400640 -1 osd.3 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T01:38:07.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:06 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:06.840+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:08.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:07 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:07.861+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:09.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:08 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:08.854+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:09.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:09 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:09.453+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:09.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:09 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:09.471+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:10.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:09 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:09.865+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:10.576 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:10 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:10.427+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:10.577 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:10 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5[46675]: 2024-09-12T01:38:10.219+0000 7f586703b640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.703841+0000 front 2024-09-12T01:37:44.703958+0000 (oldest deadline 2024-09-12T01:38:10.003735+0000) 2024-09-12T01:38:10.599 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:10 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:10.521+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:10.599 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 12 01:38:10 smithi001 podman[109320]: 2024-09-12 01:38:10.288333305 +0000 UTC m=+5.836950366 container died 9010efe13821a475e483542089c581e22767a023df1eec355eb2a934f78da4d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-3, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-12T01:38:10.928 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 12 01:38:10 smithi001 podman[109320]: 2024-09-12 01:38:10.723597286 +0000 UTC m=+6.272214337 container cleanup 9010efe13821a475e483542089c581e22767a023df1eec355eb2a934f78da4d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-12T01:38:10.928 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 12 01:38:10 smithi001 bash[109320]: ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-3 2024-09-12T01:38:11.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:10 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:10.830+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:11.428 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 12 01:38:11 smithi001 podman[109336]: 2024-09-12 01:38:11.017073324 +0000 UTC m=+0.712892289 container remove 9010efe13821a475e483542089c581e22767a023df1eec355eb2a934f78da4d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-3, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T01:38:11.432 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:11 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5[46675]: 2024-09-12T01:38:11.181+0000 7f586703b640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.703841+0000 front 2024-09-12T01:37:44.703958+0000 (oldest deadline 2024-09-12T01:38:10.003735+0000) 2024-09-12T01:38:11.793 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:11 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:11.500+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:11.794 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 12 01:38:11 smithi001 podman[109426]: 2024-09-12 01:38:11.549048734 +0000 UTC m=+0.027870997 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:38:11.794 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 12 01:38:11 smithi001 podman[109426]: 2024-09-12 01:38:11.659567042 +0000 UTC m=+0.138389284 container create a85b9fc3908edb69e1aab75cb4d196f4241205c0dff8970f2749fe6abd3e74d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T01:38:11.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:11 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:11.433+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:12.178 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 12 01:38:11 smithi001 podman[109426]: 2024-09-12 01:38:11.92941085 +0000 UTC m=+0.408233092 container init a85b9fc3908edb69e1aab75cb4d196f4241205c0dff8970f2749fe6abd3e74d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-12T01:38:12.179 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 12 01:38:11 smithi001 podman[109426]: 2024-09-12 01:38:11.93617425 +0000 UTC m=+0.414996509 container start a85b9fc3908edb69e1aab75cb4d196f4241205c0dff8970f2749fe6abd3e74d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-12T01:38:12.179 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 12 01:38:12 smithi001 podman[109426]: 2024-09-12 01:38:12.029080903 +0000 UTC m=+0.507903152 container attach a85b9fc3908edb69e1aab75cb4d196f4241205c0dff8970f2749fe6abd3e74d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T01:38:12.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:11 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:11.805+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:12.418 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:12 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5[46675]: 2024-09-12T01:38:12.157+0000 7f586703b640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.703841+0000 front 2024-09-12T01:37:44.703958+0000 (oldest deadline 2024-09-12T01:38:10.003735+0000) 2024-09-12T01:38:12.549 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 12 01:38:12 smithi001 podman[109426]: 2024-09-12 01:38:12.204324498 +0000 UTC m=+0.683146741 container died a85b9fc3908edb69e1aab75cb4d196f4241205c0dff8970f2749fe6abd3e74d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-12T01:38:12.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:12 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:12.418+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:12.907 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:12 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:12.548+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:12.938 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@osd.3.service' 2024-09-12T01:38:13.015 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:38:13.015 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-12T01:38:13.015 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-12T01:38:13.016 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@osd.4 2024-09-12T01:38:13.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:12 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:12.834+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:13.240 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 12 01:38:13 smithi053 systemd[1]: Stopping Ceph osd.4 for c0e4f180-70a4-11ef-bcea-c7b262605968... 2024-09-12T01:38:13.443 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:13 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5[46675]: 2024-09-12T01:38:13.128+0000 7f586703b640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.703841+0000 front 2024-09-12T01:37:44.703958+0000 (oldest deadline 2024-09-12T01:38:10.003735+0000) 2024-09-12T01:38:13.740 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 12 01:38:13 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-4[46372]: 2024-09-12T01:38:13.456+0000 7f358dad7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:38:13.740 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 12 01:38:13 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-4[46372]: 2024-09-12T01:38:13.456+0000 7f358dad7640 -1 osd.4 102 *** Got signal Terminated *** 2024-09-12T01:38:13.740 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 12 01:38:13 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-4[46372]: 2024-09-12T01:38:13.456+0000 7f358dad7640 -1 osd.4 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T01:38:13.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:13 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:13.443+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:13.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:13 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:13.567+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:14.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:13 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:13.831+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:14.477 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:14 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5[46675]: 2024-09-12T01:38:14.090+0000 7f586703b640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.703841+0000 front 2024-09-12T01:37:44.703958+0000 (oldest deadline 2024-09-12T01:38:10.003735+0000) 2024-09-12T01:38:14.478 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:14 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5[46675]: 2024-09-12T01:38:14.090+0000 7f586703b640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.104737+0000 front 2024-09-12T01:37:51.104755+0000 (oldest deadline 2024-09-12T01:38:14.004876+0000) 2024-09-12T01:38:14.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:14 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:14.477+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:14.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:14 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:14.477+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:14.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:14 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:14.575+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:14.990 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:14 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:14.877+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:14.990 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:14 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:14.634+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.309568+0000 front 2024-09-12T01:37:51.309537+0000 (oldest deadline 2024-09-12T01:38:14.209738+0000) 2024-09-12T01:38:15.325 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:15 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5[46675]: 2024-09-12T01:38:15.129+0000 7f586703b640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.703841+0000 front 2024-09-12T01:37:44.703958+0000 (oldest deadline 2024-09-12T01:38:10.003735+0000) 2024-09-12T01:38:15.325 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:15 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5[46675]: 2024-09-12T01:38:15.129+0000 7f586703b640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.104737+0000 front 2024-09-12T01:37:51.104755+0000 (oldest deadline 2024-09-12T01:38:14.004876+0000) 2024-09-12T01:38:15.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:14 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:14.926+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:48.737024+0000 front 2024-09-12T01:37:48.736899+0000 (oldest deadline 2024-09-12T01:38:14.036808+0000) 2024-09-12T01:38:15.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:15 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:15.437+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:15.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:15 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:15.437+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:15.885 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:15 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:15.647+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.309568+0000 front 2024-09-12T01:37:51.309537+0000 (oldest deadline 2024-09-12T01:38:14.209738+0000) 2024-09-12T01:38:15.886 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 12 01:38:15 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-4[46372]: 2024-09-12T01:38:15.615+0000 7f358a0ed640 -1 osd.4 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:49.590309+0000 front 2024-09-12T01:37:49.590410+0000 (oldest deadline 2024-09-12T01:38:15.490433+0000) 2024-09-12T01:38:15.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:15 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:15.602+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:16.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:15 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:15.885+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:16.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:15 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:15.944+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:48.737024+0000 front 2024-09-12T01:37:48.736899+0000 (oldest deadline 2024-09-12T01:38:14.036808+0000) 2024-09-12T01:38:16.325 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:16 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5[46675]: 2024-09-12T01:38:16.175+0000 7f586703b640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.703841+0000 front 2024-09-12T01:37:44.703958+0000 (oldest deadline 2024-09-12T01:38:10.003735+0000) 2024-09-12T01:38:16.325 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:16 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5[46675]: 2024-09-12T01:38:16.175+0000 7f586703b640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.104737+0000 front 2024-09-12T01:37:51.104755+0000 (oldest deadline 2024-09-12T01:38:14.004876+0000) 2024-09-12T01:38:16.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:16 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:16.480+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:16.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:16 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:16.480+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:16.913 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:16 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:16.635+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.309568+0000 front 2024-09-12T01:37:51.309537+0000 (oldest deadline 2024-09-12T01:38:14.209738+0000) 2024-09-12T01:38:16.913 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 12 01:38:16 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-4[46372]: 2024-09-12T01:38:16.620+0000 7f358a0ed640 -1 osd.4 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:49.590309+0000 front 2024-09-12T01:37:49.590410+0000 (oldest deadline 2024-09-12T01:38:15.490433+0000) 2024-09-12T01:38:16.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:16 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:16.607+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:16.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:16 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:16.607+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.476477+0000 front 2024-09-12T01:37:51.476500+0000 (oldest deadline 2024-09-12T01:38:16.077208+0000) 2024-09-12T01:38:17.220 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:16 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:16.930+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:48.737024+0000 front 2024-09-12T01:37:48.736899+0000 (oldest deadline 2024-09-12T01:38:14.036808+0000) 2024-09-12T01:38:17.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:16 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:16.913+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:17.507 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:17 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5[46675]: 2024-09-12T01:38:17.220+0000 7f586703b640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.703841+0000 front 2024-09-12T01:37:44.703958+0000 (oldest deadline 2024-09-12T01:38:10.003735+0000) 2024-09-12T01:38:17.507 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:17 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5[46675]: 2024-09-12T01:38:17.221+0000 7f586703b640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.104737+0000 front 2024-09-12T01:37:51.104755+0000 (oldest deadline 2024-09-12T01:38:14.004876+0000) 2024-09-12T01:38:17.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:17 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:17.507+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:17.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:17 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:17.507+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:17.898 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 12 01:38:17 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-4[46372]: 2024-09-12T01:38:17.650+0000 7f358a0ed640 -1 osd.4 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:49.590309+0000 front 2024-09-12T01:37:49.590410+0000 (oldest deadline 2024-09-12T01:38:15.490433+0000) 2024-09-12T01:38:17.898 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:17 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:17.617+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.309568+0000 front 2024-09-12T01:37:51.309537+0000 (oldest deadline 2024-09-12T01:38:14.209738+0000) 2024-09-12T01:38:17.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:17 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:17.616+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:17.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:17 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:17.616+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.476477+0000 front 2024-09-12T01:37:51.476500+0000 (oldest deadline 2024-09-12T01:38:16.077208+0000) 2024-09-12T01:38:18.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:17 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:17.898+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:18.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:17 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:17.898+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:38:18.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:17 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:17.965+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:48.737024+0000 front 2024-09-12T01:37:48.736899+0000 (oldest deadline 2024-09-12T01:38:14.036808+0000) 2024-09-12T01:38:18.325 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:18 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5[46675]: 2024-09-12T01:38:18.213+0000 7f586703b640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.703841+0000 front 2024-09-12T01:37:44.703958+0000 (oldest deadline 2024-09-12T01:38:10.003735+0000) 2024-09-12T01:38:18.325 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:18 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5[46675]: 2024-09-12T01:38:18.213+0000 7f586703b640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.104737+0000 front 2024-09-12T01:37:51.104755+0000 (oldest deadline 2024-09-12T01:38:14.004876+0000) 2024-09-12T01:38:18.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:18 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:18.509+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:18.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:18 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:18.509+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:18.874 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 12 01:38:18 smithi053 podman[69552]: 2024-09-12 01:38:18.802830357 +0000 UTC m=+5.583978358 container died 3831e199beb4408e6753a6403e423f8abeb04ca63fd3c0bdfe7b9087738b6fba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-4, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0) 2024-09-12T01:38:18.875 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:18 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:18.575+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.309568+0000 front 2024-09-12T01:37:51.309537+0000 (oldest deadline 2024-09-12T01:38:14.209738+0000) 2024-09-12T01:38:18.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:18 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:18.637+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:18.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:18 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:18.637+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.476477+0000 front 2024-09-12T01:37:51.476500+0000 (oldest deadline 2024-09-12T01:38:16.077208+0000) 2024-09-12T01:38:19.140 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:18 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:18.874+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:19.140 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:18 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:18.874+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:38:19.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:18 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:18.984+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:48.737024+0000 front 2024-09-12T01:37:48.736899+0000 (oldest deadline 2024-09-12T01:38:14.036808+0000) 2024-09-12T01:38:19.325 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:19 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5[46675]: 2024-09-12T01:38:19.177+0000 7f586703b640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.703841+0000 front 2024-09-12T01:37:44.703958+0000 (oldest deadline 2024-09-12T01:38:10.003735+0000) 2024-09-12T01:38:19.325 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:19 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5[46675]: 2024-09-12T01:38:19.177+0000 7f586703b640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.104737+0000 front 2024-09-12T01:37:51.104755+0000 (oldest deadline 2024-09-12T01:38:14.004876+0000) 2024-09-12T01:38:19.490 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 12 01:38:19 smithi053 podman[69552]: 2024-09-12 01:38:19.140984539 +0000 UTC m=+5.922132528 container cleanup 3831e199beb4408e6753a6403e423f8abeb04ca63fd3c0bdfe7b9087738b6fba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-4, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T01:38:19.490 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 12 01:38:19 smithi053 bash[69552]: ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-4 2024-09-12T01:38:19.490 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 12 01:38:19 smithi053 podman[69568]: 2024-09-12 01:38:19.367464743 +0000 UTC m=+0.559012353 container remove 3831e199beb4408e6753a6403e423f8abeb04ca63fd3c0bdfe7b9087738b6fba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-4, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-12T01:38:19.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:19 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:19.545+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:19.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:19 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:19.545+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:19.891 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:19 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:19.592+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.309568+0000 front 2024-09-12T01:37:51.309537+0000 (oldest deadline 2024-09-12T01:38:14.209738+0000) 2024-09-12T01:38:19.891 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 12 01:38:19 smithi053 podman[69657]: 2024-09-12 01:38:19.715596698 +0000 UTC m=+0.016132397 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:38:19.891 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 12 01:38:19 smithi053 podman[69657]: 2024-09-12 01:38:19.826773769 +0000 UTC m=+0.127309459 container create e2e5b9a08ee884c506b976e62776b7373d25c8a210af419e01f96a1f723ce437 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True) 2024-09-12T01:38:19.915 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:19 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:19.612+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:19.915 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:19 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:19.612+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.476477+0000 front 2024-09-12T01:37:51.476500+0000 (oldest deadline 2024-09-12T01:38:16.077208+0000) 2024-09-12T01:38:20.181 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 12 01:38:19 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-6[57769]: 2024-09-12T01:38:19.914+0000 7fabfbe00640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.155703+0000 front 2024-09-12T01:37:53.155459+0000 (oldest deadline 2024-09-12T01:38:19.055328+0000) 2024-09-12T01:38:20.207 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:19 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:19.943+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:48.737024+0000 front 2024-09-12T01:37:48.736899+0000 (oldest deadline 2024-09-12T01:38:14.036808+0000) 2024-09-12T01:38:20.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:19 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:19.891+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:20.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:19 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:19.891+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:38:20.240 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 12 01:38:20 smithi053 podman[69657]: 2024-09-12 01:38:20.096177933 +0000 UTC m=+0.396713622 container init e2e5b9a08ee884c506b976e62776b7373d25c8a210af419e01f96a1f723ce437 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0) 2024-09-12T01:38:20.240 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 12 01:38:20 smithi053 podman[69657]: 2024-09-12 01:38:20.099375773 +0000 UTC m=+0.399911470 container start e2e5b9a08ee884c506b976e62776b7373d25c8a210af419e01f96a1f723ce437 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T01:38:20.536 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:20 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5[46675]: 2024-09-12T01:38:20.208+0000 7f586703b640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.703841+0000 front 2024-09-12T01:37:44.703958+0000 (oldest deadline 2024-09-12T01:38:10.003735+0000) 2024-09-12T01:38:20.536 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:20 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5[46675]: 2024-09-12T01:38:20.208+0000 7f586703b640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.104737+0000 front 2024-09-12T01:37:51.104755+0000 (oldest deadline 2024-09-12T01:38:14.004876+0000) 2024-09-12T01:38:20.545 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 12 01:38:20 smithi053 podman[69657]: 2024-09-12 01:38:20.329783733 +0000 UTC m=+0.630319504 container attach e2e5b9a08ee884c506b976e62776b7373d25c8a210af419e01f96a1f723ce437 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-4-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3) 2024-09-12T01:38:20.545 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 12 01:38:20 smithi053 podman[69657]: 2024-09-12 01:38:20.330316391 +0000 UTC m=+0.630852090 container died e2e5b9a08ee884c506b976e62776b7373d25c8a210af419e01f96a1f723ce437 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-4-deactivate, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-12T01:38:20.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:20 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:20.536+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:20.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:20 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:20.536+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:20.907 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:20 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:20.545+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.309568+0000 front 2024-09-12T01:37:51.309537+0000 (oldest deadline 2024-09-12T01:38:14.209738+0000) 2024-09-12T01:38:20.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:20 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:20.581+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:20.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:20 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:20.581+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.476477+0000 front 2024-09-12T01:37:51.476500+0000 (oldest deadline 2024-09-12T01:38:16.077208+0000) 2024-09-12T01:38:21.099 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@osd.4.service' 2024-09-12T01:38:21.141 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:38:21.141 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-12T01:38:21.142 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-12T01:38:21.142 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@osd.5 2024-09-12T01:38:21.168 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:20 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:20.917+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:48.737024+0000 front 2024-09-12T01:37:48.736899+0000 (oldest deadline 2024-09-12T01:38:14.036808+0000) 2024-09-12T01:38:21.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:20 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:20.907+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:21.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:20 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:20.907+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:38:21.428 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 12 01:38:20 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-6[57769]: 2024-09-12T01:38:20.952+0000 7fabfbe00640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.155703+0000 front 2024-09-12T01:37:53.155459+0000 (oldest deadline 2024-09-12T01:38:19.055328+0000) 2024-09-12T01:38:21.442 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:21 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5[46675]: 2024-09-12T01:38:21.168+0000 7f586703b640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.703841+0000 front 2024-09-12T01:37:44.703958+0000 (oldest deadline 2024-09-12T01:38:10.003735+0000) 2024-09-12T01:38:21.442 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:21 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5[46675]: 2024-09-12T01:38:21.168+0000 7f586703b640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.104737+0000 front 2024-09-12T01:37:51.104755+0000 (oldest deadline 2024-09-12T01:38:14.004876+0000) 2024-09-12T01:38:21.442 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:21 smithi104 systemd[1]: Stopping Ceph osd.5 for c0e4f180-70a4-11ef-bcea-c7b262605968... 2024-09-12T01:38:21.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:21 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:21.582+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:21.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:21 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:21.582+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:21.825 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:21 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5[46675]: 2024-09-12T01:38:21.442+0000 7f586ea2d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:38:21.825 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:21 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5[46675]: 2024-09-12T01:38:21.442+0000 7f586ea2d640 -1 osd.5 102 *** Got signal Terminated *** 2024-09-12T01:38:21.825 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:21 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5[46675]: 2024-09-12T01:38:21.442+0000 7f586ea2d640 -1 osd.5 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T01:38:21.922 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:21 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:21.514+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.309568+0000 front 2024-09-12T01:37:51.309537+0000 (oldest deadline 2024-09-12T01:38:14.209738+0000) 2024-09-12T01:38:21.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:21 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:21.619+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:21.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:21 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:21.619+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.476477+0000 front 2024-09-12T01:37:51.476500+0000 (oldest deadline 2024-09-12T01:38:16.077208+0000) 2024-09-12T01:38:22.167 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:21 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:21.903+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:48.737024+0000 front 2024-09-12T01:37:48.736899+0000 (oldest deadline 2024-09-12T01:38:14.036808+0000) 2024-09-12T01:38:22.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:21 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:21.922+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:22.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:21 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:21.922+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:38:22.428 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 12 01:38:21 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-6[57769]: 2024-09-12T01:38:21.974+0000 7fabfbe00640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.155703+0000 front 2024-09-12T01:37:53.155459+0000 (oldest deadline 2024-09-12T01:38:19.055328+0000) 2024-09-12T01:38:22.549 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:22 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5[46675]: 2024-09-12T01:38:22.167+0000 7f586703b640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.703841+0000 front 2024-09-12T01:37:44.703958+0000 (oldest deadline 2024-09-12T01:38:10.003735+0000) 2024-09-12T01:38:22.549 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:22 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5[46675]: 2024-09-12T01:38:22.167+0000 7f586703b640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.104737+0000 front 2024-09-12T01:37:51.104755+0000 (oldest deadline 2024-09-12T01:38:14.004876+0000) 2024-09-12T01:38:22.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:22 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:22.549+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:22.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:22 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:22.549+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:22.908 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:22 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:22.501+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.309568+0000 front 2024-09-12T01:37:51.309537+0000 (oldest deadline 2024-09-12T01:38:14.209738+0000) 2024-09-12T01:38:22.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:22 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:22.632+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:22.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:22 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:22.632+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.476477+0000 front 2024-09-12T01:37:51.476500+0000 (oldest deadline 2024-09-12T01:38:16.077208+0000) 2024-09-12T01:38:23.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:22 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:22.908+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:23.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:22 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:22.908+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:38:23.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:22 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:22.902+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:48.737024+0000 front 2024-09-12T01:37:48.736899+0000 (oldest deadline 2024-09-12T01:38:14.036808+0000) 2024-09-12T01:38:23.325 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:23 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5[46675]: 2024-09-12T01:38:23.148+0000 7f586703b640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.703841+0000 front 2024-09-12T01:37:44.703958+0000 (oldest deadline 2024-09-12T01:38:10.003735+0000) 2024-09-12T01:38:23.325 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:23 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5[46675]: 2024-09-12T01:38:23.148+0000 7f586703b640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.104737+0000 front 2024-09-12T01:37:51.104755+0000 (oldest deadline 2024-09-12T01:38:14.004876+0000) 2024-09-12T01:38:23.428 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 12 01:38:23 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-6[57769]: 2024-09-12T01:38:23.011+0000 7fabfbe00640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.155703+0000 front 2024-09-12T01:37:53.155459+0000 (oldest deadline 2024-09-12T01:38:19.055328+0000) 2024-09-12T01:38:23.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:23 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:23.534+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:23.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:23 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:23.534+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:23.912 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:23 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:23.500+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.309568+0000 front 2024-09-12T01:37:51.309537+0000 (oldest deadline 2024-09-12T01:38:14.209738+0000) 2024-09-12T01:38:23.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:23 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:23.662+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:23.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:23 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:23.662+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.476477+0000 front 2024-09-12T01:37:51.476500+0000 (oldest deadline 2024-09-12T01:38:16.077208+0000) 2024-09-12T01:38:24.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:23 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:23.912+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:24.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:23 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:23.912+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:38:24.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:23 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:23.902+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:48.737024+0000 front 2024-09-12T01:37:48.736899+0000 (oldest deadline 2024-09-12T01:38:14.036808+0000) 2024-09-12T01:38:24.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:23 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:23.902+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.338640+0000 front 2024-09-12T01:37:57.338659+0000 (oldest deadline 2024-09-12T01:38:23.238871+0000) 2024-09-12T01:38:24.325 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:24 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5[46675]: 2024-09-12T01:38:24.127+0000 7f586703b640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.703841+0000 front 2024-09-12T01:37:44.703958+0000 (oldest deadline 2024-09-12T01:38:10.003735+0000) 2024-09-12T01:38:24.325 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:24 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5[46675]: 2024-09-12T01:38:24.127+0000 7f586703b640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.104737+0000 front 2024-09-12T01:37:51.104755+0000 (oldest deadline 2024-09-12T01:38:14.004876+0000) 2024-09-12T01:38:24.325 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:24 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5[46675]: 2024-09-12T01:38:24.127+0000 7f586703b640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:58.705974+0000 front 2024-09-12T01:37:58.706001+0000 (oldest deadline 2024-09-12T01:38:24.006212+0000) 2024-09-12T01:38:24.428 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 12 01:38:24 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-6[57769]: 2024-09-12T01:38:24.051+0000 7fabfbe00640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.155703+0000 front 2024-09-12T01:37:53.155459+0000 (oldest deadline 2024-09-12T01:38:19.055328+0000) 2024-09-12T01:38:24.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:24 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:24.510+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:24.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:24 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:24.510+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:24.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:24 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:24.510+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:38:24.929 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:24 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:24.521+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.309568+0000 front 2024-09-12T01:37:51.309537+0000 (oldest deadline 2024-09-12T01:38:14.209738+0000) 2024-09-12T01:38:25.068 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:24 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:24.679+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:25.068 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:24 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:24.679+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.476477+0000 front 2024-09-12T01:37:51.476500+0000 (oldest deadline 2024-09-12T01:38:16.077208+0000) 2024-09-12T01:38:25.068 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:24 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:24.679+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.778585+0000 front 2024-09-12T01:37:57.778240+0000 (oldest deadline 2024-09-12T01:38:23.678115+0000) 2024-09-12T01:38:25.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:24 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:24.930+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:25.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:24 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:24.930+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:38:25.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:24 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:24.930+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.101595+0000 front 2024-09-12T01:38:00.101475+0000 (oldest deadline 2024-09-12T01:38:24.201476+0000) 2024-09-12T01:38:25.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:24 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:24.874+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:48.737024+0000 front 2024-09-12T01:37:48.736899+0000 (oldest deadline 2024-09-12T01:38:14.036808+0000) 2024-09-12T01:38:25.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:24 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:24.874+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.338640+0000 front 2024-09-12T01:37:57.338659+0000 (oldest deadline 2024-09-12T01:38:23.238871+0000) 2024-09-12T01:38:25.325 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:25 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5[46675]: 2024-09-12T01:38:25.113+0000 7f586703b640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.703841+0000 front 2024-09-12T01:37:44.703958+0000 (oldest deadline 2024-09-12T01:38:10.003735+0000) 2024-09-12T01:38:25.326 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:25 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5[46675]: 2024-09-12T01:38:25.113+0000 7f586703b640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.104737+0000 front 2024-09-12T01:37:51.104755+0000 (oldest deadline 2024-09-12T01:38:14.004876+0000) 2024-09-12T01:38:25.326 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:25 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5[46675]: 2024-09-12T01:38:25.113+0000 7f586703b640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:58.705974+0000 front 2024-09-12T01:37:58.706001+0000 (oldest deadline 2024-09-12T01:38:24.006212+0000) 2024-09-12T01:38:25.428 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 12 01:38:25 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-6[57769]: 2024-09-12T01:38:25.067+0000 7fabfbe00640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.155703+0000 front 2024-09-12T01:37:53.155459+0000 (oldest deadline 2024-09-12T01:38:19.055328+0000) 2024-09-12T01:38:25.428 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 12 01:38:25 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-6[57769]: 2024-09-12T01:38:25.067+0000 7fabfbe00640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:59.056032+0000 front 2024-09-12T01:37:59.056071+0000 (oldest deadline 2024-09-12T01:38:24.356013+0000) 2024-09-12T01:38:25.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:25 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:25.538+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:25.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:25 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:25.538+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:25.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:25 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:25.538+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:38:25.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:25 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:25.645+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:25.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:25 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:25.645+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.476477+0000 front 2024-09-12T01:37:51.476500+0000 (oldest deadline 2024-09-12T01:38:16.077208+0000) 2024-09-12T01:38:25.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:25 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:25.645+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.778585+0000 front 2024-09-12T01:37:57.778240+0000 (oldest deadline 2024-09-12T01:38:23.678115+0000) 2024-09-12T01:38:25.963 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:25 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:25.538+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.309568+0000 front 2024-09-12T01:37:51.309537+0000 (oldest deadline 2024-09-12T01:38:14.209738+0000) 2024-09-12T01:38:25.963 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:25 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:25.538+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:59.510715+0000 front 2024-09-12T01:37:59.510935+0000 (oldest deadline 2024-09-12T01:38:25.410761+0000) 2024-09-12T01:38:26.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:25 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:25.963+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:26.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:25 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:25.963+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:38:26.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:25 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:25.963+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.101595+0000 front 2024-09-12T01:38:00.101475+0000 (oldest deadline 2024-09-12T01:38:24.201476+0000) 2024-09-12T01:38:26.325 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:26 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5[46675]: 2024-09-12T01:38:26.106+0000 7f586703b640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.703841+0000 front 2024-09-12T01:37:44.703958+0000 (oldest deadline 2024-09-12T01:38:10.003735+0000) 2024-09-12T01:38:26.325 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:26 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5[46675]: 2024-09-12T01:38:26.106+0000 7f586703b640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.104737+0000 front 2024-09-12T01:37:51.104755+0000 (oldest deadline 2024-09-12T01:38:14.004876+0000) 2024-09-12T01:38:26.325 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:26 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5[46675]: 2024-09-12T01:38:26.106+0000 7f586703b640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:58.705974+0000 front 2024-09-12T01:37:58.706001+0000 (oldest deadline 2024-09-12T01:38:24.006212+0000) 2024-09-12T01:38:26.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:25 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:25.911+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:48.737024+0000 front 2024-09-12T01:37:48.736899+0000 (oldest deadline 2024-09-12T01:38:14.036808+0000) 2024-09-12T01:38:26.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:25 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:25.911+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.338640+0000 front 2024-09-12T01:37:57.338659+0000 (oldest deadline 2024-09-12T01:38:23.238871+0000) 2024-09-12T01:38:26.428 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 12 01:38:26 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-6[57769]: 2024-09-12T01:38:26.068+0000 7fabfbe00640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.155703+0000 front 2024-09-12T01:37:53.155459+0000 (oldest deadline 2024-09-12T01:38:19.055328+0000) 2024-09-12T01:38:26.428 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 12 01:38:26 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-6[57769]: 2024-09-12T01:38:26.068+0000 7fabfbe00640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:59.056032+0000 front 2024-09-12T01:37:59.056071+0000 (oldest deadline 2024-09-12T01:38:24.356013+0000) 2024-09-12T01:38:26.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:26 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:26.570+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:26.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:26 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:26.570+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:26.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:26 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:26.570+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:38:26.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:26 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:26.632+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:26.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:26 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:26.632+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.476477+0000 front 2024-09-12T01:37:51.476500+0000 (oldest deadline 2024-09-12T01:38:16.077208+0000) 2024-09-12T01:38:26.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:26 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:26.632+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.778585+0000 front 2024-09-12T01:37:57.778240+0000 (oldest deadline 2024-09-12T01:38:23.678115+0000) 2024-09-12T01:38:26.978 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:26 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:26.582+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.309568+0000 front 2024-09-12T01:37:51.309537+0000 (oldest deadline 2024-09-12T01:38:14.209738+0000) 2024-09-12T01:38:26.978 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:26 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:26.582+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:59.510715+0000 front 2024-09-12T01:37:59.510935+0000 (oldest deadline 2024-09-12T01:38:25.410761+0000) 2024-09-12T01:38:27.142 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:26 smithi104 podman[69801]: 2024-09-12 01:38:26.832363578 +0000 UTC m=+5.499708091 container died 5219a8f6c82fdb35192b1325eb1477d6958f0dd2a067dd8af1081b46e8ed52d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-12T01:38:27.142 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:26 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:26.934+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:48.737024+0000 front 2024-09-12T01:37:48.736899+0000 (oldest deadline 2024-09-12T01:38:14.036808+0000) 2024-09-12T01:38:27.142 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:26 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:26.934+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.338640+0000 front 2024-09-12T01:37:57.338659+0000 (oldest deadline 2024-09-12T01:38:23.238871+0000) 2024-09-12T01:38:27.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:26 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:26.978+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:27.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:26 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:26.978+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:38:27.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:26 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:26.978+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.101595+0000 front 2024-09-12T01:38:00.101475+0000 (oldest deadline 2024-09-12T01:38:24.201476+0000) 2024-09-12T01:38:27.428 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 12 01:38:27 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-6[57769]: 2024-09-12T01:38:27.079+0000 7fabfbe00640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.155703+0000 front 2024-09-12T01:37:53.155459+0000 (oldest deadline 2024-09-12T01:38:19.055328+0000) 2024-09-12T01:38:27.428 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 12 01:38:27 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-6[57769]: 2024-09-12T01:38:27.079+0000 7fabfbe00640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:59.056032+0000 front 2024-09-12T01:37:59.056071+0000 (oldest deadline 2024-09-12T01:38:24.356013+0000) 2024-09-12T01:38:27.484 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:27 smithi104 podman[69801]: 2024-09-12 01:38:27.233858842 +0000 UTC m=+5.901203355 container cleanup 5219a8f6c82fdb35192b1325eb1477d6958f0dd2a067dd8af1081b46e8ed52d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-12T01:38:27.484 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:27 smithi104 bash[69801]: ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5 2024-09-12T01:38:27.825 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:27 smithi104 podman[69819]: 2024-09-12 01:38:27.485060454 +0000 UTC m=+0.655530826 container remove 5219a8f6c82fdb35192b1325eb1477d6958f0dd2a067dd8af1081b46e8ed52d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T01:38:27.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:27 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:27.563+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:27.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:27 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:27.563+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:27.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:27 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:27.563+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:38:27.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:27 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:27.615+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:27.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:27 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:27.615+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.476477+0000 front 2024-09-12T01:37:51.476500+0000 (oldest deadline 2024-09-12T01:38:16.077208+0000) 2024-09-12T01:38:27.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:27 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:27.615+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.778585+0000 front 2024-09-12T01:37:57.778240+0000 (oldest deadline 2024-09-12T01:38:23.678115+0000) 2024-09-12T01:38:27.960 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:27 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:27.552+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.309568+0000 front 2024-09-12T01:37:51.309537+0000 (oldest deadline 2024-09-12T01:38:14.209738+0000) 2024-09-12T01:38:27.960 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:27 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:27.552+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:59.510715+0000 front 2024-09-12T01:37:59.510935+0000 (oldest deadline 2024-09-12T01:38:25.410761+0000) 2024-09-12T01:38:28.188 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:27 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:27.892+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:48.737024+0000 front 2024-09-12T01:37:48.736899+0000 (oldest deadline 2024-09-12T01:38:14.036808+0000) 2024-09-12T01:38:28.188 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:27 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:27.892+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.338640+0000 front 2024-09-12T01:37:57.338659+0000 (oldest deadline 2024-09-12T01:38:23.238871+0000) 2024-09-12T01:38:28.188 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:27 smithi104 podman[69907]: 2024-09-12 01:38:27.802650005 +0000 UTC m=+0.015540813 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:38:28.188 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:27 smithi104 podman[69907]: 2024-09-12 01:38:27.919781353 +0000 UTC m=+0.132672203 container create 17b82048b38161007e2117a610c1f694eebd971bf8cc05429a2481c56898c1a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-12T01:38:28.189 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:28 smithi104 podman[69907]: 2024-09-12 01:38:28.189223063 +0000 UTC m=+0.402113871 container init 17b82048b38161007e2117a610c1f694eebd971bf8cc05429a2481c56898c1a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-12T01:38:28.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:27 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:27.960+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:28.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:27 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:27.960+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:38:28.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:27 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:27.960+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.101595+0000 front 2024-09-12T01:38:00.101475+0000 (oldest deadline 2024-09-12T01:38:24.201476+0000) 2024-09-12T01:38:28.428 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 12 01:38:28 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-6[57769]: 2024-09-12T01:38:28.066+0000 7fabfbe00640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.155703+0000 front 2024-09-12T01:37:53.155459+0000 (oldest deadline 2024-09-12T01:38:19.055328+0000) 2024-09-12T01:38:28.428 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 12 01:38:28 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-6[57769]: 2024-09-12T01:38:28.066+0000 7fabfbe00640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:59.056032+0000 front 2024-09-12T01:37:59.056071+0000 (oldest deadline 2024-09-12T01:38:24.356013+0000) 2024-09-12T01:38:28.541 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:28 smithi104 podman[69907]: 2024-09-12 01:38:28.192686105 +0000 UTC m=+0.405576914 container start 17b82048b38161007e2117a610c1f694eebd971bf8cc05429a2481c56898c1a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-12T01:38:28.541 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:28 smithi104 podman[69907]: 2024-09-12 01:38:28.289219273 +0000 UTC m=+0.502110096 container attach 17b82048b38161007e2117a610c1f694eebd971bf8cc05429a2481c56898c1a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-12T01:38:28.541 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:28 smithi104 podman[69907]: 2024-09-12 01:38:28.389818886 +0000 UTC m=+0.602709695 container died 17b82048b38161007e2117a610c1f694eebd971bf8cc05429a2481c56898c1a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-12T01:38:28.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:28 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:28.540+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:28.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:28 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:28.540+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:28.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:28 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:28.540+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:38:28.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:28 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:28.581+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:28.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:28 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:28.581+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.476477+0000 front 2024-09-12T01:37:51.476500+0000 (oldest deadline 2024-09-12T01:38:16.077208+0000) 2024-09-12T01:38:28.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:28 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:28.581+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.778585+0000 front 2024-09-12T01:37:57.778240+0000 (oldest deadline 2024-09-12T01:38:23.678115+0000) 2024-09-12T01:38:28.959 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:28 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:28.570+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.309568+0000 front 2024-09-12T01:37:51.309537+0000 (oldest deadline 2024-09-12T01:38:14.209738+0000) 2024-09-12T01:38:28.959 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:28 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:28.570+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:59.510715+0000 front 2024-09-12T01:37:59.510935+0000 (oldest deadline 2024-09-12T01:38:25.410761+0000) 2024-09-12T01:38:29.100 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@osd.5.service' 2024-09-12T01:38:29.165 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:28 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:28.887+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:48.737024+0000 front 2024-09-12T01:37:48.736899+0000 (oldest deadline 2024-09-12T01:38:14.036808+0000) 2024-09-12T01:38:29.165 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:28 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:28.887+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.338640+0000 front 2024-09-12T01:37:57.338659+0000 (oldest deadline 2024-09-12T01:38:23.238871+0000) 2024-09-12T01:38:29.166 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:29 smithi104 podman[69930]: 2024-09-12 01:38:29.076260493 +0000 UTC m=+0.677935868 container remove 17b82048b38161007e2117a610c1f694eebd971bf8cc05429a2481c56898c1a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-12T01:38:29.166 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:29 smithi104 systemd[1]: ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@osd.5.service: Deactivated successfully. 2024-09-12T01:38:29.166 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:29 smithi104 systemd[1]: Stopped Ceph osd.5 for c0e4f180-70a4-11ef-bcea-c7b262605968. 2024-09-12T01:38:29.166 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 12 01:38:29 smithi104 systemd[1]: ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@osd.5.service: Consumed 2min 36.826s CPU time. 2024-09-12T01:38:29.181 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:38:29.181 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-12T01:38:29.182 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-12T01:38:29.182 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@osd.6 2024-09-12T01:38:29.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:28 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:28.959+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:29.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:28 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:28.959+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:38:29.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:28 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:28.959+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.101595+0000 front 2024-09-12T01:38:00.101475+0000 (oldest deadline 2024-09-12T01:38:24.201476+0000) 2024-09-12T01:38:29.428 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 12 01:38:29 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-6[57769]: 2024-09-12T01:38:29.046+0000 7fabfbe00640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.155703+0000 front 2024-09-12T01:37:53.155459+0000 (oldest deadline 2024-09-12T01:38:19.055328+0000) 2024-09-12T01:38:29.428 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 12 01:38:29 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-6[57769]: 2024-09-12T01:38:29.046+0000 7fabfbe00640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:59.056032+0000 front 2024-09-12T01:37:59.056071+0000 (oldest deadline 2024-09-12T01:38:24.356013+0000) 2024-09-12T01:38:29.428 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 12 01:38:29 smithi001 systemd[1]: Stopping Ceph osd.6 for c0e4f180-70a4-11ef-bcea-c7b262605968... 2024-09-12T01:38:29.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:29 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:29.495+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:29.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:29 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:29.495+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:29.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:29 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:29.495+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:38:29.921 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:29 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:29.596+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.309568+0000 front 2024-09-12T01:37:51.309537+0000 (oldest deadline 2024-09-12T01:38:14.209738+0000) 2024-09-12T01:38:29.921 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:29 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:29.596+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:59.510715+0000 front 2024-09-12T01:37:59.510935+0000 (oldest deadline 2024-09-12T01:38:25.410761+0000) 2024-09-12T01:38:29.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:29 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:29.572+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:29.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:29 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:29.572+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.476477+0000 front 2024-09-12T01:37:51.476500+0000 (oldest deadline 2024-09-12T01:38:16.077208+0000) 2024-09-12T01:38:29.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:29 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:29.572+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.778585+0000 front 2024-09-12T01:37:57.778240+0000 (oldest deadline 2024-09-12T01:38:23.678115+0000) 2024-09-12T01:38:29.928 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 12 01:38:29 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-6[57769]: 2024-09-12T01:38:29.579+0000 7fac00400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:38:29.928 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 12 01:38:29 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-6[57769]: 2024-09-12T01:38:29.579+0000 7fac00400640 -1 osd.6 102 *** Got signal Terminated *** 2024-09-12T01:38:29.928 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 12 01:38:29 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-6[57769]: 2024-09-12T01:38:29.579+0000 7fac00400640 -1 osd.6 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T01:38:30.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:29 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:29.921+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:30.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:29 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:29.921+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:38:30.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:29 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:29.921+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.101595+0000 front 2024-09-12T01:38:00.101475+0000 (oldest deadline 2024-09-12T01:38:24.201476+0000) 2024-09-12T01:38:30.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:29 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:29.907+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:48.737024+0000 front 2024-09-12T01:37:48.736899+0000 (oldest deadline 2024-09-12T01:38:14.036808+0000) 2024-09-12T01:38:30.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:29 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:29.907+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.338640+0000 front 2024-09-12T01:37:57.338659+0000 (oldest deadline 2024-09-12T01:38:23.238871+0000) 2024-09-12T01:38:30.428 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 12 01:38:30 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-6[57769]: 2024-09-12T01:38:30.038+0000 7fabfbe00640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.155703+0000 front 2024-09-12T01:37:53.155459+0000 (oldest deadline 2024-09-12T01:38:19.055328+0000) 2024-09-12T01:38:30.428 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 12 01:38:30 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-6[57769]: 2024-09-12T01:38:30.038+0000 7fabfbe00640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:59.056032+0000 front 2024-09-12T01:37:59.056071+0000 (oldest deadline 2024-09-12T01:38:24.356013+0000) 2024-09-12T01:38:30.829 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:30 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:30.458+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:30.829 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:30 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:30.458+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:30.829 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:30 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:30.458+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:38:30.881 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:30 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:30.623+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.309568+0000 front 2024-09-12T01:37:51.309537+0000 (oldest deadline 2024-09-12T01:38:14.209738+0000) 2024-09-12T01:38:30.881 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:30 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:30.623+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:59.510715+0000 front 2024-09-12T01:37:59.510935+0000 (oldest deadline 2024-09-12T01:38:25.410761+0000) 2024-09-12T01:38:30.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:30 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:30.592+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:30.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:30 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:30.592+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.476477+0000 front 2024-09-12T01:37:51.476500+0000 (oldest deadline 2024-09-12T01:38:16.077208+0000) 2024-09-12T01:38:30.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:30 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:30.592+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.778585+0000 front 2024-09-12T01:37:57.778240+0000 (oldest deadline 2024-09-12T01:38:23.678115+0000) 2024-09-12T01:38:31.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:30 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:30.881+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:31.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:30 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:30.881+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:38:31.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:30 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:30.881+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.101595+0000 front 2024-09-12T01:38:00.101475+0000 (oldest deadline 2024-09-12T01:38:24.201476+0000) 2024-09-12T01:38:31.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:30 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:30.912+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:48.737024+0000 front 2024-09-12T01:37:48.736899+0000 (oldest deadline 2024-09-12T01:38:14.036808+0000) 2024-09-12T01:38:31.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:30 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:30.912+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.338640+0000 front 2024-09-12T01:37:57.338659+0000 (oldest deadline 2024-09-12T01:38:23.238871+0000) 2024-09-12T01:38:31.428 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 12 01:38:31 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-6[57769]: 2024-09-12T01:38:31.014+0000 7fabfbe00640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.155703+0000 front 2024-09-12T01:37:53.155459+0000 (oldest deadline 2024-09-12T01:38:19.055328+0000) 2024-09-12T01:38:31.428 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 12 01:38:31 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-6[57769]: 2024-09-12T01:38:31.014+0000 7fabfbe00640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:59.056032+0000 front 2024-09-12T01:37:59.056071+0000 (oldest deadline 2024-09-12T01:38:24.356013+0000) 2024-09-12T01:38:31.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:31 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:31.476+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:31.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:31 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:31.476+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:31.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:31 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:31.476+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:38:31.920 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:31 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:31.620+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.309568+0000 front 2024-09-12T01:37:51.309537+0000 (oldest deadline 2024-09-12T01:38:14.209738+0000) 2024-09-12T01:38:31.920 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:31 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:31.620+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:59.510715+0000 front 2024-09-12T01:37:59.510935+0000 (oldest deadline 2024-09-12T01:38:25.410761+0000) 2024-09-12T01:38:31.920 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:31 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:31.620+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:06.511690+0000 front 2024-09-12T01:38:06.511734+0000 (oldest deadline 2024-09-12T01:38:31.211659+0000) 2024-09-12T01:38:31.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:31 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:31.565+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:31.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:31 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:31.565+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.476477+0000 front 2024-09-12T01:37:51.476500+0000 (oldest deadline 2024-09-12T01:38:16.077208+0000) 2024-09-12T01:38:31.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:31 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:31.565+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.778585+0000 front 2024-09-12T01:37:57.778240+0000 (oldest deadline 2024-09-12T01:38:23.678115+0000) 2024-09-12T01:38:32.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:31 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:31.920+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:32.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:31 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:31.920+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:38:32.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:31 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:31.920+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.101595+0000 front 2024-09-12T01:38:00.101475+0000 (oldest deadline 2024-09-12T01:38:24.201476+0000) 2024-09-12T01:38:32.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:31 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:31.920+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.902499+0000 front 2024-09-12T01:38:08.902599+0000 (oldest deadline 2024-09-12T01:38:31.202668+0000) 2024-09-12T01:38:32.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:31 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:31.874+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:48.737024+0000 front 2024-09-12T01:37:48.736899+0000 (oldest deadline 2024-09-12T01:38:14.036808+0000) 2024-09-12T01:38:32.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:31 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:31.874+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.338640+0000 front 2024-09-12T01:37:57.338659+0000 (oldest deadline 2024-09-12T01:38:23.238871+0000) 2024-09-12T01:38:32.428 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 12 01:38:32 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-6[57769]: 2024-09-12T01:38:32.028+0000 7fabfbe00640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.155703+0000 front 2024-09-12T01:37:53.155459+0000 (oldest deadline 2024-09-12T01:38:19.055328+0000) 2024-09-12T01:38:32.428 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 12 01:38:32 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-6[57769]: 2024-09-12T01:38:32.028+0000 7fabfbe00640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:59.056032+0000 front 2024-09-12T01:37:59.056071+0000 (oldest deadline 2024-09-12T01:38:24.356013+0000) 2024-09-12T01:38:32.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:32 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:32.526+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:32.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:32 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:32.526+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:32.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:32 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:32.526+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:38:32.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:32 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:32.615+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:32.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:32 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:32.615+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.476477+0000 front 2024-09-12T01:37:51.476500+0000 (oldest deadline 2024-09-12T01:38:16.077208+0000) 2024-09-12T01:38:32.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:32 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:32.615+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.778585+0000 front 2024-09-12T01:37:57.778240+0000 (oldest deadline 2024-09-12T01:38:23.678115+0000) 2024-09-12T01:38:32.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:32 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:32.615+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.780419+0000 front 2024-09-12T01:38:08.780480+0000 (oldest deadline 2024-09-12T01:38:31.680454+0000) 2024-09-12T01:38:32.990 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:32 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:32.631+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.309568+0000 front 2024-09-12T01:37:51.309537+0000 (oldest deadline 2024-09-12T01:38:14.209738+0000) 2024-09-12T01:38:32.990 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:32 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:32.631+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:59.510715+0000 front 2024-09-12T01:37:59.510935+0000 (oldest deadline 2024-09-12T01:38:25.410761+0000) 2024-09-12T01:38:32.990 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:32 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:32.631+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:06.511690+0000 front 2024-09-12T01:38:06.511734+0000 (oldest deadline 2024-09-12T01:38:31.211659+0000) 2024-09-12T01:38:32.991 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:32 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:32.877+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:32.991 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:32 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:32.877+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:38:32.991 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:32 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:32.877+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.101595+0000 front 2024-09-12T01:38:00.101475+0000 (oldest deadline 2024-09-12T01:38:24.201476+0000) 2024-09-12T01:38:32.991 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:32 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:32.877+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.902499+0000 front 2024-09-12T01:38:08.902599+0000 (oldest deadline 2024-09-12T01:38:31.202668+0000) 2024-09-12T01:38:33.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:32 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:32.918+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:48.737024+0000 front 2024-09-12T01:37:48.736899+0000 (oldest deadline 2024-09-12T01:38:14.036808+0000) 2024-09-12T01:38:33.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:32 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:32.918+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.338640+0000 front 2024-09-12T01:37:57.338659+0000 (oldest deadline 2024-09-12T01:38:23.238871+0000) 2024-09-12T01:38:33.428 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 12 01:38:33 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-6[57769]: 2024-09-12T01:38:33.046+0000 7fabfbe00640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.155703+0000 front 2024-09-12T01:37:53.155459+0000 (oldest deadline 2024-09-12T01:38:19.055328+0000) 2024-09-12T01:38:33.428 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 12 01:38:33 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-6[57769]: 2024-09-12T01:38:33.046+0000 7fabfbe00640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:59.056032+0000 front 2024-09-12T01:37:59.056071+0000 (oldest deadline 2024-09-12T01:38:24.356013+0000) 2024-09-12T01:38:33.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:33 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:33.568+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:33.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:33 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:33.568+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:33.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:33 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:33.568+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:38:33.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:33 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:33.640+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:33.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:33 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:33.640+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.476477+0000 front 2024-09-12T01:37:51.476500+0000 (oldest deadline 2024-09-12T01:38:16.077208+0000) 2024-09-12T01:38:33.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:33 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:33.640+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.778585+0000 front 2024-09-12T01:37:57.778240+0000 (oldest deadline 2024-09-12T01:38:23.678115+0000) 2024-09-12T01:38:33.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:33 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:33.640+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.780419+0000 front 2024-09-12T01:38:08.780480+0000 (oldest deadline 2024-09-12T01:38:31.680454+0000) 2024-09-12T01:38:33.990 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:33 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:33.630+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.309568+0000 front 2024-09-12T01:37:51.309537+0000 (oldest deadline 2024-09-12T01:38:14.209738+0000) 2024-09-12T01:38:33.990 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:33 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:33.630+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:59.510715+0000 front 2024-09-12T01:37:59.510935+0000 (oldest deadline 2024-09-12T01:38:25.410761+0000) 2024-09-12T01:38:33.990 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:33 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:33.630+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:06.511690+0000 front 2024-09-12T01:38:06.511734+0000 (oldest deadline 2024-09-12T01:38:31.211659+0000) 2024-09-12T01:38:33.990 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:33 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:33.878+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:33.990 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:33 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:33.878+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:38:33.990 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:33 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:33.878+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.101595+0000 front 2024-09-12T01:38:00.101475+0000 (oldest deadline 2024-09-12T01:38:24.201476+0000) 2024-09-12T01:38:33.991 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:33 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:33.878+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.902499+0000 front 2024-09-12T01:38:08.902599+0000 (oldest deadline 2024-09-12T01:38:31.202668+0000) 2024-09-12T01:38:34.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:33 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:33.923+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:48.737024+0000 front 2024-09-12T01:37:48.736899+0000 (oldest deadline 2024-09-12T01:38:14.036808+0000) 2024-09-12T01:38:34.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:33 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:33.923+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.338640+0000 front 2024-09-12T01:37:57.338659+0000 (oldest deadline 2024-09-12T01:38:23.238871+0000) 2024-09-12T01:38:34.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:33 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:33.923+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:09.139766+0000 front 2024-09-12T01:38:09.139677+0000 (oldest deadline 2024-09-12T01:38:33.839904+0000) 2024-09-12T01:38:34.428 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 12 01:38:34 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-6[57769]: 2024-09-12T01:38:34.068+0000 7fabfbe00640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.155703+0000 front 2024-09-12T01:37:53.155459+0000 (oldest deadline 2024-09-12T01:38:19.055328+0000) 2024-09-12T01:38:34.428 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 12 01:38:34 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-6[57769]: 2024-09-12T01:38:34.068+0000 7fabfbe00640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:59.056032+0000 front 2024-09-12T01:37:59.056071+0000 (oldest deadline 2024-09-12T01:38:24.356013+0000) 2024-09-12T01:38:34.856 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:34 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:34.601+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.309568+0000 front 2024-09-12T01:37:51.309537+0000 (oldest deadline 2024-09-12T01:38:14.209738+0000) 2024-09-12T01:38:34.856 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:34 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:34.601+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:59.510715+0000 front 2024-09-12T01:37:59.510935+0000 (oldest deadline 2024-09-12T01:38:25.410761+0000) 2024-09-12T01:38:34.856 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:34 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:34.601+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:06.511690+0000 front 2024-09-12T01:38:06.511734+0000 (oldest deadline 2024-09-12T01:38:31.211659+0000) 2024-09-12T01:38:34.897 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:34 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:34.593+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:34.897 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:34 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:34.593+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:34.897 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:34 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:34.593+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:38:34.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:34 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:34.605+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:34.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:34 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:34.605+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.476477+0000 front 2024-09-12T01:37:51.476500+0000 (oldest deadline 2024-09-12T01:38:16.077208+0000) 2024-09-12T01:38:34.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:34 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:34.605+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.778585+0000 front 2024-09-12T01:37:57.778240+0000 (oldest deadline 2024-09-12T01:38:23.678115+0000) 2024-09-12T01:38:34.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:34 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:34.605+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.780419+0000 front 2024-09-12T01:38:08.780480+0000 (oldest deadline 2024-09-12T01:38:31.680454+0000) 2024-09-12T01:38:35.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:34 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:34.856+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:35.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:34 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:34.856+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:38:35.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:34 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:34.856+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.101595+0000 front 2024-09-12T01:38:00.101475+0000 (oldest deadline 2024-09-12T01:38:24.201476+0000) 2024-09-12T01:38:35.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:34 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:34.856+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.902499+0000 front 2024-09-12T01:38:08.902599+0000 (oldest deadline 2024-09-12T01:38:31.202668+0000) 2024-09-12T01:38:35.319 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 12 01:38:34 smithi001 podman[109526]: 2024-09-12 01:38:34.987736823 +0000 UTC m=+5.520147925 container died f2b9464762aa4e38699e250168334b55be16d206a8867bcccb9d4f49e672bc24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-12T01:38:35.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:34 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:34.897+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:48.737024+0000 front 2024-09-12T01:37:48.736899+0000 (oldest deadline 2024-09-12T01:38:14.036808+0000) 2024-09-12T01:38:35.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:34 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:34.897+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.338640+0000 front 2024-09-12T01:37:57.338659+0000 (oldest deadline 2024-09-12T01:38:23.238871+0000) 2024-09-12T01:38:35.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:34 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:34.897+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:09.139766+0000 front 2024-09-12T01:38:09.139677+0000 (oldest deadline 2024-09-12T01:38:33.839904+0000) 2024-09-12T01:38:35.631 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 12 01:38:35 smithi001 podman[109526]: 2024-09-12 01:38:35.452401278 +0000 UTC m=+5.984812378 container cleanup f2b9464762aa4e38699e250168334b55be16d206a8867bcccb9d4f49e672bc24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-6, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T01:38:35.632 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 12 01:38:35 smithi001 bash[109526]: ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-6 2024-09-12T01:38:35.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:35 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:35.548+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:35.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:35 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:35.548+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:35.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:35 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:35.548+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:38:35.928 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 12 01:38:35 smithi001 podman[109545]: 2024-09-12 01:38:35.703810025 +0000 UTC m=+0.706614592 container remove f2b9464762aa4e38699e250168334b55be16d206a8867bcccb9d4f49e672bc24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-6, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True) 2024-09-12T01:38:35.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:35 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:35.631+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:35.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:35 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:35.631+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.476477+0000 front 2024-09-12T01:37:51.476500+0000 (oldest deadline 2024-09-12T01:38:16.077208+0000) 2024-09-12T01:38:35.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:35 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:35.631+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.778585+0000 front 2024-09-12T01:37:57.778240+0000 (oldest deadline 2024-09-12T01:38:23.678115+0000) 2024-09-12T01:38:35.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:35 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:35.631+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.780419+0000 front 2024-09-12T01:38:08.780480+0000 (oldest deadline 2024-09-12T01:38:31.680454+0000) 2024-09-12T01:38:35.990 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:35 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:35.640+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.309568+0000 front 2024-09-12T01:37:51.309537+0000 (oldest deadline 2024-09-12T01:38:14.209738+0000) 2024-09-12T01:38:35.990 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:35 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:35.640+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:59.510715+0000 front 2024-09-12T01:37:59.510935+0000 (oldest deadline 2024-09-12T01:38:25.410761+0000) 2024-09-12T01:38:35.990 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:35 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:35.640+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:06.511690+0000 front 2024-09-12T01:38:06.511734+0000 (oldest deadline 2024-09-12T01:38:31.211659+0000) 2024-09-12T01:38:35.990 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:35 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:35.862+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:35.991 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:35 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:35.862+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:38:35.991 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:35 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:35.862+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.101595+0000 front 2024-09-12T01:38:00.101475+0000 (oldest deadline 2024-09-12T01:38:24.201476+0000) 2024-09-12T01:38:35.991 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:35 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:35.862+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.902499+0000 front 2024-09-12T01:38:08.902599+0000 (oldest deadline 2024-09-12T01:38:31.202668+0000) 2024-09-12T01:38:36.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:35 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:35.891+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:48.737024+0000 front 2024-09-12T01:37:48.736899+0000 (oldest deadline 2024-09-12T01:38:14.036808+0000) 2024-09-12T01:38:36.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:35 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:35.891+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.338640+0000 front 2024-09-12T01:37:57.338659+0000 (oldest deadline 2024-09-12T01:38:23.238871+0000) 2024-09-12T01:38:36.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:35 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:35.891+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:09.139766+0000 front 2024-09-12T01:38:09.139677+0000 (oldest deadline 2024-09-12T01:38:33.839904+0000) 2024-09-12T01:38:36.574 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 12 01:38:36 smithi001 podman[109630]: 2024-09-12 01:38:36.171591987 +0000 UTC m=+0.027347172 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:38:36.575 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 12 01:38:36 smithi001 podman[109630]: 2024-09-12 01:38:36.313223724 +0000 UTC m=+0.168978912 container create e78d9ac32e2f0eba312ab191978929c68f7496717543cb9480b3f8e412b70c62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T01:38:36.575 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 12 01:38:36 smithi001 podman[109630]: 2024-09-12 01:38:36.575284433 +0000 UTC m=+0.431039642 container init e78d9ac32e2f0eba312ab191978929c68f7496717543cb9480b3f8e412b70c62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-12T01:38:36.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:36 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:36.570+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:36.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:36 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:36.570+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:36.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:36 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:36.570+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:38:36.847 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 12 01:38:36 smithi001 podman[109630]: 2024-09-12 01:38:36.582146906 +0000 UTC m=+0.437902091 container start e78d9ac32e2f0eba312ab191978929c68f7496717543cb9480b3f8e412b70c62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-6-deactivate, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T01:38:36.847 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 12 01:38:36 smithi001 podman[109630]: 2024-09-12 01:38:36.690568376 +0000 UTC m=+0.546323562 container attach e78d9ac32e2f0eba312ab191978929c68f7496717543cb9480b3f8e412b70c62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T01:38:36.848 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:36 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:36.632+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:36.848 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:36 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:36.632+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.476477+0000 front 2024-09-12T01:37:51.476500+0000 (oldest deadline 2024-09-12T01:38:16.077208+0000) 2024-09-12T01:38:36.848 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:36 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:36.632+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.778585+0000 front 2024-09-12T01:37:57.778240+0000 (oldest deadline 2024-09-12T01:38:23.678115+0000) 2024-09-12T01:38:36.848 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:36 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:36.632+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.780419+0000 front 2024-09-12T01:38:08.780480+0000 (oldest deadline 2024-09-12T01:38:31.680454+0000) 2024-09-12T01:38:36.880 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:36 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:36.595+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.309568+0000 front 2024-09-12T01:37:51.309537+0000 (oldest deadline 2024-09-12T01:38:14.209738+0000) 2024-09-12T01:38:36.880 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:36 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:36.595+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:59.510715+0000 front 2024-09-12T01:37:59.510935+0000 (oldest deadline 2024-09-12T01:38:25.410761+0000) 2024-09-12T01:38:36.880 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:36 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:36.595+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:06.511690+0000 front 2024-09-12T01:38:06.511734+0000 (oldest deadline 2024-09-12T01:38:31.211659+0000) 2024-09-12T01:38:37.178 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 12 01:38:36 smithi001 podman[109653]: 2024-09-12 01:38:36.887812052 +0000 UTC m=+0.026125547 container died e78d9ac32e2f0eba312ab191978929c68f7496717543cb9480b3f8e412b70c62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-12T01:38:37.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:36 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:36.880+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:37.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:36 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:36.880+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:38:37.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:36 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:36.880+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.101595+0000 front 2024-09-12T01:38:00.101475+0000 (oldest deadline 2024-09-12T01:38:24.201476+0000) 2024-09-12T01:38:37.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:36 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:36.880+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.902499+0000 front 2024-09-12T01:38:08.902599+0000 (oldest deadline 2024-09-12T01:38:31.202668+0000) 2024-09-12T01:38:37.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:36 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:36.911+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:48.737024+0000 front 2024-09-12T01:37:48.736899+0000 (oldest deadline 2024-09-12T01:38:14.036808+0000) 2024-09-12T01:38:37.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:36 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:36.911+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.338640+0000 front 2024-09-12T01:37:57.338659+0000 (oldest deadline 2024-09-12T01:38:23.238871+0000) 2024-09-12T01:38:37.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:36 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:36.911+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:09.139766+0000 front 2024-09-12T01:38:09.139677+0000 (oldest deadline 2024-09-12T01:38:33.839904+0000) 2024-09-12T01:38:37.643 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@osd.6.service' 2024-09-12T01:38:37.719 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:38:37.719 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-12T01:38:37.719 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-12T01:38:37.719 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@osd.7 2024-09-12T01:38:37.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:37 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:37.651+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:37.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:37 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:37.651+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.476477+0000 front 2024-09-12T01:37:51.476500+0000 (oldest deadline 2024-09-12T01:38:16.077208+0000) 2024-09-12T01:38:37.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:37 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:37.651+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.778585+0000 front 2024-09-12T01:37:57.778240+0000 (oldest deadline 2024-09-12T01:38:23.678115+0000) 2024-09-12T01:38:37.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:37 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:37.651+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.780419+0000 front 2024-09-12T01:38:08.780480+0000 (oldest deadline 2024-09-12T01:38:31.680454+0000) 2024-09-12T01:38:37.934 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:37 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:37.612+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:37.934 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:37 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:37.612+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:37.934 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:37 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:37.612+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:38:37.990 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:37 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:37.582+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.309568+0000 front 2024-09-12T01:37:51.309537+0000 (oldest deadline 2024-09-12T01:38:14.209738+0000) 2024-09-12T01:38:37.990 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:37 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:37.582+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:59.510715+0000 front 2024-09-12T01:37:59.510935+0000 (oldest deadline 2024-09-12T01:38:25.410761+0000) 2024-09-12T01:38:37.990 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:37 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:37.582+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:06.511690+0000 front 2024-09-12T01:38:06.511734+0000 (oldest deadline 2024-09-12T01:38:31.211659+0000) 2024-09-12T01:38:37.990 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:37 smithi053 systemd[1]: Stopping Ceph osd.7 for c0e4f180-70a4-11ef-bcea-c7b262605968... 2024-09-12T01:38:37.990 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:37 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:37.830+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:37.990 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:37 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:37.830+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:38:37.991 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:37 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:37.830+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.101595+0000 front 2024-09-12T01:38:00.101475+0000 (oldest deadline 2024-09-12T01:38:24.201476+0000) 2024-09-12T01:38:37.991 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:37 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:37.830+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.902499+0000 front 2024-09-12T01:38:08.902599+0000 (oldest deadline 2024-09-12T01:38:31.202668+0000) 2024-09-12T01:38:38.261 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:38 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:38.045+0000 7fe01991a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:38:38.262 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:38 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:38.045+0000 7fe01991a640 -1 osd.7 102 *** Got signal Terminated *** 2024-09-12T01:38:38.262 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:38 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:38.045+0000 7fe01991a640 -1 osd.7 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T01:38:38.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:37 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:37.935+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:48.737024+0000 front 2024-09-12T01:37:48.736899+0000 (oldest deadline 2024-09-12T01:38:14.036808+0000) 2024-09-12T01:38:38.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:37 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:37.935+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.338640+0000 front 2024-09-12T01:37:57.338659+0000 (oldest deadline 2024-09-12T01:38:23.238871+0000) 2024-09-12T01:38:38.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:37 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:37.935+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:09.139766+0000 front 2024-09-12T01:38:09.139677+0000 (oldest deadline 2024-09-12T01:38:33.839904+0000) 2024-09-12T01:38:38.835 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:38 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:38.535+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.309568+0000 front 2024-09-12T01:37:51.309537+0000 (oldest deadline 2024-09-12T01:38:14.209738+0000) 2024-09-12T01:38:38.836 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:38 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:38.535+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:59.510715+0000 front 2024-09-12T01:37:59.510935+0000 (oldest deadline 2024-09-12T01:38:25.410761+0000) 2024-09-12T01:38:38.836 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:38 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:38.535+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:06.511690+0000 front 2024-09-12T01:38:06.511734+0000 (oldest deadline 2024-09-12T01:38:31.211659+0000) 2024-09-12T01:38:38.889 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:38 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:38.613+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:38.889 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:38 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:38.613+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:38.889 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:38 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:38.613+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:38:38.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:38 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:38.641+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:38.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:38 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:38.641+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.476477+0000 front 2024-09-12T01:37:51.476500+0000 (oldest deadline 2024-09-12T01:38:16.077208+0000) 2024-09-12T01:38:38.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:38 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:38.641+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.778585+0000 front 2024-09-12T01:37:57.778240+0000 (oldest deadline 2024-09-12T01:38:23.678115+0000) 2024-09-12T01:38:38.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:38 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:38.641+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.780419+0000 front 2024-09-12T01:38:08.780480+0000 (oldest deadline 2024-09-12T01:38:31.680454+0000) 2024-09-12T01:38:39.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:38 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:38.835+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:39.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:38 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:38.835+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:38:39.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:38 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:38.835+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.101595+0000 front 2024-09-12T01:38:00.101475+0000 (oldest deadline 2024-09-12T01:38:24.201476+0000) 2024-09-12T01:38:39.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:38 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:38.835+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.902499+0000 front 2024-09-12T01:38:08.902599+0000 (oldest deadline 2024-09-12T01:38:31.202668+0000) 2024-09-12T01:38:39.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:38 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:38.889+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:48.737024+0000 front 2024-09-12T01:37:48.736899+0000 (oldest deadline 2024-09-12T01:38:14.036808+0000) 2024-09-12T01:38:39.337 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:38 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:38.889+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.338640+0000 front 2024-09-12T01:37:57.338659+0000 (oldest deadline 2024-09-12T01:38:23.238871+0000) 2024-09-12T01:38:39.337 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:38 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:38.889+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:09.139766+0000 front 2024-09-12T01:38:09.139677+0000 (oldest deadline 2024-09-12T01:38:33.839904+0000) 2024-09-12T01:38:39.806 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:39 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:39.547+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.309568+0000 front 2024-09-12T01:37:51.309537+0000 (oldest deadline 2024-09-12T01:38:14.209738+0000) 2024-09-12T01:38:39.806 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:39 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:39.547+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:59.510715+0000 front 2024-09-12T01:37:59.510935+0000 (oldest deadline 2024-09-12T01:38:25.410761+0000) 2024-09-12T01:38:39.806 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:39 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:39.547+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:06.511690+0000 front 2024-09-12T01:38:06.511734+0000 (oldest deadline 2024-09-12T01:38:31.211659+0000) 2024-09-12T01:38:39.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:39 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:39.637+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:39.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:39 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:39.637+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.476477+0000 front 2024-09-12T01:37:51.476500+0000 (oldest deadline 2024-09-12T01:38:16.077208+0000) 2024-09-12T01:38:39.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:39 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:39.637+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.778585+0000 front 2024-09-12T01:37:57.778240+0000 (oldest deadline 2024-09-12T01:38:23.678115+0000) 2024-09-12T01:38:39.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:39 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:39.637+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.780419+0000 front 2024-09-12T01:38:08.780480+0000 (oldest deadline 2024-09-12T01:38:31.680454+0000) 2024-09-12T01:38:40.075 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:39 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:39.895+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:48.737024+0000 front 2024-09-12T01:37:48.736899+0000 (oldest deadline 2024-09-12T01:38:14.036808+0000) 2024-09-12T01:38:40.075 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:39 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:39.895+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.338640+0000 front 2024-09-12T01:37:57.338659+0000 (oldest deadline 2024-09-12T01:38:23.238871+0000) 2024-09-12T01:38:40.075 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:39 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:39.895+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:09.139766+0000 front 2024-09-12T01:38:09.139677+0000 (oldest deadline 2024-09-12T01:38:33.839904+0000) 2024-09-12T01:38:40.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:39 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:39.649+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:40.076 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:39 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:39.649+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:40.076 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:39 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:39.649+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:38:40.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:39 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:39.806+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:40.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:39 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:39.806+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:38:40.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:39 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:39.806+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.101595+0000 front 2024-09-12T01:38:00.101475+0000 (oldest deadline 2024-09-12T01:38:24.201476+0000) 2024-09-12T01:38:40.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:39 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:39.806+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.902499+0000 front 2024-09-12T01:38:08.902599+0000 (oldest deadline 2024-09-12T01:38:31.202668+0000) 2024-09-12T01:38:40.830 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:40 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:40.540+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.309568+0000 front 2024-09-12T01:37:51.309537+0000 (oldest deadline 2024-09-12T01:38:14.209738+0000) 2024-09-12T01:38:40.830 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:40 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:40.540+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:59.510715+0000 front 2024-09-12T01:37:59.510935+0000 (oldest deadline 2024-09-12T01:38:25.410761+0000) 2024-09-12T01:38:40.830 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:40 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:40.540+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:06.511690+0000 front 2024-09-12T01:38:06.511734+0000 (oldest deadline 2024-09-12T01:38:31.211659+0000) 2024-09-12T01:38:40.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:40 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:40.620+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:40.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:40 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:40.620+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.476477+0000 front 2024-09-12T01:37:51.476500+0000 (oldest deadline 2024-09-12T01:38:16.077208+0000) 2024-09-12T01:38:40.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:40 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:40.620+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.778585+0000 front 2024-09-12T01:37:57.778240+0000 (oldest deadline 2024-09-12T01:38:23.678115+0000) 2024-09-12T01:38:40.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:40 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:40.620+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.780419+0000 front 2024-09-12T01:38:08.780480+0000 (oldest deadline 2024-09-12T01:38:31.680454+0000) 2024-09-12T01:38:41.075 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:40 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:40.849+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:48.737024+0000 front 2024-09-12T01:37:48.736899+0000 (oldest deadline 2024-09-12T01:38:14.036808+0000) 2024-09-12T01:38:41.075 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:40 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:40.849+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.338640+0000 front 2024-09-12T01:37:57.338659+0000 (oldest deadline 2024-09-12T01:38:23.238871+0000) 2024-09-12T01:38:41.075 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:40 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:40.849+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:09.139766+0000 front 2024-09-12T01:38:09.139677+0000 (oldest deadline 2024-09-12T01:38:33.839904+0000) 2024-09-12T01:38:41.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:40 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:40.634+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:41.076 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:40 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:40.634+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:41.076 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:40 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:40.634+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:38:41.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:40 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:40.830+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:41.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:40 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:40.830+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:38:41.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:40 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:40.830+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.101595+0000 front 2024-09-12T01:38:00.101475+0000 (oldest deadline 2024-09-12T01:38:24.201476+0000) 2024-09-12T01:38:41.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:40 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:40.830+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.902499+0000 front 2024-09-12T01:38:08.902599+0000 (oldest deadline 2024-09-12T01:38:31.202668+0000) 2024-09-12T01:38:41.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:40 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:40.830+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:18.204266+0000 front 2024-09-12T01:38:18.204233+0000 (oldest deadline 2024-09-12T01:38:40.504381+0000) 2024-09-12T01:38:41.840 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:41 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:41.562+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.309568+0000 front 2024-09-12T01:37:51.309537+0000 (oldest deadline 2024-09-12T01:38:14.209738+0000) 2024-09-12T01:38:41.840 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:41 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:41.562+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:59.510715+0000 front 2024-09-12T01:37:59.510935+0000 (oldest deadline 2024-09-12T01:38:25.410761+0000) 2024-09-12T01:38:41.840 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:41 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:41.562+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:06.511690+0000 front 2024-09-12T01:38:06.511734+0000 (oldest deadline 2024-09-12T01:38:31.211659+0000) 2024-09-12T01:38:41.878 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:41 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:41.589+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:41.879 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:41 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:41.589+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:41.879 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:41 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:41.589+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:38:41.879 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:41 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:41.589+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:15.274333+0000 front 2024-09-12T01:38:15.274465+0000 (oldest deadline 2024-09-12T01:38:41.174237+0000) 2024-09-12T01:38:41.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:41 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:41.578+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:41.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:41 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:41.578+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.476477+0000 front 2024-09-12T01:37:51.476500+0000 (oldest deadline 2024-09-12T01:38:16.077208+0000) 2024-09-12T01:38:41.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:41 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:41.578+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.778585+0000 front 2024-09-12T01:37:57.778240+0000 (oldest deadline 2024-09-12T01:38:23.678115+0000) 2024-09-12T01:38:41.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:41 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:41.578+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.780419+0000 front 2024-09-12T01:38:08.780480+0000 (oldest deadline 2024-09-12T01:38:31.680454+0000) 2024-09-12T01:38:42.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:41 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:41.840+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:42.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:41 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:41.840+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:38:42.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:41 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:41.840+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.101595+0000 front 2024-09-12T01:38:00.101475+0000 (oldest deadline 2024-09-12T01:38:24.201476+0000) 2024-09-12T01:38:42.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:41 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:41.840+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.902499+0000 front 2024-09-12T01:38:08.902599+0000 (oldest deadline 2024-09-12T01:38:31.202668+0000) 2024-09-12T01:38:42.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:41 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:41.840+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:18.204266+0000 front 2024-09-12T01:38:18.204233+0000 (oldest deadline 2024-09-12T01:38:40.504381+0000) 2024-09-12T01:38:42.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:41 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:41.879+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:48.737024+0000 front 2024-09-12T01:37:48.736899+0000 (oldest deadline 2024-09-12T01:38:14.036808+0000) 2024-09-12T01:38:42.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:41 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:41.879+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.338640+0000 front 2024-09-12T01:37:57.338659+0000 (oldest deadline 2024-09-12T01:38:23.238871+0000) 2024-09-12T01:38:42.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:41 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:41.879+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:09.139766+0000 front 2024-09-12T01:38:09.139677+0000 (oldest deadline 2024-09-12T01:38:33.839904+0000) 2024-09-12T01:38:42.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:42 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:42.550+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:42.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:42 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:42.550+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:42.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:42 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:42.550+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:38:42.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:42 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:42.550+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:15.274333+0000 front 2024-09-12T01:38:15.274465+0000 (oldest deadline 2024-09-12T01:38:41.174237+0000) 2024-09-12T01:38:42.840 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:42 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:42.576+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.309568+0000 front 2024-09-12T01:37:51.309537+0000 (oldest deadline 2024-09-12T01:38:14.209738+0000) 2024-09-12T01:38:42.840 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:42 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:42.576+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:59.510715+0000 front 2024-09-12T01:37:59.510935+0000 (oldest deadline 2024-09-12T01:38:25.410761+0000) 2024-09-12T01:38:42.840 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:42 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:42.576+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:06.511690+0000 front 2024-09-12T01:38:06.511734+0000 (oldest deadline 2024-09-12T01:38:31.211659+0000) 2024-09-12T01:38:42.840 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:42 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7[53295]: 2024-09-12T01:38:42.576+0000 7fe011f28640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:17.613126+0000 front 2024-09-12T01:38:17.613158+0000 (oldest deadline 2024-09-12T01:38:41.713205+0000) 2024-09-12T01:38:42.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:42 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:42.564+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:42.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:42 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:42.564+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.476477+0000 front 2024-09-12T01:37:51.476500+0000 (oldest deadline 2024-09-12T01:38:16.077208+0000) 2024-09-12T01:38:42.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:42 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:42.564+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.778585+0000 front 2024-09-12T01:37:57.778240+0000 (oldest deadline 2024-09-12T01:38:23.678115+0000) 2024-09-12T01:38:42.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:42 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:42.564+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.780419+0000 front 2024-09-12T01:38:08.780480+0000 (oldest deadline 2024-09-12T01:38:31.680454+0000) 2024-09-12T01:38:43.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:42 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:42.841+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:43.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:42 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:42.841+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:38:43.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:42 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:42.841+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.101595+0000 front 2024-09-12T01:38:00.101475+0000 (oldest deadline 2024-09-12T01:38:24.201476+0000) 2024-09-12T01:38:43.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:42 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:42.841+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.902499+0000 front 2024-09-12T01:38:08.902599+0000 (oldest deadline 2024-09-12T01:38:31.202668+0000) 2024-09-12T01:38:43.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:42 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:42.841+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:18.204266+0000 front 2024-09-12T01:38:18.204233+0000 (oldest deadline 2024-09-12T01:38:40.504381+0000) 2024-09-12T01:38:43.329 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:42 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:42.837+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:48.737024+0000 front 2024-09-12T01:37:48.736899+0000 (oldest deadline 2024-09-12T01:38:14.036808+0000) 2024-09-12T01:38:43.329 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:42 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:42.837+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.338640+0000 front 2024-09-12T01:37:57.338659+0000 (oldest deadline 2024-09-12T01:38:23.238871+0000) 2024-09-12T01:38:43.329 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:42 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:42.837+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:09.139766+0000 front 2024-09-12T01:38:09.139677+0000 (oldest deadline 2024-09-12T01:38:33.839904+0000) 2024-09-12T01:38:43.329 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:42 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:42.837+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:13.840610+0000 front 2024-09-12T01:38:13.840490+0000 (oldest deadline 2024-09-12T01:38:42.640999+0000) 2024-09-12T01:38:43.740 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:43 smithi053 podman[69760]: 2024-09-12 01:38:43.488825948 +0000 UTC m=+5.570701716 container died c2ca342fd2b5edabf4f749c97483f289bd4f77c91062f2375f79bfe1aa2dae83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-12T01:38:43.825 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:43 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:43.811+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:48.737024+0000 front 2024-09-12T01:37:48.736899+0000 (oldest deadline 2024-09-12T01:38:14.036808+0000) 2024-09-12T01:38:43.825 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:43 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:43.811+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.338640+0000 front 2024-09-12T01:37:57.338659+0000 (oldest deadline 2024-09-12T01:38:23.238871+0000) 2024-09-12T01:38:43.825 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:43 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:43.811+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:09.139766+0000 front 2024-09-12T01:38:09.139677+0000 (oldest deadline 2024-09-12T01:38:33.839904+0000) 2024-09-12T01:38:43.825 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:43 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:43.811+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:13.840610+0000 front 2024-09-12T01:38:13.840490+0000 (oldest deadline 2024-09-12T01:38:42.640999+0000) 2024-09-12T01:38:43.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:43 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:43.575+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:43.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:43 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:43.575+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:43.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:43 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:43.575+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:38:43.826 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:43 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:43.575+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:15.274333+0000 front 2024-09-12T01:38:15.274465+0000 (oldest deadline 2024-09-12T01:38:41.174237+0000) 2024-09-12T01:38:43.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:43 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:43.529+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:43.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:43 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:43.529+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.476477+0000 front 2024-09-12T01:37:51.476500+0000 (oldest deadline 2024-09-12T01:38:16.077208+0000) 2024-09-12T01:38:43.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:43 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:43.529+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.778585+0000 front 2024-09-12T01:37:57.778240+0000 (oldest deadline 2024-09-12T01:38:23.678115+0000) 2024-09-12T01:38:43.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:43 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:43.529+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.780419+0000 front 2024-09-12T01:38:08.780480+0000 (oldest deadline 2024-09-12T01:38:31.680454+0000) 2024-09-12T01:38:44.112 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:43 smithi053 podman[69760]: 2024-09-12 01:38:43.903972839 +0000 UTC m=+5.985848602 container cleanup c2ca342fd2b5edabf4f749c97483f289bd4f77c91062f2375f79bfe1aa2dae83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-12T01:38:44.113 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:43 smithi053 bash[69760]: ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7 2024-09-12T01:38:44.113 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:43 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:43.818+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:44.113 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:43 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:43.818+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:38:44.113 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:43 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:43.818+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.101595+0000 front 2024-09-12T01:38:00.101475+0000 (oldest deadline 2024-09-12T01:38:24.201476+0000) 2024-09-12T01:38:44.113 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:43 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:43.818+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.902499+0000 front 2024-09-12T01:38:08.902599+0000 (oldest deadline 2024-09-12T01:38:31.202668+0000) 2024-09-12T01:38:44.113 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:43 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:43.818+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:18.204266+0000 front 2024-09-12T01:38:18.204233+0000 (oldest deadline 2024-09-12T01:38:40.504381+0000) 2024-09-12T01:38:44.492 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:44 smithi053 podman[69786]: 2024-09-12 01:38:44.113472692 +0000 UTC m=+0.621762904 container remove c2ca342fd2b5edabf4f749c97483f289bd4f77c91062f2375f79bfe1aa2dae83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731) 2024-09-12T01:38:44.804 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:44 smithi053 podman[69874]: 2024-09-12 01:38:44.430001924 +0000 UTC m=+0.015891974 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:38:44.804 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:44 smithi053 podman[69874]: 2024-09-12 01:38:44.547821368 +0000 UTC m=+0.133711450 container create 7721b0076f143f6b2502275a103700c0e5d2a5f3c97ff561089cf2392642cf8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-12T01:38:44.805 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:44 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:44.774+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:44.805 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:44 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:44.774+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:38:44.805 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:44 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:44.774+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.101595+0000 front 2024-09-12T01:38:00.101475+0000 (oldest deadline 2024-09-12T01:38:24.201476+0000) 2024-09-12T01:38:44.805 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:44 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:44.774+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.902499+0000 front 2024-09-12T01:38:08.902599+0000 (oldest deadline 2024-09-12T01:38:31.202668+0000) 2024-09-12T01:38:44.805 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:44 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:44.774+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:18.204266+0000 front 2024-09-12T01:38:18.204233+0000 (oldest deadline 2024-09-12T01:38:40.504381+0000) 2024-09-12T01:38:44.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:44 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:44.522+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:44.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:44 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:44.522+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.476477+0000 front 2024-09-12T01:37:51.476500+0000 (oldest deadline 2024-09-12T01:38:16.077208+0000) 2024-09-12T01:38:44.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:44 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:44.522+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.778585+0000 front 2024-09-12T01:37:57.778240+0000 (oldest deadline 2024-09-12T01:38:23.678115+0000) 2024-09-12T01:38:44.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:44 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:44.522+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.780419+0000 front 2024-09-12T01:38:08.780480+0000 (oldest deadline 2024-09-12T01:38:31.680454+0000) 2024-09-12T01:38:44.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:44 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:44.522+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:18.082372+0000 front 2024-09-12T01:38:18.082350+0000 (oldest deadline 2024-09-12T01:38:43.982544+0000) 2024-09-12T01:38:45.075 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:44 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:44.774+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:48.737024+0000 front 2024-09-12T01:37:48.736899+0000 (oldest deadline 2024-09-12T01:38:14.036808+0000) 2024-09-12T01:38:45.075 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:44 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:44.774+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.338640+0000 front 2024-09-12T01:37:57.338659+0000 (oldest deadline 2024-09-12T01:38:23.238871+0000) 2024-09-12T01:38:45.075 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:44 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:44.774+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:09.139766+0000 front 2024-09-12T01:38:09.139677+0000 (oldest deadline 2024-09-12T01:38:33.839904+0000) 2024-09-12T01:38:45.075 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:44 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:44.774+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:13.840610+0000 front 2024-09-12T01:38:13.840490+0000 (oldest deadline 2024-09-12T01:38:42.640999+0000) 2024-09-12T01:38:45.076 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:44 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:44.579+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:45.076 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:44 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:44.579+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:45.076 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:44 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:44.579+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:38:45.076 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:44 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:44.579+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:15.274333+0000 front 2024-09-12T01:38:15.274465+0000 (oldest deadline 2024-09-12T01:38:41.174237+0000) 2024-09-12T01:38:45.162 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:44 smithi053 podman[69874]: 2024-09-12 01:38:44.967011942 +0000 UTC m=+0.552901982 container init 7721b0076f143f6b2502275a103700c0e5d2a5f3c97ff561089cf2392642cf8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731) 2024-09-12T01:38:45.162 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:44 smithi053 podman[69874]: 2024-09-12 01:38:44.970414237 +0000 UTC m=+0.556304286 container start 7721b0076f143f6b2502275a103700c0e5d2a5f3c97ff561089cf2392642cf8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7-deactivate, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-12T01:38:45.162 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:45 smithi053 podman[69874]: 2024-09-12 01:38:45.050218449 +0000 UTC m=+0.636108501 container attach 7721b0076f143f6b2502275a103700c0e5d2a5f3c97ff561089cf2392642cf8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-12T01:38:45.490 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 12 01:38:45 smithi053 podman[69874]: 2024-09-12 01:38:45.162576901 +0000 UTC m=+0.748466955 container died 7721b0076f143f6b2502275a103700c0e5d2a5f3c97ff561089cf2392642cf8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-12T01:38:45.819 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@osd.7.service' 2024-09-12T01:38:45.822 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:45 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:45.540+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:45.822 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:45 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:45.540+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:45.822 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:45 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:45.540+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:38:45.822 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:45 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:45.540+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:15.274333+0000 front 2024-09-12T01:38:15.274465+0000 (oldest deadline 2024-09-12T01:38:41.174237+0000) 2024-09-12T01:38:45.863 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:38:45.864 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-12T01:38:45.864 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-12T01:38:45.864 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@osd.8 2024-09-12T01:38:45.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:45 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:45.567+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:45.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:45 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:45.567+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.476477+0000 front 2024-09-12T01:37:51.476500+0000 (oldest deadline 2024-09-12T01:38:16.077208+0000) 2024-09-12T01:38:45.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:45 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:45.567+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.778585+0000 front 2024-09-12T01:37:57.778240+0000 (oldest deadline 2024-09-12T01:38:23.678115+0000) 2024-09-12T01:38:45.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:45 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:45.567+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.780419+0000 front 2024-09-12T01:38:08.780480+0000 (oldest deadline 2024-09-12T01:38:31.680454+0000) 2024-09-12T01:38:45.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:45 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:45.567+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:18.082372+0000 front 2024-09-12T01:38:18.082350+0000 (oldest deadline 2024-09-12T01:38:43.982544+0000) 2024-09-12T01:38:46.075 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:45 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:45.822+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:48.737024+0000 front 2024-09-12T01:37:48.736899+0000 (oldest deadline 2024-09-12T01:38:14.036808+0000) 2024-09-12T01:38:46.075 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:45 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:45.822+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.338640+0000 front 2024-09-12T01:37:57.338659+0000 (oldest deadline 2024-09-12T01:38:23.238871+0000) 2024-09-12T01:38:46.075 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:45 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:45.822+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:09.139766+0000 front 2024-09-12T01:38:09.139677+0000 (oldest deadline 2024-09-12T01:38:33.839904+0000) 2024-09-12T01:38:46.075 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:45 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:45.822+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:13.840610+0000 front 2024-09-12T01:38:13.840490+0000 (oldest deadline 2024-09-12T01:38:42.640999+0000) 2024-09-12T01:38:46.075 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:45 smithi104 systemd[1]: Stopping Ceph osd.8 for c0e4f180-70a4-11ef-bcea-c7b262605968... 2024-09-12T01:38:46.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:45 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:45.811+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:46.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:45 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:45.811+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:38:46.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:45 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:45.811+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.101595+0000 front 2024-09-12T01:38:00.101475+0000 (oldest deadline 2024-09-12T01:38:24.201476+0000) 2024-09-12T01:38:46.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:45 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:45.811+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.902499+0000 front 2024-09-12T01:38:08.902599+0000 (oldest deadline 2024-09-12T01:38:31.202668+0000) 2024-09-12T01:38:46.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:45 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:45.811+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:18.204266+0000 front 2024-09-12T01:38:18.204233+0000 (oldest deadline 2024-09-12T01:38:40.504381+0000) 2024-09-12T01:38:46.562 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:46 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:46.169+0000 7ff70de2d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:38:46.562 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:46 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:46.169+0000 7ff70de2d640 -1 osd.8 102 *** Got signal Terminated *** 2024-09-12T01:38:46.562 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:46 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:46.169+0000 7ff70de2d640 -1 osd.8 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T01:38:46.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:46 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:46.562+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:46.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:46 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:46.562+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:46.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:46 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:46.562+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:38:46.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:46 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:46.562+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:15.274333+0000 front 2024-09-12T01:38:15.274465+0000 (oldest deadline 2024-09-12T01:38:41.174237+0000) 2024-09-12T01:38:46.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:46 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:46.570+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:46.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:46 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:46.570+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.476477+0000 front 2024-09-12T01:37:51.476500+0000 (oldest deadline 2024-09-12T01:38:16.077208+0000) 2024-09-12T01:38:46.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:46 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:46.570+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.778585+0000 front 2024-09-12T01:37:57.778240+0000 (oldest deadline 2024-09-12T01:38:23.678115+0000) 2024-09-12T01:38:46.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:46 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:46.570+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.780419+0000 front 2024-09-12T01:38:08.780480+0000 (oldest deadline 2024-09-12T01:38:31.680454+0000) 2024-09-12T01:38:46.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:46 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:46.570+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:18.082372+0000 front 2024-09-12T01:38:18.082350+0000 (oldest deadline 2024-09-12T01:38:43.982544+0000) 2024-09-12T01:38:47.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:46 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:46.785+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:47.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:46 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:46.785+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:38:47.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:46 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:46.785+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.101595+0000 front 2024-09-12T01:38:00.101475+0000 (oldest deadline 2024-09-12T01:38:24.201476+0000) 2024-09-12T01:38:47.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:46 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:46.785+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.902499+0000 front 2024-09-12T01:38:08.902599+0000 (oldest deadline 2024-09-12T01:38:31.202668+0000) 2024-09-12T01:38:47.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:46 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:46.785+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:18.204266+0000 front 2024-09-12T01:38:18.204233+0000 (oldest deadline 2024-09-12T01:38:40.504381+0000) 2024-09-12T01:38:47.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:46 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:46.858+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:48.737024+0000 front 2024-09-12T01:37:48.736899+0000 (oldest deadline 2024-09-12T01:38:14.036808+0000) 2024-09-12T01:38:47.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:46 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:46.858+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.338640+0000 front 2024-09-12T01:37:57.338659+0000 (oldest deadline 2024-09-12T01:38:23.238871+0000) 2024-09-12T01:38:47.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:46 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:46.858+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:09.139766+0000 front 2024-09-12T01:38:09.139677+0000 (oldest deadline 2024-09-12T01:38:33.839904+0000) 2024-09-12T01:38:47.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:46 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:46.858+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:13.840610+0000 front 2024-09-12T01:38:13.840490+0000 (oldest deadline 2024-09-12T01:38:42.640999+0000) 2024-09-12T01:38:47.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:47 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:47.570+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:47.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:47 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:47.570+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:47.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:47 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:47.570+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:38:47.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:47 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:47.570+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:15.274333+0000 front 2024-09-12T01:38:15.274465+0000 (oldest deadline 2024-09-12T01:38:41.174237+0000) 2024-09-12T01:38:47.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:47 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:47.561+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:47.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:47 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:47.561+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.476477+0000 front 2024-09-12T01:37:51.476500+0000 (oldest deadline 2024-09-12T01:38:16.077208+0000) 2024-09-12T01:38:47.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:47 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:47.561+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.778585+0000 front 2024-09-12T01:37:57.778240+0000 (oldest deadline 2024-09-12T01:38:23.678115+0000) 2024-09-12T01:38:47.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:47 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:47.561+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.780419+0000 front 2024-09-12T01:38:08.780480+0000 (oldest deadline 2024-09-12T01:38:31.680454+0000) 2024-09-12T01:38:47.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:47 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:47.561+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:18.082372+0000 front 2024-09-12T01:38:18.082350+0000 (oldest deadline 2024-09-12T01:38:43.982544+0000) 2024-09-12T01:38:48.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:47 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:47.758+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:48.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:47 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:47.758+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:38:48.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:47 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:47.758+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.101595+0000 front 2024-09-12T01:38:00.101475+0000 (oldest deadline 2024-09-12T01:38:24.201476+0000) 2024-09-12T01:38:48.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:47 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:47.758+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.902499+0000 front 2024-09-12T01:38:08.902599+0000 (oldest deadline 2024-09-12T01:38:31.202668+0000) 2024-09-12T01:38:48.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:47 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:47.758+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:18.204266+0000 front 2024-09-12T01:38:18.204233+0000 (oldest deadline 2024-09-12T01:38:40.504381+0000) 2024-09-12T01:38:48.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:47 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:47.853+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:48.737024+0000 front 2024-09-12T01:37:48.736899+0000 (oldest deadline 2024-09-12T01:38:14.036808+0000) 2024-09-12T01:38:48.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:47 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:47.853+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.338640+0000 front 2024-09-12T01:37:57.338659+0000 (oldest deadline 2024-09-12T01:38:23.238871+0000) 2024-09-12T01:38:48.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:47 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:47.853+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:09.139766+0000 front 2024-09-12T01:38:09.139677+0000 (oldest deadline 2024-09-12T01:38:33.839904+0000) 2024-09-12T01:38:48.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:47 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:47.853+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:13.840610+0000 front 2024-09-12T01:38:13.840490+0000 (oldest deadline 2024-09-12T01:38:42.640999+0000) 2024-09-12T01:38:48.830 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:48 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:48.521+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:48.830 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:48 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:48.521+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.476477+0000 front 2024-09-12T01:37:51.476500+0000 (oldest deadline 2024-09-12T01:38:16.077208+0000) 2024-09-12T01:38:48.830 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:48 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:48.521+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.778585+0000 front 2024-09-12T01:37:57.778240+0000 (oldest deadline 2024-09-12T01:38:23.678115+0000) 2024-09-12T01:38:48.830 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:48 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:48.521+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.780419+0000 front 2024-09-12T01:38:08.780480+0000 (oldest deadline 2024-09-12T01:38:31.680454+0000) 2024-09-12T01:38:48.830 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:48 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:48.521+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:18.082372+0000 front 2024-09-12T01:38:18.082350+0000 (oldest deadline 2024-09-12T01:38:43.982544+0000) 2024-09-12T01:38:48.872 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:48 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:48.607+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:48.873 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:48 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:48.607+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:48.873 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:48 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:48.607+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:38:48.873 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:48 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:48.607+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:15.274333+0000 front 2024-09-12T01:38:15.274465+0000 (oldest deadline 2024-09-12T01:38:41.174237+0000) 2024-09-12T01:38:49.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:48 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:48.803+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:49.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:48 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:48.803+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:38:49.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:48 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:48.803+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.101595+0000 front 2024-09-12T01:38:00.101475+0000 (oldest deadline 2024-09-12T01:38:24.201476+0000) 2024-09-12T01:38:49.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:48 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:48.803+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.902499+0000 front 2024-09-12T01:38:08.902599+0000 (oldest deadline 2024-09-12T01:38:31.202668+0000) 2024-09-12T01:38:49.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:48 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:48.803+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:18.204266+0000 front 2024-09-12T01:38:18.204233+0000 (oldest deadline 2024-09-12T01:38:40.504381+0000) 2024-09-12T01:38:49.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:48 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:48.869+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:48.737024+0000 front 2024-09-12T01:37:48.736899+0000 (oldest deadline 2024-09-12T01:38:14.036808+0000) 2024-09-12T01:38:49.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:48 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:48.869+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.338640+0000 front 2024-09-12T01:37:57.338659+0000 (oldest deadline 2024-09-12T01:38:23.238871+0000) 2024-09-12T01:38:49.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:48 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:48.869+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:09.139766+0000 front 2024-09-12T01:38:09.139677+0000 (oldest deadline 2024-09-12T01:38:33.839904+0000) 2024-09-12T01:38:49.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:48 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:48.869+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:13.840610+0000 front 2024-09-12T01:38:13.840490+0000 (oldest deadline 2024-09-12T01:38:42.640999+0000) 2024-09-12T01:38:49.857 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:49 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:49.505+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:49.857 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:49 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:49.505+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.476477+0000 front 2024-09-12T01:37:51.476500+0000 (oldest deadline 2024-09-12T01:38:16.077208+0000) 2024-09-12T01:38:49.858 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:49 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:49.505+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.778585+0000 front 2024-09-12T01:37:57.778240+0000 (oldest deadline 2024-09-12T01:38:23.678115+0000) 2024-09-12T01:38:49.858 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:49 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:49.505+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.780419+0000 front 2024-09-12T01:38:08.780480+0000 (oldest deadline 2024-09-12T01:38:31.680454+0000) 2024-09-12T01:38:49.858 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:49 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:49.505+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:18.082372+0000 front 2024-09-12T01:38:18.082350+0000 (oldest deadline 2024-09-12T01:38:43.982544+0000) 2024-09-12T01:38:49.858 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:49 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:49.505+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:25.083904+0000 front 2024-09-12T01:38:25.083768+0000 (oldest deadline 2024-09-12T01:38:48.583768+0000) 2024-09-12T01:38:50.077 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:49 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:49.873+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:48.737024+0000 front 2024-09-12T01:37:48.736899+0000 (oldest deadline 2024-09-12T01:38:14.036808+0000) 2024-09-12T01:38:50.078 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:49 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:49.873+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.338640+0000 front 2024-09-12T01:37:57.338659+0000 (oldest deadline 2024-09-12T01:38:23.238871+0000) 2024-09-12T01:38:50.078 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:49 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:49.873+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:09.139766+0000 front 2024-09-12T01:38:09.139677+0000 (oldest deadline 2024-09-12T01:38:33.839904+0000) 2024-09-12T01:38:50.078 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:49 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:49.873+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:13.840610+0000 front 2024-09-12T01:38:13.840490+0000 (oldest deadline 2024-09-12T01:38:42.640999+0000) 2024-09-12T01:38:50.078 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:49 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:49.628+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:50.078 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:49 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:49.628+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:50.078 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:49 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:49.628+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:38:50.078 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:49 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:49.628+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:15.274333+0000 front 2024-09-12T01:38:15.274465+0000 (oldest deadline 2024-09-12T01:38:41.174237+0000) 2024-09-12T01:38:50.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:49 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:49.825+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:50.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:49 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:49.825+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:38:50.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:49 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:49.825+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.101595+0000 front 2024-09-12T01:38:00.101475+0000 (oldest deadline 2024-09-12T01:38:24.201476+0000) 2024-09-12T01:38:50.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:49 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:49.825+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.902499+0000 front 2024-09-12T01:38:08.902599+0000 (oldest deadline 2024-09-12T01:38:31.202668+0000) 2024-09-12T01:38:50.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:49 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:49.825+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:18.204266+0000 front 2024-09-12T01:38:18.204233+0000 (oldest deadline 2024-09-12T01:38:40.504381+0000) 2024-09-12T01:38:50.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:50 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:50.545+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:50.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:50 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:50.545+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.476477+0000 front 2024-09-12T01:37:51.476500+0000 (oldest deadline 2024-09-12T01:38:16.077208+0000) 2024-09-12T01:38:50.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:50 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:50.545+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.778585+0000 front 2024-09-12T01:37:57.778240+0000 (oldest deadline 2024-09-12T01:38:23.678115+0000) 2024-09-12T01:38:50.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:50 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:50.545+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.780419+0000 front 2024-09-12T01:38:08.780480+0000 (oldest deadline 2024-09-12T01:38:31.680454+0000) 2024-09-12T01:38:50.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:50 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:50.545+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:18.082372+0000 front 2024-09-12T01:38:18.082350+0000 (oldest deadline 2024-09-12T01:38:43.982544+0000) 2024-09-12T01:38:50.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:50 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:50.545+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:25.083904+0000 front 2024-09-12T01:38:25.083768+0000 (oldest deadline 2024-09-12T01:38:48.583768+0000) 2024-09-12T01:38:51.075 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:50 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:50.837+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:48.737024+0000 front 2024-09-12T01:37:48.736899+0000 (oldest deadline 2024-09-12T01:38:14.036808+0000) 2024-09-12T01:38:51.075 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:50 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:50.837+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.338640+0000 front 2024-09-12T01:37:57.338659+0000 (oldest deadline 2024-09-12T01:38:23.238871+0000) 2024-09-12T01:38:51.075 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:50 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:50.837+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:09.139766+0000 front 2024-09-12T01:38:09.139677+0000 (oldest deadline 2024-09-12T01:38:33.839904+0000) 2024-09-12T01:38:51.075 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:50 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:50.837+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:13.840610+0000 front 2024-09-12T01:38:13.840490+0000 (oldest deadline 2024-09-12T01:38:42.640999+0000) 2024-09-12T01:38:51.075 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:50 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8[53169]: 2024-09-12T01:38:50.837+0000 7ff70a443640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:24.341729+0000 front 2024-09-12T01:38:24.341782+0000 (oldest deadline 2024-09-12T01:38:50.241826+0000) 2024-09-12T01:38:51.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:50 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:50.589+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:51.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:50 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:50.589+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:51.076 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:50 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:50.589+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:38:51.076 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:50 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:50.589+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:15.274333+0000 front 2024-09-12T01:38:15.274465+0000 (oldest deadline 2024-09-12T01:38:41.174237+0000) 2024-09-12T01:38:51.076 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:50 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:50.589+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:26.375623+0000 front 2024-09-12T01:38:26.375659+0000 (oldest deadline 2024-09-12T01:38:49.875890+0000) 2024-09-12T01:38:51.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:50 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:50.872+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:51.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:50 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:50.872+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:38:51.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:50 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:50.872+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.101595+0000 front 2024-09-12T01:38:00.101475+0000 (oldest deadline 2024-09-12T01:38:24.201476+0000) 2024-09-12T01:38:51.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:50 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:50.872+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.902499+0000 front 2024-09-12T01:38:08.902599+0000 (oldest deadline 2024-09-12T01:38:31.202668+0000) 2024-09-12T01:38:51.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:50 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:50.872+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:18.204266+0000 front 2024-09-12T01:38:18.204233+0000 (oldest deadline 2024-09-12T01:38:40.504381+0000) 2024-09-12T01:38:51.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:50 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:50.872+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:25.205737+0000 front 2024-09-12T01:38:25.205624+0000 (oldest deadline 2024-09-12T01:38:49.905511+0000) 2024-09-12T01:38:51.678 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:51 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:51.587+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:51.678 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:51 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:51.587+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.476477+0000 front 2024-09-12T01:37:51.476500+0000 (oldest deadline 2024-09-12T01:38:16.077208+0000) 2024-09-12T01:38:51.678 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:51 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:51.587+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.778585+0000 front 2024-09-12T01:37:57.778240+0000 (oldest deadline 2024-09-12T01:38:23.678115+0000) 2024-09-12T01:38:51.678 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:51 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:51.587+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.780419+0000 front 2024-09-12T01:38:08.780480+0000 (oldest deadline 2024-09-12T01:38:31.680454+0000) 2024-09-12T01:38:51.679 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:51 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:51.587+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:18.082372+0000 front 2024-09-12T01:38:18.082350+0000 (oldest deadline 2024-09-12T01:38:43.982544+0000) 2024-09-12T01:38:51.679 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:51 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:51.587+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:25.083904+0000 front 2024-09-12T01:38:25.083768+0000 (oldest deadline 2024-09-12T01:38:48.583768+0000) 2024-09-12T01:38:52.045 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:51 smithi104 podman[70008]: 2024-09-12 01:38:51.738020069 +0000 UTC m=+5.684251789 container died 6a4500edb1bebf9f9b40abcb7b99fb2a5f186a7b55faa7d1488c0ceee8ecd96a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-12T01:38:52.045 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:51 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:51.623+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:52.045 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:51 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:51.623+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:52.045 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:51 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:51.623+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:38:52.045 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:51 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:51.623+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:15.274333+0000 front 2024-09-12T01:38:15.274465+0000 (oldest deadline 2024-09-12T01:38:41.174237+0000) 2024-09-12T01:38:52.045 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:51 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:51.623+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:26.375623+0000 front 2024-09-12T01:38:26.375659+0000 (oldest deadline 2024-09-12T01:38:49.875890+0000) 2024-09-12T01:38:52.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:51 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:51.843+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:52.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:51 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:51.843+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:38:52.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:51 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:51.843+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.101595+0000 front 2024-09-12T01:38:00.101475+0000 (oldest deadline 2024-09-12T01:38:24.201476+0000) 2024-09-12T01:38:52.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:51 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:51.843+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.902499+0000 front 2024-09-12T01:38:08.902599+0000 (oldest deadline 2024-09-12T01:38:31.202668+0000) 2024-09-12T01:38:52.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:51 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:51.843+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:18.204266+0000 front 2024-09-12T01:38:18.204233+0000 (oldest deadline 2024-09-12T01:38:40.504381+0000) 2024-09-12T01:38:52.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:51 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:51.843+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:25.205737+0000 front 2024-09-12T01:38:25.205624+0000 (oldest deadline 2024-09-12T01:38:49.905511+0000) 2024-09-12T01:38:52.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:52 smithi104 podman[70008]: 2024-09-12 01:38:52.144093467 +0000 UTC m=+6.090325187 container cleanup 6a4500edb1bebf9f9b40abcb7b99fb2a5f186a7b55faa7d1488c0ceee8ecd96a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-12T01:38:52.325 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:52 smithi104 bash[70008]: ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8 2024-09-12T01:38:52.753 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:52 smithi104 podman[70023]: 2024-09-12 01:38:52.403530764 +0000 UTC m=+0.664179218 container remove 6a4500edb1bebf9f9b40abcb7b99fb2a5f186a7b55faa7d1488c0ceee8ecd96a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T01:38:52.754 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:52 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:52.599+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:52.754 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:52 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:52.599+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:52.754 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:52 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:52.599+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:38:52.754 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:52 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:52.599+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:15.274333+0000 front 2024-09-12T01:38:15.274465+0000 (oldest deadline 2024-09-12T01:38:41.174237+0000) 2024-09-12T01:38:52.754 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:52 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:52.599+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:26.375623+0000 front 2024-09-12T01:38:26.375659+0000 (oldest deadline 2024-09-12T01:38:49.875890+0000) 2024-09-12T01:38:52.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:52 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:52.545+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:52.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:52 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:52.545+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.476477+0000 front 2024-09-12T01:37:51.476500+0000 (oldest deadline 2024-09-12T01:38:16.077208+0000) 2024-09-12T01:38:52.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:52 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:52.545+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.778585+0000 front 2024-09-12T01:37:57.778240+0000 (oldest deadline 2024-09-12T01:38:23.678115+0000) 2024-09-12T01:38:52.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:52 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:52.545+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.780419+0000 front 2024-09-12T01:38:08.780480+0000 (oldest deadline 2024-09-12T01:38:31.680454+0000) 2024-09-12T01:38:52.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:52 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:52.545+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:18.082372+0000 front 2024-09-12T01:38:18.082350+0000 (oldest deadline 2024-09-12T01:38:43.982544+0000) 2024-09-12T01:38:52.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:52 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:52.545+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:25.083904+0000 front 2024-09-12T01:38:25.083768+0000 (oldest deadline 2024-09-12T01:38:48.583768+0000) 2024-09-12T01:38:53.039 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:52 smithi104 podman[70113]: 2024-09-12 01:38:52.65727985 +0000 UTC m=+0.015710646 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:38:53.039 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:52 smithi104 podman[70113]: 2024-09-12 01:38:52.762606003 +0000 UTC m=+0.121036797 container create 57cfc740ee5870ee5cfe41d5eec1499e7896e2f6425765936ae3e6105045091d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-09-12T01:38:53.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:52 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:52.877+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:53.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:52 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:52.877+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:38:53.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:52 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:52.877+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.101595+0000 front 2024-09-12T01:38:00.101475+0000 (oldest deadline 2024-09-12T01:38:24.201476+0000) 2024-09-12T01:38:53.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:52 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:52.877+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.902499+0000 front 2024-09-12T01:38:08.902599+0000 (oldest deadline 2024-09-12T01:38:31.202668+0000) 2024-09-12T01:38:53.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:52 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:52.877+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:18.204266+0000 front 2024-09-12T01:38:18.204233+0000 (oldest deadline 2024-09-12T01:38:40.504381+0000) 2024-09-12T01:38:53.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:52 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:52.877+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:25.205737+0000 front 2024-09-12T01:38:25.205624+0000 (oldest deadline 2024-09-12T01:38:49.905511+0000) 2024-09-12T01:38:53.311 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:53 smithi104 podman[70113]: 2024-09-12 01:38:53.040581506 +0000 UTC m=+0.399012304 container init 57cfc740ee5870ee5cfe41d5eec1499e7896e2f6425765936ae3e6105045091d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-12T01:38:53.312 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:53 smithi104 podman[70113]: 2024-09-12 01:38:53.044303489 +0000 UTC m=+0.402734285 container start 57cfc740ee5870ee5cfe41d5eec1499e7896e2f6425765936ae3e6105045091d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3) 2024-09-12T01:38:53.312 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:53 smithi104 podman[70113]: 2024-09-12 01:38:53.123809467 +0000 UTC m=+0.482240276 container attach 57cfc740ee5870ee5cfe41d5eec1499e7896e2f6425765936ae3e6105045091d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8-deactivate, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-12T01:38:53.312 INFO:journalctl@ceph.osd.8.smithi104.stdout:Sep 12 01:38:53 smithi104 podman[70113]: 2024-09-12 01:38:53.252884877 +0000 UTC m=+0.611315672 container died 57cfc740ee5870ee5cfe41d5eec1499e7896e2f6425765936ae3e6105045091d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-12T01:38:53.885 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:53 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:53.607+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:53.885 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:53 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:53.607+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:53.885 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:53 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:53.607+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:38:53.885 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:53 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:53.607+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:15.274333+0000 front 2024-09-12T01:38:15.274465+0000 (oldest deadline 2024-09-12T01:38:41.174237+0000) 2024-09-12T01:38:53.885 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:53 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:53.607+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:26.375623+0000 front 2024-09-12T01:38:26.375659+0000 (oldest deadline 2024-09-12T01:38:49.875890+0000) 2024-09-12T01:38:53.906 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@osd.8.service' 2024-09-12T01:38:53.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:53 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:53.552+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:53.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:53 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:53.552+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.476477+0000 front 2024-09-12T01:37:51.476500+0000 (oldest deadline 2024-09-12T01:38:16.077208+0000) 2024-09-12T01:38:53.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:53 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:53.552+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.778585+0000 front 2024-09-12T01:37:57.778240+0000 (oldest deadline 2024-09-12T01:38:23.678115+0000) 2024-09-12T01:38:53.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:53 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:53.552+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.780419+0000 front 2024-09-12T01:38:08.780480+0000 (oldest deadline 2024-09-12T01:38:31.680454+0000) 2024-09-12T01:38:53.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:53 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:53.552+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:18.082372+0000 front 2024-09-12T01:38:18.082350+0000 (oldest deadline 2024-09-12T01:38:43.982544+0000) 2024-09-12T01:38:53.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:53 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:53.552+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:25.083904+0000 front 2024-09-12T01:38:25.083768+0000 (oldest deadline 2024-09-12T01:38:48.583768+0000) 2024-09-12T01:38:53.943 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:38:53.943 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-12T01:38:53.944 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-12T01:38:53.944 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@osd.9 2024-09-12T01:38:54.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:53 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:53.884+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:54.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:53 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:53.884+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:38:54.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:53 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:53.884+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.101595+0000 front 2024-09-12T01:38:00.101475+0000 (oldest deadline 2024-09-12T01:38:24.201476+0000) 2024-09-12T01:38:54.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:53 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:53.884+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.902499+0000 front 2024-09-12T01:38:08.902599+0000 (oldest deadline 2024-09-12T01:38:31.202668+0000) 2024-09-12T01:38:54.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:53 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:53.884+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:18.204266+0000 front 2024-09-12T01:38:18.204233+0000 (oldest deadline 2024-09-12T01:38:40.504381+0000) 2024-09-12T01:38:54.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:53 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:53.884+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:25.205737+0000 front 2024-09-12T01:38:25.205624+0000 (oldest deadline 2024-09-12T01:38:49.905511+0000) 2024-09-12T01:38:54.391 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:54 smithi001 systemd[1]: Stopping Ceph osd.9 for c0e4f180-70a4-11ef-bcea-c7b262605968... 2024-09-12T01:38:54.678 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:54 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:54.391+0000 7f7de1a00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:38:54.678 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:54 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:54.391+0000 7f7de1a00640 -1 osd.9 102 *** Got signal Terminated *** 2024-09-12T01:38:54.678 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:54 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:54.391+0000 7f7de1a00640 -1 osd.9 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T01:38:54.678 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:54 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:54.533+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:54.678 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:54 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:54.533+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.476477+0000 front 2024-09-12T01:37:51.476500+0000 (oldest deadline 2024-09-12T01:38:16.077208+0000) 2024-09-12T01:38:54.678 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:54 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:54.533+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.778585+0000 front 2024-09-12T01:37:57.778240+0000 (oldest deadline 2024-09-12T01:38:23.678115+0000) 2024-09-12T01:38:54.679 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:54 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:54.533+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.780419+0000 front 2024-09-12T01:38:08.780480+0000 (oldest deadline 2024-09-12T01:38:31.680454+0000) 2024-09-12T01:38:54.679 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:54 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:54.533+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:18.082372+0000 front 2024-09-12T01:38:18.082350+0000 (oldest deadline 2024-09-12T01:38:43.982544+0000) 2024-09-12T01:38:54.679 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:54 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:54.533+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:25.083904+0000 front 2024-09-12T01:38:25.083768+0000 (oldest deadline 2024-09-12T01:38:48.583768+0000) 2024-09-12T01:38:55.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:54 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:54.657+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:55.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:54 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:54.657+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:55.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:54 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:54.657+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:38:55.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:54 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:54.657+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:15.274333+0000 front 2024-09-12T01:38:15.274465+0000 (oldest deadline 2024-09-12T01:38:41.174237+0000) 2024-09-12T01:38:55.076 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:54 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:54.657+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:26.375623+0000 front 2024-09-12T01:38:26.375659+0000 (oldest deadline 2024-09-12T01:38:49.875890+0000) 2024-09-12T01:38:55.230 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:54 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:54.850+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:55.230 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:54 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:54.850+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:38:55.230 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:54 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:54.850+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.101595+0000 front 2024-09-12T01:38:00.101475+0000 (oldest deadline 2024-09-12T01:38:24.201476+0000) 2024-09-12T01:38:55.230 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:54 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:54.850+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.902499+0000 front 2024-09-12T01:38:08.902599+0000 (oldest deadline 2024-09-12T01:38:31.202668+0000) 2024-09-12T01:38:55.230 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:54 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:54.850+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:18.204266+0000 front 2024-09-12T01:38:18.204233+0000 (oldest deadline 2024-09-12T01:38:40.504381+0000) 2024-09-12T01:38:55.230 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:54 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:54.850+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:25.205737+0000 front 2024-09-12T01:38:25.205624+0000 (oldest deadline 2024-09-12T01:38:49.905511+0000) 2024-09-12T01:38:55.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:55 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:55.556+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:55.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:55 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:55.556+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.476477+0000 front 2024-09-12T01:37:51.476500+0000 (oldest deadline 2024-09-12T01:38:16.077208+0000) 2024-09-12T01:38:55.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:55 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:55.556+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.778585+0000 front 2024-09-12T01:37:57.778240+0000 (oldest deadline 2024-09-12T01:38:23.678115+0000) 2024-09-12T01:38:55.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:55 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:55.556+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.780419+0000 front 2024-09-12T01:38:08.780480+0000 (oldest deadline 2024-09-12T01:38:31.680454+0000) 2024-09-12T01:38:55.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:55 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:55.556+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:18.082372+0000 front 2024-09-12T01:38:18.082350+0000 (oldest deadline 2024-09-12T01:38:43.982544+0000) 2024-09-12T01:38:55.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:55 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:55.556+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:25.083904+0000 front 2024-09-12T01:38:25.083768+0000 (oldest deadline 2024-09-12T01:38:48.583768+0000) 2024-09-12T01:38:56.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:55 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:55.634+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:56.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:55 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:55.634+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:56.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:55 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:55.634+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:38:56.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:55 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:55.634+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:15.274333+0000 front 2024-09-12T01:38:15.274465+0000 (oldest deadline 2024-09-12T01:38:41.174237+0000) 2024-09-12T01:38:56.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:55 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:55.634+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:26.375623+0000 front 2024-09-12T01:38:26.375659+0000 (oldest deadline 2024-09-12T01:38:49.875890+0000) 2024-09-12T01:38:56.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:55 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:55.880+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:56.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:55 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:55.880+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:38:56.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:55 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:55.880+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.101595+0000 front 2024-09-12T01:38:00.101475+0000 (oldest deadline 2024-09-12T01:38:24.201476+0000) 2024-09-12T01:38:56.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:55 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:55.880+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.902499+0000 front 2024-09-12T01:38:08.902599+0000 (oldest deadline 2024-09-12T01:38:31.202668+0000) 2024-09-12T01:38:56.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:55 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:55.880+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:18.204266+0000 front 2024-09-12T01:38:18.204233+0000 (oldest deadline 2024-09-12T01:38:40.504381+0000) 2024-09-12T01:38:56.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:55 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:55.880+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:25.205737+0000 front 2024-09-12T01:38:25.205624+0000 (oldest deadline 2024-09-12T01:38:49.905511+0000) 2024-09-12T01:38:56.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:56 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:56.594+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:56.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:56 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:56.594+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.476477+0000 front 2024-09-12T01:37:51.476500+0000 (oldest deadline 2024-09-12T01:38:16.077208+0000) 2024-09-12T01:38:56.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:56 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:56.594+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.778585+0000 front 2024-09-12T01:37:57.778240+0000 (oldest deadline 2024-09-12T01:38:23.678115+0000) 2024-09-12T01:38:56.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:56 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:56.594+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.780419+0000 front 2024-09-12T01:38:08.780480+0000 (oldest deadline 2024-09-12T01:38:31.680454+0000) 2024-09-12T01:38:56.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:56 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:56.594+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:18.082372+0000 front 2024-09-12T01:38:18.082350+0000 (oldest deadline 2024-09-12T01:38:43.982544+0000) 2024-09-12T01:38:56.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:56 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:56.594+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:25.083904+0000 front 2024-09-12T01:38:25.083768+0000 (oldest deadline 2024-09-12T01:38:48.583768+0000) 2024-09-12T01:38:56.976 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:56 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:56.591+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:56.977 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:56 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:56.591+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:56.977 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:56 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:56.591+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:38:56.977 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:56 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:56.591+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:15.274333+0000 front 2024-09-12T01:38:15.274465+0000 (oldest deadline 2024-09-12T01:38:41.174237+0000) 2024-09-12T01:38:56.977 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:56 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:56.591+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:26.375623+0000 front 2024-09-12T01:38:26.375659+0000 (oldest deadline 2024-09-12T01:38:49.875890+0000) 2024-09-12T01:38:57.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:56 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:56.853+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:57.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:56 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:56.853+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:38:57.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:56 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:56.853+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.101595+0000 front 2024-09-12T01:38:00.101475+0000 (oldest deadline 2024-09-12T01:38:24.201476+0000) 2024-09-12T01:38:57.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:56 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:56.853+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.902499+0000 front 2024-09-12T01:38:08.902599+0000 (oldest deadline 2024-09-12T01:38:31.202668+0000) 2024-09-12T01:38:57.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:56 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:56.853+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:18.204266+0000 front 2024-09-12T01:38:18.204233+0000 (oldest deadline 2024-09-12T01:38:40.504381+0000) 2024-09-12T01:38:57.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:56 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:56.853+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:25.205737+0000 front 2024-09-12T01:38:25.205624+0000 (oldest deadline 2024-09-12T01:38:49.905511+0000) 2024-09-12T01:38:57.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:57 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:57.624+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:57.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:57 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:57.624+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.476477+0000 front 2024-09-12T01:37:51.476500+0000 (oldest deadline 2024-09-12T01:38:16.077208+0000) 2024-09-12T01:38:57.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:57 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:57.624+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.778585+0000 front 2024-09-12T01:37:57.778240+0000 (oldest deadline 2024-09-12T01:38:23.678115+0000) 2024-09-12T01:38:57.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:57 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:57.624+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.780419+0000 front 2024-09-12T01:38:08.780480+0000 (oldest deadline 2024-09-12T01:38:31.680454+0000) 2024-09-12T01:38:57.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:57 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:57.624+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:18.082372+0000 front 2024-09-12T01:38:18.082350+0000 (oldest deadline 2024-09-12T01:38:43.982544+0000) 2024-09-12T01:38:57.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:57 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:57.624+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:25.083904+0000 front 2024-09-12T01:38:25.083768+0000 (oldest deadline 2024-09-12T01:38:48.583768+0000) 2024-09-12T01:38:58.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:57 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:57.603+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:58.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:57 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:57.603+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:58.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:57 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:57.603+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:38:58.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:57 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:57.603+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:15.274333+0000 front 2024-09-12T01:38:15.274465+0000 (oldest deadline 2024-09-12T01:38:41.174237+0000) 2024-09-12T01:38:58.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:57 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:57.603+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:26.375623+0000 front 2024-09-12T01:38:26.375659+0000 (oldest deadline 2024-09-12T01:38:49.875890+0000) 2024-09-12T01:38:58.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:57 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:57.603+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-12T01:38:32.176992+0000 front 2024-09-12T01:38:32.177018+0000 (oldest deadline 2024-09-12T01:38:57.476816+0000) 2024-09-12T01:38:58.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:57 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:57.823+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:58.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:57 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:57.823+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:38:58.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:57 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:57.823+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.101595+0000 front 2024-09-12T01:38:00.101475+0000 (oldest deadline 2024-09-12T01:38:24.201476+0000) 2024-09-12T01:38:58.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:57 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:57.823+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.902499+0000 front 2024-09-12T01:38:08.902599+0000 (oldest deadline 2024-09-12T01:38:31.202668+0000) 2024-09-12T01:38:58.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:57 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:57.823+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:18.204266+0000 front 2024-09-12T01:38:18.204233+0000 (oldest deadline 2024-09-12T01:38:40.504381+0000) 2024-09-12T01:38:58.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:57 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:57.823+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:25.205737+0000 front 2024-09-12T01:38:25.205624+0000 (oldest deadline 2024-09-12T01:38:49.905511+0000) 2024-09-12T01:38:58.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:58 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:58.611+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:43.275395+0000 front 2024-09-12T01:37:43.275372+0000 (oldest deadline 2024-09-12T01:38:09.175270+0000) 2024-09-12T01:38:58.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:58 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:58.611+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:51.476477+0000 front 2024-09-12T01:37:51.476500+0000 (oldest deadline 2024-09-12T01:38:16.077208+0000) 2024-09-12T01:38:58.928 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:58 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:58.611+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:37:57.778585+0000 front 2024-09-12T01:37:57.778240+0000 (oldest deadline 2024-09-12T01:38:23.678115+0000) 2024-09-12T01:38:58.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:58 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:58.611+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.780419+0000 front 2024-09-12T01:38:08.780480+0000 (oldest deadline 2024-09-12T01:38:31.680454+0000) 2024-09-12T01:38:58.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:58 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:58.611+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:18.082372+0000 front 2024-09-12T01:38:18.082350+0000 (oldest deadline 2024-09-12T01:38:43.982544+0000) 2024-09-12T01:38:58.929 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:38:58 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9[64597]: 2024-09-12T01:38:58.611+0000 7f7ddd400640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:25.083904+0000 front 2024-09-12T01:38:25.083768+0000 (oldest deadline 2024-09-12T01:38:48.583768+0000) 2024-09-12T01:38:59.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:58 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:58.642+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:38:59.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:58 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:58.642+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:38:59.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:58 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:58.642+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:38:59.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:58 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:58.642+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:15.274333+0000 front 2024-09-12T01:38:15.274465+0000 (oldest deadline 2024-09-12T01:38:41.174237+0000) 2024-09-12T01:38:59.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:58 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:58.642+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:26.375623+0000 front 2024-09-12T01:38:26.375659+0000 (oldest deadline 2024-09-12T01:38:49.875890+0000) 2024-09-12T01:38:59.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:58 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:58.642+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-12T01:38:32.176992+0000 front 2024-09-12T01:38:32.177018+0000 (oldest deadline 2024-09-12T01:38:57.476816+0000) 2024-09-12T01:38:59.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:58 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:58.851+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:38:59.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:58 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:58.851+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:38:59.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:58 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:58.851+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.101595+0000 front 2024-09-12T01:38:00.101475+0000 (oldest deadline 2024-09-12T01:38:24.201476+0000) 2024-09-12T01:38:59.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:58 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:58.851+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.902499+0000 front 2024-09-12T01:38:08.902599+0000 (oldest deadline 2024-09-12T01:38:31.202668+0000) 2024-09-12T01:38:59.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:58 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:58.851+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:18.204266+0000 front 2024-09-12T01:38:18.204233+0000 (oldest deadline 2024-09-12T01:38:40.504381+0000) 2024-09-12T01:38:59.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:58 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:58.851+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:25.205737+0000 front 2024-09-12T01:38:25.205624+0000 (oldest deadline 2024-09-12T01:38:49.905511+0000) 2024-09-12T01:38:59.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:58 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:58.851+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-12T01:38:32.806518+0000 front 2024-09-12T01:38:32.806495+0000 (oldest deadline 2024-09-12T01:38:58.706478+0000) 2024-09-12T01:39:00.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:59 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:59.616+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:39:00.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:59 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:59.616+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:39:00.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:59 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:59.616+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:39:00.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:59 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:59.616+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:15.274333+0000 front 2024-09-12T01:38:15.274465+0000 (oldest deadline 2024-09-12T01:38:41.174237+0000) 2024-09-12T01:39:00.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:59 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:59.616+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:26.375623+0000 front 2024-09-12T01:38:26.375659+0000 (oldest deadline 2024-09-12T01:38:49.875890+0000) 2024-09-12T01:39:00.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:38:59 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:38:59.616+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-12T01:38:32.176992+0000 front 2024-09-12T01:38:32.177018+0000 (oldest deadline 2024-09-12T01:38:57.476816+0000) 2024-09-12T01:39:00.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:59 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:59.862+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:39:00.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:59 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:59.862+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:39:00.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:59 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:59.862+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.101595+0000 front 2024-09-12T01:38:00.101475+0000 (oldest deadline 2024-09-12T01:38:24.201476+0000) 2024-09-12T01:39:00.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:59 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:59.862+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.902499+0000 front 2024-09-12T01:38:08.902599+0000 (oldest deadline 2024-09-12T01:38:31.202668+0000) 2024-09-12T01:39:00.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:59 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:59.862+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:18.204266+0000 front 2024-09-12T01:38:18.204233+0000 (oldest deadline 2024-09-12T01:38:40.504381+0000) 2024-09-12T01:39:00.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:59 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:59.862+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:25.205737+0000 front 2024-09-12T01:38:25.205624+0000 (oldest deadline 2024-09-12T01:38:49.905511+0000) 2024-09-12T01:39:00.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:38:59 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:38:59.862+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-12T01:38:32.806518+0000 front 2024-09-12T01:38:32.806495+0000 (oldest deadline 2024-09-12T01:38:58.706478+0000) 2024-09-12T01:39:00.416 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:39:00 smithi001 podman[109743]: 2024-09-12 01:39:00.106257686 +0000 UTC m=+5.859954750 container died df85cdd4409f147eeadedbcc3fa7136f74d670983f2d9e8068ab8ea5664a3515 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T01:39:00.678 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:39:00 smithi001 podman[109743]: 2024-09-12 01:39:00.538230297 +0000 UTC m=+6.291927369 container cleanup df85cdd4409f147eeadedbcc3fa7136f74d670983f2d9e8068ab8ea5664a3515 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3) 2024-09-12T01:39:00.679 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:39:00 smithi001 bash[109743]: ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9 2024-09-12T01:39:01.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:00 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:00.636+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:39:01.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:00 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:00.636+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:39:01.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:00 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:00.636+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:39:01.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:00 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:00.636+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:15.274333+0000 front 2024-09-12T01:38:15.274465+0000 (oldest deadline 2024-09-12T01:38:41.174237+0000) 2024-09-12T01:39:01.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:00 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:00.636+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:26.375623+0000 front 2024-09-12T01:38:26.375659+0000 (oldest deadline 2024-09-12T01:38:49.875890+0000) 2024-09-12T01:39:01.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:00 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:00.636+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-12T01:38:32.176992+0000 front 2024-09-12T01:38:32.177018+0000 (oldest deadline 2024-09-12T01:38:57.476816+0000) 2024-09-12T01:39:01.178 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:39:00 smithi001 podman[109760]: 2024-09-12 01:39:00.800150773 +0000 UTC m=+0.682192893 container remove df85cdd4409f147eeadedbcc3fa7136f74d670983f2d9e8068ab8ea5664a3515 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-12T01:39:01.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:00 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:00.876+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:39:01.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:00 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:00.876+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:39:01.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:00 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:00.876+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.101595+0000 front 2024-09-12T01:38:00.101475+0000 (oldest deadline 2024-09-12T01:38:24.201476+0000) 2024-09-12T01:39:01.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:00 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:00.876+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.902499+0000 front 2024-09-12T01:38:08.902599+0000 (oldest deadline 2024-09-12T01:38:31.202668+0000) 2024-09-12T01:39:01.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:00 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:00.876+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:18.204266+0000 front 2024-09-12T01:38:18.204233+0000 (oldest deadline 2024-09-12T01:38:40.504381+0000) 2024-09-12T01:39:01.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:00 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:00.876+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:25.205737+0000 front 2024-09-12T01:38:25.205624+0000 (oldest deadline 2024-09-12T01:38:49.905511+0000) 2024-09-12T01:39:01.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:00 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:00.876+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-12T01:38:32.806518+0000 front 2024-09-12T01:38:32.806495+0000 (oldest deadline 2024-09-12T01:38:58.706478+0000) 2024-09-12T01:39:01.678 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:39:01 smithi001 podman[109851]: 2024-09-12 01:39:01.275691964 +0000 UTC m=+0.027211434 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:39:01.678 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:39:01 smithi001 podman[109851]: 2024-09-12 01:39:01.442818202 +0000 UTC m=+0.194337647 container create 007c3dd94da5a0f8d25b86fd20be15e4598cc03d1144f483fffd06919180d2e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2) 2024-09-12T01:39:02.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:01 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:01.630+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:39:02.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:01 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:01.630+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:39:02.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:01 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:01.630+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:39:02.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:01 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:01.630+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:15.274333+0000 front 2024-09-12T01:38:15.274465+0000 (oldest deadline 2024-09-12T01:38:41.174237+0000) 2024-09-12T01:39:02.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:01 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:01.630+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:26.375623+0000 front 2024-09-12T01:38:26.375659+0000 (oldest deadline 2024-09-12T01:38:49.875890+0000) 2024-09-12T01:39:02.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:01 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:01.630+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-12T01:38:32.176992+0000 front 2024-09-12T01:38:32.177018+0000 (oldest deadline 2024-09-12T01:38:57.476816+0000) 2024-09-12T01:39:02.109 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:39:01 smithi001 podman[109851]: 2024-09-12 01:39:01.837028944 +0000 UTC m=+0.588548413 container init 007c3dd94da5a0f8d25b86fd20be15e4598cc03d1144f483fffd06919180d2e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-12T01:39:02.109 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:39:01 smithi001 podman[109851]: 2024-09-12 01:39:01.843982279 +0000 UTC m=+0.595501725 container start 007c3dd94da5a0f8d25b86fd20be15e4598cc03d1144f483fffd06919180d2e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T01:39:02.109 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:39:01 smithi001 podman[109851]: 2024-09-12 01:39:01.928275418 +0000 UTC m=+0.679794881 container attach 007c3dd94da5a0f8d25b86fd20be15e4598cc03d1144f483fffd06919180d2e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2) 2024-09-12T01:39:02.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:01 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:01.841+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:39:02.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:01 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:01.841+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:39:02.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:01 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:01.841+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.101595+0000 front 2024-09-12T01:38:00.101475+0000 (oldest deadline 2024-09-12T01:38:24.201476+0000) 2024-09-12T01:39:02.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:01 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:01.841+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.902499+0000 front 2024-09-12T01:38:08.902599+0000 (oldest deadline 2024-09-12T01:38:31.202668+0000) 2024-09-12T01:39:02.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:01 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:01.841+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:18.204266+0000 front 2024-09-12T01:38:18.204233+0000 (oldest deadline 2024-09-12T01:38:40.504381+0000) 2024-09-12T01:39:02.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:01 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:01.841+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:25.205737+0000 front 2024-09-12T01:38:25.205624+0000 (oldest deadline 2024-09-12T01:38:49.905511+0000) 2024-09-12T01:39:02.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:01 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:01.841+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-12T01:38:32.806518+0000 front 2024-09-12T01:38:32.806495+0000 (oldest deadline 2024-09-12T01:38:58.706478+0000) 2024-09-12T01:39:02.428 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 12 01:39:02 smithi001 podman[109874]: 2024-09-12 01:39:02.155404827 +0000 UTC m=+0.026910056 container died 007c3dd94da5a0f8d25b86fd20be15e4598cc03d1144f483fffd06919180d2e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-9-deactivate, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T01:39:02.786 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@osd.9.service' 2024-09-12T01:39:02.861 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:39:02.861 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-12T01:39:02.861 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-12T01:39:02.861 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@osd.10 2024-09-12T01:39:03.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:02 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:02.604+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:39:03.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:02 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:02.604+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:39:03.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:02 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:02.604+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:39:03.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:02 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:02.604+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:15.274333+0000 front 2024-09-12T01:38:15.274465+0000 (oldest deadline 2024-09-12T01:38:41.174237+0000) 2024-09-12T01:39:03.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:02 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:02.604+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:26.375623+0000 front 2024-09-12T01:38:26.375659+0000 (oldest deadline 2024-09-12T01:38:49.875890+0000) 2024-09-12T01:39:03.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:02 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:02.604+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-12T01:38:32.176992+0000 front 2024-09-12T01:38:32.177018+0000 (oldest deadline 2024-09-12T01:38:57.476816+0000) 2024-09-12T01:39:03.206 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:02 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:02.826+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:39:03.206 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:02 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:02.826+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:39:03.206 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:02 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:02.826+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.101595+0000 front 2024-09-12T01:38:00.101475+0000 (oldest deadline 2024-09-12T01:38:24.201476+0000) 2024-09-12T01:39:03.207 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:02 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:02.826+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.902499+0000 front 2024-09-12T01:38:08.902599+0000 (oldest deadline 2024-09-12T01:38:31.202668+0000) 2024-09-12T01:39:03.207 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:02 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:02.826+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:18.204266+0000 front 2024-09-12T01:38:18.204233+0000 (oldest deadline 2024-09-12T01:38:40.504381+0000) 2024-09-12T01:39:03.207 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:02 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:02.826+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:25.205737+0000 front 2024-09-12T01:38:25.205624+0000 (oldest deadline 2024-09-12T01:38:49.905511+0000) 2024-09-12T01:39:03.207 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:02 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:02.826+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-12T01:38:32.806518+0000 front 2024-09-12T01:38:32.806495+0000 (oldest deadline 2024-09-12T01:38:58.706478+0000) 2024-09-12T01:39:03.207 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:02 smithi053 systemd[1]: Stopping Ceph osd.10 for c0e4f180-70a4-11ef-bcea-c7b262605968... 2024-09-12T01:39:03.207 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:03 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:03.206+0000 7f2a1fc27640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:39:03.207 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:03 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:03.206+0000 7f2a1fc27640 -1 osd.10 102 *** Got signal Terminated *** 2024-09-12T01:39:03.207 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:03 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:03.206+0000 7f2a1fc27640 -1 osd.10 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T01:39:04.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:03 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:03.587+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:39:04.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:03 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:03.587+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:39:04.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:03 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:03.587+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:39:04.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:03 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:03.587+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:15.274333+0000 front 2024-09-12T01:38:15.274465+0000 (oldest deadline 2024-09-12T01:38:41.174237+0000) 2024-09-12T01:39:04.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:03 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:03.587+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:26.375623+0000 front 2024-09-12T01:38:26.375659+0000 (oldest deadline 2024-09-12T01:38:49.875890+0000) 2024-09-12T01:39:04.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:03 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:03.587+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-12T01:38:32.176992+0000 front 2024-09-12T01:38:32.177018+0000 (oldest deadline 2024-09-12T01:38:57.476816+0000) 2024-09-12T01:39:04.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:03 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:03.864+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:39:04.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:03 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:03.864+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:39:04.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:03 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:03.864+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.101595+0000 front 2024-09-12T01:38:00.101475+0000 (oldest deadline 2024-09-12T01:38:24.201476+0000) 2024-09-12T01:39:04.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:03 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:03.864+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.902499+0000 front 2024-09-12T01:38:08.902599+0000 (oldest deadline 2024-09-12T01:38:31.202668+0000) 2024-09-12T01:39:04.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:03 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:03.864+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:18.204266+0000 front 2024-09-12T01:38:18.204233+0000 (oldest deadline 2024-09-12T01:38:40.504381+0000) 2024-09-12T01:39:04.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:03 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:03.864+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:25.205737+0000 front 2024-09-12T01:38:25.205624+0000 (oldest deadline 2024-09-12T01:38:49.905511+0000) 2024-09-12T01:39:04.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:03 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:03.864+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-12T01:38:32.806518+0000 front 2024-09-12T01:38:32.806495+0000 (oldest deadline 2024-09-12T01:38:58.706478+0000) 2024-09-12T01:39:05.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:04 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:04.632+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:39:05.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:04 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:04.632+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:39:05.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:04 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:04.632+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:39:05.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:04 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:04.632+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:15.274333+0000 front 2024-09-12T01:38:15.274465+0000 (oldest deadline 2024-09-12T01:38:41.174237+0000) 2024-09-12T01:39:05.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:04 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:04.632+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:26.375623+0000 front 2024-09-12T01:38:26.375659+0000 (oldest deadline 2024-09-12T01:38:49.875890+0000) 2024-09-12T01:39:05.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:04 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:04.632+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-12T01:38:32.176992+0000 front 2024-09-12T01:38:32.177018+0000 (oldest deadline 2024-09-12T01:38:57.476816+0000) 2024-09-12T01:39:05.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:04 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:04.873+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:39:05.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:04 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:04.873+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:39:05.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:04 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:04.873+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.101595+0000 front 2024-09-12T01:38:00.101475+0000 (oldest deadline 2024-09-12T01:38:24.201476+0000) 2024-09-12T01:39:05.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:04 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:04.873+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.902499+0000 front 2024-09-12T01:38:08.902599+0000 (oldest deadline 2024-09-12T01:38:31.202668+0000) 2024-09-12T01:39:05.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:04 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:04.873+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:18.204266+0000 front 2024-09-12T01:38:18.204233+0000 (oldest deadline 2024-09-12T01:38:40.504381+0000) 2024-09-12T01:39:05.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:04 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:04.873+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:25.205737+0000 front 2024-09-12T01:38:25.205624+0000 (oldest deadline 2024-09-12T01:38:49.905511+0000) 2024-09-12T01:39:05.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:04 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:04.873+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-12T01:38:32.806518+0000 front 2024-09-12T01:38:32.806495+0000 (oldest deadline 2024-09-12T01:38:58.706478+0000) 2024-09-12T01:39:06.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:05 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:05.651+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:39:06.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:05 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:05.651+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:39:06.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:05 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:05.651+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:39:06.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:05 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:05.651+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:15.274333+0000 front 2024-09-12T01:38:15.274465+0000 (oldest deadline 2024-09-12T01:38:41.174237+0000) 2024-09-12T01:39:06.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:05 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:05.651+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:26.375623+0000 front 2024-09-12T01:38:26.375659+0000 (oldest deadline 2024-09-12T01:38:49.875890+0000) 2024-09-12T01:39:06.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:05 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:05.651+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-12T01:38:32.176992+0000 front 2024-09-12T01:38:32.177018+0000 (oldest deadline 2024-09-12T01:38:57.476816+0000) 2024-09-12T01:39:06.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:05 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:05.852+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:39:06.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:05 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:05.852+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:39:06.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:05 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:05.852+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.101595+0000 front 2024-09-12T01:38:00.101475+0000 (oldest deadline 2024-09-12T01:38:24.201476+0000) 2024-09-12T01:39:06.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:05 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:05.852+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.902499+0000 front 2024-09-12T01:38:08.902599+0000 (oldest deadline 2024-09-12T01:38:31.202668+0000) 2024-09-12T01:39:06.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:05 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:05.852+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:18.204266+0000 front 2024-09-12T01:38:18.204233+0000 (oldest deadline 2024-09-12T01:38:40.504381+0000) 2024-09-12T01:39:06.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:05 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:05.852+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:25.205737+0000 front 2024-09-12T01:38:25.205624+0000 (oldest deadline 2024-09-12T01:38:49.905511+0000) 2024-09-12T01:39:06.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:05 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:05.852+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-12T01:38:32.806518+0000 front 2024-09-12T01:38:32.806495+0000 (oldest deadline 2024-09-12T01:38:58.706478+0000) 2024-09-12T01:39:07.029 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:06 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:06.667+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:39:07.029 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:06 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:06.667+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:39:07.029 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:06 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:06.667+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:39:07.029 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:06 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:06.667+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:15.274333+0000 front 2024-09-12T01:38:15.274465+0000 (oldest deadline 2024-09-12T01:38:41.174237+0000) 2024-09-12T01:39:07.029 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:06 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:06.667+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:26.375623+0000 front 2024-09-12T01:38:26.375659+0000 (oldest deadline 2024-09-12T01:38:49.875890+0000) 2024-09-12T01:39:07.029 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:06 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:06.667+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-12T01:38:32.176992+0000 front 2024-09-12T01:38:32.177018+0000 (oldest deadline 2024-09-12T01:38:57.476816+0000) 2024-09-12T01:39:07.029 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:06 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:06.667+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-09-12T01:38:40.377822+0000 front 2024-09-12T01:38:40.377898+0000 (oldest deadline 2024-09-12T01:39:05.677634+0000) 2024-09-12T01:39:07.141 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:06 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:06.818+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:39:07.142 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:06 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:06.818+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:39:07.142 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:06 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:06.818+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.101595+0000 front 2024-09-12T01:38:00.101475+0000 (oldest deadline 2024-09-12T01:38:24.201476+0000) 2024-09-12T01:39:07.142 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:06 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:06.818+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.902499+0000 front 2024-09-12T01:38:08.902599+0000 (oldest deadline 2024-09-12T01:38:31.202668+0000) 2024-09-12T01:39:07.142 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:06 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:06.818+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:18.204266+0000 front 2024-09-12T01:38:18.204233+0000 (oldest deadline 2024-09-12T01:38:40.504381+0000) 2024-09-12T01:39:07.142 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:06 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:06.818+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:25.205737+0000 front 2024-09-12T01:38:25.205624+0000 (oldest deadline 2024-09-12T01:38:49.905511+0000) 2024-09-12T01:39:07.142 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:06 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:06.818+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-12T01:38:32.806518+0000 front 2024-09-12T01:38:32.806495+0000 (oldest deadline 2024-09-12T01:38:58.706478+0000) 2024-09-12T01:39:08.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:07 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:07.636+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:39:08.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:07 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:07.636+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:39:08.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:07 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:07.636+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:39:08.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:07 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:07.636+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:15.274333+0000 front 2024-09-12T01:38:15.274465+0000 (oldest deadline 2024-09-12T01:38:41.174237+0000) 2024-09-12T01:39:08.076 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:07 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:07.636+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:26.375623+0000 front 2024-09-12T01:38:26.375659+0000 (oldest deadline 2024-09-12T01:38:49.875890+0000) 2024-09-12T01:39:08.076 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:07 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:07.636+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-12T01:38:32.176992+0000 front 2024-09-12T01:38:32.177018+0000 (oldest deadline 2024-09-12T01:38:57.476816+0000) 2024-09-12T01:39:08.076 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:07 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:07.636+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-09-12T01:38:40.377822+0000 front 2024-09-12T01:38:40.377898+0000 (oldest deadline 2024-09-12T01:39:05.677634+0000) 2024-09-12T01:39:08.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:07 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:07.848+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:40.798798+0000 front 2024-09-12T01:37:40.798714+0000 (oldest deadline 2024-09-12T01:38:06.098620+0000) 2024-09-12T01:39:08.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:07 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:07.848+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:52.500416+0000 front 2024-09-12T01:37:52.500447+0000 (oldest deadline 2024-09-12T01:38:17.200528+0000) 2024-09-12T01:39:08.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:07 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:07.848+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.101595+0000 front 2024-09-12T01:38:00.101475+0000 (oldest deadline 2024-09-12T01:38:24.201476+0000) 2024-09-12T01:39:08.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:07 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:07.848+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-12T01:38:08.902499+0000 front 2024-09-12T01:38:08.902599+0000 (oldest deadline 2024-09-12T01:38:31.202668+0000) 2024-09-12T01:39:08.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:07 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:07.848+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:18.204266+0000 front 2024-09-12T01:38:18.204233+0000 (oldest deadline 2024-09-12T01:38:40.504381+0000) 2024-09-12T01:39:08.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:07 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:07.848+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:25.205737+0000 front 2024-09-12T01:38:25.205624+0000 (oldest deadline 2024-09-12T01:38:49.905511+0000) 2024-09-12T01:39:08.241 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:07 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10[59833]: 2024-09-12T01:39:07.848+0000 7f2a18235640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-12T01:38:32.806518+0000 front 2024-09-12T01:38:32.806495+0000 (oldest deadline 2024-09-12T01:38:58.706478+0000) 2024-09-12T01:39:08.882 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:08 smithi053 podman[69975]: 2024-09-12 01:39:08.616696015 +0000 UTC m=+5.541841541 container died c9ea596a889bab74053ef97728bcf1609d6fb53d690fe355b3210c7bc4543fc7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-12T01:39:09.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:08 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:08.638+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:39:09.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:08 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:08.638+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:39:09.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:08 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:08.638+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:39:09.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:08 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:08.638+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:15.274333+0000 front 2024-09-12T01:38:15.274465+0000 (oldest deadline 2024-09-12T01:38:41.174237+0000) 2024-09-12T01:39:09.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:08 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:08.638+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:26.375623+0000 front 2024-09-12T01:38:26.375659+0000 (oldest deadline 2024-09-12T01:38:49.875890+0000) 2024-09-12T01:39:09.076 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:08 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:08.638+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-12T01:38:32.176992+0000 front 2024-09-12T01:38:32.177018+0000 (oldest deadline 2024-09-12T01:38:57.476816+0000) 2024-09-12T01:39:09.076 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:08 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:08.638+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-09-12T01:38:40.377822+0000 front 2024-09-12T01:38:40.377898+0000 (oldest deadline 2024-09-12T01:39:05.677634+0000) 2024-09-12T01:39:09.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:08 smithi053 podman[69975]: 2024-09-12 01:39:08.989769354 +0000 UTC m=+5.914914869 container cleanup c9ea596a889bab74053ef97728bcf1609d6fb53d690fe355b3210c7bc4543fc7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T01:39:09.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:08 smithi053 bash[69975]: ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10 2024-09-12T01:39:09.595 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:09 smithi053 podman[69993]: 2024-09-12 01:39:09.257578236 +0000 UTC m=+0.637036855 container remove c9ea596a889bab74053ef97728bcf1609d6fb53d690fe355b3210c7bc4543fc7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0) 2024-09-12T01:39:09.902 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:09 smithi053 podman[70083]: 2024-09-12 01:39:09.499505719 +0000 UTC m=+0.015498230 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:39:09.902 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:09 smithi053 podman[70083]: 2024-09-12 01:39:09.608664603 +0000 UTC m=+0.124657144 container create 126c687186d3bc16b9d86e424962123713f67f1222f0b2d8a8c5f9e01dd9c46b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T01:39:09.902 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:09 smithi053 podman[70083]: 2024-09-12 01:39:09.819201587 +0000 UTC m=+0.335194086 container init 126c687186d3bc16b9d86e424962123713f67f1222f0b2d8a8c5f9e01dd9c46b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-12T01:39:09.902 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:09 smithi053 podman[70083]: 2024-09-12 01:39:09.822670856 +0000 UTC m=+0.338663356 container start 126c687186d3bc16b9d86e424962123713f67f1222f0b2d8a8c5f9e01dd9c46b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-12T01:39:09.902 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:09 smithi053 podman[70083]: 2024-09-12 01:39:09.902511311 +0000 UTC m=+0.418503819 container attach 126c687186d3bc16b9d86e424962123713f67f1222f0b2d8a8c5f9e01dd9c46b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True) 2024-09-12T01:39:10.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:09 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:09.642+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:39:10.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:09 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:09.642+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:39:10.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:09 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:09.642+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:39:10.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:09 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:09.642+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:15.274333+0000 front 2024-09-12T01:38:15.274465+0000 (oldest deadline 2024-09-12T01:38:41.174237+0000) 2024-09-12T01:39:10.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:09 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:09.642+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:26.375623+0000 front 2024-09-12T01:38:26.375659+0000 (oldest deadline 2024-09-12T01:38:49.875890+0000) 2024-09-12T01:39:10.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:09 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:09.642+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-12T01:38:32.176992+0000 front 2024-09-12T01:38:32.177018+0000 (oldest deadline 2024-09-12T01:38:57.476816+0000) 2024-09-12T01:39:10.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:09 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:09.642+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-09-12T01:38:40.377822+0000 front 2024-09-12T01:38:40.377898+0000 (oldest deadline 2024-09-12T01:39:05.677634+0000) 2024-09-12T01:39:10.240 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 12 01:39:10 smithi053 podman[70083]: 2024-09-12 01:39:10.009949694 +0000 UTC m=+0.525942194 container died 126c687186d3bc16b9d86e424962123713f67f1222f0b2d8a8c5f9e01dd9c46b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3) 2024-09-12T01:39:10.571 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@osd.10.service' 2024-09-12T01:39:10.609 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:39:10.609 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-12T01:39:10.609 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-12T01:39:10.609 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@osd.11 2024-09-12T01:39:10.903 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:10 smithi104 systemd[1]: Stopping Ceph osd.11 for c0e4f180-70a4-11ef-bcea-c7b262605968... 2024-09-12T01:39:10.903 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:10 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:10.676+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:39:10.903 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:10 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:10.676+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:39:10.903 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:10 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:10.676+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:39:10.904 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:10 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:10.676+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:15.274333+0000 front 2024-09-12T01:38:15.274465+0000 (oldest deadline 2024-09-12T01:38:41.174237+0000) 2024-09-12T01:39:10.904 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:10 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:10.676+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:26.375623+0000 front 2024-09-12T01:38:26.375659+0000 (oldest deadline 2024-09-12T01:38:49.875890+0000) 2024-09-12T01:39:10.904 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:10 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:10.676+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-12T01:38:32.176992+0000 front 2024-09-12T01:38:32.177018+0000 (oldest deadline 2024-09-12T01:38:57.476816+0000) 2024-09-12T01:39:10.904 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:10 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:10.676+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-09-12T01:38:40.377822+0000 front 2024-09-12T01:38:40.377898+0000 (oldest deadline 2024-09-12T01:39:05.677634+0000) 2024-09-12T01:39:10.904 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:10 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:10.902+0000 7f19f251b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:39:10.904 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:10 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:10.902+0000 7f19f251b640 -1 osd.11 102 *** Got signal Terminated *** 2024-09-12T01:39:10.904 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:10 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:10.902+0000 7f19f251b640 -1 osd.11 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T01:39:12.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:11 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:11.683+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:39:12.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:11 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:11.683+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:39:12.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:11 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:11.683+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:39:12.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:11 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:11.683+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:15.274333+0000 front 2024-09-12T01:38:15.274465+0000 (oldest deadline 2024-09-12T01:38:41.174237+0000) 2024-09-12T01:39:12.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:11 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:11.683+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:26.375623+0000 front 2024-09-12T01:38:26.375659+0000 (oldest deadline 2024-09-12T01:38:49.875890+0000) 2024-09-12T01:39:12.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:11 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:11.683+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-12T01:38:32.176992+0000 front 2024-09-12T01:38:32.177018+0000 (oldest deadline 2024-09-12T01:38:57.476816+0000) 2024-09-12T01:39:12.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:11 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:11.683+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-09-12T01:38:40.377822+0000 front 2024-09-12T01:38:40.377898+0000 (oldest deadline 2024-09-12T01:39:05.677634+0000) 2024-09-12T01:39:13.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:12 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:12.649+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:39:13.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:12 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:12.649+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:39:13.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:12 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:12.649+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:39:13.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:12 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:12.649+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:15.274333+0000 front 2024-09-12T01:38:15.274465+0000 (oldest deadline 2024-09-12T01:38:41.174237+0000) 2024-09-12T01:39:13.076 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:12 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:12.649+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:26.375623+0000 front 2024-09-12T01:38:26.375659+0000 (oldest deadline 2024-09-12T01:38:49.875890+0000) 2024-09-12T01:39:13.076 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:12 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:12.649+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-12T01:38:32.176992+0000 front 2024-09-12T01:38:32.177018+0000 (oldest deadline 2024-09-12T01:38:57.476816+0000) 2024-09-12T01:39:13.076 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:12 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:12.649+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-09-12T01:38:40.377822+0000 front 2024-09-12T01:38:40.377898+0000 (oldest deadline 2024-09-12T01:39:05.677634+0000) 2024-09-12T01:39:14.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:13 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:13.666+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:39:14.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:13 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:13.666+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:39:14.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:13 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:13.666+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:39:14.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:13 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:13.666+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:15.274333+0000 front 2024-09-12T01:38:15.274465+0000 (oldest deadline 2024-09-12T01:38:41.174237+0000) 2024-09-12T01:39:14.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:13 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:13.666+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:26.375623+0000 front 2024-09-12T01:38:26.375659+0000 (oldest deadline 2024-09-12T01:38:49.875890+0000) 2024-09-12T01:39:14.076 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:13 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:13.666+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-12T01:38:32.176992+0000 front 2024-09-12T01:38:32.177018+0000 (oldest deadline 2024-09-12T01:38:57.476816+0000) 2024-09-12T01:39:14.076 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:13 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:13.666+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-09-12T01:38:40.377822+0000 front 2024-09-12T01:38:40.377898+0000 (oldest deadline 2024-09-12T01:39:05.677634+0000) 2024-09-12T01:39:15.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:14 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:14.655+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:39:15.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:14 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:14.655+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:39:15.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:14 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:14.655+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:39:15.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:14 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:14.655+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:15.274333+0000 front 2024-09-12T01:38:15.274465+0000 (oldest deadline 2024-09-12T01:38:41.174237+0000) 2024-09-12T01:39:15.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:14 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:14.655+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:26.375623+0000 front 2024-09-12T01:38:26.375659+0000 (oldest deadline 2024-09-12T01:38:49.875890+0000) 2024-09-12T01:39:15.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:14 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:14.655+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-12T01:38:32.176992+0000 front 2024-09-12T01:38:32.177018+0000 (oldest deadline 2024-09-12T01:38:57.476816+0000) 2024-09-12T01:39:15.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:14 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:14.655+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-09-12T01:38:40.377822+0000 front 2024-09-12T01:38:40.377898+0000 (oldest deadline 2024-09-12T01:39:05.677634+0000) 2024-09-12T01:39:16.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:15 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:15.643+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-12T01:37:44.468058+0000 front 2024-09-12T01:37:44.468088+0000 (oldest deadline 2024-09-12T01:38:08.468780+0000) 2024-09-12T01:39:16.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:15 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:15.643+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-12T01:37:53.070409+0000 front 2024-09-12T01:37:53.070343+0000 (oldest deadline 2024-09-12T01:38:14.170514+0000) 2024-09-12T01:39:16.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:15 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:15.643+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-09-12T01:38:00.071599+0000 front 2024-09-12T01:38:00.071579+0000 (oldest deadline 2024-09-12T01:38:24.171838+0000) 2024-09-12T01:39:16.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:15 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:15.643+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-12T01:38:15.274333+0000 front 2024-09-12T01:38:15.274465+0000 (oldest deadline 2024-09-12T01:38:41.174237+0000) 2024-09-12T01:39:16.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:15 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:15.643+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-09-12T01:38:26.375623+0000 front 2024-09-12T01:38:26.375659+0000 (oldest deadline 2024-09-12T01:38:49.875890+0000) 2024-09-12T01:39:16.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:15 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:15.643+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-12T01:38:32.176992+0000 front 2024-09-12T01:38:32.177018+0000 (oldest deadline 2024-09-12T01:38:57.476816+0000) 2024-09-12T01:39:16.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:15 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11[60124]: 2024-09-12T01:39:15.643+0000 7f19eab29640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-09-12T01:38:40.377822+0000 front 2024-09-12T01:38:40.377898+0000 (oldest deadline 2024-09-12T01:39:05.677634+0000) 2024-09-12T01:39:16.554 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:16 smithi104 podman[70230]: 2024-09-12 01:39:16.294148153 +0000 UTC m=+5.492618059 container died de906ca44a3991b81c2266ec63cdec4c89b659c2b565689fd393190b58bfe5df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-12T01:39:16.820 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:16 smithi104 podman[70230]: 2024-09-12 01:39:16.636276004 +0000 UTC m=+5.834745908 container cleanup de906ca44a3991b81c2266ec63cdec4c89b659c2b565689fd393190b58bfe5df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-12T01:39:16.820 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:16 smithi104 bash[70230]: ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11 2024-09-12T01:39:17.075 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:16 smithi104 podman[70245]: 2024-09-12 01:39:16.820774687 +0000 UTC m=+0.523820378 container remove de906ca44a3991b81c2266ec63cdec4c89b659c2b565689fd393190b58bfe5df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-12T01:39:17.539 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:17 smithi104 podman[70334]: 2024-09-12 01:39:17.140557697 +0000 UTC m=+0.015774326 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:39:17.539 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:17 smithi104 podman[70334]: 2024-09-12 01:39:17.246304203 +0000 UTC m=+0.121520816 container create 949b0c1b02d6e4b62cc14c6b005626ae6b3d352220e32de8d3832208bfa4ea35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS) 2024-09-12T01:39:17.539 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:17 smithi104 podman[70334]: 2024-09-12 01:39:17.473694521 +0000 UTC m=+0.348911130 container init 949b0c1b02d6e4b62cc14c6b005626ae6b3d352220e32de8d3832208bfa4ea35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-12T01:39:17.539 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:17 smithi104 podman[70334]: 2024-09-12 01:39:17.477125068 +0000 UTC m=+0.352341686 container start 949b0c1b02d6e4b62cc14c6b005626ae6b3d352220e32de8d3832208bfa4ea35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS) 2024-09-12T01:39:17.539 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:17 smithi104 podman[70334]: 2024-09-12 01:39:17.540111656 +0000 UTC m=+0.415328271 container attach 949b0c1b02d6e4b62cc14c6b005626ae6b3d352220e32de8d3832208bfa4ea35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11-deactivate, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3) 2024-09-12T01:39:17.825 INFO:journalctl@ceph.osd.11.smithi104.stdout:Sep 12 01:39:17 smithi104 podman[70334]: 2024-09-12 01:39:17.66483466 +0000 UTC m=+0.540051288 container died 949b0c1b02d6e4b62cc14c6b005626ae6b3d352220e32de8d3832208bfa4ea35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-12T01:39:18.236 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@osd.11.service' 2024-09-12T01:39:18.274 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:39:18.274 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-12T01:39:18.274 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-12T01:39:18.275 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mds.a 2024-09-12T01:39:18.678 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 12 01:39:18 smithi001 systemd[1]: Stopping Ceph mds.a for c0e4f180-70a4-11ef-bcea-c7b262605968... 2024-09-12T01:39:19.157 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 12 01:39:18 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-a[71628]: 2024-09-12T01:39:18.769+0000 7f8679e00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:39:19.157 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 12 01:39:18 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-a[71628]: 2024-09-12T01:39:18.769+0000 7f8679e00640 -1 mds.a *** got signal Terminated *** 2024-09-12T01:39:20.678 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 12 01:39:20 smithi001 podman[109951]: 2024-09-12 01:39:20.422299717 +0000 UTC m=+1.849981856 container died 7f31d72433303eb83f41f3ce88a9b7abdfd908912ef44b6f8d7125dfff41ab25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-a, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True) 2024-09-12T01:39:21.178 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 12 01:39:20 smithi001 podman[109951]: 2024-09-12 01:39:20.991854986 +0000 UTC m=+2.419537131 container cleanup 7f31d72433303eb83f41f3ce88a9b7abdfd908912ef44b6f8d7125dfff41ab25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-12T01:39:21.178 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 12 01:39:20 smithi001 bash[109951]: ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-a 2024-09-12T01:39:21.470 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 12 01:39:21 smithi001 podman[109973]: 2024-09-12 01:39:21.201266484 +0000 UTC m=+0.761343672 container remove 7f31d72433303eb83f41f3ce88a9b7abdfd908912ef44b6f8d7125dfff41ab25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-a, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-12T01:39:21.500 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mds.a.service' 2024-09-12T01:39:21.572 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:39:21.572 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-12T01:39:21.573 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-12T01:39:21.573 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mds.d 2024-09-12T01:39:21.928 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 12 01:39:21 smithi001 systemd[1]: Stopping Ceph mds.d for c0e4f180-70a4-11ef-bcea-c7b262605968... 2024-09-12T01:39:22.428 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 12 01:39:22 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-d[70460]: 2024-09-12T01:39:21.997+0000 7f99efe00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:39:22.428 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 12 01:39:22 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-d[70460]: 2024-09-12T01:39:21.997+0000 7f99efe00640 -1 mds.d *** got signal Terminated *** 2024-09-12T01:39:23.428 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 12 01:39:23 smithi001 podman[110083]: 2024-09-12 01:39:23.142055798 +0000 UTC m=+1.278281311 container died d1561eab7856fc786a147f1877d513844b637834fc303ef308e48eea9cf82292 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-d, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-12T01:39:23.838 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 12 01:39:23 smithi001 podman[110083]: 2024-09-12 01:39:23.578798628 +0000 UTC m=+1.715024141 container cleanup d1561eab7856fc786a147f1877d513844b637834fc303ef308e48eea9cf82292 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_CLEAN=True) 2024-09-12T01:39:23.838 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 12 01:39:23 smithi001 bash[110083]: ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-d 2024-09-12T01:39:24.114 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 12 01:39:23 smithi001 podman[110095]: 2024-09-12 01:39:23.83847201 +0000 UTC m=+0.683956669 container remove d1561eab7856fc786a147f1877d513844b637834fc303ef308e48eea9cf82292 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-d, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0) 2024-09-12T01:39:24.138 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mds.d.service' 2024-09-12T01:39:24.208 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:39:24.209 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-12T01:39:24.209 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-12T01:39:24.209 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mds.g 2024-09-12T01:39:24.428 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 12 01:39:24 smithi001 systemd[1]: Stopping Ceph mds.g for c0e4f180-70a4-11ef-bcea-c7b262605968... 2024-09-12T01:39:25.178 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 12 01:39:24 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-g[71269]: 2024-09-12T01:39:24.758+0000 7f4edaa00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:39:25.178 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 12 01:39:24 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-g[71269]: 2024-09-12T01:39:24.758+0000 7f4edaa00640 -1 mds.g *** got signal Terminated *** 2024-09-12T01:39:28.678 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 12 01:39:28 smithi001 podman[110202]: 2024-09-12 01:39:28.427453533 +0000 UTC m=+3.926610658 container died 801be67e9eb3ad63d9b2e957ae148b23f887b7228fba5bba5115bca9d0a3a435 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-12T01:39:29.091 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 12 01:39:28 smithi001 podman[110202]: 2024-09-12 01:39:28.823954318 +0000 UTC m=+4.323111445 container cleanup 801be67e9eb3ad63d9b2e957ae148b23f887b7228fba5bba5115bca9d0a3a435 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-g, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-12T01:39:29.091 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 12 01:39:28 smithi001 bash[110202]: ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-g 2024-09-12T01:39:29.277 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mds.g.service' 2024-09-12T01:39:29.346 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:39:29.347 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-12T01:39:29.347 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-12T01:39:29.347 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mds.j 2024-09-12T01:39:29.678 INFO:journalctl@ceph.mds.j.smithi001.stdout:Sep 12 01:39:29 smithi001 systemd[1]: Stopping Ceph mds.j for c0e4f180-70a4-11ef-bcea-c7b262605968... 2024-09-12T01:39:30.178 INFO:journalctl@ceph.mds.j.smithi001.stdout:Sep 12 01:39:29 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-j[70080]: 2024-09-12T01:39:29.914+0000 7f6423200640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:39:30.178 INFO:journalctl@ceph.mds.j.smithi001.stdout:Sep 12 01:39:29 smithi001 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-j[70080]: 2024-09-12T01:39:29.914+0000 7f6423200640 -1 mds.j *** got signal Terminated *** 2024-09-12T01:39:34.178 INFO:journalctl@ceph.mds.j.smithi001.stdout:Sep 12 01:39:33 smithi001 podman[110322]: 2024-09-12 01:39:33.925860323 +0000 UTC m=+4.303383823 container died e2640934b85c0dc469b0ba69f292c5fc55b25ecc83dcdee2d1720c53a0b8694c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-j, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-12T01:39:34.678 INFO:journalctl@ceph.mds.j.smithi001.stdout:Sep 12 01:39:34 smithi001 podman[110322]: 2024-09-12 01:39:34.336138841 +0000 UTC m=+4.713662330 container cleanup e2640934b85c0dc469b0ba69f292c5fc55b25ecc83dcdee2d1720c53a0b8694c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-j, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-12T01:39:34.678 INFO:journalctl@ceph.mds.j.smithi001.stdout:Sep 12 01:39:34 smithi001 bash[110322]: ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-j 2024-09-12T01:39:35.067 INFO:journalctl@ceph.mds.j.smithi001.stdout:Sep 12 01:39:34 smithi001 podman[110333]: 2024-09-12 01:39:34.782502483 +0000 UTC m=+0.843485876 container remove e2640934b85c0dc469b0ba69f292c5fc55b25ecc83dcdee2d1720c53a0b8694c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-j, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-12T01:39:35.098 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mds.j.service' 2024-09-12T01:39:35.166 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:39:35.166 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-12T01:39:35.167 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-12T01:39:35.167 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mds.b 2024-09-12T01:39:35.481 INFO:journalctl@ceph.mds.b.smithi053.stdout:Sep 12 01:39:35 smithi053 systemd[1]: Stopping Ceph mds.b for c0e4f180-70a4-11ef-bcea-c7b262605968... 2024-09-12T01:39:35.740 INFO:journalctl@ceph.mds.b.smithi053.stdout:Sep 12 01:39:35 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-b[65497]: 2024-09-12T01:39:35.481+0000 7fe06a580640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:39:35.740 INFO:journalctl@ceph.mds.b.smithi053.stdout:Sep 12 01:39:35 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-b[65497]: 2024-09-12T01:39:35.481+0000 7fe06a580640 -1 mds.b *** got signal Terminated *** 2024-09-12T01:39:39.773 INFO:journalctl@ceph.mds.b.smithi053.stdout:Sep 12 01:39:39 smithi053 podman[70184]: 2024-09-12 01:39:39.491010134 +0000 UTC m=+4.120562687 container died be93d6a775616c3a3974b8bfc3c984b360dd8ba8e5aed9c6ac3423a03f508449 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True) 2024-09-12T01:39:40.107 INFO:journalctl@ceph.mds.b.smithi053.stdout:Sep 12 01:39:39 smithi053 podman[70184]: 2024-09-12 01:39:39.87308613 +0000 UTC m=+4.502638682 container cleanup be93d6a775616c3a3974b8bfc3c984b360dd8ba8e5aed9c6ac3423a03f508449 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-b, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-12T01:39:40.107 INFO:journalctl@ceph.mds.b.smithi053.stdout:Sep 12 01:39:39 smithi053 bash[70184]: ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-b 2024-09-12T01:39:40.262 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mds.b.service' 2024-09-12T01:39:40.295 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:39:40.296 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-12T01:39:40.296 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-12T01:39:40.296 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mds.e 2024-09-12T01:39:40.626 INFO:journalctl@ceph.mds.e.smithi053.stdout:Sep 12 01:39:40 smithi053 systemd[1]: Stopping Ceph mds.e for c0e4f180-70a4-11ef-bcea-c7b262605968... 2024-09-12T01:39:40.990 INFO:journalctl@ceph.mds.e.smithi053.stdout:Sep 12 01:39:40 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-e[64882]: 2024-09-12T01:39:40.625+0000 7f7c2bf30640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:39:40.990 INFO:journalctl@ceph.mds.e.smithi053.stdout:Sep 12 01:39:40 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-e[64882]: 2024-09-12T01:39:40.625+0000 7f7c2bf30640 -1 mds.e *** got signal Terminated *** 2024-09-12T01:39:44.668 INFO:journalctl@ceph.mds.e.smithi053.stdout:Sep 12 01:39:44 smithi053 podman[70303]: 2024-09-12 01:39:44.396156936 +0000 UTC m=+3.873536671 container died a74684ef297f069a2e7e78492019653b137c7ca071217a4df3ad11a4db81cff8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-12T01:39:44.990 INFO:journalctl@ceph.mds.e.smithi053.stdout:Sep 12 01:39:44 smithi053 podman[70303]: 2024-09-12 01:39:44.751489431 +0000 UTC m=+4.228869155 container cleanup a74684ef297f069a2e7e78492019653b137c7ca071217a4df3ad11a4db81cff8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-e, ceph=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-12T01:39:44.990 INFO:journalctl@ceph.mds.e.smithi053.stdout:Sep 12 01:39:44 smithi053 bash[70303]: ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-e 2024-09-12T01:39:45.184 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mds.e.service' 2024-09-12T01:39:45.216 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:39:45.217 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-12T01:39:45.217 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-12T01:39:45.217 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mds.h 2024-09-12T01:39:45.740 INFO:journalctl@ceph.mds.h.smithi053.stdout:Sep 12 01:39:45 smithi053 systemd[1]: Stopping Ceph mds.h for c0e4f180-70a4-11ef-bcea-c7b262605968... 2024-09-12T01:39:45.740 INFO:journalctl@ceph.mds.h.smithi053.stdout:Sep 12 01:39:45 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-h[65182]: 2024-09-12T01:39:45.538+0000 7fd6844e4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:39:45.740 INFO:journalctl@ceph.mds.h.smithi053.stdout:Sep 12 01:39:45 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-h[65182]: 2024-09-12T01:39:45.538+0000 7fd6844e4640 -1 mds.h *** got signal Terminated *** 2024-09-12T01:39:48.672 INFO:journalctl@ceph.mds.h.smithi053.stdout:Sep 12 01:39:48 smithi053 podman[70424]: 2024-09-12 01:39:48.398914351 +0000 UTC m=+2.965917561 container died c5a1c644c4e5956f91f4d2fe71cb138c9c5e9cc036a18b08f4305fbab8d4048e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-h, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-12T01:39:48.990 INFO:journalctl@ceph.mds.h.smithi053.stdout:Sep 12 01:39:48 smithi053 podman[70424]: 2024-09-12 01:39:48.755610953 +0000 UTC m=+3.322614164 container cleanup c5a1c644c4e5956f91f4d2fe71cb138c9c5e9cc036a18b08f4305fbab8d4048e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-h, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-12T01:39:48.990 INFO:journalctl@ceph.mds.h.smithi053.stdout:Sep 12 01:39:48 smithi053 bash[70424]: ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-h 2024-09-12T01:39:49.224 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mds.h.service' 2024-09-12T01:39:49.255 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:39:49.256 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-12T01:39:49.256 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-12T01:39:49.256 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mds.k 2024-09-12T01:39:49.575 INFO:journalctl@ceph.mds.k.smithi053.stdout:Sep 12 01:39:49 smithi053 systemd[1]: Stopping Ceph mds.k for c0e4f180-70a4-11ef-bcea-c7b262605968... 2024-09-12T01:39:49.990 INFO:journalctl@ceph.mds.k.smithi053.stdout:Sep 12 01:39:49 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-k[64570]: 2024-09-12T01:39:49.575+0000 7f687d662640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:39:49.990 INFO:journalctl@ceph.mds.k.smithi053.stdout:Sep 12 01:39:49 smithi053 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-k[64570]: 2024-09-12T01:39:49.575+0000 7f687d662640 -1 mds.k *** got signal Terminated *** 2024-09-12T01:39:52.660 INFO:journalctl@ceph.mds.k.smithi053.stdout:Sep 12 01:39:52 smithi053 podman[70544]: 2024-09-12 01:39:52.404487714 +0000 UTC m=+2.931684778 container died cc65a2bf2316dce33bd40d309b2bd67232e80e947fd0ad9cfe4c0e820739629e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-k, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-12T01:39:52.990 INFO:journalctl@ceph.mds.k.smithi053.stdout:Sep 12 01:39:52 smithi053 podman[70544]: 2024-09-12 01:39:52.76761634 +0000 UTC m=+3.294813398 container cleanup cc65a2bf2316dce33bd40d309b2bd67232e80e947fd0ad9cfe4c0e820739629e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-k, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-12T01:39:52.990 INFO:journalctl@ceph.mds.k.smithi053.stdout:Sep 12 01:39:52 smithi053 bash[70544]: ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-k 2024-09-12T01:39:53.190 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mds.k.service' 2024-09-12T01:39:53.221 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:39:53.222 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-12T01:39:53.222 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-12T01:39:53.222 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mds.c 2024-09-12T01:39:53.575 INFO:journalctl@ceph.mds.c.smithi104.stdout:Sep 12 01:39:53 smithi104 systemd[1]: Stopping Ceph mds.c for c0e4f180-70a4-11ef-bcea-c7b262605968... 2024-09-12T01:39:54.075 INFO:journalctl@ceph.mds.c.smithi104.stdout:Sep 12 01:39:53 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-c[65214]: 2024-09-12T01:39:53.611+0000 7f7cf311a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:39:54.075 INFO:journalctl@ceph.mds.c.smithi104.stdout:Sep 12 01:39:53 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-c[65214]: 2024-09-12T01:39:53.611+0000 7f7cf311a640 -1 mds.c *** got signal Terminated *** 2024-09-12T01:39:56.673 INFO:journalctl@ceph.mds.c.smithi104.stdout:Sep 12 01:39:56 smithi104 podman[70437]: 2024-09-12 01:39:56.395317194 +0000 UTC m=+2.979125984 container died fdc976a557b2baea7786bfbdf2461f4240448ea036e530e2a1df439f2bc93d9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-c, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-12T01:39:57.056 INFO:journalctl@ceph.mds.c.smithi104.stdout:Sep 12 01:39:56 smithi104 podman[70437]: 2024-09-12 01:39:56.823062299 +0000 UTC m=+3.406871090 container cleanup fdc976a557b2baea7786bfbdf2461f4240448ea036e530e2a1df439f2bc93d9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-c, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-12T01:39:57.056 INFO:journalctl@ceph.mds.c.smithi104.stdout:Sep 12 01:39:56 smithi104 bash[70437]: ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-c 2024-09-12T01:39:57.308 INFO:journalctl@ceph.mds.c.smithi104.stdout:Sep 12 01:39:57 smithi104 podman[70461]: 2024-09-12 01:39:57.057545914 +0000 UTC m=+0.663523117 container remove fdc976a557b2baea7786bfbdf2461f4240448ea036e530e2a1df439f2bc93d9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-c, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-12T01:39:57.313 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mds.c.service' 2024-09-12T01:39:57.349 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:39:57.349 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-12T01:39:57.349 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-12T01:39:57.350 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mds.f 2024-09-12T01:39:57.576 INFO:journalctl@ceph.mds.f.smithi104.stdout:Sep 12 01:39:57 smithi104 systemd[1]: Stopping Ceph mds.f for c0e4f180-70a4-11ef-bcea-c7b262605968... 2024-09-12T01:39:58.075 INFO:journalctl@ceph.mds.f.smithi104.stdout:Sep 12 01:39:57 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-f[64914]: 2024-09-12T01:39:57.668+0000 7f6271822640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:39:58.075 INFO:journalctl@ceph.mds.f.smithi104.stdout:Sep 12 01:39:57 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-f[64914]: 2024-09-12T01:39:57.668+0000 7f6271822640 -1 mds.f *** got signal Terminated *** 2024-09-12T01:40:00.709 INFO:journalctl@ceph.mds.f.smithi104.stdout:Sep 12 01:40:00 smithi104 podman[70570]: 2024-09-12 01:40:00.391247446 +0000 UTC m=+2.820363051 container died 053c647063aced76c0d4d83caea83bc24b246a57d2de4738bc9e61bf89a6bfae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-f, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T01:40:00.709 INFO:journalctl@ceph.mds.f.smithi104.stdout:Sep 12 01:40:00 smithi104 podman[70570]: 2024-09-12 01:40:00.710088532 +0000 UTC m=+3.139204131 container cleanup 053c647063aced76c0d4d83caea83bc24b246a57d2de4738bc9e61bf89a6bfae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-f, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-12T01:40:01.075 INFO:journalctl@ceph.mds.f.smithi104.stdout:Sep 12 01:40:00 smithi104 bash[70570]: ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-f 2024-09-12T01:40:01.075 INFO:journalctl@ceph.mds.f.smithi104.stdout:Sep 12 01:40:00 smithi104 podman[70583]: 2024-09-12 01:40:00.936167523 +0000 UTC m=+0.538457755 container remove 053c647063aced76c0d4d83caea83bc24b246a57d2de4738bc9e61bf89a6bfae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-f, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-12T01:40:01.139 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mds.f.service' 2024-09-12T01:40:01.173 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:40:01.173 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-12T01:40:01.173 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-12T01:40:01.173 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mds.i 2024-09-12T01:40:01.488 INFO:journalctl@ceph.mds.i.smithi104.stdout:Sep 12 01:40:01 smithi104 systemd[1]: Stopping Ceph mds.i for c0e4f180-70a4-11ef-bcea-c7b262605968... 2024-09-12T01:40:01.825 INFO:journalctl@ceph.mds.i.smithi104.stdout:Sep 12 01:40:01 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-i[65514]: 2024-09-12T01:40:01.488+0000 7f9bb14a4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:40:01.825 INFO:journalctl@ceph.mds.i.smithi104.stdout:Sep 12 01:40:01 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-i[65514]: 2024-09-12T01:40:01.488+0000 7f9bb14a4640 -1 mds.i *** got signal Terminated *** 2024-09-12T01:40:04.703 INFO:journalctl@ceph.mds.i.smithi104.stdout:Sep 12 01:40:04 smithi104 podman[70693]: 2024-09-12 01:40:04.388447145 +0000 UTC m=+2.986424667 container died ee23f250750b6cc74e5969642ad3af70c874ed32e4c93c0175e44377f9bf53e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-i, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T01:40:04.703 INFO:journalctl@ceph.mds.i.smithi104.stdout:Sep 12 01:40:04 smithi104 podman[70693]: 2024-09-12 01:40:04.704430419 +0000 UTC m=+3.302407931 container cleanup ee23f250750b6cc74e5969642ad3af70c874ed32e4c93c0175e44377f9bf53e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-i, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-12T01:40:05.075 INFO:journalctl@ceph.mds.i.smithi104.stdout:Sep 12 01:40:04 smithi104 bash[70693]: ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-i 2024-09-12T01:40:05.075 INFO:journalctl@ceph.mds.i.smithi104.stdout:Sep 12 01:40:04 smithi104 podman[70705]: 2024-09-12 01:40:04.914099495 +0000 UTC m=+0.520623767 container remove ee23f250750b6cc74e5969642ad3af70c874ed32e4c93c0175e44377f9bf53e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-i, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T01:40:05.132 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mds.i.service' 2024-09-12T01:40:05.164 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:40:05.165 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-12T01:40:05.165 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-12T01:40:05.165 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mds.l 2024-09-12T01:40:05.541 INFO:journalctl@ceph.mds.l.smithi104.stdout:Sep 12 01:40:05 smithi104 systemd[1]: Stopping Ceph mds.l for c0e4f180-70a4-11ef-bcea-c7b262605968... 2024-09-12T01:40:05.826 INFO:journalctl@ceph.mds.l.smithi104.stdout:Sep 12 01:40:05 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-l[64617]: 2024-09-12T01:40:05.541+0000 7f1cce172640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:40:05.827 INFO:journalctl@ceph.mds.l.smithi104.stdout:Sep 12 01:40:05 smithi104 ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-l[64617]: 2024-09-12T01:40:05.541+0000 7f1cce172640 -1 mds.l *** got signal Terminated *** 2024-09-12T01:40:08.674 INFO:journalctl@ceph.mds.l.smithi104.stdout:Sep 12 01:40:08 smithi104 podman[70816]: 2024-09-12 01:40:08.391657015 +0000 UTC m=+2.997114973 container died 53a28893b85e8d057fc1c1d89ec9d0ff54aaf2830560fbca9bea04e6feb740e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-l, GIT_CLEAN=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-12T01:40:09.024 INFO:journalctl@ceph.mds.l.smithi104.stdout:Sep 12 01:40:08 smithi104 podman[70816]: 2024-09-12 01:40:08.758097253 +0000 UTC m=+3.363555209 container cleanup 53a28893b85e8d057fc1c1d89ec9d0ff54aaf2830560fbca9bea04e6feb740e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-12T01:40:09.024 INFO:journalctl@ceph.mds.l.smithi104.stdout:Sep 12 01:40:08 smithi104 bash[70816]: ceph-c0e4f180-70a4-11ef-bcea-c7b262605968-mds-l 2024-09-12T01:40:09.145 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e4f180-70a4-11ef-bcea-c7b262605968@mds.l.service' 2024-09-12T01:40:09.177 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:40:09.178 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-12T01:40:09.178 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T01:40:09.460 INFO:teuthology.orchestra.run.smithi001.stdout:Deleting cluster with fsid: c0e4f180-70a4-11ef-bcea-c7b262605968 2024-09-12T01:40:13.492 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T01:40:13.701 INFO:teuthology.orchestra.run.smithi053.stdout:Deleting cluster with fsid: c0e4f180-70a4-11ef-bcea-c7b262605968 2024-09-12T01:40:16.184 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T01:40:16.384 INFO:teuthology.orchestra.run.smithi104.stdout:Deleting cluster with fsid: c0e4f180-70a4-11ef-bcea-c7b262605968 2024-09-12T01:40:18.754 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T01:40:18.821 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T01:40:18.865 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T01:40:19.382 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-12T01:40:19.384 DEBUG:teuthology.misc:Transferring archived files from smithi001:/var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901536/remote/smithi001/crash 2024-09-12T01:40:19.385 DEBUG:teuthology.orchestra.run.smithi001:> sudo tar c -f - -C /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/crash -- . 2024-09-12T01:40:19.472 INFO:teuthology.orchestra.run.smithi001.stderr:tar: /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T01:40:19.473 INFO:teuthology.orchestra.run.smithi001.stderr:tar: Error is not recoverable: exiting now 2024-09-12T01:40:19.476 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901536/remote/smithi053/crash 2024-09-12T01:40:19.477 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/crash -- . 2024-09-12T01:40:19.526 INFO:teuthology.orchestra.run.smithi053.stderr:tar: /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T01:40:19.526 INFO:teuthology.orchestra.run.smithi053.stderr:tar: Error is not recoverable: exiting now 2024-09-12T01:40:19.528 DEBUG:teuthology.misc:Transferring archived files from smithi104:/var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901536/remote/smithi104/crash 2024-09-12T01:40:19.529 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/crash -- . 2024-09-12T01:40:19.567 INFO:teuthology.orchestra.run.smithi104.stderr:tar: /var/lib/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T01:40:19.568 INFO:teuthology.orchestra.run.smithi104.stderr:tar: Error is not recoverable: exiting now 2024-09-12T01:40:19.569 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-12T01:40:19.569 DEBUG:teuthology.orchestra.run.smithi001:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-12T01:40:19.652 INFO:tasks.cephadm:Compressing logs... 2024-09-12T01:40:19.652 DEBUG:teuthology.orchestra.run.smithi001:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T01:40:19.656 DEBUG:teuthology.orchestra.run.smithi053:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T01:40:19.659 DEBUG:teuthology.orchestra.run.smithi104:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T01:40:19.684 INFO:teuthology.orchestra.run.smithi053.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-osd.10.log 2024-09-12T01:40:19.684 INFO:teuthology.orchestra.run.smithi053.stderr:: No such file or directory 2024-09-12T01:40:19.684 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-12T01:40:19.684 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-12T01:40:19.687 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T01:40:19.687 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mds.b.log 2024-09-12T01:40:19.687 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mds.k.log 2024-09-12T01:40:19.687 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-12T01:40:19.687 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T01:40:19.687 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T01:40:19.689 INFO:teuthology.orchestra.run.smithi104.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T01:40:19.689 INFO:teuthology.orchestra.run.smithi104.stderr:: No such file or directory 2024-09-12T01:40:19.689 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T01:40:19.689 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mds.i.log 2024-09-12T01:40:19.689 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-12T01:40:19.689 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T01:40:19.691 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-osd.11.log 2024-09-12T01:40:19.691 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mon.c.log: /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-12T01:40:19.691 INFO:teuthology.orchestra.run.smithi104.stderr: 82.2% -- replaced with /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-12T01:40:19.691 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-12T01:40:19.691 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph.log 2024-09-12T01:40:19.691 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-osd.2.log: /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mds.l.log 2024-09-12T01:40:19.691 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph.log: 90.4% -- replaced with /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T01:40:19.692 INFO:teuthology.orchestra.run.smithi104.stderr: 86.5% -- replaced with /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T01:40:19.692 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-osd.8.log 2024-09-12T01:40:19.694 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph.audit.log: 90.3% -- replaced with /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T01:40:19.694 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-12T01:40:19.694 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph.cephadm.log: 82.3% -- replaced with /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-12T01:40:19.694 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph.log 2024-09-12T01:40:19.694 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mgr.z.log 2024-09-12T01:40:19.697 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mds.l.log: /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mds.c.log 2024-09-12T01:40:19.700 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mgr.y.log: 92.6% -- replaced with /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-12T01:40:19.700 INFO:teuthology.orchestra.run.smithi053.stderr: 90.9% -- replaced with /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T01:40:19.701 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mds.h.log 2024-09-12T01:40:19.702 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mgr.z.log: 90.9% -- replaced with /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T01:40:19.703 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph.log: 86.5% -- replaced with /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T01:40:19.704 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-12T01:40:19.705 INFO:teuthology.orchestra.run.smithi104.stderr: 92.8% -- replaced with /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mgr.z.log.gz 2024-09-12T01:40:19.705 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mds.e.log 2024-09-12T01:40:19.706 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mds.f.log 2024-09-12T01:40:19.708 INFO:teuthology.orchestra.run.smithi001.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory /var/log/ceph/ceph-client.admin.log 2024-09-12T01:40:19.708 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-09-12T01:40:19.709 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-12T01:40:19.709 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-osd.9.log 2024-09-12T01:40:19.710 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T01:40:19.711 INFO:teuthology.orchestra.run.smithi001.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-09-12T01:40:19.711 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T01:40:19.712 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mds.a.log 2024-09-12T01:40:19.714 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T01:40:19.715 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-12T01:40:19.716 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mds.g.log 2024-09-12T01:40:19.717 INFO:teuthology.orchestra.run.smithi001.stderr: 82.3% -- replaced with /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-12T01:40:19.717 INFO:teuthology.orchestra.run.smithi001.stderr: 90.2% -- replaced with /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T01:40:19.718 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mon.b.log: /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T01:40:19.718 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-12T01:40:19.719 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph.log 2024-09-12T01:40:19.721 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T01:40:19.722 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-osd.6.log: /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mds.d.log 2024-09-12T01:40:19.724 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-12T01:40:19.728 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mds.d.log: 90.9% -- replaced with /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T01:40:19.730 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mgr.x.log: 86.5% -- replaced with /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T01:40:19.730 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mds.j.log 2024-09-12T01:40:19.732 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-12T01:40:19.734 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mds.j.log: 94.5% -- replaced with /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mds.d.log.gz 2024-09-12T01:40:19.738 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T01:40:19.739 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T01:40:19.742 INFO:teuthology.orchestra.run.smithi053.stderr: 94.4% -- replaced with /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mds.k.log.gz 2024-09-12T01:40:19.746 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.2% 93.0% -- replaced with /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mds.i.log.gz 2024-09-12T01:40:19.749 INFO:teuthology.orchestra.run.smithi104.stderr: -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T01:40:19.753 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/cephadm.log: 94.6% -- replaced with /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mds.j.log.gz 2024-09-12T01:40:19.766 INFO:teuthology.orchestra.run.smithi001.stderr: 93.6% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T01:40:19.778 INFO:teuthology.orchestra.run.smithi001.stderr: 94.2% -- replaced with /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mds.g.log.gz 2024-09-12T01:40:19.780 INFO:teuthology.orchestra.run.smithi001.stderr: 93.8% -- replaced with /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mds.a.log.gz 2024-09-12T01:40:19.787 INFO:teuthology.orchestra.run.smithi104.stderr: 93.2% -- replaced with /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mds.f.log.gz 2024-09-12T01:40:19.823 INFO:teuthology.orchestra.run.smithi104.stderr: 92.9% -- replaced with /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mds.c.log.gz 2024-09-12T01:40:20.012 INFO:teuthology.orchestra.run.smithi001.stderr: 90.0% -- replaced with /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-12T01:40:20.100 INFO:teuthology.orchestra.run.smithi053.stderr: 94.9% -- replaced with /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mds.e.log.gz 2024-09-12T01:40:20.102 INFO:teuthology.orchestra.run.smithi104.stderr: 91.7% -- replaced with /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-12T01:40:20.127 INFO:teuthology.orchestra.run.smithi053.stderr: 91.7% -- replaced with /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-12T01:40:20.850 INFO:teuthology.orchestra.run.smithi001.stderr: 90.4% -- replaced with /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-12T01:40:21.741 INFO:teuthology.orchestra.run.smithi104.stderr: 95.9% -- replaced with /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mds.l.log.gz 2024-09-12T01:40:22.703 INFO:teuthology.orchestra.run.smithi053.stderr: 93.9% -- replaced with /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mds.b.log.gz 2024-09-12T01:40:27.966 INFO:teuthology.orchestra.run.smithi104.stderr: 92.0% -- replaced with /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-osd.11.log.gz 2024-09-12T01:40:29.084 INFO:teuthology.orchestra.run.smithi053.stderr: 92.4% -- replaced with /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-12T01:40:29.490 INFO:teuthology.orchestra.run.smithi053.stderr: 92.2% -- replaced with /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-osd.10.log.gz 2024-09-12T01:40:31.946 INFO:teuthology.orchestra.run.smithi104.stderr: 91.6% -- replaced with /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-osd.8.log.gz 2024-09-12T01:40:33.245 INFO:teuthology.orchestra.run.smithi001.stderr: 92.1% -- replaced with /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-12T01:40:35.546 INFO:teuthology.orchestra.run.smithi053.stderr: 92.1% -- replaced with /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-12T01:40:37.641 INFO:teuthology.orchestra.run.smithi001.stderr: 91.7% -- replaced with /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-12T01:40:38.930 INFO:teuthology.orchestra.run.smithi001.stderr: 91.3% -- replaced with /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-12T01:40:42.394 INFO:teuthology.orchestra.run.smithi053.stderr: 91.9% -- replaced with /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-12T01:40:43.440 INFO:teuthology.orchestra.run.smithi104.stderr: 91.6% -- replaced with /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-12T01:40:46.382 INFO:teuthology.orchestra.run.smithi104.stderr: 91.6% -- replaced with /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-12T01:40:46.384 INFO:teuthology.orchestra.run.smithi104.stderr: 2024-09-12T01:40:46.384 INFO:teuthology.orchestra.run.smithi104.stderr:real 0m26.709s 2024-09-12T01:40:46.384 INFO:teuthology.orchestra.run.smithi104.stderr:user 1m11.340s 2024-09-12T01:40:46.384 INFO:teuthology.orchestra.run.smithi104.stderr:sys 0m2.231s 2024-09-12T01:40:51.619 INFO:teuthology.orchestra.run.smithi053.stderr: 93.3% -- replaced with /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-mds.h.log.gz 2024-09-12T01:40:51.620 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-09-12T01:40:51.621 INFO:teuthology.orchestra.run.smithi053.stderr:real 0m31.948s 2024-09-12T01:40:51.621 INFO:teuthology.orchestra.run.smithi053.stderr:user 1m30.706s 2024-09-12T01:40:51.621 INFO:teuthology.orchestra.run.smithi053.stderr:sys 0m2.662s 2024-09-12T01:40:57.302 INFO:teuthology.orchestra.run.smithi001.stderr: 91.4% -- replaced with /var/log/ceph/c0e4f180-70a4-11ef-bcea-c7b262605968/ceph-osd.9.log.gz 2024-09-12T01:40:57.305 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-09-12T01:40:57.305 INFO:teuthology.orchestra.run.smithi001.stderr:real 0m37.620s 2024-09-12T01:40:57.306 INFO:teuthology.orchestra.run.smithi001.stderr:user 1m23.299s 2024-09-12T01:40:57.306 INFO:teuthology.orchestra.run.smithi001.stderr:sys 0m6.584s 2024-09-12T01:40:57.307 INFO:tasks.cephadm:Archiving logs... 2024-09-12T01:40:57.308 DEBUG:teuthology.misc:Transferring archived files from smithi001:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901536/remote/smithi001/log 2024-09-12T01:40:57.309 DEBUG:teuthology.orchestra.run.smithi001:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T01:41:11.074 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901536/remote/smithi053/log 2024-09-12T01:41:11.075 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T01:41:23.682 DEBUG:teuthology.misc:Transferring archived files from smithi104:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901536/remote/smithi104/log 2024-09-12T01:41:23.683 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T01:41:35.814 INFO:tasks.cephadm:Removing cluster... 2024-09-12T01:41:35.815 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 --force 2024-09-12T01:41:36.099 INFO:teuthology.orchestra.run.smithi001.stdout:Deleting cluster with fsid: c0e4f180-70a4-11ef-bcea-c7b262605968 2024-09-12T01:41:36.903 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 --force 2024-09-12T01:41:37.112 INFO:teuthology.orchestra.run.smithi053.stdout:Deleting cluster with fsid: c0e4f180-70a4-11ef-bcea-c7b262605968 2024-09-12T01:41:37.474 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c0e4f180-70a4-11ef-bcea-c7b262605968 --force 2024-09-12T01:41:37.673 INFO:teuthology.orchestra.run.smithi104.stdout:Deleting cluster with fsid: c0e4f180-70a4-11ef-bcea-c7b262605968 2024-09-12T01:41:38.043 INFO:tasks.cephadm:Removing cephadm ... 2024-09-12T01:41:38.043 DEBUG:teuthology.orchestra.run.smithi001:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T01:41:38.080 DEBUG:teuthology.orchestra.run.smithi053:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T01:41:38.098 DEBUG:teuthology.orchestra.run.smithi104:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T01:41:38.114 INFO:tasks.cephadm:Teardown complete 2024-09-12T01:41:38.114 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-12T01:41:38.127 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-12T01:41:38.128 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T01:41:38.132 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T01:41:38.141 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T01:41:38.290 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-12T01:41:38.300 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-12T01:41:38.301 DEBUG:teuthology.orchestra.run.smithi001:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T01:41:38.306 DEBUG:teuthology.orchestra.run.smithi053:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T01:41:38.309 DEBUG:teuthology.orchestra.run.smithi104:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T01:41:38.322 INFO:teuthology.orchestra.run.smithi053.stderr:bash: line 1: ntpq: command not found 2024-09-12T01:41:38.324 INFO:teuthology.orchestra.run.smithi104.stderr:bash: line 1: ntpq: command not found 2024-09-12T01:41:38.327 INFO:teuthology.orchestra.run.smithi053.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T01:41:38.327 INFO:teuthology.orchestra.run.smithi053.stdout:=============================================================================== 2024-09-12T01:41:38.327 INFO:teuthology.orchestra.run.smithi053.stdout:^* hv01.front.sepia.ceph.com 3 6 377 43 -2052us[-2056us] +/- 61ms 2024-09-12T01:41:38.327 INFO:teuthology.orchestra.run.smithi053.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 45 -92us[ -96us] +/- 59ms 2024-09-12T01:41:38.327 INFO:teuthology.orchestra.run.smithi053.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 44 +2624us[+2620us] +/- 71ms 2024-09-12T01:41:38.327 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:41:38.329 INFO:teuthology.orchestra.run.smithi104.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T01:41:38.329 INFO:teuthology.orchestra.run.smithi104.stdout:=============================================================================== 2024-09-12T01:41:38.329 INFO:teuthology.orchestra.run.smithi104.stdout:^* hv01.front.sepia.ceph.com 3 6 377 47 -2149us[-2161us] +/- 61ms 2024-09-12T01:41:38.329 INFO:teuthology.orchestra.run.smithi104.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 42 -157us[ -157us] +/- 59ms 2024-09-12T01:41:38.329 INFO:teuthology.orchestra.run.smithi104.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 46 +2551us[+2551us] +/- 71ms 2024-09-12T01:41:38.329 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:41:38.334 INFO:teuthology.orchestra.run.smithi001.stderr:bash: line 1: ntpq: command not found 2024-09-12T01:41:38.341 INFO:teuthology.orchestra.run.smithi001.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T01:41:38.341 INFO:teuthology.orchestra.run.smithi001.stdout:=============================================================================== 2024-09-12T01:41:38.341 INFO:teuthology.orchestra.run.smithi001.stdout:^* hv01.front.sepia.ceph.com 3 6 377 45 -2071us[-2094us] +/- 61ms 2024-09-12T01:41:38.341 INFO:teuthology.orchestra.run.smithi001.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 43 -93us[ -93us] +/- 59ms 2024-09-12T01:41:38.341 INFO:teuthology.orchestra.run.smithi001.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 44 +2630us[+2630us] +/- 71ms 2024-09-12T01:41:38.341 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:41:38.343 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-12T01:41:38.353 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-12T01:41:38.353 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-12T01:41:38.450 DEBUG:teuthology.orchestra.run.smithi001:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T01:41:38.578 DEBUG:teuthology.orchestra.run.smithi053:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T01:41:38.654 DEBUG:teuthology.orchestra.run.smithi104:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T01:41:38.722 DEBUG:teuthology.orchestra.run.smithi001:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T01:41:38.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T01:41:38.784 DEBUG:teuthology.orchestra.run.smithi053:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T01:41:38.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T01:41:38.815 DEBUG:teuthology.orchestra.run.smithi104:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T01:41:38.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T01:41:38.843 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-12T01:41:38.854 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-12T01:41:38.864 INFO:teuthology.task.internal:Duration was 1888.610779 seconds 2024-09-12T01:41:38.865 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-12T01:41:38.874 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-12T01:41:38.874 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T01:41:38.879 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T01:41:38.881 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T01:41:38.921 INFO:teuthology.orchestra.run.smithi053.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T01:41:38.921 INFO:teuthology.orchestra.run.smithi104.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T01:41:38.962 INFO:teuthology.orchestra.run.smithi001.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T01:41:39.281 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-12T01:41:39.281 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi001.front.sepia.ceph.com 2024-09-12T01:41:39.281 DEBUG:teuthology.orchestra.run.smithi001:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T01:41:39.331 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi053.front.sepia.ceph.com 2024-09-12T01:41:39.331 DEBUG:teuthology.orchestra.run.smithi053:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T01:41:39.362 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi104.front.sepia.ceph.com 2024-09-12T01:41:39.362 DEBUG:teuthology.orchestra.run.smithi104:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T01:41:39.393 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-12T01:41:39.394 DEBUG:teuthology.orchestra.run.smithi001:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T01:41:39.398 DEBUG:teuthology.orchestra.run.smithi053:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T01:41:39.405 DEBUG:teuthology.orchestra.run.smithi104:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T01:41:39.459 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-12T01:41:39.460 DEBUG:teuthology.orchestra.run.smithi001:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T01:41:39.463 DEBUG:teuthology.orchestra.run.smithi053:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T01:41:39.474 DEBUG:teuthology.orchestra.run.smithi104:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T01:41:40.066 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-12T01:41:40.077 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-12T01:41:40.078 DEBUG:teuthology.orchestra.run.smithi001:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T01:41:40.134 DEBUG:teuthology.orchestra.run.smithi053:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T01:41:40.159 DEBUG:teuthology.orchestra.run.smithi104:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T01:41:40.186 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-12T01:41:40.197 DEBUG:teuthology.orchestra.run.smithi001:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T01:41:40.202 DEBUG:teuthology.orchestra.run.smithi053:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T01:41:40.205 DEBUG:teuthology.orchestra.run.smithi104:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T01:41:40.226 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern = core 2024-09-12T01:41:40.250 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern = core 2024-09-12T01:41:40.269 INFO:teuthology.orchestra.run.smithi001.stdout:kernel.core_pattern = core 2024-09-12T01:41:40.298 DEBUG:teuthology.orchestra.run.smithi001:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T01:41:40.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T01:41:40.331 DEBUG:teuthology.orchestra.run.smithi053:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T01:41:40.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T01:41:40.346 DEBUG:teuthology.orchestra.run.smithi104:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T01:41:40.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T01:41:40.361 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-12T01:41:40.373 INFO:teuthology.task.internal:Transferring archived files... 2024-09-12T01:41:40.373 DEBUG:teuthology.misc:Transferring archived files from smithi001:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901536/remote/smithi001 2024-09-12T01:41:40.374 DEBUG:teuthology.orchestra.run.smithi001:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T01:41:40.450 DEBUG:teuthology.misc:Transferring archived files from smithi053:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901536/remote/smithi053 2024-09-12T01:41:40.451 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T01:41:40.486 DEBUG:teuthology.misc:Transferring archived files from smithi104:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901536/remote/smithi104 2024-09-12T01:41:40.486 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T01:41:40.522 INFO:teuthology.task.internal:Removing archive directory... 2024-09-12T01:41:40.522 DEBUG:teuthology.orchestra.run.smithi001:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T01:41:40.526 DEBUG:teuthology.orchestra.run.smithi053:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T01:41:40.529 DEBUG:teuthology.orchestra.run.smithi104:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T01:41:40.572 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-12T01:41:40.586 INFO:teuthology.task.internal:Not uploading archives. 2024-09-12T01:41:40.586 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-12T01:41:40.597 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-12T01:41:40.597 DEBUG:teuthology.orchestra.run.smithi001:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T01:41:40.602 DEBUG:teuthology.orchestra.run.smithi053:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T01:41:40.604 DEBUG:teuthology.orchestra.run.smithi104:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T01:41:40.617 INFO:teuthology.orchestra.run.smithi053.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 01:41 /home/ubuntu/cephtest 2024-09-12T01:41:40.628 INFO:teuthology.orchestra.run.smithi104.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 01:41 /home/ubuntu/cephtest 2024-09-12T01:41:40.632 INFO:teuthology.orchestra.run.smithi001.stdout: 393233 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 01:41 /home/ubuntu/cephtest 2024-09-12T01:41:40.635 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-12T01:41:40.648 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-12T01:41:40.679 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/suites/fsx}} duration: 1888.6107790470123 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-12T01:41:40.679 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-12T01:41:40.767 INFO:teuthology.run:pass